GORM support for Hadoop HBase
Dependency :
compile ":gorm-hbase:0.2.4"
Summary
Installation
Pre-requisites
- Grails 1.3.1
- Java 6
- Hadoop Common 0.20.2 (which must be run as pseudo or fully distributed, not standalone)
- HBase 0.20.4 (which must be run as pseudo or fully distributed, not standalone)
Install
- Remove the default Hibernate plug-in
grails uninstall-plugin hibernate
- Install the GORM-HBase plugin
grails install-plugin gorm-hbase
Required Configuration
- HBase CLASSPATH element in
hbase-env.shto include the transactional jar file, e.g.
export HBASE_CLASSPATH=$HBASE_INSTALL/contrib/transactional/hbase-0.20.4-transactional.jar
- If Grails is not running on the same machine as your ZooKeeper Master configure
<grails-appl-base-dir>/conf/hbase-site.xmlto point to the name or address of the machine running your ZooKeeper Master,
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
<description>Comma separated list of servers in the ZooKeeper Quorum.
For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
By default this is set to localhost for local and pseudo-distributed modes
of operation. For a fully-distributed setup, this should be set to a full
list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh
this is the list of servers which we will start/stop ZooKeeper on.
</description>
</property>Optional Configuration
- Edit
<grails-appl-base-dir>/conf/DataSource.groovyto control whether HBase tables are, - either dropped (if they already exist) then created, or, left untouched, and whether
- sorted queries are acticated by enabling indexing on the HBase datasource when the application or tests start.
dbCreate element is honored. By default both tables and data are not dropped, created, or updated in any way unless this one option is coded,dbCreate = "create-drop"dataSource {
dbCreate = "update"
indexed = true
}- Edit
<grails-appl-base-dir>/conf/Config.groovyto control logging. Until you are comfortable with the behavior of this plug-in it is recommended you at least configureinfolevel debugging,
info 'org.grails.hbase'
Description
Overview
This plug-in persists domain class data to a Hadoop HBase 'nosql' database and implements the GORM api to facilitate retrieval of persisted data.Release 0.2.4
Upgraded to run on the following,- Grails 1.3.2
Release 0.2.3
Upgraded to run on the following,- Grails 1.3.1
- Hadoop 0.20.2
- HBase 0.20.4
Release 0.2.2
Initial SQL-like capabilities provided in extended,- findAll() support
- find() support
Release 0.2.1
- Dynamic Finders
- Ability to define finder filters as native HBase filters
- find() support
- Multiple And/Or operators in method names now understood by dynamic finders
- 'Like' and 'NotLike' are now valid operators in dynamic finder names and use regular expressions
- 'Operator.LIKE' and 'Operator.NOT_LIKE' are supported by FinderFilters
Release 0.2
- Dynamic Finders,
- findAll()
- findAllBy()
- findBy()
- Improved consistency with the default Hibernate based GORM implementation,
- Return null when get() fails to find an object
- Return correct values from save()
- Return null from delete()
- Support getters on one-to-many association
- Support of standard HBase 0.20.3 instead of custom version of 0.20.0
- All Grails supported basic types are persisted in an upgradeable manner
Release 0.1
- DataSource.groovy - the
dbCreatevalue can be set to"create-drop"to drop existing HBase tables and create new tables. Any other setting should leave the HBase database untouched. - A subset of GORM related Domain class methods and properties,
- 'count()' with no arguments
- 'delete()' with no arguments
- 'get()' with the
idargument - 'hasErrors()' with no arguments
- 'list()' with support for
max,offsetandsortarguments - 'save()' with no arguments
- the
errorsproperty - Optimistic locking
- Lazy loading
- Associations between domain classes,
- One to One
- One to Many
- All Grails supported basic types can be persisted
- Secondary index support using HBase Indexed Tables
- Table name spaces - a similar idea to schema names in RDBMS - whereby each table created has a name prefixed with an identifier constructed from the Grails application name
findAll()
Instead of the Hibernate Query Language (HQL) being supported this plug-in provides two alternate options,- Small number of api classes
- HBase filters
- SQL-like capabilities
DynamicFinderFilter, FinderFilter, FinderFilterList and Operator
// The domain class class Book {
String title
String author
Date releaseDate
}// everything
def results = Book.findAll()// a basic example using built-in filter support
import org.grails.hbase.api.finders.FinderFilter
import org.grails.hbase.api.finders.FinderFilterList
import org.grails.hbase.api.finders.Operator // defaults to 'Operator.EQUAL',
DynamicFinderFilter filter = new FinderFilter('author', 'Dan Brown')
results = Book.findAll(filter)// the same finder with a native HBase filter,
import org.apache.hadoop.hbase.filter.Filter
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter
import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp
import org.apache.hadoop.hbase.util.Bytes import org.grails.hbase.store.Constants
import org.grails.hbase.util.HBaseNameUtils def columnName = Bytes.toBytes(HBaseNameUtils.getDomainColumnName('name'))
def value = Bytes.toBytes('Dan Brown')
Filter filter = new SingleColumnValueFilter(Constants.DEFAULT_DATA_FAMILY, columnName, CompareOp.EQUAL, value) results = Book.findAll(filter)// examples with max/offset usage DynamicFinderFilter filter = new FinderFilter('author', Operator.NOT_EQUAL, 'Dan Brown') // first 10 books without Dan Brown as the author
results = Book.findAll(filter, [max:10]) // 10 books without Dan Brown as the author starting from the 5th book
results = Book.findAll(filter, [max:10, offset:5]) // 10 books from Dan Brown staring from 5th book ordered by release date
DynamicFinderFilter filter = new FinderFilter('author', 'Dan Brown')
results = Book.findAll(filter, [max:10, offset:5, sort:'releaseDate'])// example with multiple filters // all books written by Dan Brown or J K Rowling
DynamicFinderFilter filterList = new FinderFilterList(Operator.OR) DynamicFinderFilter filter1 = new Filter('author', 'Dan Brown')
filterList.addFilter(filter1) DynamicFinderFilter filter2 = new Filter('author', 'J K Rowling')
filterList.addFilter(filter12) results = Book.findAll(filterList)// examples using SQl-like syntax
// All books written by Terry Pratchett
def results = Book.findAll("author = 'Terry Pratchett’", [sort:'title']) // All books written by Terry Pratchett or Dan Brown
results = Book.findAll("author = 'Terry Pratchett’ or author = ‘Dan Brown’“) // All books written by Terry Pratchett from May 31st, 2005
results = Book.findAll("author = 'Terry Pratchett’ and releaseDate >= ‘2005-05-31’“) // All books written by Dan Brown or books written in 2000 by Terry Pratchett
results = Book.findAll(“â€â€
author = ‘Dan Brown’
or
( author = 'Terry Pratchett’
and
releaseDate >= ‘2000-01-01’
and
releaseDate <= ‘2000-12-31’
)
“â€â€)EQUAL, NOT_EQUAL, GREATER, GREATER_OR_EQUAL, LESS, LESS_OR_EQUAL, LIKE and NOT_LIKE
AND and OR
=, <, <=, >, >=, !=
find()
The same syntax as findAll() above, the only difference is that this method returns just one object not a list of objects,// Find Dan Brown's third book with a single filter,
DynamicFinderFilter filter = new FinderFilter('author', 'Dan Brown')
results = Book.find(filter, [offset:3])findAllBy()
This finder is very similar to the default Hibernate based implementation. For example,// The domain class
class Book {
String title
String author
Date releaseDate
}// basic examples
def results = Book.findAllByAuthor('Dan Brown', [max:10, sort:'title', offset:2] )
results = Book.findAllByTitleNotEqualAndAuthor('Digital Fortress', 'Dan Brown')
results = Book.findAllByReleaseDateGreaterThanEquals(firstDate)// example with Like using a regular expression
// All books written by an author called Brown
Book.findAllByAuthorLike(".+Brown") // All books not written by authors called Brown or Rowling
Book.findAllByAuthorNotLike(".+(Brown|Rowling)")// multiple and/or conditions
// 'and' is resolved before 'or' (c.f. Java) so this finds all books released in 2001 by J K Rowling books or
// in 2002 by Dan Brown
Book.findAllByAuthorAndReleaseDateOrAuthorAndReleaseDate("J K Rowling", date2001, "Dan Brown", date 2002)findBy()
This finder is very similar to the default Hibernate based implementation. Effectively it is exactly the same as the findAllBy() finder above,// The domain class
class Book {
String title
String author
Date releaseDate
}Book result = Book.findByAuthor('Dan Brown', [max:10, sort:'title', offset:2] )
result = Book.findByTitleNotEqualAndAuthor('Digital Fortress', 'Dan Brown')
result = Book.findByReleaseDateGreaterThanEquals(firstDate)