1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
/** ? *?coprocessor
? *?当用户在使用get命令从表中取特定的row时,就会触发这个自定义的observer?coprocessor
? *?触发条件是用户使用get指定的rowkey与程序中指定的FIXED_ROW一致为@@@GETTIME@@@时
? *?触发后的操作是程序会在服务端生成一个keyvalue实例,并将这个实例返回给客户端。这个kv实例是以
? *?@@@GETTIME@@@为rowkey,列族和列标识符均为@@@GETTIME@@@,列值为服务器端的时间
? */
?
package?org.apache.hbase.kora.coprocessor; ?
import?java.io.IOException; import?java.util.List; ?
import?org.apache.commons.logging.Log; import?org.apache.commons.logging.LogFactory; import?org.apache.hadoop.hbase.KeyValue; import?org.apache.hadoop.hbase.client.Get; import?org.apache.hadoop.hbase.coprocessor.BaseRegionObserver; import?org.apache.hadoop.hbase.coprocessor.ObserverContext; import?org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; import?org.apache.hadoop.hbase.regionserver.HRegion; import?org.apache.hadoop.hbase.util.Bytes; ?
public ?class ?RegionObserverExample? extends ?BaseRegionObserver?{
???? public ?static ?final ?Log?LOG?=?LogFactory.getLog(HRegion. class );
???? public ?static ?final ?byte[]?FIXED_ROW?=?Bytes.toBytes( "@@@GETTIME@@@" );
?
???? @Override
???? public ?void?preGet(ObserverContext<RegionCoprocessorEnvironment>?c,
???????????? Get?get,?List<KeyValue>?result)?throws?IOException?{
???????? LOG.debug( "Got?preGet?for?row:?" ?+?Bytes.toStringBinary(get.getRow()));
???????? ????????? if ?(Bytes.equals(get.getRow(),?FIXED_ROW))?{
???????????? KeyValue?kv?=? new ?KeyValue(get.getRow(),?FIXED_ROW,?FIXED_ROW,
???????????????????? Bytes.toBytes(System.currentTimeMillis()));
???????????? LOG.debug( "Had?a?match,?adding?fake?kv:?" ?+?kv);
???????????? result.add(kv);
???????? }
???? }
} |
1
2
3
4
5
6
7
8
9
|
##?已经上传到hadoop1上 [grid@hadoop1?~]$?ls?/ var /ftp/pub/RegionObserverExample.jar?
/ var /ftp/pub/RegionObserverExample.jar
##?由于是完全分布式系统,为了方便管理,我们将jar包存放到hadoop?hdfs的根目录下的jars目录下 [grid@hadoop1?~]$?hdfs?dfs?-put?/ var /ftp/pub/RegionObserverExample.jar?/jars
##?OK,验证已经上传成功 [grid@hadoop1?~]$?hdfs?dfs?-ls?/jars???????????????????????????????????????? Found?1?items -rw-r--r--???4?grid?supergroup???????3884?2014-11-15?04:46?/jars/RegionObserverExample.jar |
1
2
3
4
5
6
7
8
9
|
##?首先修改hbase-env.sh中的这行,将存放Coprocessor?jar文件的目录添加到Hbase的classpath中 export?HBASE_CLASSPATH=hdfs: //hadoop1:8020/jars
?
##?然后修改hbase-site.xml文件,添加一个选项 ???? <!--?这里要注意value中一定要写完整的类名(即把包名写全),否则报ClassNotFound错?-->
???? <property>
???????? <name>hbase.coprocessor.region.classes</name>
???????? <value>org.apache.hbase.kora.coprocessor.RegionObserverExample</value>
???? </property>
|
1
2
3
4
5
6
7
8
9
10
|
##?使用get命令从kora表中取rowkey为@@@GETTIME@@@的行 hbase(main):014:0>?get? ‘kora‘ ,? ‘@@@GETTIME@@@‘
COLUMN???????????????????CELL???????????????????????????????????????????????????????????????? ? @@@GETTIME@@@:@@@GETTIM?timestamp=9223372036854775807,?value=\x00\x00\x01I\xB0@\xA0\xE0?????
? E@@@????????????????????????????????????????????????????????????????????????????????????????
1?row(s)?in?0.0420?seconds ##?将列值转化为uninx?时间 hbase(main):015:0>?Time.at(Bytes.toLong( "\x00\x00\x01I\xB0\x0BZ\x0B" .to_java_bytes)/?1000)
=>?Sat?Nov?15?04:42:54?+0800?2014???? ##?从上面的测试中看出,我们自定义的Coprocessor已经成功的部署到分布式系统中了。 |
1
2
3
4
5
6
7
8
9
10
11
12
13
|
##?看hbase-env.sh的配置 [grid@hadoop1?~]$?grep? "jars" ?/opt/hbase-0.98.4-hadoop2/conf/hbase-env.sh?
export?HBASE_CLASSPATH=hdfs: //hadoop1:8020/jars
?
##?注释掉hbase-site.xml中的Coprocessor的配置 [grid@hadoop1?~]$?tail?-7??/opt/hbase-0.98.4-hadoop2/conf/hbase-site.xml? <!-- ???? <property>
???????? <name>hbase.coprocessor.region.classes</name>
???????? <value>org.apache.hbase.kora.coprocessor.RegionObserverExample</value>
???? </property>
? -->
</configuration> |
1
2
3
4
5
6
7
8
9
10
11
|
##?格式:[coprocessor?jar?file?location]?|? class ?name?|?[priority]?|?[arguments]
##?列子:hbase>?alter? ‘t1‘ ,
##??????? ‘coprocessor‘ => ‘hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2‘
##?由于一定设置了classpath,所以可以忽略jar?file?location,如下: hbase(main):101:0>?alter? ‘kora‘ ,
hbase(main):102:0*? ‘coprocessor‘ ?=>? ‘|org.apache.hbase.kora.coprocessor.RegionObserverExample|‘
Updating?all?regions?with?the? new ?schema...
0/1?regions?updated. 1/1?regions?updated. Done. 0?row(s)?in?2.5670?seconds |
1
2
3
4
5
6
7
|
hbase(main):103:0>?describe? ‘kora‘
DESCRIPTION???????????????????????????????????????????????????????????????????????????????????????????ENABLED???????????????????????????????????????????????? ? ‘kora‘ ,?{TABLE_ATTRIBUTES?=>?{coprocessor $1 ?=>?‘|org.apache.hbase.kora.coprocessor.RegionObserverExa?true???????????????????????????????????????????????????
? mple| ‘},?{NAME?=>?‘ project ‘,?DATA_BLOCK_ENCODING?=>?‘ NONE ‘,?BLOOMFILTER?=>?‘ ROW‘,?REPLICATION_SCOPE?????????????????????????????????????????????????????????
? =>? ‘0‘ ,?VERSIONS?=>? ‘1‘ ,?COMPRESSION?=>? ‘NONE‘ ,?MIN_VERSIONS?=>? ‘0‘ ,?TTL?=>? ‘FOREVER‘ ,?KEEP_DELETED_????????????????????????????????????????????????????????
? CELLS?=>? ‘false‘ ,?BLOCKSIZE?=>? ‘65536‘ ,?IN_MEMORY?=>? ‘false‘ ,?BLOCKCACHE?=>? ‘true‘ }?????????????????????????????????????????????????????????????????????????
1?row(s)?in?0.0580?seconds |
1
2
3
4
5
6
7
8
9
|
##?kora表,指定了Coprocessor hbase(main):104:0>?get? ‘kora‘ ,? ‘@@@GETTIME@@@‘
COLUMN???????????????????????????????????CELL???????????????????????????????????????????????????????????????????????????????????????????????????????????????? ? @@@GETTIME@@@:@@@GETTIME@@@?????????????timestamp=9223372036854775807,?value=\x00\x00\x01I\xB0\x985W????????????????????????????????????????????????????????
1?row(s)?in?0.0360?seconds ##?testtable,没有指定Coprocessor hbase(main):105:0>?get? ‘testtable‘ ,? ‘@@@GETTIME@@@‘
COLUMN???????????????????????????????????CELL???????????????????????????????????????????????????????????????????????????????????????????????????????????????? 0?row(s)?in?0.0180?seconds |
1
|
‘coprocessor‘ ?=>? ‘|org.apache.hbase.kora.coprocessor.RegionObserverExample|USER|‘
|
Hbase0.98.4中部署用户自定义的Observer Coprocessor
原文:http://jlins.iteye.com/blog/2159018