MAC 북(10.8)에서 hdfs가 아닌 file단위로 저장하게 해서 standalone으로 설치했다. 

간단한 구성이지만 hbase api를 이해할 수 있다. 


1. hbase 설치

hbase download url에서 설치

(http://www.apache.org/dyn/closer.cgi/hbase/ )

최신 버전은 0.94.6 이고, hbase/lib에 있는 hadoop은 1.0.4 이용  




2. 환경 수정


$ vi conf/hbase-env.sh


#export JAVA_HOME=/usr/java/jdk1.6.0/

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home





$ vi conf/hbase-site.xml  


<configuration>

                <property>

                 <name>hbase.rootdir</name>

                 <value>file:///nosql/hbase/rootdir</value>

                 <description>The directory shared by region servers.

                 </description>

               </property>


</configuration>





3.hbase 데몬 실행


$ bin/start-hbase.sh


starting master, logging to /nosql/hbase-0.94.6/bin/../logs/hbase-knight-master.local.out





4. shell 테스트


$ ./bin/hbase shell


HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.94.6, r1456690, Thu Mar 14 16:32:05 PDT 2013



hbase(main):002:0> create 'key1','cf'

0 row(s) in 1.0420 seconds


hbase(main):004:0> put 'key1','row1', 'cf:a', 'value1'

0 row(s) in 0.0530 seconds


hbase(main):005:0> put 'key1', 'row2', 'cf:b', 'value2

0 row(s) in 0.0190 seconds


hbase(main):006:0> put 'key1', 'row3', 'cf:c', 'value3'

0 row(s) in 0.0210 seconds


hbase(main):007:0> scan key1

NameError: undefined local variable or method `key1' for #<Object:0x3b4d679>


hbase(main):008:0> scan 'key1'

ROW                                                                   COLUMN+CELL                                                                                                                                                                                               

 row1                                                                 column=cf:a, timestamp=1365045469807, value=value1                                                                                                                                                        

 row2                                                                 column=cf:b, timestamp=1365045488451, value=value2                                                                                                                                                        

 row3                                                                 column=cf:c, timestamp=1365045506088, value=value3                                                                                                                                                        

3 row(s) in 0.0510 seconds


hbase(main):009:0> get 'key1', 'row1'

COLUMN                                                                CELL                                                                                                                                                                                                      

 cf:a                                                                 timestamp=1365045469807, value=value1                                                                                                                                                                     

1 row(s) in 0.0260 seconds


hbase(main):010:0> get 'key1'


ERROR: wrong number of arguments (1 for 2)


Here is some help for this command:

Get row or cell contents; pass table name, row, and optionally

a dictionary of column(s), timestamp, timerange and versions. Examples:


  hbase> get 't1', 'r1'

  hbase> get 't1', 'r1', {TIMERANGE => [ts1, ts2]}

  hbase> get 't1', 'r1', {COLUMN => 'c1'}

  hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']}

  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}

  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4}

  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4}

  hbase> get 't1', 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"}

  hbase> get 't1', 'r1', 'c1'

  hbase> get 't1', 'r1', 'c1', 'c2'

  hbase> get 't1', 'r1', ['c1', 'c2']


hbase(main):011:0> disable 'key1'

0 row(s) in 7.0560 seconds


hbase(main):013:0> drop 'key1'

0 row(s) in 1.0570 seconds


hbase(main):015:0> exit




5. 사용 jar 확인


$ ps -ef | grep java


  502  5999     1   0  2:21PM ttys001    0:01.92 /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -XX:+UseConcMarkSweepGC -Dhbase.log.dir=/nosql/hbase-0.94.6/bin/../logs -Dhbase.log.file=hbase-knight-master.local.log -Dhbase.home.dir=/nosql/hbase-0.94.6/bin/.. -Dhbase.id.str=knight -Dhbase.root.logger=INFO,DRFA -Djava.library.path=/nosql/hbase-0.94.6/bin/../lib/native/Mac_OS_X-x86_64-64 -Dhbase.security.logger=INFO,DRFAS -classpath /nosql/hbase-0.94.6/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/lib/tools.jar:/nosql/hbase-0.94.6/bin/..:/nosql/hbase-0.94.6/bin/../hbase-0.94.6-tests.jar:/nosql/hbase-0.94.6/bin/../hbase-0.94.6.jar:/nosql/hbase-0.94.6/bin/../lib/activation-1.1.jar:/nosql/hbase-0.94.6/bin/../lib/asm-3.1.jar:/nosql/hbase-0.94.6/bin/../lib/avro-1.5.3.jar:/nosql/hbase-0.94.6/bin/../lib/avro-ipc-1.5.3.jar:/nosql/hbase-0.94.6/bin/../lib/commons-beanutils-1.7.0.jar:/nosql/hbase-0.94.6/bin/../lib/commons-beanutils-core-1.8.0.jar:/nosql/hbase-0.94.6/bin/../lib/commons-cli-1.2.jar:/nosql/hbase-0.94.6/bin/../lib/commons-codec-1.4.jar:/nosql/hbase-0.94.6/bin/../lib/commons-collections-3.2.1.jar:/nosql/hbase-0.94.6/bin/../lib/commons-configuration-1.6.jar:/nosql/hbase-0.94.6/bin/../lib/commons-digester-1.8.jar:/nosql/hbase-0.94.6/bin/../lib/commons-el-1.0.jar:/nosql/hbase-0.94.6/bin/../lib/commons-httpclient-3.1.jar:/nosql/hbase-0.94.6/bin/../lib/commons-io-2.1.jar:/nosql/hbase-0.94.6/bin/../lib/commons-lang-2.5.jar:/nosql/hbase-0.94.6/bin/../lib/commons-logging-1.1.1.jar:/nosql/hbase-0.94.6/bin/../lib/commons-math-2.1.jar:/nosql/hbase-0.94.6/bin/../lib/commons-net-1.4.1.jar:/nosql/hbase-0.94.6/bin/../lib/core-3.1.1.jar:/nosql/hbase-0.94.6/bin/../lib/guava-11.0.2.jar:/nosql/hbase-0.94.6/bin/../lib/hadoop-core-1.0.4.jar:/nosql/hbase-0.94.6/bin/../lib/high-scale-lib-1.1.1.jar:/nosql/hbase-0.94.6/bin/../lib/httpclient-4.1.2.jar:/nosql/hbase-0.94.6/bin/../lib/httpcore-4.1.3.jar:/nosql/hbase-0.94.6/bin/../lib/jackson-core-asl-1.8.8.jar:/nosql/hbase-0.94.6/bin/../lib/jackson-jaxrs-1.8.8.jar:/nosql/hbase-0.94.6/bin/../lib/jackson-mapper-asl-1.8.8.jar:/nosql/hbase-0.94.6/bin/../lib/jackson-xc-1.8.8.jar:/nosql/hbase-0.94.6/bin/../lib/jamon-runtime-2.3.1.jar:/nosql/hbase-0.94.6/bin/../lib/jasper-compiler-5.5.23.jar:/nosql/hbase-0.94.6/bin/../lib/jasper-runtime-5.5.23.jar:/nosql/hbase-0.94.6/bin/../lib/jaxb-api-2.1.jar:/nosql/hbase-0.94.6/bin/../lib/jaxb-impl-2.2.3-1.jar:/nosql/hbase-0.94.6/bin/../lib/jersey-core-1.8.jar:/nosql/hbase-0.94.6/bin/../lib/jersey-json-1.8.jar:/nosql/hbase-0.94.6/bin/../lib/jersey-server-1.8.jar:/nosql/hbase-0.94.6/bin/../lib/jettison-1.1.jar:/nosql/hbase-0.94.6/bin/../lib/jetty-6.1.26.jar:/nosql/hbase-0.94.6/bin/../lib/jetty-util-6.1.26.jar:/nosql/hbase-0.94.6/bin/../lib/jruby-complete-1.6.5.jar:/nosql/hbase-0.94.6/bin/../lib/jsp-2.1-6.1.14.jar:/nosql/hbase-0.94.6/bin/../lib/jsp-api-2.1-6.1.14.jar:/nosql/hbase-0.94.6/bin/../lib/jsr305-1.3.9.jar:/nosql/hbase-0.94.6/bin/../lib/junit-4.10-HBASE-1.jar:/nosql/hbase-0.94.6/bin/../lib/libthrift-0.8.0.jar:/nosql/hbase-0.94.6/bin/../lib/log4j-1.2.16.jar:/nosql/hbase-0.94.6/bin/../lib/metrics-core-2.1.2.jar:/nosql/hbase-0.94.6/bin/../lib/netty-3.2.4.Final.jar:/nosql/hbase-0.94.6/bin/../lib/protobuf-java-2.4.0a.jar:/nosql/hbase-0.94.6/bin/../lib/servlet-api-2.5-6.1.14.jar:/nosql/hbase-0.94.6/bin/../lib/slf4j-api-1.4.3.jar:/nosql/hbase-0.94.6/bin/../lib/slf4j-log4j12-1.4.3.jar:/nosql/hbase-0.94.6/bin/../lib/snappy-java-1.0.3.2.jar:/nosql/hbase-0.94.6/bin/../lib/stax-api-1.0.1.jar:/nosql/hbase-0.94.6/bin/../lib/velocity-1.7.jar:/nosql/hbase-0.94.6/bin/../lib/xmlenc-0.52.jar:/nosql/hbase-0.94.6/bin/../lib/zookeeper-3.4.5.jar: org.apache.hadoop.hbase.master.HMaster start




6. 종료


$ bin/stop-hbase.sh 

stopping hbase................





<웹확인>

http://localhost:60010/master-status




* 이슈 (개인적인 생각)

mac 에서 standalone으로 설치하면 hadoop client api를 쉽게 쓸 수 있으나, hdfs 프로토콜로 연결해야 하는 spring data 규격에는 맞지 않는듯 하다. spring data hadoop 소스를 가지고 테스트를 했는데. standalone으로 설치한 hbase connection실패가 계속 일어난다. 따라서 hadoop 을 설치해서 hbase에서 hdfs설정이 가능케 하는 것이 그나마 개발환경이 편할듯 싶다.  


Posted by 김용환 '김용환'

댓글을 달아 주세요