smizy / docker-hbase

Apache HBase docker image based on alpine

docker-hbase

Apache HBase docker image based on alpine

Small setup

# load default env as needed
eval $(docker-machine env default)

# network 
docker network create vnet

# make docker-compose.yml with small size (no redudency)
zookeeper=1 namenode=1 datanode=1 ./make_docker_compose_yml.sh hdfs hbase > docker-compose.yml

# or with default size(zookeeper=3, namenode=2, journalnode=3, datanode=3, hmaster=2, regionserver=3)  
./make_docker_compose_yml.sh hdfs hbase > docker-compose.yml

# hadoop+hbase startup
docker-compose up -d

# tail logs for a while
docker-compose logs -f

# check ps
docker-compose ps

     Name                   Command               State                  Ports                
---------------------------------------------------------------------------------------------
datanode-1       entrypoint.sh datanode           Up      50010/tcp, 50020/tcp, 50075/tcp     
hmaster-1        entrypoint.sh hmaster-1          Up      16000/tcp, 0.0.0.0:32771->16010/tcp 
namenode-1       entrypoint.sh namenode-1         Up      0.0.0.0:32770->50070/tcp, 8020/tcp  
regionserver-1   entrypoint.sh regionserver       Up      16020/tcp, 16030/tcp                
zookeeper-1      entrypoint.sh -server 1 1 vnet   Up      2181/tcp, 2888/tcp, 3888/tcp

# check stats
docker ps --format {{.Names}} | xargs docker stats

# hbase shell
docker exec -it -u hbase regionserver-1 hbase shell
hbase(main):001:0> create 'test', 'cf'
hbase(main):002:0> list 'test'
hbase(main):003:0> put 'test', 'row1', 'cf:a', 'value1'
hbase(main):004:0> put 'test', 'row2', 'cf:b', 'value2'
hbase(main):005:0> put 'test', 'row3', 'cf:c', 'value3'
hbase(main):006:0> scan 'test'
hbase(main):007:0> get 'test', 'row1'
hbase(main):008:0> disable 'test'
hbase(main):009:0> drop 'test'
hbase(main):010:0> exit

# hadoop/hbase shutdown  
docker-compose stop

# cleanup container
docker-compose rm -v

docker-hbase's Issues

Docker-compose version '3.3' - volume_from not supported

Hello @smizy,

Thanks for sharing!

The volume_from is not supported in docker-compose 3.x version.

I tried to replace volume_from :

 namenode-1:
    container_name: namenode-1
    networks: ["vnet"]
    hostname: namenode-1.vnet
    image: smizy/hadoop-base:2.7.3-alpine
    expose: ["8020"]
    ports:  ["50070"]
    environment:
      - SERVICE_8020_NAME=namenode
      - SERVICE_50070_IGNORE=true
      - HADOOP_ZOOKEEPER_QUORUM=zookeeper-1.vnet:2181 
      - HADOOP_HEAPSIZE=1000
      - HADOOP_NAMENODE_HA=    
    entrypoint: entrypoint.sh
    volumes:
      - namenode:/usr/local/hadoop-2.7.3
    command: namenode-1

hmaster-1:
    container_name: hmaster-1
    networks: ["vnet"]
    hostname: hmaster-1.vnet
    image: smizy/hbase:1.2.6-alpine
    expose: [16000]
    ports:  [16010]
    depends_on: ["zookeeper-1"]
    environment:
      - SERVICE_16000_NAME=hmaster
      - SERVICE_16010_IGNORE=true
      - HBASE_ZOOKEEPER_QUORUM=zookeeper-1.vnet:2181   
    volumes:
      - namenode:/usr/local/hadoop-2.7.3
    command: hmaster-1
volumes:
  namenode:

And I got this error :

namenode-1        | ls: /usr/local/hadoop-2.7.3/etc/hadoop/*.mustache: No such file or directory
namenode-1        | su-exec: hdfs: No such file or directory

Do you have any idea?

Thanks!

2 nodes with inconsistent version

got hint on hbase master web console:
2 nodes with inconsistent version

env:
docker 1.12.4
hmaster on 1 server
3 regionserver node on 3 server

Change hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact to True?

Thanks for providing the container! After pulling the image we notice that hbase.regionserver.thrift.framed is set to False in hbase-default.xml.

However, the HBase official document and Cloudera troubleshooting page recommends to set hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact to True :

• This is the recommended transport for thrift servers and requires a similar setting on the client side. Changing this to false will select the default transport, vulnerable to DoS when malformed requests are issued due to THRIFT-601.

• To prevent the possibility of crashes due to buffer overruns, use the framed and compact transport protocols by setting hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact to true in hbase-site.xml.

Maybe it's better to enable these two parameters? Thank you!