kunickiaj / kudu-docker

Docker Image for Kudu

Docker image for Apache Kudu

What is Kudu?

Kudu is an open source storage engine for structured data which supports low-latency random access together with efficient analytical access patterns. Kudu distributes data using horizontal partitioning and replicates each partition using Raft consensus, providing low mean-time-to-recovery and low tail latencies. Kudu is designed within the context of the Hadoop ecosystem and supports many modes of access via tools such as Apache Impala (incubating), Apache Spark, and MapReduce.

https://kudu.apache.org/

How to use this image?

The Easy Way

Using docker-compose

docker-compose up -d

Starting a Kudu console

docker run --rm -it --link kududocker_kudu-tserver_1:kudu_tserver -e KUDU_TSERVER=kudu_tserver kunickiaj/kudu cli status

The Long Way

Pull this Docker image

docker pull kunickiaj/kudu

(Optional) Building this Docker image

docker build -t kunickiaj/kudu .

(Optional) Create Data containers

docker create --name kudu-master-data -v /var/lib/kudu/master kunickiaj/kudu
docker create --name kudu-tserver-data -v /var/lib/kudu/tserver kunickiaj/kudu

Starting the Kudu Master

docker run -d --name kudu-master -p 8051:8051 kunickiaj/kudu master

Starting the Kudu TabletServer

docker run -d --name kudu-tserver -p 8050:8050 --link kudu-master \
  -e KUDU_MASTER=kudu-master kunickiaj/kudu tserver

Tailing the logs

docker logs -f kudu-master
docker logs -f kudu-tserver

Starting a Kudu console

docker run --rm -it --link kudu-tserver -e KUDU_TSERVER=kudu-tserver kunickiaj/kudu kudu tserver status kudu-tserver

Accessing the web interfaces

Each component provide its own web UI. Open you browser at one of the URLs below, where dockerhost is the name / IP of the host running the docker daemon. If using Linux, this is the IP of your linux box. If using OS X or Windows (via Docker-Machine), you can find out your docker host by typing docker-machine ip default.

Component	Port
Master	http://dockerhost:8051
TabletServer	http://dockerhost:8050

Other links

• This docker image (and README) inspired by https://github.com/bigdatafoundation/docker-kudu
• https://github.com/cloudera/kudu-examples/wiki/Docker-based-tutorial

kudu-docker's Issues

Could you update for Kudu v1.0.1

Pulling gives unauthorized error

Calling

docker pull kunickiaj/kudu
or
docker run --rm -it --link kududocker_kudu-tserver_1:kudu_tserver -e KUDU_TSERVER=kudu_tserver kunickiaj/kudu cli status

both result in an authorization error

Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/kunickiaj/kudu/manifests/latest: unauthorized: incorrect username or password

This has been tested both on windows and mac.

the volume mount file not exists

hello, I mount the volumes on the windows10：

services:
kudu-master-data:
image: kunickiaj/kudu
volumes:
- /d/var/lib/kudu/master:/var/lib/kudu/master
kudu-tserver-data:
image: kunickiaj/kudu
volumes:
- /d/var/lib/kudu/tserver:/var/lib/kudu/tserver

It occur error when I start the container：

I0422 15:22:09.402210 1 server_base.cc:215] Could not load existing FS layout: Not found: /var/lib/kudu/master/instance: No such file or directory (error 2)

I0422 15:22:09.402236 1 server_base.cc:216] Creating new FS layout

I0422 15:22:09.440258 1 fs_manager.cc:377] Generated new instance metadata in path /var/lib/kudu/master/instance:

uuid: "ca19722d255b4261a463c8796700db42"

format_stamp: "Formatted at 2022-04-22 15:22:09 on boot2docker"

W0422 15:22:09.468214 1 env_posix.cc:700] The filesystem does not support fallocate().

F0422 15:22:09.475258 1 master_main.cc:68] Check failed: _s.ok() Bad status: IO error: Could not create new FS layout: Unable to create block manager: Error during hole punch test. The log block manager requires a filesystem with hole punching support such as ext4 or xfs. On el6, kernel version 2.6.32-358 or newer is required. To run without hole punching (at the cost of some efficiency and scalability), reconfigure Kudu with --block_manager=file. Refer to the Kudu documentation for more details. Raw error message follows: Unexpected pre-punch file size for /var/lib/kudu/master/data/hole_punch_test_file: expected 16384 but got 0

how to use impala

how to use impala in docker?

Upgrade kudu version 1.10.0