Coder Social home page Coder Social logo

docker-riak's Introduction

docker-riak Build Status

This is a Docker project to bring up a local Riak cluster.

Prerequisites

Install Docker

Follow the instructions on Docker's website to install Docker.

Many (but not all) Docker environments set the DOCKER_HOST environment variable to help the client find the Docker host. Some environments use a Unix domain socket by default.

If your Docker client connects to the Docker host via TCP, ensure that your DOCKER_HOST environmental variable is set correctly:

$ export DOCKER_HOST="tcp://127.0.0.1:2375"

Note: If you're using boot2docker ensure that you forward the virtual machine port range (49000-49900). If you want to set DOCKER_RIAK_BASE_HTTP_PORT, ensure that you forward that port range instead:

$ for i in {49000..49900}; do
 VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port$i,tcp,,$i,,$i";
 VBoxManage modifyvm "boot2docker-vm" --natpf1 "udp-port$i,udp,,$i,,$i";
done

sysctl

In order to tune the Docker host housing Riak containers, consider applying the following sysctl settings:

vm.swappiness = 0
net.ipv4.tcp_max_syn_backlog = 40000
net.core.somaxconn = 40000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_moderate_rcvbuf = 1

Running

Clone repository and build Riak image

$ git clone https://github.com/hectcastro/docker-riak.git
$ cd docker-riak
$ make build

Environmental variables

  • DOCKER_RIAK_CLUSTER_SIZE – The number of nodes in your Riak cluster (default: 5)
  • DOCKER_RIAK_AUTOMATIC_CLUSTERING – A flag to automatically cluster Riak (default: false)
  • DOCKER_RIAK_DEBUG – A flag to set -x on the cluster management scripts (default: false)
  • DOCKER_RIAK_BASE_HTTP_PORT - A flag to use fixed port assignment. If set, manually forward port DOCKER_RIAK_BASE_HTTP_PORT + $index to 8098 (Riak's HTTP port) and forward DOCKER_RIAK_BASE_HTTP_PORT + $index + DOCKER_RIAK_PROTO_BUF_PORT_OFFSET to 8087 (Riak's Protocol Buffers port).
  • DOCKER_RIAK_PROTO_BUF_PORT_OFFSET - Optional port offset (default: 100)
  • DOCKER_RIAK_BACKEND - Optional Riak backend to use (default: bitcask)
  • DOCKER_RIAK_STRONG_CONSISTENCY - Enables strong consistency subsystem (values: on or off, default: off)
  • DOCKER_RIAK_SEARCH - Enables search (values: on or off, default: off)

Launch cluster

$ DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 DOCKER_RIAK_BACKEND=leveldb make start-cluster
./bin/start-cluster.sh

Bringing up cluster nodes:

Starting riak01..... Completed
Starting riak02..... Completed
Starting riak03..... Completed
Starting riak04..... Completed
Starting riak05..... Completed

Please wait approximately 30 seconds for the cluster to stabilize.

Testing

From outside the container, we can interact with the HTTP or Protocol Buffers interfaces.

HTTP

The HTTP interface has an endpoint called /stats that emits Riak statistics. The test-cluster Makefile target hits a random container's /stats endpoint and pretty-prints its output to the console.

The most interesting attributes for testing cluster membership are ring_members:

$ make test-cluster | egrep -A6 "ring_members"
    "ring_members": [
        "[email protected]",
        "[email protected]",
        "[email protected]",
        "[email protected]",
        "[email protected]"
    ],

And ring_ownership:

$ make test-cluster | egrep "ring_ownership"
    "ring_ownership": "[{'[email protected]',3},\n {'[email protected]',4},\n {'[email protected]',3},\n {'[email protected]',4},\n {'[email protected]',3},\n {'[email protected]',4},\n {'[email protected]',3},\n {'[email protected]',4},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3},\n {'[email protected]',3}]",

Together, these attributes let us know that this particular Riak node knows about all of the other Riak instances.

SSH

The phusion/baseimage-docker image has the ability to enable an insecure key for conveniently logging into a container via SSH. It is enabled in the Dockerfile by default here:

RUN /usr/sbin/enable_insecure_key

In order to login to the container via SSH using the insecure key, follow the steps below.

Use docker inspect to determine the container IP address:

$ docker inspect $CONTAINER_ID | grep IPAddress
        "IPAddress": "172.17.0.2",

Download the insecure key, alter its permissions, and use it to SSH into the container via its IP address:

$ curl -o insecure_key -fSL https://github.com/phusion/baseimage-docker/raw/master/image/services/sshd/keys/insecure_key
$ chmod 600 insecure_key
$ ssh -i insecure_key [email protected]

Note: If you're using boot2docker, ensure that you're issuing the SSH command from within the virtual machine running boot2docker.

Stopping

$ make stop-cluster
./bin/stop-cluster.sh
Stopped all of the running containers.

Destroying

$ make remove-cluster
./bin/remove-cluster.sh
Stopped the cluster and cleared all of the running containers.

docker-riak's People

Contributors

ecolabardini avatar hectcastro avatar hnakamur avatar jaseg avatar jplock avatar ksato9700 avatar lgustafson avatar mitchellwrosen avatar sofuture avatar whummer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-riak's Issues

Ring ownership after start

So starting riak cluster as per documention
`DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 DOCKER_RIAK_BACKEND=leveldb make start-cluster``

After stabilization I'll check for ring ownership;
"ring_ownership": "[{'[email protected]',64}]",

Thats uncool.

So docker-enter into one of the nodes to see what riak-admin cluster plan shows

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      20.3%     20.3%    '[email protected]'
valid      20.3%     20.3%    '[email protected]'
valid      20.3%     20.3%    '[email protected]'
valid      20.3%     20.3%    '[email protected]'
valid      18.8%     18.8%    '[email protected]'
-------------------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Outstading changes, I'll commit them and check the status again

root@5ec57bbf8295:~# riak-admin cluster commit
Cluster changes committed
root@5ec57bbf8295:~# riak-admin cluster plan
There are no staged changes

All good. And obviously ring ownership also cool now
"ring_ownership": "[{'[email protected]',13},\n {'[email protected]',13},\n {'[email protected]',13},\n {'[email protected]',13},\n {'[email protected]',12}]",

So something with automatic_clustering fails here.

make build crashes

Somewhere in building it tries to
/usr/bin/chfn -f Riak user riak
which is of course impossible. I'm guessing someone forget to escape the space,
or forgot an underscore as in Riak_user.
After that, the dpkg also crashes, but that might be related.
How can I fix this?

start-cluster script hangs indefinitely

We are attempting to bootstrap riak under coreos/docker for evaluation purposes.

The build process seemed to go ok, but bootstrapping the actual cluster hangs until SIGKILL:

~/sandbox/riak/docker-riak$ sudo DOCKER_HOST="unix:///var/run/docker.sock" DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 make start-cluster
./bin/start-cluster.sh

Bringing up cluster nodes:

hangs right here ^^

When examining docker status, it looks like riak01 is running:

594da7c70e46 hectcastro/riak:latest "/sbin/my_init --qui 5 minutes ago Up 5 minutes 0.0.0.0:49155->8087/tcp, 0.0.0.0:49156->8098/tcp riak01

And when checking the logs for that container, I see:

No SSH host key available. Generating one...
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Creating SSH2 ED25519 key; this may take some time ...
invoke-rc.d: policy-rc.d denied execution of restart.

And that's it. I can't think of any more debug info I can give on this issue, unless there is an undocumented verbose or debug flag in the script itself.

This is under coreos alpha channel. The full dump of docker version info follows:

Client version: 1.3.0
Client API version: 1.15
Go version (client): go1.3.2
Git commit (client): c78088f
OS/Arch (client): linux/amd64
Server version: 1.3.0
Server API version: 1.15
Go version (server): go1.3.2
Git commit (server): c78088f

Error with oracle-java7-installer package

Tried with tag: 2.0.0

$ make build
docker build -t "hectcastro/riak" .
Sending build context to Docker daemon 196.1 kB
Step 1/14 : FROM phusion/baseimage:0.9.14
 ---> fc382bb53394
Step 2/14 : MAINTAINER Hector Castro [email protected]
 ---> Using cache
 ---> 342b0c0e3361
Step 3/14 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> 89122c8fe754
Step 4/14 : ENV RIAK_VERSION 2.0.2-1
 ---> Running in 4652004026d5
 ---> 485d3bdfec8a
Removing intermediate container 4652004026d5
Step 5/14 : RUN sed -i.bak 's/main$/main universe/' /etc/apt/sources.list &&     apt-get update -qq && apt-get install -y software-properties-common &&     apt-add-repository ppa:webupd8team/java -y && apt-get update -qq &&     echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections &&     apt-get install -y oracle-java7-installer &&     curl https://packagecloud.io/install/repositories/basho/riak/script.deb.sh | bash &&     apt-get install -y riak=${RIAK_VERSION} &&     apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 ---> Running in 9651f1d3fe80
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  python3-software-properties
The following packages will be upgraded:
  python3-software-properties software-properties-common
2 upgraded, 0 newly installed, 0 to remove and 125 not upgraded.
Need to get 28.6 kB of archives.
After this operation, 1024 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty-updates/main software-properties-common all 0.92.37.8 [9384 B]
Get:2 http://archive.ubuntu.com/ubuntu/ trusty-updates/main python3-software-properties all 0.92.37.8 [19.2 kB]
Fetched 28.6 kB in 0s (432 kB/s)
(Reading database ... 15989 files and directories currently installed.)
Preparing to unpack .../software-properties-common_0.92.37.8_all.deb ...
Unpacking software-properties-common (0.92.37.8) over (0.92.37.1) ...
Preparing to unpack .../python3-software-properties_0.92.37.8_all.deb ...
Unpacking python3-software-properties (0.92.37.8) over (0.92.37.1) ...
Setting up python3-software-properties (0.92.37.8) ...
Setting up software-properties-common (0.92.37.8) ...
gpg: keyring `/tmp/tmpwjcfevcj/secring.gpg' created
gpg: keyring `/tmp/tmpwjcfevcj/pubring.gpg' created
gpg: requesting key EEA14886 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpwjcfevcj/trustdb.gpg: trustdb created
gpg: key EEA14886: public key "Launchpad VLC" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  binutils gsfonts gsfonts-x11 java-common libfontenc1 libfreetype6 libxfont1
  wget x11-common xfonts-encodings xfonts-utils
Suggested packages:
  binutils-doc default-jre equivs binfmt-support visualvm ttf-baekmuk
  ttf-unfonts ttf-unfonts-core ttf-kochi-gothic ttf-sazanami-gothic
  ttf-kochi-mincho ttf-sazanami-mincho ttf-arphic-uming firefox firefox-2
  iceweasel mozilla-firefox iceape-browser mozilla-browser epiphany-gecko
  epiphany-webkit epiphany-browser galeon midbrowser moblin-web-browser
  xulrunner xulrunner-1.9 konqueror chromium-browser midori google-chrome
The following NEW packages will be installed:
  binutils gsfonts gsfonts-x11 java-common libfontenc1 libfreetype6 libxfont1
  oracle-java7-installer wget x11-common xfonts-encodings xfonts-utils
0 upgraded, 12 newly installed, 0 to remove and 125 not upgraded.
Need to get 7003 kB of archives.
After this operation, 20.6 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main libfontenc1 amd64 1:1.1.2-1 [15.6 kB]
Get:2 http://archive.ubuntu.com/ubuntu/ trusty-updates/main libfreetype6 amd64 2.5.2-1ubuntu2.8 [304 kB]
Get:3 http://archive.ubuntu.com/ubuntu/ trusty-updates/main libxfont1 amd64 1:1.4.7-1ubuntu0.2 [95.0 kB]
Get:4 http://archive.ubuntu.com/ubuntu/ trusty-updates/main wget amd64 1.15-1ubuntu1.14.04.2 [271 kB]
Get:5 http://archive.ubuntu.com/ubuntu/ trusty-updates/main binutils amd64 2.24-5ubuntu14.2 [2076 kB]
Get:6 http://archive.ubuntu.com/ubuntu/ trusty/main java-common all 0.51 [130 kB]
Get:7 http://archive.ubuntu.com/ubuntu/ trusty/main gsfonts all 1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1 [3374 kB]
Get:8 http://archive.ubuntu.com/ubuntu/ trusty-updates/main x11-common all 1:7.7+1ubuntu8.1 [49.5 kB]
Get:9 http://archive.ubuntu.com/ubuntu/ trusty/main xfonts-encodings all 1:1.0.4-1ubuntu1 [583 kB]
Get:10 http://archive.ubuntu.com/ubuntu/ trusty/main xfonts-utils amd64 1:7.7+1 [73.9 kB]
Get:11 http://ppa.launchpad.net/webupd8team/java/ubuntu/ trusty/main oracle-java7-installer all 7u80+7u60arm-0~webupd8~1 [22.4 kB]
Get:12 http://archive.ubuntu.com/ubuntu/ trusty/main gsfonts-x11 all 0.22 [9108 B]
Preconfiguring packages ...
Fetched 7003 kB in 2s (3131 kB/s)
Selecting previously unselected package libfontenc1:amd64.
(Reading database ... 15989 files and directories currently installed.)
Preparing to unpack .../libfontenc1_1%3a1.1.2-1_amd64.deb ...
Unpacking libfontenc1:amd64 (1:1.1.2-1) ...
Selecting previously unselected package libfreetype6:amd64.
Preparing to unpack .../libfreetype6_2.5.2-1ubuntu2.8_amd64.deb ...
Unpacking libfreetype6:amd64 (2.5.2-1ubuntu2.8) ...
Selecting previously unselected package libxfont1:amd64.
Preparing to unpack .../libxfont1_1%3a1.4.7-1ubuntu0.2_amd64.deb ...
Unpacking libxfont1:amd64 (1:1.4.7-1ubuntu0.2) ...
Selecting previously unselected package wget.
Preparing to unpack .../wget_1.15-1ubuntu1.14.04.2_amd64.deb ...
Unpacking wget (1.15-1ubuntu1.14.04.2) ...
Selecting previously unselected package binutils.
Preparing to unpack .../binutils_2.24-5ubuntu14.2_amd64.deb ...
Unpacking binutils (2.24-5ubuntu14.2) ...
Selecting previously unselected package java-common.
Preparing to unpack .../java-common_0.51_all.deb ...
Unpacking java-common (0.51) ...
Setting up wget (1.15-1ubuntu1.14.04.2) ...
Setting up binutils (2.24-5ubuntu14.2) ...
Processing triggers for libc-bin (2.19-0ubuntu6.3) ...
Selecting previously unselected package oracle-java7-installer.
(Reading database ... 16253 files and directories currently installed.)
Preparing to unpack .../oracle-java7-installer_7u80+7u60arm-0~webupd8~1_all.deb ...
oracle-license-v1-1 license has already been accepted
Unpacking oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ...
Selecting previously unselected package gsfonts.
Preparing to unpack .../gsfonts_1%3a8.11+urwcyr1.0.7~pre44-4.2ubuntu1_all.deb ...
Unpacking gsfonts (1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1) ...
Selecting previously unselected package x11-common.
Preparing to unpack .../x11-common_1%3a7.7+1ubuntu8.1_all.deb ...
Unpacking x11-common (1:7.7+1ubuntu8.1) ...
Selecting previously unselected package xfonts-encodings.
Preparing to unpack .../xfonts-encodings_1%3a1.0.4-1ubuntu1_all.deb ...
Unpacking xfonts-encodings (1:1.0.4-1ubuntu1) ...
Selecting previously unselected package xfonts-utils.
Preparing to unpack .../xfonts-utils_1%3a7.7+1_amd64.deb ...
Unpacking xfonts-utils (1:7.7+1) ...
Selecting previously unselected package gsfonts-x11.
Preparing to unpack .../gsfonts-x11_0.22_all.deb ...
Unpacking gsfonts-x11 (0.22) ...
Processing triggers for mime-support (3.54ubuntu1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up libfontenc1:amd64 (1:1.1.2-1) ...
Setting up libfreetype6:amd64 (2.5.2-1ubuntu2.8) ...
Setting up libxfont1:amd64 (1:1.4.7-1ubuntu0.2) ...
Setting up java-common (0.51) ...
Setting up oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ...
Downloading Oracle Java 7...
--2017-07-28 14:34:55--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
Resolving download.oracle.com (download.oracle.com)... 23.1.242.90, 23.1.242.8
Connecting to download.oracle.com (download.oracle.com)|23.1.242.90|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz [following]
--2017-07-28 14:34:55--  https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
Resolving edelivery.oracle.com (edelivery.oracle.com)... 2.19.158.221, 2a02:26f0:ec:384::2d3e, 2a02:26f0:ec:38b::2d3e
Connecting to edelivery.oracle.com (edelivery.oracle.com)|2.19.158.221|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1501252615_ee4189153567f58c09735918eafe7b10 [following]
--2017-07-28 14:34:55--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1501252615_ee4189153567f58c09735918eafe7b10
Connecting to download.oracle.com (download.oracle.com)|23.1.242.90|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-07-28 14:34:57 ERROR 404: Not Found.

download failed
Oracle JDK 7 is NOT installed.
dpkg: error processing package oracle-java7-installer (--configure):
 subprocess installed post-installation script returned error exit status 1
Setting up gsfonts (1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1) ...
Setting up x11-common (1:7.7+1ubuntu8.1) ...
invoke-rc.d: policy-rc.d denied execution of start.
Processing triggers for ureadahead (0.100.0-16) ...
Setting up xfonts-encodings (1:1.0.4-1ubuntu1) ...
Setting up xfonts-utils (1:7.7+1) ...
Setting up gsfonts-x11 (0.22) ...
Processing triggers for libc-bin (2.19-0ubuntu6.3) ...
Errors were encountered while processing:
 oracle-java7-installer
E: Sub-process /usr/bin/dpkg returned an error code (1)
The command '/bin/sh -c sed -i.bak 's/main$/main universe/' /etc/apt/sources.list &&     apt-get update -qq && apt-get install -y software-properties-common &&     apt-add-repository ppa:webupd8team/java -y && apt-get update -qq &&     echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections &&     apt-get install -y oracle-java7-installer &&     curl https://packagecloud.io/install/repositories/basho/riak/script.deb.sh | bash &&     apt-get install -y riak=${RIAK_VERSION} &&     apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*' returned a non-zero code: 100
make: *** [build] Error 100

make build error with redirecting script.dev.sh

I had redirection error with make build.

Step 6 : RUN curl https://packagecloud.io/install/repositories/basho/riak/script.deb | bash
 ---> Running in 0f866c961169
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   135    0   135    0     0    209      0 --:--:-- --:--:-- --:--:--   209
bash: line 1: syntax error near unexpected token `<'
bash: line 1: `<html><body>You are being <a href="https://packagecloud.io/install/repositories/basho/riak/script.deb.sh">redirected</a>.</body></html>'
The command '/bin/sh -c curl https://packagecloud.io/install/repositories/basho/riak/script.deb | bash' returned a non-zero code: 2
make: *** [build] Error 1

Update README.md to contemplate unix sockets

When installing Docker for Ubuntu 14.04 LTS (1) with apt, the docker daemon is using unix sockets. Therefore using something like

export DOCKER_HOST="tcp://127.0.0.1:2375"

will result in

$ DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 make start-cluster
./bin/start-cluster.sh
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

Bringing up cluster nodes:

make: *** [start-cluster] Error 125`

For future installers would be great that the "Install Docker" section on the README.md contemplate Unix sockets, e.g.:

export DOCKER_HOST="unix:///var/run/docker.sock"

(1) https://docs.docker.com/engine/installation/linux/ubuntulinux/

Mounting Volume takes infinite time

I am binding volume at time of starting RIAK nodes to reuse data created by RIAK . I have attached this line to start-cluster.sh to mount volume " -v /riak${index}:/var/lib/riak:rw \ " .
It works fine when I start it for first time. it creates riak01 folder and puts data to it. But once when you kills all running riak containers and start it again it takes infinite time to bind mounted volume "/riak01" to container. although its shows something is continuously running in background which occupies 20-25% of memory..

start script breaks when using unix sockets with Docker

The start script assumes that the DOCKER_HOST env var contains the hostname which it then parses out to curl against to see if the container has come up. This isn't necessarily the case when the DOCKER_HOST is a unix socket (unix:///var/run/docker.sock), which results in an infinite loop of curling an invalid address, then sleeping.

Read-only file system errors during image build on Docker 0.11.1

On Docker 0.11.0 and above, some sysctl related steps in the Dockerfile fail with a read-only file system error:

$ make build
docker build -t "hectcastro/riak" .
Uploading context 892.4 kB
Uploading context
Step 0 : FROM phusion/baseimage:0.9.9
Pulling repository phusion/baseimage
745d3ac92697: Download complete
511136ea3c5a: Download complete
6170bb7b0ad1: Download complete
9cd978db300e: Download complete
a6d44c263269: Download complete
13e42d0c2a51: Download complete
846e143f3fab: Download complete
10ebd1d649cb: Download complete
5db917407faa: Download complete
 ---> 745d3ac92697
Step 1 : MAINTAINER Hector Castro [email protected]
 ---> Running in 734c2e8d1465
 ---> f5e5d9646d4a
Removing intermediate container 734c2e8d1465
Step 2 : ENV DEBIAN_FRONTEND noninteractive
 ---> Running in 3b3320786e09
 ---> 2cb6ba3be28e
Removing intermediate container 3b3320786e09
Step 3 : ENV RIAK_VERSION 1.4.8
 ---> Running in 438bd4976a5a
 ---> 6271107b7db0
Removing intermediate container 438bd4976a5a
Step 4 : ENV RIAK_SHORT_VERSION 1.4
 ---> Running in 82bc122a22c1
 ---> 04cea8151e88
Removing intermediate container 82bc122a22c1
Step 5 : ADD http://s3.amazonaws.com/downloads.basho.com/riak/${RIAK_SHORT_VERSION}/${RIAK_VERSION}/ubuntu/precise/riak_${RIAK_VERSION}-1_amd64.deb /
 ---> 435d530e7e41
Removing intermediate container c2da170361dc
Step 6 : RUN (cd / && dpkg -i "riak_${RIAK_VERSION}-1_amd64.deb")
 ---> Running in 5efd67d2d4a0
Selecting previously unselected package riak.
(Reading database ... 12731 files and directories currently installed.)
Unpacking riak (from riak_1.4.8-1_amd64.deb) ...
Setting up riak (1.4.8-1) ...
Adding group `riak' (GID 105) ...
Done.
Adding system user `riak' (UID 103) ...
Adding new user `riak' (UID 103) with group `riak' ...
Not creating home directory `/var/lib/riak'.
Processing triggers for ureadahead ...
 ---> 79104c324bb6
Removing intermediate container 5efd67d2d4a0
Step 7 : RUN mkdir -p /etc/service/riak
 ---> Running in 9ec98048eac9
 ---> 23d480f20e56
Removing intermediate container 9ec98048eac9
Step 8 : ADD bin/riak.sh /etc/service/riak/run
 ---> 6be33e4bff83
Removing intermediate container 7fd0a06e5ddd
Step 9 : ADD bin/automatic_clustering.sh /etc/my_init.d/99_automatic_clustering.sh
 ---> 9890e5bd1845
Removing intermediate container bb9d94cbb48d
Step 10 : RUN sed -i.bak 's/127.0.0.1/0.0.0.0/' /etc/riak/app.config &&    sed -i.bak 's/{anti_entropy_concurrency, 2}/{anti_entropy_concurrency, 1}/' /etc/riak/app.config &&    sed -i.bak 's/{map_js_vm_count, 8 }/{map_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak 's/{reduce_js_vm_count, 6 }/{reduce_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak 's/{hook_js_vm_count, 2 }/{hook_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak "s/##+zdbbl/+zdbbl/" /etc/riak/vm.args
 ---> Running in f3b130d124c2
 ---> b021e433fece
Removing intermediate container f3b130d124c2
Step 11 : RUN echo "vm.swappiness = 0" > /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_max_syn_backlog = 40000" >> /etc/sysctl.d/riak.conf &&    echo "net.core.somaxconn = 40000" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_sack = 1" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_window_scaling = 1" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_fin_timeout = 15" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_keepalive_intvl = 30" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_tw_reuse = 1" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_moderate_rcvbuf = 1" >> /etc/sysctl.d/riak.conf &&    sysctl -e -p /etc/sysctl.d/riak.conf
 ---> Running in 11d999b30f83
error: "Read-only file system" setting key "vm.swappiness"
error: "Read-only file system" setting key "net.core.somaxconn"
2014/05/09 12:55:49 The command [/bin/sh -c echo "vm.swappiness = 0" > /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_max_syn_backlog = 40000" >> /etc/sysctl.d/riak.conf &&    echo "net.core.somaxconn = 40000" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_sack = 1" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_window_scaling = 1" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_fin_timeout = 15" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_keepalive_intvl = 30" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_tw_reuse = 1" >> /etc/sysctl.d/riak.conf &&    echo "net.ipv4.tcp_moderate_rcvbuf = 1" >> /etc/sysctl.d/riak.conf &&    sysctl -e -p /etc/sysctl.d/riak.conf] returned a non-zero code: 255
make: *** [build] Error 1

As of now, the errors appear to be isolated to setting vm.swappiness and net.core.somaxconn.

riak node fail to start along with the container

When i stop one of the container and start then riak fails to start. looks like some services required for riak are not getting started properly hence riak not able to bring up the node. Ping itself not working. Could you please help me in getting around of this issue

problem setting up multi host cluster

Hey there,

I am playing around with the docker-riak repo and have some problems. I hope it is ok to address my issue here, even if it is not the correct platform (maybe the riak/coreos mailing list would be more appropriate). If so, sorry!

Anyway, I am trying to figure out how to set up a docker-riak cluster across multiple hosts. The idea is the following. Having two coreos [1] machines running, the cluster should be created using one riak node on the first coreos machine, and one riak node on the second coreos machine. Linking should be enabled using the ambassador pattern [2]. I have a gist [3] describing how it should work. Everything comes up properly, but the riak nodes don't get linked to one cluster. It is possible to access the riak nodes from inside the containers, but for some reason the riak-admin cluster join <node> does not do what it should.

I don't understand the problem, because the riak nodes can access the correct IP's and be able to communicate. For me there is no obvious reason why that scenario fails. Maybe somebody could take a look to get that to work. Would be really cool to be able to do this cluster thing across multiple hosts.

[1] https://coreos.com/
[2] http://coreos.com/blog/docker-dynamic-ambassador-powered-by-etcd/
[3] https://gist.github.com/zyndiecate/74e8df820ccee60f67ae

All the best, Tim

some inital experiences as feedback

Some fine-tuning seems to be appropriate, but what then?

TL;DR:

Maybe this will help others struggling to learn like me and also you developers and evangelists from basho to be able to understand and help us newbies better.

It is a long story, but quite interesting and instructive, I hope.

In the end, I finally gave up, however.

Setup

I work with Vagrant/Virtualbox on XP and managed to set up two data containers (dbvx1, app), a MySQL server container ms working on dbvx1, a memcached container mc and an Apache/PHP container ap making use of app and linked to ms and mc.

CONTAINER ID        IMAGE                       COMMAND                CREATED             STATUS              PORTS                         NAMES
5185cbf11593        kklepper/AP:latest          /start.sh /bin/sh -c   42 hours ago        Up 42 hours         192.168.0.42:49154->80/tcp    ap
b0ebad4826db        kklepper/memcached:latest   memcached -u daemon    45 hours ago        Up 45 hours         0.0.0.0:49156->11211/tcp      ap/mc,mc
7530e02913fd        kklepper/Ms:latest          /bin/bash /start.sh    45 hours ago        Up 45 hours         192.168.0.42:3306->3306/tcp   ap/db,ms
0b56d0060c76        busybox:latest              true                   2 days ago          Exit 0                                            app
8b6572266d74        busybox:latest              true                   2 days ago          Exit 0                                            dbvx1                 

The app running on ap (192.168.0.42:49154) shows that even compressed key-value pairs on MySQL with query cache are significantly slower than memcached uncompressed, with values as strings of around 60 KB, so this is nice as far as it goes and shows that things are working as expected.

Readings vary greatly, though, but the follwing is quite typical and instructive:

Did mysql_query in 48.332929611206 milliseconds
Did memcached in 14.45198059082 milliseconds
Result: memcached 3 times faster: 33 milliseconds saved: 48 :: 14

Did memcached zipped in 10.414123535156 milliseconds
Result: memcached zipped / mysql 4 times faster: 37 milliseconds saved: 48 :: 10
Result: memcached/memcached zipped 1 times faster: 4 milliseconds saved: 14 :: 10

As an aside: It does not matter if I start timing before or after connecting to mysql or memcached, connection is fast.

The MySQL database could be replicated ad libitum, reads should by far outweigh writes in our app. I know this technique very well, although I don't know about Galera Cluster yet, which looks very interesting indeed.

The question remains: is MySQL the best choice for persistent storage in our case?

Next I wanted to have a look at Riak (see also the case study Inaka/Boundary, where they switched from Riak to MySQL).


First issue:

process hangs

vagrant@precise64:/vagrant/docker-lampstack-master/docker-riak$  make start-cluster
./bin/start-cluster.sh
Started [riak1] and assigned it the IP [33.33.33.10]

no more, process hangs

testing delivers

vagrant@precise64:/vagrant/docker-lampstack-master/docker-riak$ make test-cluster
./bin/test-cluster.sh
Testing [riak1]...
Testing [riak2]...
Testing [riak3]...
Testing [riak4]...
Testing [riak5]...

This behavior seemingly was caused by running an instance from executing the sample from this tutorial Riak Service.

Stopping and removing this container resolved the problem, so maybe a word of caution in the readme would help.


Next issue:

sshpass missing

Started [riak2] and assigned it the IP [33.33.33.20]
Requesting that [riak2] join the cluster..
./bin/start-cluster.sh: line 44: sshpass: command not found
make: *** [start-cluster] Error 127

So, the Dockerfile line

RUN apt-get install -y curl lsb-release supervisor openssh-server

should better be augmented with sshpass


Next issue:

time delay

Everything worked like a charm. But on repeat, I experienced this twice:

Requesting that [riak4] join the cluster..
Success: staged join request for '[email protected]' to '[email protected]'
2014/01/18 12:49:48 Error: start: Cannot start container 69b42cdb901c7c34e063cf3c769bbb9ad13cc273426a35bb6d625383c1da0d6b: The container failed to start due to timed out.
make: *** [start-cluster] Error 1

So even a timeout occurs. Why?

From the docker ps -a output it might seem be a kind of memory swapping problem due to the amount of time needed to complete an operation, but a simple restart resolved the problem altogether (why?):

CONTAINER ID        IMAGE                       COMMAND                CREATED              STATUS              PORTS                         NAMES
de5d6feb9fcb        hectcastro/riak:latest      /usr/bin/supervisord   47 seconds ago       Up 47 seconds       22/tcp, 8087/tcp, 8098/tcp    nostalgic_poincare
4ee5baa6dcb8        hectcastro/riak:latest      /usr/bin/supervisord   About a minute ago   Up About a minute   22/tcp, 8087/tcp, 8098/tcp    hungry_lumiere
73a949a40029        hectcastro/riak:latest      /usr/bin/supervisord   3 minutes ago        Up 3 minutes        22/tcp, 8087/tcp, 8098/tcp    elegant_ritchie
64cd80d9d6ac        hectcastro/riak:latest      /usr/bin/supervisord   3 minutes ago        Up 3 minutes        22/tcp, 8087/tcp, 8098/tcp    sad_davinci
0e77bc096995        hectcastro/riak:latest      /usr/bin/supervisord   3 minutes ago        Up 3 minutes        22/tcp, 8087/tcp, 8098/tcp    mad_curie

Now on repeat, I experienced again a similar time delay:

vagrant@precise64:/vagrant/docker-lampstack-master/docker-riak$ make start-cluster
./bin/start-cluster.sh
Started [riak1] and assigned it the IP [33.33.33.10]
Started [riak2] and assigned it the IP [33.33.33.20]
Requesting that [riak2] join the cluster..
Success: staged join request for '[email protected]' to '[email protected]'
Started [riak3] and assigned it the IP [33.33.33.30]
Requesting that [riak3] join the cluster..
Success: staged join request for '[email protected]' to '[email protected]'
Started [riak4] and assigned it the IP [33.33.33.40]
Requesting that [riak4] join the cluster..
Success: staged join request for '[email protected]' to '[email protected]'
Started [riak5] and assigned it the IP [33.33.33.50]
Requesting that [riak5] join the cluster..
Success: staged join request for '[email protected]' to '[email protected]'

etc. It took 7 minutes this time for the process to complete.

To test my memory/swapping hypothesis, I killed my RAM-hog Firefox and tried again:

fcef2725bde9        hectcastro/riak:latest      /usr/bin/supervisord   1 seconds ago       Up Less than a second   22/tcp, 8087/tcp, 8098/tcp    elegant_wright
f2f5a6a714d9        hectcastro/riak:latest      /usr/bin/supervisord   21 seconds ago      Up 20 seconds           22/tcp, 8087/tcp, 8098/tcp    ecstatic_wozniak
a5efc0e0f293        hectcastro/riak:latest      /usr/bin/supervisord   27 seconds ago      Up 27 seconds           22/tcp, 8087/tcp, 8098/tcp    berserk_mccarthy

As can be seen, the second container came quickly, the third took some more time, but then...

fcef2725bde9        hectcastro/riak:latest      /usr/bin/supervisord   33 seconds ago      Up 33 seconds       22/tcp, 8087/tcp, 8098/tcp    elegant_wright
f2f5a6a714d9        hectcastro/riak:latest      /usr/bin/supervisord   53 seconds ago      Up 53 seconds       22/tcp, 8087/tcp, 8098/tcp    ecstatic_wozniak
a5efc0e0f293        hectcastro/riak:latest      /usr/bin/supervisord   59 seconds ago      Up 59 seconds       22/tcp, 8087/tcp, 8098/tcp    berserk_mccarthy

After that, my second shell didn't even react at all... VBoxHeadless takes about 50% CPU, memory obviously isn't the problem, there is no I/O activity. And finally...

e027dec47311        hectcastro/riak:latest      /usr/bin/supervisord   About a minute ago   Up About a minute   22/tcp, 8087/tcp, 8098/tcp    sharp_engelbart
a3938391363f        hectcastro/riak:latest      /usr/bin/supervisord   2 minutes ago        Up 2 minutes        22/tcp, 8087/tcp, 8098/tcp    condescending_tesla
fcef2725bde9        hectcastro/riak:latest      /usr/bin/supervisord   11 minutes ago       Up 11 minutes       22/tcp, 8087/tcp, 8098/tcp    elegant_wright
f2f5a6a714d9        hectcastro/riak:latest      /usr/bin/supervisord   11 minutes ago       Up 11 minutes       22/tcp, 8087/tcp, 8098/tcp    ecstatic_wozniak
a5efc0e0f293        hectcastro/riak:latest      /usr/bin/supervisord   11 minutes ago       Up 11 minutes       22/tcp, 8087/tcp, 8098/tcp    berserk_mccarthy

So we had about 9 minutes of seeming inactivity...

I already made up my mind to reload vagrant... memory doesn't seem to be the problem, does it?

What causes this random annoying delay? Should we be warned?


Last issue:

No idea what to do with this wonderful setup.

It is all very instructive, thank you very much - but what to do now? Considering the diligence of this example/tutorial, the next step seems to be missing. How about adding at least a link to http://docs.basho.com/riak/latest/quickstart/? But even then...

Apparently, I don't understand the model and concept here. So for your information this is what a newbie might do and experience:

I first tried to attach to the first container, berserk_mccarthy, saw nothing, hit Ctrl+c, and stopped it -- supervisord was running, my interaction caused it to stop. Ok, not a good idea.

So I started the container again and supervisord resumed operation, as it seems, but nevertheless I decided to experiment with the next one, ecstatic_wozniak, further on to be sure, and leave berserk_mccarthy alone.

vagrant@precise64:~$ docker inspect ecstatic_wozniak # revealed the IP 172.17.0.150
vagrant@precise64:~$ curl -XPUT http://172.17.0.150:8087/riak/images/1.jpg -H "Content-type: image/jpeg" --data-binary @test.jpg
curl: (7) couldn't connect to host
vagrant@precise64:~$ curl -XPUT http://192.168.0.42:8087/riak/images/1.jpg -H "Content-type: image/jpeg" --data-binary @test.jpg
curl: (7) couldn't connect to host

Now I deduce from this that I have to start from a container linked to one of the riak containers. (Later: these ports are exposed, so they should not be seen from the host, but from other containers, see below.)

vagrant@precise64:~$ docker run -i -t -rm -link /ecstatic_wozniak:r2 ubuntu bash
root@379409e588b6:/# echo $R2_PORT_8087_TCP_ADDR
172.17.0.150 // ok, works, I can use that
root@379409e588b6:/# curl -XPUT http://$R2_PORT_8087_TCP_ADDR:8087/riak/images/1.jpg -H "Content-type: image/jpeg" --data-binary @test.jpg
bash: curl: command not found // ok, fix that
root@379409e588b6:/# apt-get install -y curl
...
root@379409e588b6:/# curl -XPUT http://$R2_PORT_8087_TCP_ADDR:8087/riak/images/1.jpg -H "Content-type: image/jpeg" --data-binary @test.jpg
Warning: Couldn't read data from file "test.jpg", this makes an empty POST.
curl: (7) couldn't connect to host

Well, the first statement is absolutely correct...

I definitely have to make sure I have this file, so I'll start anew and add a volume to my container...

vagrant@precise64:~$ docker run -i -t -rm  -w /var/www -volumes-from app -link /ecstatic_wozniak:r2 ubuntu bash
root@c218b49c4d1d:/var/www# ls *.j*
composer.json  test.jpg # here we have it
root@c218b49c4d1d:/var/www# apt-get install -y curl
... # again
root@c218b49c4d1d:/var/www# curl -XPUT http://$R2_PORT_8087_TCP_ADDR:8087/riak/images/1.jpg -H "Content-type: image/jpeg" --data-binary @test.jpg
curl: (7) couldn't connect to host # now why is that?
root@c218b49c4d1d:/var/www# ping $R2_PORT_8087_TCP_ADDR
bash: ping: command not found
root@c218b49c4d1d:/var/www# apt-get install -y ping
root@c218b49c4d1d:/var/www# ping $R2_PORT_8087_TCP_ADDR // ok, we can ping - now what? Get more information?
root@c218b49c4d1d:/var/www# curl -vvv http://$R2_PORT_8087_TCP_ADDR:8087
* About to connect() to 172.17.0.150 port 8087 (#0)
*   Trying 172.17.0.150... Connection refused
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host
root@c218b49c4d1d:/var/www# curl -vvv -noproxy http://$R2_PORT_8087_TCP_ADDR:8087
* Couldn't find host 172.17.0.150 in the .netrc file; using defaults
* About to connect() to 172.17.0.150 port 8087 (#0)
*   Trying 172.17.0.150...   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Connection refused
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host

What now?

As an aside: When I tried to attach to the first riak container, berserk_mccarthy, I saw that the riak service was running on it:

vagrant@precise64:~$ docker attach berserk_mccarthy
^C2014-01-18 13:32:34,272 WARN received SIGINT indicating exit request
2014-01-18 13:32:34,659 INFO waiting for sshd, riak to die
2014-01-18 13:32:35,745 INFO stopped: riak (terminated by SIGTERM)
2014-01-18 13:32:36,528 INFO stopped: sshd (exit status 0)

When I started again, I got

vagrant@precise64:~$ docker start berserk_mccarthy
berserk_mccarthy
vagrant@precise64:~$ docker attach berserk_mccarthy
2014-01-18 13:33:05,932 CRIT reaped unknown pid 60)
2014-01-18 13:33:06,113 CRIT reaped unknown pid 63)
2014-01-18 13:33:07,116 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-01-18 13:33:07,116 INFO success: riak entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-01-18 13:33:12,175 CRIT reaped unknown pid 66)
2014-01-18 13:33:12,178 CRIT reaped unknown pid 65)
2014-01-18 13:33:12,313 CRIT reaped unknown pid 81)
2014-01-18 13:33:12,453 CRIT reaped unknown pid 84)
2014-01-18 13:33:12,462 CRIT reaped unknown pid 85)
2014-01-18 13:33:12,515 CRIT reaped unknown pid 94)
2014-01-18 13:33:12,518 CRIT reaped unknown pid 95)
2014-01-18 13:33:12,758 CRIT reaped unknown pid 167)

So there might not be everything ok with the first, although riak is running

Back to my question:

http://stackoverflow.com/questions/19679803/how-to-seed-data-to-riak-using-rest-api-from-remote-host
The cluster built in the Riak quick start is intended as a local development cluster and it is therefore by default set up to only accept connections from 127.0.0.1. You can change this in the app.config file for each node, which can be found in the /etc directory, and instead make it bind to e.g. 0.0.0.0.

Interesting. First I tried to get into my running first riak container berserk_mccarthy to find out, to no avail.

^C2014-01-18 14:19:55,875 WARN received SIGINT indicating exit request
2014-01-18 14:19:55,890 INFO waiting for sshd, riak to die
2014-01-18 14:19:56,892 INFO stopped: riak (terminated by SIGTERM)
2014-01-18 14:19:57,000 INFO stopped: sshd (exit status 0)
vagrant@precise64:~$ docker start berserk_mccarthy
berserk_mccarthy
vagrant@precise64:~$ docker attach berserk_mccarthy
2014-01-18 14:20:15,517 CRIT reaped unknown pid 66)
2014-01-18 14:20:15,634 CRIT reaped unknown pid 82)
2014-01-18 14:20:15,787 CRIT reaped unknown pid 85)
2014-01-18 14:20:15,795 CRIT reaped unknown pid 86)
2014-01-18 14:20:15,850 CRIT reaped unknown pid 95)
2014-01-18 14:20:15,850 CRIT reaped unknown pid 96)
2014-01-18 14:20:16,073 CRIT reaped unknown pid 168)


^C2014-01-18 14:20:24,323 WARN received SIGINT indicating exit request
2014-01-18 14:20:24,323 INFO waiting for sshd, riak to die
2014-01-18 14:20:25,326 INFO stopped: riak (terminated by SIGTERM)
2014-01-18 14:20:25,389 INFO stopped: sshd (exit status 0)
vagrant@precise64:~$ docker start berserk_mccarthy
berserk_mccarthy
vagrant@precise64:~$ docker ps -a
CONTAINER ID        IMAGE                       COMMAND                CREATED             STATUS              PORTS                         NAMES
c218b49c4d1d        ubuntu:12.04                bash                   33 minutes ago      Up 33 minutes                                     cranky_turing
e027dec47311        hectcastro/riak:latest      /usr/bin/supervisord   About an hour ago   Up About an hour    22/tcp, 8087/tcp, 8098/tcp    sharp_engelbart
a3938391363f        hectcastro/riak:latest      /usr/bin/supervisord   About an hour ago   Up About an hour    22/tcp, 8087/tcp, 8098/tcp    condescending_tesla
fcef2725bde9        hectcastro/riak:latest      /usr/bin/supervisord   About an hour ago   Up About an hour    22/tcp, 8087/tcp, 8098/tcp    elegant_wright
f2f5a6a714d9        hectcastro/riak:latest      /usr/bin/supervisord   About an hour ago   Up About an hour    22/tcp, 8087/tcp, 8098/tcp    cranky_turing/r2,ecstatic_wozniak
a5efc0e0f293        hectcastro/riak:latest      /usr/bin/supervisord   About an hour ago   Up 6 seconds        22/tcp, 8087/tcp, 8098/tcp    berserk_mccarthy

So this time it seems to have started as a daemon as expected...

Ok, so I start another copy with shell to find out:

vagrant@precise64:~$ docker run -i -t -rm -name test_riak hectcastro/riak bash

Didn't find anything in /etc/riak/app.config as indicated above... So this is unrewarding.

Well, never mind, I don't really want to connect from the host anyway, so I'll give it another try on my ubuntu container cranky_turing with link to ecstatic_wozniak.

Actually I don't really understand that bridge thing; maybe give that a try:

root@c218b49c4d1d:/var/www# curl -i -X GET http://33.33.33.20:10018/riak/stats
curl: (7) couldn't connect to host
root@c218b49c4d1d:/var/www# ping 33.33.33.20
PING 33.33.33.20 (33.33.33.20) 56(84) bytes of data.
64 bytes from 33.33.33.20: icmp_req=1 ttl=63 time=0.169 ms
64 bytes from 33.33.33.20: icmp_req=2 ttl=63 time=0.156 ms
64 bytes from 33.33.33.20: icmp_req=3 ttl=63 time=0.125 ms
64 bytes from 33.33.33.20: icmp_req=4 ttl=63 time=0.122 ms
^C
--- 33.33.33.20 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.122/0.143/0.169/0.020 ms

Any hint? Googling doesn't help.

Now surprise, surprise, by chance I tried port 22 from my container and got:

root@c218b49c4d1d:/var/www# curl -vvv https://$R2_PORT_8087_TCP_ADDR:22
* About to connect() to 172.17.0.150 port 22 (#0)
*   Trying 172.17.0.150... connected
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
* Closing connection #0
curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

Doesn't this look promising??? I was able to connect to ecstatic_wozniak from cranky_turing on port 22!

Do I need a certificate for this to be totally successful? I don't think so. Still: why can't I connect normally? We have 3 ports exposed, don't we?

This will fail, but may teach me something:

root@c218b49c4d1d:/var/www# curl -vvv http://$R2_PORT_8087_TCP_ADDR:22
* About to connect() to 172.17.0.150 port 22 (#0)
*   Trying 172.17.0.150... connected
> GET / HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: 172.17.0.150:22
> Accept: */*
>
SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
Protocol mismatch.
* Recv failure: Connection reset by peer
* Closing connection #0
curl: (56) Recv failure: Connection reset by peer

At least this was expected. So: port 22 is ok.

root@c218b49c4d1d:/var/www# curl -vvv https://$R2_PORT_8087_TCP_ADDR:8087
* About to connect() to 172.17.0.150 port 8087 (#0)
*   Trying 172.17.0.150... Connection refused
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host
root@c218b49c4d1d:/var/www# curl -vvv http://$R2_PORT_8087_TCP_ADDR:8098
* About to connect() to 172.17.0.150 port 8098 (#0)
*   Trying 172.17.0.150... Connection refused
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host

This was expected as well.

The port rather than the protocol seems to be the issue. Ports 8087 and 8098 don't work. Why?

root@c218b49c4d1d:/var/www# echo $R2_PORT
tcp://172.17.0.150:22

Here we have it. Why do I see only this port, not the others? Aren't they exposed just as well?

CONTAINER ID        IMAGE                       COMMAND                CREATED             STATUS              PORTS                         NAMES
f2f5a6a714d9        hectcastro/riak:latest      /usr/bin/supervisord   5 hours ago         Up 5 hours          22/tcp, 8087/tcp, 8098/tcp    cranky_turing/r2,ecstatic_wozniak

vagrant@precise64:~$ docker port ecstatic_wozniak  22
2014/01/18 19:15:00 Error: No public port '22' published for ecstatic_wozniak

The same for the ports 8087, 8098. What is this?? (Later: they are exposed, i.e. not visible from the host, see below.)


The next day, I had to take my machine down due to a monitor failure. After vagrant up the riak containers and memcached were restartet automatically; I had to start my Ms and AP manually, so my installation must be improved in this respect.

And then:

vagrant@precise64:~$ curl http://192.168.0.42:49154
curl: (7) couldn't connect to host

What is this?

vagrant@precise64:~$ docker port ap 80
192.168.0.42:49154

Well, obviously Apache wasn't running, so stopping, removing and running ap again solved this problem. Aha. Apache responds to curl. Who responds to curl on the riak containers? Hopefully riak, or what?

Let's see what my test_riak says:

vagrant@precise64:~$ docker port test_riak  22
2014/01/19 12:11:46 Error: No public port '22' published for test_riak
vagrant@precise64:~$ docker port test_riak  8087
2014/01/19 12:11:51 Error: No public port '8087' published for test_riak
vagrant@precise64:~$ docker port test_riak  8098
2014/01/19 12:11:55 Error: No public port '8098' published for test_riak

Could that message be true???

Dockerfile says:
EXPOSE 8087 8098 22

That means: other containers can see this, but not the host... if not exposed. What did I do?

vagrant@precise64:~$ docker run -i -t -rm -name test_riak hectcastro/riak bash

Ok, so this is that. Easy to fix, I guess.

vagrant@precise64:~$ docker stop test_riak
test_riak
vagrant@precise64:~$ docker rm test_riak
test_riak
vagrant@precise64:~$ docker run -i -t -rm -p 22 8087 8098 -name test_riak hectcastro/riak bash
2014/01/19 12:17:02 Error: create: ExportEnv json: cannot unmarshal number into Go value of type string
vagrant@precise64:~$ docker run -i -t -rm -p 22 -p 8087 -p 8098 -name test_riak hectcastro/riak bash
2014/01/19 12:17:35 Error: start: Cannot start container cd1ee0f6a6ec1287f2ff4d60058bcc36cc97a927c02175adc0e81137d5d75134: listen tcp 0.0.0.0:49154: bind: address already in use

What is this? 49154 belongs to AP, I know. docker makes a good guess for the IP, so another try might help.

vagrant@precise64:~$ docker run -i -t -rm -p 22 -p 8087 -p 8098 -name test_riak hectcastro/riak bash
2014/01/19 12:18:55 Error: create: Conflict, The name test_riak is already assigned to cd1ee0f6a6ec. You have to delete (or rename) that container to be able to assign test_riak to a container again.
vagrant@precise64:~$ docker stop test_riak
test_riak
vagrant@precise64:~$ docker rm test_riak
test_riak
vagrant@precise64:~$ docker run -i -t -rm -p 22 -p 8087 -p 8098 -name test_riak hectcastro/riak bash
root@83f31aaf6a51:/#

Bingo.

vagrant@precise64:~$ docker port test_riak 22
0.0.0.0:49155
vagrant@precise64:~$ docker port test_riak 8087
0.0.0.0:49157
vagrant@precise64:~$ docker port test_riak 8098
0.0.0.0:49158

Bingo.

vagrant@precise64:~$ curl -i -X GET http://172.17.0.18:8087/riak/stats
curl: (7) couldn't connect to host
vagrant@precise64:~$ curl -i -X GET https://172.17.0.18:22/riak/stats
curl: (7) couldn't connect to host

No chance from the host. Start a linked container again:

vagrant@precise64:~$ docker run -i -t -rm  -w /var/www -volumes-from app -h test_u -name test_u -link /test_riak:r2 ubuntu bash
root@test_u:/var/www# apt-get install -y curl

Wait... test_riak exposed the shell. supervisord & leads to loop.

Kill all, start new:

vagrant@precise64:~$ docker run -d -name t_riak hectcastro/riak
ff753acf43575f9dab2dd462ffc931d51ebe6cfdc61af1a00f97bcefa3395be0
vagrant@precise64:~$ docker run -i -t -rm  -w /var/www -volumes-from app -h t_u -name test_u -link /t_riak:r2 ubuntu bash
root@t_u:/var/www# env
R2_PORT_22_TCP_PORT=22
HOSTNAME=t_u
R2_PORT_8087_TCP_ADDR=172.17.0.23
R2_PORT_8087_TCP_PROTO=tcp
TERM=xterm
R2_PORT_22_TCP_ADDR=172.17.0.23
R2_PORT_22_TCP_PROTO=tcp
R2_PORT_8098_TCP_ADDR=172.17.0.23
R2_PORT_8098_TCP=tcp://172.17.0.23:8098
R2_NAME=/test_u/r2
R2_PORT_8098_TCP_PROTO=tcp
R2_PORT_22_TCP=tcp://172.17.0.23:22
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/var/www
R2_PORT_8087_TCP_PORT=8087
R2_PORT_8087_TCP=tcp://172.17.0.23:8087
SHLVL=1
HOME=/
R2_PORT_8098_TCP_PORT=8098
container=lxc
R2_PORT=tcp://172.17.0.23:22
_=/usr/bin/env
root@t_u:/var/www# apt-get install -y curl
root@t_u:/var/www# curl -i -X GET http://$R2_PORT_8098_TCP/riak/stats
curl: (6) Couldn't resolve host 'tcp'

Oh, something new.

root@t_u:/var/www# echo $R2_PORT_8098_TCP
tcp://172.17.0.23:8098

Ok, no wonder...

root@t_u:/var/www# curl -i -X GET http://$R2_PORT_8098_TCP_ADDR:8098/riak/stats
curl: (7) couldn't connect to host

Again. This is the problem, no matter if I start from the host or another container. Most probably port 22 will do, though.

root@t_u:/var/www# curl -i -X GET https://$R2_PORT_22_TCP_ADDR:22/riak/stats
curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

I had this before. Googling didn'thelp here.

root@t_u:/var/www# curl -i -X GET http://$R2_PORT_22_TCP_ADDR:22/riak/stats
SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
Protocol mismatch.
curl: (56) Recv failure: Connection reset by peer

Ok, this was expected.

Well, for the time being, I'll give up. I tried my best and learned a lot. Thanks again.


PS: To give you some more feedback:

As a last resort, I tried http://docs.basho.com/riak/latest/quickstart/, but failed miserably as well. I didn't have a problem to install riak on my vagrant machine. The problem started with Now that Riak is built, we are going to use Rebar,....

I didn't know really how to install rebar on my system. Downloading the executable from https://github.com/rebar/rebar/wiki/rebar didn't really help. A call to rebar delivered

/usr/bin/env: escript: No such file or directory

I spent quite some time trying to find out how to get this escript and finally gave up.

riak resides in /usr/sbin, so I switched there.

As expected,

make devrel DEVNODES=5

gave an error:

make: *** No rule to make target `devrel'.  Stop.

Sorry, I'm not going to waste more time here. Obviously, this tutorial is made for people who know a lot of things I don't.


PPS: Exploring LXC Networking explains bridging etc. very good.

'make build' broken due to ppa:webupd8team/java

I'm setting up a freshly cloned docker-riak. 'make build' fails because apt-add-repository can't find the webupd8team/java PPA. Can I omit installing Java 7? Here is the output from 'make build':

dtalmage@reasonable:~/repos/docker-riak$ sudo make build
docker build -t "hectcastro/riak" .                                                                                 
Sending build context to Docker daemon 159.2 kB                                                                     
Sending build context to Docker daemon                                                                              
Step 0 : FROM phusion/baseimage:0.9.14                                                                              
 ---> e74fe19c755c                                                                                                  
Step 1 : MAINTAINER Hector Castro [email protected]                                                              
 ---> Using cache                                                                                                   
 ---> 4b9708d2ae1f                                                                                                  
Step 2 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> 827d1658c4b4
Step 3 : ENV RIAK_VERSION 2.0.1-1
 ---> Using cache
 ---> 3002456bc7e0
Step 4 : RUN sed -i.bak 's/main$/main universe/' /etc/apt/sources.list
 ---> Using cache
 ---> 3f672725b8ea
Step 5 : RUN apt-get update -qq && apt-get install -y software-properties-common &&     apt-add-repository ppa:webupd8team/java -y && apt-get update -qq &&     echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections &&     apt-get install -y oracle-java7-installer
 ---> Running in b143ee4dc4a8
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease  

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/InRelease  

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-security/InRelease  

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg  Could not resolve 'archive.ubuntu.com'

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg  Could not resolve 'archive.ubuntu.com'

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg  Could not resolve 'archive.ubuntu.com'

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
software-properties-common is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Cannot add PPA: 'ppa:webupd8team/java'.
Please check that the PPA name or format is correct.
2014/10/29 10:36:20 The command [/bin/sh -c apt-get update -qq && apt-get install -y software-properties-common &&     apt-add-repository ppa:webupd8team/java -y && apt-get update -qq &&     echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections &&     apt-get install -y oracle-java7-installer] returned a non-zero code: 1
Makefile:6: recipe for target 'build' failed
make: *** [build] Error 1
dtalmage@reasonable:~/repos/docker-riak$ 

`docker run` fails when there is a dead container named riak0*

If the names riak01, riak02, etc. are already taken (even if the container is dead, so it doesn't show up in docker ps but does show up in docker ps -a), docker run seems to just silently exit with status 1:

λ DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=3 DOCKER_RIAK_DEBUG=1 bin/start-cluster.sh
+ DOCKER_RIAK_CLUSTER_SIZE=3
+ docker ps
+ grep hectcastro/riak
+ echo

+ echo 'Bringing up cluster nodes:'
Bringing up cluster nodes:
+ echo

++ seq -f %02g 1 3
+ for index in '$(seq -f "%02g" "1" "${DOCKER_RIAK_CLUSTER_SIZE}")'
+ '[' 01 -gt 1 ']'
+ docker run -e DOCKER_RIAK_CLUSTER_SIZE=3 -e DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 -P --name riak01 -d hectcastro/riak

When running without debug info, nothing at all is printed to the output, it just fails to run. Removing the dead containers fixes the issue.

make: *** [Makefile:9: start-cluster] Error 125

Cannot run DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 DOCKER_RIAK_BACKEND=leveldb make start-cluster ./bin/start-cluster.sh

Output:

Bringing up cluster nodes:

make: *** [Makefile:9: start-cluster] Error 125

Running arch linux
4.10.5-1-ARCH #1 SMP PREEMPT Wed Mar 22 14:42:03 CET 2017 x86_64 GNU/Linux

Support for other backends?

Would it be possible to expose the Riak backend used in the cluster as an environment variable? DOCKER_RIAK_BACKEND defaulting to bitcask but also supporting memory or leveldb?

can't ssh to container

→  docker ps
.....
099dc3966110        hectcastro/riak:latest   /sbin/my_init --quie   8 minutes ago       Up 8 minutes        0.0.0.0:49163->8087/tcp, 0.0.0.0:49164->8098/tcp   riak01,riak02/seed,riak03/seed,riak04/seed,riak05/seed   

→  docker inspect 099dc3966110 | grep IP
        "IPAddress": "172.17.0.7",
        "IPPrefixLen": 16,
→  ssh -i insecure_key [email protected]
The authenticity of host '172.17.0.7 (172.17.0.7)' can't be established.
ECDSA key fingerprint is f1:56:8c:fc:8b:95:75:ff:5f:bc:4a:53:72:3d:da:0a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.17.0.7' (ECDSA) to the list of known hosts.
Connection to 172.17.0.7 closed by remote host.
Connection to 172.17.0.7 closed.

Docker Riak between multiple hosts with Docker Swarm?

I'm trying to set up something similar to #10. Thanks, that thread definitely gives me a good place to start.

I know a recent release of Docker introduced "Docker Swarm" which seems aimed to address this kind of issue — allowing docker containers running on different hosts to communicate. Anyone know more about this new feature and know how to use it to run Riak between containers on multiple hosts?

Problem in the config file (storage_backend =)

Riak was not starting so I logged into the container and did:

riak console
/etc/riak/riak.conf after line 334 column 1, parsing incomplete

vim /etc/riak/riak.conf

## Acceptable values:                                                                                                                                                                 
##   - one of: bitcask, leveldb, memory, multi                                                                                                                                        
storage_backend =

So I commented the entire line and it works.

better error reporting

Is it possible to debug why start-cluster failed?

→  DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 DOCKER_RIAK_BACKEND=riak_kv_eleveldb_backend make start-cluster
./bin/start-cluster.sh

Bringing up cluster nodes:

  Successfully brought up [riak01]
  Successfully brought up [riak02]
make: *** [start-cluster] Error 1
→  docker ps -a
CONTAINER ID        IMAGE                    COMMAND                CREATED             STATUS              PORTS                                              NAMES
f5c2cfd5229b        hectcastro/riak:latest   /sbin/my_init --quie   2 minutes ago                                                                                                   
d1eeed766851        hectcastro/riak:latest   /sbin/my_init --quie   4 minutes ago       Up 3 minutes        0.0.0.0:49157->8087/tcp, 0.0.0.0:49158->8098/tcp   riak02               
f30ce6ca8db0        hectcastro/riak:latest   /sbin/my_init --quie   4 minutes ago       Up 4 minutes        0.0.0.0:49155->8087/tcp, 0.0.0.0:49156->8098/tcp   riak01,riak02/seed   

apt-get upgrade returns an error

Tried to install riak using your procedure but it does work because apt-get upgrade returns an error and exit.
Providing below the end of the log of the apt-get upgrade. I remain available if you need further information:
Setting up initscripts (2.88dsf-13.10ubuntu11.1) ...
mount: block device /dev/shm is write-protected, mounting read-only
mount: cannot mount block device /dev/shm read-only
dpkg: error processing initscripts (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of ifupdown:
ifupdown depends on initscripts (>= 2.88dsf-13.3); however:
Package initscripts is not configured yet.
dpkg: error processing ifupdown (--configure):
dependency problems - leaving unconfigured
Processing triggers for libc-bin ...
dpkg: dependency problems prevent configuration of procps:
procps depends on initscripts; however:
Package initscripts is not configured yet.
dpkg: error processing procps (--configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
No apport report written because the error message indicates its a followup error from a previous failure.
dpkg: dependency problems prevent configuration of udev:
udev depends on procps; however:
Package procps is not configured yet.
dpkg: error processing udev (--configure):
dependency problems - leaving unconfigured
No apport report written because MaxReports is reached already
dpkg: dependency problems prevent configuration of plymouth:
plymouth depends on udev (>= 166-0ubuntu4); however:
Package udev is not configured yet.
dpkg: error processing plymouth (--configure):
dependency problems - leaving unconfigured
No apport report written because MaxReports is reached already
dpkg: dependency problems prevent configuration of mountall:
mountall depends on udev; however:
Package udev is not configured yet.
mountall depends on plymouth; however:
Package plymouth is not configured yet.
dpkg: error processing mountall (--configure):
dependency problems - leaving unconfigured
No apport report written because MaxReports is reached already
dpkg: dependency problems prevent configuration of initramfs-tools:
initramfs-tools depends on udev (>= 147~-5); however:
Package udev is not configured yet.
dpkg: error processing initramfs-tools (--configure):
dependency problems - leaving unconfigured
No apport report written because MaxReports is reached already
ldconfig deferred processing now taking place
Errors were encountered while processing:
initscripts
ifupdown
procps
udev
plymouth
mountall
initramfs-tools

2014/02/26 11:03:09 build: The command [/bin/sh -c apt-get upgrade -y] returned a non-zero code: 255
make: *** [riak-container] Error 1

2.0.0 beta doesn't seem to work

Could be doing something wrong but this is what happens when I try to build 2.0.0beta

$ git diff Dockerfile
diff --git a/Dockerfile b/Dockerfile
index 05ac064..8089d30 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -7,8 +7,8 @@ MAINTAINER Hector Castro [email protected]

 # Environmental variables
 ENV DEBIAN_FRONTEND noninteractive
-ENV RIAK_VERSION 1.4.8
-ENV RIAK_SHORT_VERSION 1.4
+ENV RIAK_VERSION 2.0.0beta1
+ENV RIAK_SHORT_VERSION 2.0

 # Install Riak
$ make build
docker build -t "hectcastro/riak" .
Uploading context 159.7 kB
Uploading context 
Step 0 : FROM phusion/baseimage:0.9.9
Pulling repository phusion/baseimage
745d3ac92697: Download complete 
511136ea3c5a: Download complete 
6170bb7b0ad1: Download complete 
9cd978db300e: Download complete 
a6d44c263269: Download complete 
13e42d0c2a51: Download complete 
846e143f3fab: Download complete 
10ebd1d649cb: Download complete 
5db917407faa: Download complete 
 ---> 745d3ac92697
Step 1 : MAINTAINER Hector Castro [email protected]
 ---> Using cache
 ---> cdf4f76a52ba
Step 2 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> 1d1bc50ff14c
Step 3 : ENV RIAK_VERSION 2.0.0beta1
 ---> Running in 852b3e18a7c5
 ---> 651b9aa51c71
Removing intermediate container 852b3e18a7c5
Step 4 : ENV RIAK_SHORT_VERSION 2.0
 ---> Running in 60a4fdb3f1e7
 ---> fef320a7de3e
Removing intermediate container 60a4fdb3f1e7
Step 5 : ADD http://s3.amazonaws.com/downloads.basho.com/riak/${RIAK_SHORT_VERSION}/${RIAK_VERSION}/ubuntu/precise/riak_${RIAK_VERSION}-1_amd64.deb /
 ---> 8d68aa537009
Removing intermediate container d796d0e24349
Step 6 : RUN (cd / && dpkg -i "riak_${RIAK_VERSION}-1_amd64.deb")
 ---> Running in 5a1b940801ec
Selecting previously unselected package riak.
(Reading database ... 12731 files and directories currently installed.)
Unpacking riak (from riak_2.0.0beta1-1_amd64.deb) ...
Setting up riak (2.0.0beta1-1) ...
Adding group `riak' (GID 105) ...
Done.
Adding system user `riak' (UID 103) ...
Adding new user `riak' (UID 103) with group `riak' ...
Not creating home directory `/var/lib/riak'.
Processing triggers for ureadahead ...
 ---> 826aa1451523
Removing intermediate container 5a1b940801ec
Step 7 : RUN mkdir -p /etc/service/riak
 ---> Running in ef9ea9e0c313
 ---> 8d7290bdcb8d
Removing intermediate container ef9ea9e0c313
Step 8 : ADD bin/riak.sh /etc/service/riak/run
 ---> c0265e2fb97f
Removing intermediate container 1c022ae5f1d5
Step 9 : ADD bin/automatic_clustering.sh /etc/my_init.d/99_automatic_clustering.sh
 ---> f11bb1860420
Removing intermediate container 633bcea879ae
Step 10 : RUN sed -i.bak 's/127.0.0.1/0.0.0.0/' /etc/riak/app.config &&    sed -i.bak 's/{anti_entropy_concurrency, 2}/{anti_entropy_concurrency, 1}/' /etc/riak/app.config &&    sed -i.bak 's/{map_js_vm_count, 8 }/{map_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak 's/{reduce_js_vm_count, 6 }/{reduce_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak 's/{hook_js_vm_count, 2 }/{hook_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak "s/##+zdbbl/+zdbbl/" /etc/riak/vm.args
 ---> Running in 2f93b1c39b87
sed: can't read /etc/riak/app.config: No such file or directory
2014/05/30 10:40:04 The command [/bin/sh -c sed -i.bak 's/127.0.0.1/0.0.0.0/' /etc/riak/app.config &&    sed -i.bak 's/{anti_entropy_concurrency, 2}/{anti_entropy_concurrency, 1}/' /etc/riak/app.config &&    sed -i.bak 's/{map_js_vm_count, 8 }/{map_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak 's/{reduce_js_vm_count, 6 }/{reduce_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak 's/{hook_js_vm_count, 2 }/{hook_js_vm_count, 0 }/' /etc/riak/app.config &&    sed -i.bak "s/##+zdbbl/+zdbbl/" /etc/riak/vm.args] returned a non-zero code: 2
make: *** [build] Error 1

I'm guessing its just an incompatibility with the new beta version and where it places config files

doesn't start cluster

Docker version 1.1.2, build d84a070/1.1.2
Fedora 20

Hangs after:

export DOCKER_HOST="unix:///var/run/docker.sock"
→  DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RIAK_CLUSTER_SIZE=5 DOCKER_RIAK_BACKEND=riak_kv_eleveldb_backend make start-cluster./bin/start-cluster.sh

Bringing up cluster nodes:

Here is recent debug output from sudo docker -d D:

[debug] server.go:1022 Calling POST /containers/{name:.*}/start
2014/08/20 11:24:35 POST /v1.13/containers/41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123/start
[5bf92bb2] +job start(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123)
[debug] deviceset.go:278 activateDeviceIfNeeded(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123)
[5bf92bb2] +job allocate_interface(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123)
[5bf92bb2] -job allocate_interface(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123) = OK (0)
[5bf92bb2] +job allocate_port(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123)
[debug] /sbin/iptables, [-t nat -A DOCKER -p tcp -d 0/0 --dport 49155 ! -i docker0 -j DNAT --to-destination 172.17.0.3:8087]
[debug] /sbin/iptables, [-I FORWARD ! -i docker0 -o docker0 -p tcp -d 172.17.0.3 --dport 8087 -j ACCEPT]
[5bf92bb2] -job allocate_port(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123) = OK (0)
[5bf92bb2] +job allocate_port(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123)
[debug] /sbin/iptables, [-t nat -A DOCKER -p tcp -d 0/0 --dport 49156 ! -i docker0 -j DNAT --to-destination 172.17.0.3:8098]
[debug] /sbin/iptables, [-I FORWARD ! -i docker0 -o docker0 -p tcp -d 172.17.0.3 --dport 8098 -j ACCEPT]
[5bf92bb2] -job allocate_port(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123) = OK (0)
[debug] archive.go:505 Creating dest directory: /var/lib/docker/vfs/dir/9ae11c203359ba0444b4bc8b82cfd73b4f4dd30e5799148c6433f4b6b9a60943
[debug] archive.go:509 Calling TarUntar(/var/lib/docker/devicemapper/mnt/41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123/rootfs/var/lib/riak, /var/lib/docker/vfs/dir/9ae11c203359ba0444b4bc8b82cfd73b4f4dd30e5799148c6433f4b6b9a60943)
[debug] archive.go:468 TarUntar(/var/lib/docker/devicemapper/mnt/41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123/rootfs/var/lib/riak /var/lib/docker/vfs/dir/9ae11c203359ba0444b4bc8b82cfd73b4f4dd30e5799148c6433f4b6b9a60943)
[debug] archive.go:86 [tar autodetect] n: [46 47 0 0 0 0 0 0 0 0]
[5bf92bb2] -job start(41e6325a492c547533c043bd46aefb28aa8875a1d7c87c4a499d5b954b466123) = OK (0)
[debug] server.go:1022 Calling GET /containers/json
2014/08/20 11:24:35 GET /v1.13/containers/json
[5bf92bb2] +job containers()
[5bf92bb2] -job containers() = OK (0)
[debug] server.go:1022 Calling GET /containers/{name:.*}/json
2014/08/20 11:24:35 GET /v1.13/containers/41e6325a492c/json
[5bf92bb2] +job container_inspect(41e6325a492c)
[5bf92bb2] -job container_inspect(41e6325a492c) = OK (0)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.