weiqingy / caochong Goto Github PK

View Code? Open in Web Editor NEW

78.0 78.0 35.0 87 KB

Set up a Hadoop and/or Spark cluster running within Docker containers on a single physical machine

License: Apache License 2.0

Shell 91.17% Dockerfile 8.83%

caochong's People

Stargazers

Watchers

caochong's Issues

docker ntpd service is dead, cannot restart

Just a simple test and spinning up 3 containers but ambari install check shows NTP is not running and IPTABLE is running.

I checked NTPD and try to start it but failed

[root@97ea7075ca78 ~]# service ntpd start
Starting ntpd:                                             [  OK  ]
[root@97ea7075ca78 ~]# service ntpd status
ntpd dead but pid file exists

I also checked ipables but it is NOT running.

The warning message can be ignored and proceed with final installation of HDP but in the end, most of my services are not running and I have to quit using this repo approach.

Wonder who has successfully used this repo though?

Make Ambari secure cluster work

Support HA-enabled feature

HDFS and YARN clusters can be HA enabled, and we should support this case.

Make the number of containers configurable for both from-ambari and from-source

Not working on Mac (Sierra - OSX 10.12)

First, thanks for this wonderful piece of software!

Sofware Versions
Mac OSX 10.12 (Sierra)
Docker Version 1.12.5

I'm using the ambari/run.sh script to bring up a 3 node hadoop cluster. Everything works like a charm until I reach the 9th step - i.e. the actual installation of the various services on the containers.

The scripts invoked (inside the containers) to install hadoop (and other services) get timed out.

On the host (Mac - /var/log/system.log), I get to see logs like the following:

Jan 10 14:51:08 ggnf2058s-MacBook-Pro Docker[2668]: dropping unexpected TCP packet sent from 172.18.0.3:36100 to 14.139.62.20:80 (valid subnets = 192.168.65.0/28; valid sources = 0.0.0.0)
Jan 10 14:51:09 ggnf2058s-MacBook-Pro Docker[2668]: dropping unexpected TCP packet sent from 172.18.0.4:45648 to 103.194.116.38:80 (valid subnets = 192.168.65.0/28; valid sources = 0.0.0.0)
Jan 10 14:51:23 ggnf2058s-MacBook-Pro Docker[2668]: dropping unexpected TCP packet sent from 172.18.0.3:36102 to 14.139.62.20:80 (valid subnets = 192.168.65.0/28; valid sources = 0.0.0.0)
Jan 10 14:51:48 ggnf2058s-MacBook-Pro Docker[2668]: dropping unexpected TCP packet sent from 172.18.0.4:45648 to 103.194.116.38:80 (valid subnets = 192.168.65.0/28; valid sources = 0.0.0.0)

I'm aware docker has issues working on Mac but I think this one isn't one of them. I feel there's something amiss with what I'm doing.

Can you please help?

run.sh from-source fails because the ssh service hasn't started in the docker container

Running this script (caochong/from-source/run.sh) gives this error:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [caochong-master]
caochong-master: ssh: connect to host caochong-master port 22: Connection refused
localhost: ssh: connect to host localhost port 22: Connection refused
Starting secondary namenodes [0.0.0.0]
0.0.0.0: ssh: connect to host 0.0.0.0 port 22: Connection refused
17/01/16 06:51:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
chown: missing operand after '/hadoop/logs'
Try 'chown --help' for more information.
starting resourcemanager, logging to /hadoop/logs/yarn--resourcemanager-fbad3ebe7cdf.out
localhost: ssh: connect to host localhost port 22: Connection refused
starting resourcemanager, logging to /hadoop/logs/yarn--resourcemanager-ed26fb8707b6.out

The relevant part is

localhost: ssh: connect to host localhost port 22: Connection refused

The reason is that the ssh service hasn't been started in the docker container.

The simplest fix for this would be to add a line to start ssh in the container,i.e add this line:

# Start hdfs and yarn services
docker exec -it $master_id service ssh start
docker exec -it $master_id $HADOOP_HOME/sbin/start-dfs.sh
docker exec -it $master_id $HADOOP_HOME/sbin/start-yarn.sh

I have tried this solution and it works well. Am I missing something, or should this line be added to the script?

upgrade repo for Ambari 2.4.2

When I changed the repo to http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0/hdp.repo I got the following error

saptak$ ./run.sh
Sending build context to Docker daemon 9.728 kB
Step 1 : FROM centos:6
6: Pulling from library/centos
f402e43879a8: Pull complete
Digest: sha256:f3cc24b376d42d9fd9fa914335dddee46ed7b5a2e2dd1ee6df045ce437199d79
Status: Downloaded newer image for centos:6
 ---> f07f6ca555a5
Step 2 : MAINTAINER Mingliang Liu <[email protected]>
 ---> Running in 1ea4e8799ceb
 ---> 75cde86e0b4f
Removing intermediate container 1ea4e8799ceb
Step 3 : RUN yum install -y epel-release
 ---> Running in 5fe54e8d41ef
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:6-8 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                Arch             Version         Repository        Size
================================================================================
Installing:
 epel-release           noarch           6-8             extras            14 k

Transaction Summary
================================================================================
Install       1 Package(s)

Total download size: 14 k
Installed size: 22 k
Downloading Packages:
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Importing GPG key 0xC105B9DE:
 Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <[email protected]>
 Package: centos-release-6-8.el6.centos.12.3.x86_64 (@CentOS/6.8)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : epel-release-6-8.noarch                                      1/1
  Verifying  : epel-release-6-8.noarch                                      1/1

Installed:
  epel-release.noarch 0:6-8

Complete!
 ---> 9a0961b73380
Removing intermediate container 5fe54e8d41ef
Step 4 : RUN yum -y update
 ---> Running in 5f767c1c44e8
Loaded plugins: fastestmirror, ovl
Setting up Update Process
Determining fastest mirrors
 * base: centos.eecs.wsu.edu
 * epel: mirrors.cat.pdx.edu
 * extras: mirror.5ninesolutions.com
 * updates: repos.lax.quadranet.com
Resolving Dependencies
--> Running transaction check
---> Package bind-libs.x86_64 32:9.8.2-0.47.rc1.el6 will be updated
---> Package bind-libs.x86_64 32:9.8.2-0.47.rc1.el6_8.1 will be an update
---> Package bind-utils.x86_64 32:9.8.2-0.47.rc1.el6 will be updated
---> Package bind-utils.x86_64 32:9.8.2-0.47.rc1.el6_8.1 will be an update
---> Package openssl.x86_64 0:1.0.1e-48.el6_8.1 will be updated
---> Package openssl.x86_64 0:1.0.1e-48.el6_8.3 will be an update
---> Package tzdata.noarch 0:2016f-1.el6 will be updated
---> Package tzdata.noarch 0:2016g-2.el6 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package         Arch        Version                         Repository    Size
================================================================================
Updating:
 bind-libs       x86_64      32:9.8.2-0.47.rc1.el6_8.1       updates      889 k
 bind-utils      x86_64      32:9.8.2-0.47.rc1.el6_8.1       updates      187 k
 openssl         x86_64      1.0.1e-48.el6_8.3               updates      1.5 M
 tzdata          noarch      2016g-2.el6                     updates      452 k

Transaction Summary
================================================================================
Upgrade       4 Package(s)

Total download size: 3.0 M
Downloading Packages:
--------------------------------------------------------------------------------
Total                                           787 kB/s | 3.0 MB     00:03
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating   : openssl-1.0.1e-48.el6_8.3.x86_64                             1/8
  Updating   : 32:bind-libs-9.8.2-0.47.rc1.el6_8.1.x86_64                   2/8
  Updating   : 32:bind-utils-9.8.2-0.47.rc1.el6_8.1.x86_64                  3/8
  Updating   : tzdata-2016g-2.el6.noarch                                    4/8
  Cleanup    : 32:bind-utils-9.8.2-0.47.rc1.el6.x86_64                      5/8
  Cleanup    : tzdata-2016f-1.el6.noarch                                    6/8
  Cleanup    : 32:bind-libs-9.8.2-0.47.rc1.el6.x86_64                       7/8
  Cleanup    : openssl-1.0.1e-48.el6_8.1.x86_64                             8/8
  Verifying  : 32:bind-utils-9.8.2-0.47.rc1.el6_8.1.x86_64                  1/8
  Verifying  : 32:bind-libs-9.8.2-0.47.rc1.el6_8.1.x86_64                   2/8
  Verifying  : tzdata-2016g-2.el6.noarch                                    3/8
  Verifying  : openssl-1.0.1e-48.el6_8.3.x86_64                             4/8
  Verifying  : 32:bind-libs-9.8.2-0.47.rc1.el6.x86_64                       5/8
  Verifying  : 32:bind-utils-9.8.2-0.47.rc1.el6.x86_64                      6/8
  Verifying  : tzdata-2016f-1.el6.noarch                                    7/8
  Verifying  : openssl-1.0.1e-48.el6_8.1.x86_64                             8/8

Updated:
  bind-libs.x86_64 32:9.8.2-0.47.rc1.el6_8.1
  bind-utils.x86_64 32:9.8.2-0.47.rc1.el6_8.1
  openssl.x86_64 0:1.0.1e-48.el6_8.3
  tzdata.noarch 0:2016g-2.el6

Complete!
 ---> 111bc959f0eb
Removing intermediate container 5f767c1c44e8
Step 5 : RUN yum install -y wget ntp sudo
 ---> Running in 45db9f7ca001
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: centos.eecs.wsu.edu
 * epel: mirrors.cat.pdx.edu
 * extras: mirror.5ninesolutions.com
 * updates: repos.lax.quadranet.com
Resolving Dependencies
--> Running transaction check
---> Package ntp.x86_64 0:4.2.6p5-10.el6.centos.1 will be installed
--> Processing Dependency: ntpdate = 4.2.6p5-10.el6.centos.1 for package: ntp-4.2.6p5-10.el6.centos.1.x86_64
--> Processing Dependency: /sbin/service for package: ntp-4.2.6p5-10.el6.centos.1.x86_64
--> Processing Dependency: libedit.so.0()(64bit) for package: ntp-4.2.6p5-10.el6.centos.1.x86_64
---> Package sudo.x86_64 0:1.8.6p3-24.el6 will be installed
---> Package wget.x86_64 0:1.12-8.el6 will be installed
--> Running transaction check
---> Package initscripts.x86_64 0:9.03.53-1.el6.centos.1 will be installed
--> Processing Dependency: util-linux-ng >= 2.16 for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: udev >= 125-1 for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: sysvinit-tools >= 2.87-6 for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: plymouth for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: /sbin/pidof for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: /sbin/ip for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: /sbin/blkid for package: initscripts-9.03.53-1.el6.centos.1.x86_64
--> Processing Dependency: /sbin/arping for package: initscripts-9.03.53-1.el6.centos.1.x86_64
---> Package libedit.x86_64 0:2.11-4.20080712cvs.1.el6 will be installed
---> Package ntpdate.x86_64 0:4.2.6p5-10.el6.centos.1 will be installed
--> Running transaction check
---> Package iproute.x86_64 0:2.6.32-54.el6 will be installed
--> Processing Dependency: iptables >= 1.4.5 for package: iproute-2.6.32-54.el6.x86_64
--> Processing Dependency: libxtables.so.4()(64bit) for package: iproute-2.6.32-54.el6.x86_64
---> Package iputils.x86_64 0:20071127-21.el6 will be installed
---> Package plymouth.x86_64 0:0.8.3-27.el6.centos.1 will be installed
--> Processing Dependency: system-logos for package: plymouth-0.8.3-27.el6.centos.1.x86_64
--> Processing Dependency: libdrm_radeon.so.1()(64bit) for package: plymouth-0.8.3-27.el6.centos.1.x86_64
--> Processing Dependency: libdrm_nouveau.so.1()(64bit) for package: plymouth-0.8.3-27.el6.centos.1.x86_64
--> Processing Dependency: libdrm_intel.so.1()(64bit) for package: plymouth-0.8.3-27.el6.centos.1.x86_64
--> Processing Dependency: libdrm.so.2()(64bit) for package: plymouth-0.8.3-27.el6.centos.1.x86_64
---> Package sysvinit-tools.x86_64 0:2.87-6.dsf.el6 will be installed
---> Package udev.x86_64 0:147-2.73.el6_8.2 will be installed
--> Processing Dependency: hwdata for package: udev-147-2.73.el6_8.2.x86_64
---> Package util-linux-ng.x86_64 0:2.17.2-12.24.el6 will be installed
--> Running transaction check
---> Package hwdata.noarch 0:0.233-16.1.el6 will be installed
---> Package iptables.x86_64 0:1.4.7-16.el6 will be installed
--> Processing Dependency: policycoreutils for package: iptables-1.4.7-16.el6.x86_64
---> Package libdrm.x86_64 0:2.4.65-2.el6 will be installed
--> Processing Dependency: libpciaccess.so.0()(64bit) for package: libdrm-2.4.65-2.el6.x86_64
---> Package redhat-logos.noarch 0:60.0.14-12.el6.centos will be installed
--> Running transaction check
---> Package libpciaccess.x86_64 0:0.13.4-1.el6 will be installed
---> Package policycoreutils.x86_64 0:2.0.83-29.el6 will be installed
--> Processing Dependency: libdbus-glib-1.so.2()(64bit) for package: policycoreutils-2.0.83-29.el6.x86_64
--> Running transaction check
---> Package dbus-glib.x86_64 0:0.86-6.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package             Arch       Version                       Repository   Size
================================================================================
Installing:
 ntp                 x86_64     4.2.6p5-10.el6.centos.1       updates     598 k
 sudo                x86_64     1.8.6p3-24.el6                base        710 k
 wget                x86_64     1.12-8.el6                    base        484 k
Installing for dependencies:
 dbus-glib           x86_64     0.86-6.el6                    base        170 k
 hwdata              noarch     0.233-16.1.el6                base        1.3 M
 initscripts         x86_64     9.03.53-1.el6.centos.1        updates     947 k
 iproute             x86_64     2.6.32-54.el6                 base        389 k
 iptables            x86_64     1.4.7-16.el6                  base        254 k
 iputils             x86_64     20071127-21.el6               base        120 k
 libdrm              x86_64     2.4.65-2.el6                  base        136 k
 libedit             x86_64     2.11-4.20080712cvs.1.el6      base         74 k
 libpciaccess        x86_64     0.13.4-1.el6                  base         24 k
 ntpdate             x86_64     4.2.6p5-10.el6.centos.1       updates      78 k
 plymouth            x86_64     0.8.3-27.el6.centos.1         base         89 k
 policycoreutils     x86_64     2.0.83-29.el6                 base        663 k
 redhat-logos        noarch     60.0.14-12.el6.centos         base         15 M
 sysvinit-tools      x86_64     2.87-6.dsf.el6                base         60 k
 udev                x86_64     147-2.73.el6_8.2              updates     358 k
 util-linux-ng       x86_64     2.17.2-12.24.el6              base        1.6 M

Transaction Summary
================================================================================
Install      19 Package(s)

Total download size: 22 M
Installed size: 45 M
Downloading Packages:
--------------------------------------------------------------------------------
Total                                           3.0 MB/s |  22 MB     00:07
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : hwdata-0.233-16.1.el6.noarch                                1/19
  Installing : sysvinit-tools-2.87-6.dsf.el6.x86_64                        2/19
  Installing : libpciaccess-0.13.4-1.el6.x86_64                            3/19
  Installing : dbus-glib-0.86-6.el6.x86_64                                 4/19
  Installing : redhat-logos-60.0.14-12.el6.centos.noarch                   5/19
  Installing : policycoreutils-2.0.83-29.el6.x86_64                        6/19
  Installing : iptables-1.4.7-16.el6.x86_64                                7/19
  Installing : iproute-2.6.32-54.el6.x86_64                                8/19
  Installing : libdrm-2.4.65-2.el6.x86_64                                  9/19
  Installing : plymouth-0.8.3-27.el6.centos.1.x86_64                      10/19
  Installing : iputils-20071127-21.el6.x86_64                             11/19
  Installing : util-linux-ng-2.17.2-12.24.el6.x86_64                      12/19
install-info: No such file or directory for /usr/share/info/ipc.info
  Installing : initscripts-9.03.53-1.el6.centos.1.x86_64                  13/19
  Installing : udev-147-2.73.el6_8.2.x86_64                               14/19
  Installing : ntpdate-4.2.6p5-10.el6.centos.1.x86_64                     15/19
  Installing : libedit-2.11-4.20080712cvs.1.el6.x86_64                    16/19
  Installing : ntp-4.2.6p5-10.el6.centos.1.x86_64                         17/19
  Installing : wget-1.12-8.el6.x86_64                                     18/19
install-info: No such file or directory for /usr/share/info/wget.info.gz
  Installing : sudo-1.8.6p3-24.el6.x86_64                                 19/19
  Verifying  : sudo-1.8.6p3-24.el6.x86_64                                  1/19
  Verifying  : iptables-1.4.7-16.el6.x86_64                                2/19
  Verifying  : plymouth-0.8.3-27.el6.centos.1.x86_64                       3/19
  Verifying  : libedit-2.11-4.20080712cvs.1.el6.x86_64                     4/19
  Verifying  : ntp-4.2.6p5-10.el6.centos.1.x86_64                          5/19
  Verifying  : redhat-logos-60.0.14-12.el6.centos.noarch                   6/19
  Verifying  : udev-147-2.73.el6_8.2.x86_64                                7/19
  Verifying  : iproute-2.6.32-54.el6.x86_64                                8/19
  Verifying  : iputils-20071127-21.el6.x86_64                              9/19
  Verifying  : libpciaccess-0.13.4-1.el6.x86_64                           10/19
  Verifying  : initscripts-9.03.53-1.el6.centos.1.x86_64                  11/19
  Verifying  : sysvinit-tools-2.87-6.dsf.el6.x86_64                       12/19
  Verifying  : util-linux-ng-2.17.2-12.24.el6.x86_64                      13/19
  Verifying  : policycoreutils-2.0.83-29.el6.x86_64                       14/19
  Verifying  : wget-1.12-8.el6.x86_64                                     15/19
  Verifying  : ntpdate-4.2.6p5-10.el6.centos.1.x86_64                     16/19
  Verifying  : dbus-glib-0.86-6.el6.x86_64                                17/19
  Verifying  : libdrm-2.4.65-2.el6.x86_64                                 18/19
  Verifying  : hwdata-0.233-16.1.el6.noarch                               19/19

Installed:
  ntp.x86_64 0:4.2.6p5-10.el6.centos.1       sudo.x86_64 0:1.8.6p3-24.el6
  wget.x86_64 0:1.12-8.el6

Dependency Installed:
  dbus-glib.x86_64 0:0.86-6.el6
  hwdata.noarch 0:0.233-16.1.el6
  initscripts.x86_64 0:9.03.53-1.el6.centos.1
  iproute.x86_64 0:2.6.32-54.el6
  iptables.x86_64 0:1.4.7-16.el6
  iputils.x86_64 0:20071127-21.el6
  libdrm.x86_64 0:2.4.65-2.el6
  libedit.x86_64 0:2.11-4.20080712cvs.1.el6
  libpciaccess.x86_64 0:0.13.4-1.el6
  ntpdate.x86_64 0:4.2.6p5-10.el6.centos.1
  plymouth.x86_64 0:0.8.3-27.el6.centos.1
  policycoreutils.x86_64 0:2.0.83-29.el6
  redhat-logos.noarch 0:60.0.14-12.el6.centos
  sysvinit-tools.x86_64 0:2.87-6.dsf.el6
  udev.x86_64 0:147-2.73.el6_8.2
  util-linux-ng.x86_64 0:2.17.2-12.24.el6

Complete!
 ---> e6aa2b8e3301
Removing intermediate container 45db9f7ca001
Step 6 : RUN yum install -y which openssh-clients openssh-server
 ---> Running in fac30fe7d9ba
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: centos.eecs.wsu.edu
 * epel: mirrors.cat.pdx.edu
 * extras: mirror.5ninesolutions.com
 * updates: repos.lax.quadranet.com
Package which-2.19-6.el6.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package openssh-clients.x86_64 0:5.3p1-118.1.el6_8 will be installed
--> Processing Dependency: openssh = 5.3p1-118.1.el6_8 for package: openssh-clients-5.3p1-118.1.el6_8.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-clients-5.3p1-118.1.el6_8.x86_64
---> Package openssh-server.x86_64 0:5.3p1-118.1.el6_8 will be installed
--> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-5.3p1-118.1.el6_8.x86_64
--> Running transaction check
---> Package fipscheck-lib.x86_64 0:1.2.0-7.el6 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.2.0-7.el6.x86_64
---> Package openssh.x86_64 0:5.3p1-118.1.el6_8 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-58.el6 will be installed
--> Running transaction check
---> Package fipscheck.x86_64 0:1.2.0-7.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                Arch        Version                  Repository    Size
================================================================================
Installing:
 openssh-clients        x86_64      5.3p1-118.1.el6_8        updates      442 k
 openssh-server         x86_64      5.3p1-118.1.el6_8        updates      328 k
Installing for dependencies:
 fipscheck              x86_64      1.2.0-7.el6              base          14 k
 fipscheck-lib          x86_64      1.2.0-7.el6              base         8.3 k
 openssh                x86_64      5.3p1-118.1.el6_8        updates      276 k
 tcp_wrappers-libs      x86_64      7.6-58.el6               base          62 k

Transaction Summary
================================================================================
Install       6 Package(s)

Total download size: 1.1 M
Installed size: 2.9 M
Downloading Packages:
--------------------------------------------------------------------------------
Total                                           418 kB/s | 1.1 MB     00:02
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : fipscheck-lib-1.2.0-7.el6.x86_64                             1/6
  Installing : fipscheck-1.2.0-7.el6.x86_64                                 2/6
  Installing : openssh-5.3p1-118.1.el6_8.x86_64                             3/6
  Installing : tcp_wrappers-libs-7.6-58.el6.x86_64                          4/6
  Installing : openssh-server-5.3p1-118.1.el6_8.x86_64                      5/6
  Installing : openssh-clients-5.3p1-118.1.el6_8.x86_64                     6/6
  Verifying  : tcp_wrappers-libs-7.6-58.el6.x86_64                          1/6
  Verifying  : fipscheck-1.2.0-7.el6.x86_64                                 2/6
  Verifying  : openssh-5.3p1-118.1.el6_8.x86_64                             3/6
  Verifying  : openssh-server-5.3p1-118.1.el6_8.x86_64                      4/6
  Verifying  : openssh-clients-5.3p1-118.1.el6_8.x86_64                     5/6
  Verifying  : fipscheck-lib-1.2.0-7.el6.x86_64                             6/6

Installed:
  openssh-clients.x86_64 0:5.3p1-118.1.el6_8
  openssh-server.x86_64 0:5.3p1-118.1.el6_8

Dependency Installed:
  fipscheck.x86_64 0:1.2.0-7.el6        fipscheck-lib.x86_64 0:1.2.0-7.el6
  openssh.x86_64 0:5.3p1-118.1.el6_8    tcp_wrappers-libs.x86_64 0:7.6-58.el6

Complete!
 ---> 138e6136c7b7
Removing intermediate container fac30fe7d9ba
Step 7 : RUN echo 'root:hortonworks' | chpasswd
 ---> Running in 94e01620c813
 ---> 76253858f470
Removing intermediate container 94e01620c813
Step 8 : RUN ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' &&     cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
 ---> Running in d7cb817f4fd0
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
36:82:79:4d:ea:c9:8f:91:3f:fc:66:a9:20:2e:6d:8e root@4f3bd15550ac
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|        .        |
|     o +         |
|    o + S        |
|     + = .       |
|   .. B.   .     |
|  .oo. *o +      |
|  E+o . +*.      |
+-----------------+
 ---> d15b99506522
Removing intermediate container d7cb817f4fd0
Step 9 : RUN sed -i '/pam_loginuid.so/c session    optional     pam_loginuid.so'  /etc/pam.d/sshd
 ---> Running in b627cb76a389
 ---> b8bd1b2d8357
Removing intermediate container b627cb76a389
Step 10 : RUN echo -e "Host *\n StrictHostKeyChecking no" >> /etc/ssh/ssh_config
 ---> Running in ca655a4b6c1a
 ---> be4e1be88190
Removing intermediate container ca655a4b6c1a
Step 11 : RUN yum -y install supervisor
 ---> Running in 6ea6ef3b151b
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: centos.eecs.wsu.edu
 * epel: mirrors.cat.pdx.edu
 * extras: mirror.5ninesolutions.com
 * updates: repos.lax.quadranet.com
Resolving Dependencies
--> Running transaction check
---> Package supervisor.noarch 0:2.1-9.el6 will be installed
--> Processing Dependency: python-meld3 for package: supervisor-2.1-9.el6.noarch
--> Running transaction check
---> Package python-meld3.x86_64 0:0.6.7-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package               Arch            Version              Repository     Size
================================================================================
Installing:
 supervisor            noarch          2.1-9.el6            epel          292 k
Installing for dependencies:
 python-meld3          x86_64          0.6.7-1.el6          epel           71 k

Transaction Summary
================================================================================
Install       2 Package(s)

Total download size: 364 k
Installed size: 1.4 M
Downloading Packages:
--------------------------------------------------------------------------------
Total                                           876 kB/s | 364 kB     00:00
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Importing GPG key 0x0608B895:
 Userid : EPEL (6) <[email protected]>
 Package: epel-release-6-8.noarch (@extras)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : python-meld3-0.6.7-1.el6.x86_64                              1/2
  Installing : supervisor-2.1-9.el6.noarch                                  2/2
  Verifying  : python-meld3-0.6.7-1.el6.x86_64                              1/2
  Verifying  : supervisor-2.1-9.el6.noarch                                  2/2

Installed:
  supervisor.noarch 0:2.1-9.el6

Dependency Installed:
  python-meld3.x86_64 0:0.6.7-1.el6

Complete!
 ---> 581cb336f4ee
Removing intermediate container 6ea6ef3b151b
Step 12 : RUN mkdir /etc/supervisor.d/
 ---> Running in 73a582ac6059
 ---> b97923db908d
Removing intermediate container 73a582ac6059
Step 13 : RUN echo -e "[program:sshd]\ncommand=/sbin/service sshd start" >> /etc/supervisord.conf
 ---> Running in 8b063c002ced
 ---> f5339c19ae7c
Removing intermediate container 8b063c002ced
Step 14 : RUN echo -e "[program:ntpd]\ncommand=/sbin/service ntpd start" >> /etc/supervisord.conf
 ---> Running in a9fa6ab99b78
 ---> aacacefa8916
Removing intermediate container a9fa6ab99b78
Step 15 : RUN wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0/hdp.repo
 ---> Running in 4b0766c0ff34
--2016-10-03 16:41:15--  http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0/hdp.repo
Resolving public-repo-1.hortonworks.com... 52.84.16.148, 52.84.16.114, 52.84.16.45, ...
Connecting to public-repo-1.hortonworks.com|52.84.16.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 575 [binary/octet-stream]
Saving to: `/etc/yum.repos.d/ambari.repo'

     0K                                                       100% 64.6M=0s

2016-10-03 16:41:15 (64.6 MB/s) - `/etc/yum.repos.d/ambari.repo' saved [575/575]

 ---> dd8bbc626366
Removing intermediate container 4b0766c0ff34
Step 16 : RUN yum install ambari-server -y
 ---> Running in fd381f3ffa67
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: centos.eecs.wsu.edu
 * epel: mirrors.cat.pdx.edu
 * extras: mirror.5ninesolutions.com
 * updates: repos.lax.quadranet.com
No package ambari-server available.
Error: Nothing to do
The command '/bin/sh -c yum install ambari-server -y' returned a non-zero code: 1
476c16ed12cfe011b282a1629101d2712c02cef0d26d22101f7a6b8364c0d6c5
"docker rm" requires at least 1 argument(s).
See 'docker rm --help'.

Usage:  docker rm [OPTIONS] CONTAINER [CONTAINER...]

Remove one or more containers
Unable to find image 'caochong-ambari:latest' locally
Pulling repository docker.io/library/caochong-ambari
docker: Error: image library/caochong-ambari:latest not found.
See 'docker run --help'.
Unable to find image 'caochong-ambari:latest' locally
Pulling repository docker.io/library/caochong-ambari
docker: Error: image library/caochong-ambari:latest not found.
See 'docker run --help'.
Unable to find image 'caochong-ambari:latest' locally
Pulling repository docker.io/library/caochong-ambari
docker: Error: image library/caochong-ambari:latest not found.
See 'docker run --help'.
must specify at least one container source
Using the following hostnames:
------------------------------



------------------------------
Copying back the private key...
must specify at least one container source
Error response from daemon: No such container: caochong-ambari-0

Doesn't install (see log - REASON: Unable to start PostgreSQL server.)

Sending build context to Docker daemon 7.68 kB
Step 1 : FROM centos:6
---> 30365b2e827c
Step 2 : MAINTAINER Mingliang Liu [email protected]
---> Running in f4236d506eaf
---> 260bc73a722b
Removing intermediate container f4236d506eaf
Step 3 : RUN yum install -y epel-release
---> Running in e50bba792d61
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:6-8 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size

Installing:
epel-release noarch 6-8 extras 14 k

Transaction Summary

Install 1 Package(s)

Total download size: 14 k
Installed size: 22 k
Downloading Packages:
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Importing GPG key 0xC105B9DE:
Userid : CentOS-6 Key (CentOS 6 Official Signing Key) [email protected]
Package: centos-release-6-9.el6.12.3.x86_64 (@CentOS/6.9)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : epel-release-6-8.noarch 1/1
Verifying : epel-release-6-8.noarch 1/1

Installed:
epel-release.noarch 0:6-8

Complete!
---> 9b6df53f91f2
Removing intermediate container e50bba792d61
Step 4 : RUN yum -y update
---> Running in 058cc10edf64
Loaded plugins: fastestmirror, ovl
Setting up Update Process
Determining fastest mirrors

base: mirror.mirohost.net
epel: ftp.icm.edu.pl
extras: mirror.mirohost.net
updates: mirror.mirohost.net
Resolving Dependencies
--> Running transaction check
---> Package binutils.x86_64 0:2.20.51.0.2-5.46.el6 will be updated
---> Package binutils.x86_64 0:2.20.51.0.2-5.47.el6_9.1 will be an update
---> Package glibc.x86_64 0:2.12-1.209.el6 will be updated
---> Package glibc.x86_64 0:2.12-1.209.el6_9.1 will be an update
---> Package glibc-common.x86_64 0:2.12-1.209.el6 will be updated
---> Package glibc-common.x86_64 0:2.12-1.209.el6_9.1 will be an update
---> Package nss.x86_64 0:3.27.1-13.el6 will be updated
---> Package nss.x86_64 0:3.28.3-3.el6_9 will be an update
---> Package nss-sysinit.x86_64 0:3.27.1-13.el6 will be updated
---> Package nss-sysinit.x86_64 0:3.28.3-3.el6_9 will be an update
---> Package nss-tools.x86_64 0:3.27.1-13.el6 will be updated
---> Package nss-tools.x86_64 0:3.28.3-3.el6_9 will be an update
---> Package nss-util.x86_64 0:3.27.1-3.el6 will be updated
---> Package nss-util.x86_64 0:3.28.3-1.el6_9 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size

Updating:
binutils x86_64 2.20.51.0.2-5.47.el6_9.1 updates 2.8 M
glibc x86_64 2.12-1.209.el6_9.1 updates 3.8 M
glibc-common x86_64 2.12-1.209.el6_9.1 updates 14 M
nss x86_64 3.28.3-3.el6_9 updates 880 k
nss-sysinit x86_64 3.28.3-3.el6_9 updates 51 k
nss-tools x86_64 3.28.3-3.el6_9 updates 446 k
nss-util x86_64 3.28.3-1.el6_9 updates 68 k

Transaction Summary

Upgrade 7 Package(s)

Total download size: 22 M
Downloading Packages:

Total Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : glibc-common-2.12-1.209.el6_9.1.x86_64 Updating : glibc-2.12-1.209.el6_9.1.x86_64 Updating : nss-util-3.28.3-1.el6_9.x86_64 Updating : nss-3.28.3-3.el6_9.x86_64 Updating : nss-sysinit-3.28.3-3.el6_9.x86_64 Updating : nss-tools-3.28.3-3.el6_9.x86_64 Updating : binutils-2.20.51.0.2-5.47.el6_9.1.x86_64 Cleanup : nss-tools-3.27.1-13.el6.x86_64 Cleanup : nss-3.27.1-13.el6.x86_64 Cleanup : nss-sysinit-3.27.1-13.el6.x86_64 Cleanup : nss-util-3.27.1-3.el6.x86_64 Cleanup : binutils-2.20.51.0.2-5.46.el6.x86_64 Cleanup : glibc-common-2.12-1.209.el6.x86_64 Cleanup : glibc-2.12-1.209.el6.x86_64 Verifying : binutils-2.20.51.0.2-5.47.el6_9.1.x86_64 Verifying : nss-sysinit-3.28.3-3.el6_9.x86_64 Verifying : nss-util-3.28.3-1.el6_9.x86_64 Verifying : nss-3.28.3-3.el6_9.x86_64 Verifying : glibc-2.12-1.209.el6_9.1.x86_64 Verifying : glibc-common-2.12-1.209.el6_9.1.x86_64 Verifying : nss-tools-3.28.3-3.el6_9.x86_64 Verifying : nss-3.27.1-13.el6.x86_64 Verifying : nss-util-3.27.1-3.el6.x86_64 Verifying : binutils-2.20.51.0.2-5.46.el6.x86_64 Verifying : nss-sysinit-3.27.1-13.el6.x86_64 Verifying : glibc-2.12-1.209.el6.x86_64 Verifying : glibc-common-2.12-1.209.el6.x86_64 Verifying : nss-tools-3.27.1-13.el6.x86_64 2.2 MB/s | 22 MB 00:10
1/14
2/14
3/14
4/14
5/14
6/14
7/14
8/14
9/14
10/14
11/14
12/14
13/14
14/14
1/14
2/14
3/14
4/14
5/14
6/14
7/14
8/14
9/14
10/14
11/14
12/14
13/14
14/14

Updated:
binutils.x86_64 0:2.20.51.0.2-5.47.el6_9.1 glibc.x86_64 0:2.12-1.209.el6_9.1
glibc-common.x86_64 0:2.12-1.209.el6_9.1 nss.x86_64 0:3.28.3-3.el6_9
nss-sysinit.x86_64 0:3.28.3-3.el6_9 nss-tools.x86_64 0:3.28.3-3.el6_9
nss-util.x86_64 0:3.28.3-1.el6_9

Complete!
---> 2fa070b312a7
Removing intermediate container 058cc10edf64
Step 5 : RUN yum install -y wget ntp sudo
---> Running in 319bccc4eaf4
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile

base: mirror.mirohost.net
epel: ftp.icm.edu.pl
extras: mirror.mirohost.net
updates: mirror.mirohost.net
Resolving Dependencies
--> Running transaction check
---> Package ntp.x86_64 0:4.2.6p5-10.el6.centos.2 will be installed
--> Processing Dependency: ntpdate = 4.2.6p5-10.el6.centos.2 for package: ntp-4.2.6p5-10.el6.centos.2.x86_64
--> Processing Dependency: /sbin/service for package: ntp-4.2.6p5-10.el6.centos.2.x86_64
--> Processing Dependency: libedit.so.0()(64bit) for package: ntp-4.2.6p5-10.el6.centos.2.x86_64
---> Package sudo.x86_64 0:1.8.6p3-27.el6 will be installed
---> Package wget.x86_64 0:1.12-10.el6 will be installed
--> Running transaction check
---> Package initscripts.x86_64 0:9.03.58-1.el6.centos will be installed
--> Processing Dependency: util-linux-ng >= 2.16 for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: udev >= 125-1 for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: sysvinit-tools >= 2.87-6 for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: plymouth for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: /sbin/pidof for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: /sbin/ip for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: /sbin/blkid for package: initscripts-9.03.58-1.el6.centos.x86_64
--> Processing Dependency: /sbin/arping for package: initscripts-9.03.58-1.el6.centos.x86_64
---> Package libedit.x86_64 0:2.11-4.20080712cvs.1.el6 will be installed
---> Package ntpdate.x86_64 0:4.2.6p5-10.el6.centos.2 will be installed
--> Running transaction check
---> Package iproute.x86_64 0:2.6.32-54.el6 will be installed
--> Processing Dependency: iptables >= 1.4.5 for package: iproute-2.6.32-54.el6.x86_64
--> Processing Dependency: libxtables.so.4()(64bit) for package: iproute-2.6.32-54.el6.x86_64
---> Package iputils.x86_64 0:20071127-24.el6 will be installed
---> Package plymouth.x86_64 0:0.8.3-29.el6.centos will be installed
--> Processing Dependency: system-logos for package: plymouth-0.8.3-29.el6.centos.x86_64
--> Processing Dependency: libdrm_radeon.so.1()(64bit) for package: plymouth-0.8.3-29.el6.centos.x86_64
--> Processing Dependency: libdrm_nouveau.so.1()(64bit) for package: plymouth-0.8.3-29.el6.centos.x86_64
--> Processing Dependency: libdrm_intel.so.1()(64bit) for package: plymouth-0.8.3-29.el6.centos.x86_64
--> Processing Dependency: libdrm.so.2()(64bit) for package: plymouth-0.8.3-29.el6.centos.x86_64
---> Package sysvinit-tools.x86_64 0:2.87-6.dsf.el6 will be installed
---> Package udev.x86_64 0:147-2.73.el6_8.2 will be installed
--> Processing Dependency: hwdata for package: udev-147-2.73.el6_8.2.x86_64
---> Package util-linux-ng.x86_64 0:2.17.2-12.28.el6 will be installed
--> Running transaction check
---> Package hwdata.noarch 0:0.233-18.1.el6 will be installed
---> Package iptables.x86_64 0:1.4.7-16.el6 will be installed
--> Processing Dependency: policycoreutils for package: iptables-1.4.7-16.el6.x86_64
---> Package libdrm.x86_64 0:2.4.65-2.el6 will be installed
--> Processing Dependency: libpciaccess.so.0()(64bit) for package: libdrm-2.4.65-2.el6.x86_64
---> Package redhat-logos.noarch 0:60.0.14-12.el6.centos will be installed
--> Running transaction check
---> Package libpciaccess.x86_64 0:0.13.4-1.el6 will be installed
---> Package policycoreutils.x86_64 0:2.0.83-30.1.el6_8 will be installed
--> Processing Dependency: libdbus-glib-1.so.2()(64bit) for package: policycoreutils-2.0.83-30.1.el6_8.x86_64
--> Running transaction check
---> Package dbus-glib.x86_64 0:0.86-6.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository
Size

Installing:
ntp x86_64 4.2.6p5-10.el6.centos.2 base 599 k
sudo x86_64 1.8.6p3-27.el6 base 711 k
wget x86_64 1.12-10.el6 base 484 k
Installing for dependencies:
dbus-glib x86_64 0.86-6.el6 base 170 k
hwdata noarch 0.233-18.1.el6 base 1.3 M
initscripts x86_64 9.03.58-1.el6.centos base 949 k
iproute x86_64 2.6.32-54.el6 base 389 k
iptables x86_64 1.4.7-16.el6 base 254 k
iputils x86_64 20071127-24.el6 base 121 k
libdrm x86_64 2.4.65-2.el6 base 136 k
libedit x86_64 2.11-4.20080712cvs.1.el6 base 74 k
libpciaccess x86_64 0.13.4-1.el6 base 24 k
ntpdate x86_64 4.2.6p5-10.el6.centos.2 base 78 k
plymouth x86_64 0.8.3-29.el6.centos base 89 k
policycoreutils x86_64 2.0.83-30.1.el6_8 base 663 k
redhat-logos noarch 60.0.14-12.el6.centos base 15 M
sysvinit-tools x86_64 2.87-6.dsf.el6 base 60 k
udev x86_64 147-2.73.el6_8.2 base 358 k
util-linux-ng x86_64 2.17.2-12.28.el6 base 1.6 M

Transaction Summary

Install 19 Package(s)

Total download size: 22 M
Installed size: 46 M
Downloading Packages:

Total 2.1 MB/s | 22 MB 00:10
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : hwdata-0.233-18.1.el6.noarch 1/19
Installing : sysvinit-tools-2.87-6.dsf.el6.x86_64 2/19
Installing : libpciaccess-0.13.4-1.el6.x86_64 3/19
Installing : dbus-glib-0.86-6.el6.x86_64 4/19
Installing : redhat-logos-60.0.14-12.el6.centos.noarch 5/19
Installing : policycoreutils-2.0.83-30.1.el6_8.x86_64 6/19
Installing : iptables-1.4.7-16.el6.x86_64 7/19
Installing : iproute-2.6.32-54.el6.x86_64 8/19
Installing : libdrm-2.4.65-2.el6.x86_64 9/19
Installing : iputils-20071127-24.el6.x86_64 10/19
Installing : plymouth-0.8.3-29.el6.centos.x86_64 11/19
Installing : util-linux-ng-2.17.2-12.28.el6.x86_64 12/19
install-info: No such file or directory for /usr/share/info/ipc.info
Installing : initscripts-9.03.58-1.el6.centos.x86_64 13/19
Installing : udev-147-2.73.el6_8.2.x86_64 14/19
Installing : ntpdate-4.2.6p5-10.el6.centos.2.x86_64 15/19
Installing : libedit-2.11-4.20080712cvs.1.el6.x86_64 16/19
Installing : ntp-4.2.6p5-10.el6.centos.2.x86_64 17/19
Installing : sudo-1.8.6p3-27.el6.x86_64 18/19
Installing : wget-1.12-10.el6.x86_64 19/19
install-info: No such file or directory for /usr/share/info/wget.info.gz
Verifying : wget-1.12-10.el6.x86_64 1/19
Verifying : plymouth-0.8.3-29.el6.centos.x86_64 2/19
Verifying : libedit-2.11-4.20080712cvs.1.el6.x86_64 3/19
Verifying : util-linux-ng-2.17.2-12.28.el6.x86_64 4/19
Verifying : redhat-logos-60.0.14-12.el6.centos.noarch 5/19
Verifying : udev-147-2.73.el6_8.2.x86_64 6/19
Verifying : iproute-2.6.32-54.el6.x86_64 7/19
Verifying : libdrm-2.4.65-2.el6.x86_64 8/19
Verifying : policycoreutils-2.0.83-30.1.el6_8.x86_64 9/19
Verifying : ntpdate-4.2.6p5-10.el6.centos.2.x86_64 10/19
Verifying : ntp-4.2.6p5-10.el6.centos.2.x86_64 11/19
Verifying : iptables-1.4.7-16.el6.x86_64 12/19
Verifying : sysvinit-tools-2.87-6.dsf.el6.x86_64 13/19
Verifying : iputils-20071127-24.el6.x86_64 14/19
Verifying : initscripts-9.03.58-1.el6.centos.x86_64 15/19
Verifying : dbus-glib-0.86-6.el6.x86_64 16/19
Verifying : hwdata-0.233-18.1.el6.noarch 17/19
Verifying : libpciaccess-0.13.4-1.el6.x86_64 18/19
Verifying : sudo-1.8.6p3-27.el6.x86_64 19/19

Installed:
ntp.x86_64 0:4.2.6p5-10.el6.centos.2 sudo.x86_64 0:1.8.6p3-27.el6
wget.x86_64 0:1.12-10.el6

Dependency Installed:
dbus-glib.x86_64 0:0.86-6.el6
hwdata.noarch 0:0.233-18.1.el6
initscripts.x86_64 0:9.03.58-1.el6.centos
iproute.x86_64 0:2.6.32-54.el6
iptables.x86_64 0:1.4.7-16.el6
iputils.x86_64 0:20071127-24.el6
libdrm.x86_64 0:2.4.65-2.el6
libedit.x86_64 0:2.11-4.20080712cvs.1.el6
libpciaccess.x86_64 0:0.13.4-1.el6
ntpdate.x86_64 0:4.2.6p5-10.el6.centos.2
plymouth.x86_64 0:0.8.3-29.el6.centos
policycoreutils.x86_64 0:2.0.83-30.1.el6_8
redhat-logos.noarch 0:60.0.14-12.el6.centos
sysvinit-tools.x86_64 0:2.87-6.dsf.el6
udev.x86_64 0:147-2.73.el6_8.2
util-linux-ng.x86_64 0:2.17.2-12.28.el6

Complete!
---> ef2320f3a9b5
Removing intermediate container 319bccc4eaf4
Step 6 : RUN yum install -y which openssh-clients openssh-server
---> Running in 198b8bd5b990
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile

base: mirror.mirohost.net
epel: ftp.icm.edu.pl
extras: mirror.mirohost.net
updates: mirror.mirohost.net
Package which-2.19-6.el6.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package openssh-clients.x86_64 0:5.3p1-122.el6 will be installed
--> Processing Dependency: openssh = 5.3p1-122.el6 for package: openssh-clients-5.3p1-122.el6.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-clients-5.3p1-122.el6.x86_64
---> Package openssh-server.x86_64 0:5.3p1-122.el6 will be installed
--> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-5.3p1-122.el6.x86_64
--> Running transaction check
---> Package fipscheck-lib.x86_64 0:1.2.0-7.el6 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.2.0-7.el6.x86_64
---> Package openssh.x86_64 0:5.3p1-122.el6 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-58.el6 will be installed
--> Running transaction check
---> Package fipscheck.x86_64 0:1.2.0-7.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size

Installing:
openssh-clients x86_64 5.3p1-122.el6 base 443 k
openssh-server x86_64 5.3p1-122.el6 base 329 k
Installing for dependencies:
fipscheck x86_64 1.2.0-7.el6 base 14 k
fipscheck-lib x86_64 1.2.0-7.el6 base 8.3 k
openssh x86_64 5.3p1-122.el6 base 277 k
tcp_wrappers-libs x86_64 7.6-58.el6 base 62 k

Transaction Summary

Install 6 Package(s)

Total download size: 1.1 M
Installed size: 2.9 M
Downloading Packages:

Total 1.5 MB/s | 1.1 MB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : fipscheck-lib-1.2.0-7.el6.x86_64 1/6
Installing : fipscheck-1.2.0-7.el6.x86_64 2/6
Installing : openssh-5.3p1-122.el6.x86_64 3/6
Installing : tcp_wrappers-libs-7.6-58.el6.x86_64 4/6
Installing : openssh-server-5.3p1-122.el6.x86_64 5/6
Installing : openssh-clients-5.3p1-122.el6.x86_64 6/6
Verifying : tcp_wrappers-libs-7.6-58.el6.x86_64 1/6
Verifying : openssh-server-5.3p1-122.el6.x86_64 2/6
Verifying : fipscheck-1.2.0-7.el6.x86_64 3/6
Verifying : openssh-5.3p1-122.el6.x86_64 4/6
Verifying : openssh-clients-5.3p1-122.el6.x86_64 5/6
Verifying : fipscheck-lib-1.2.0-7.el6.x86_64 6/6

Installed:
openssh-clients.x86_64 0:5.3p1-122.el6 openssh-server.x86_64 0:5.3p1-122.el6

Dependency Installed:
fipscheck.x86_64 0:1.2.0-7.el6 fipscheck-lib.x86_64 0:1.2.0-7.el6
openssh.x86_64 0:5.3p1-122.el6 tcp_wrappers-libs.x86_64 0:7.6-58.el6

Complete!
---> 735df5290c76
Removing intermediate container 198b8bd5b990
Step 7 : RUN echo 'root:hortonworks' | chpasswd
---> Running in 893b1976f914
---> 2c9bc6ba670c
Removing intermediate container 893b1976f914
Step 8 : RUN ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' && cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
---> Running in 88606172045f
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
90:56:f2:e2:3a:7b:2c:2c:e0:49:a7:e7:5c:99:02:c6 root@26ef45f4c24d
The key's randomart image is:
+--[ RSA 2048]----+
| . . |
| = |
| = . |
|. o o |
| E . S |
|o... .o |
|o.+oo= |
| +o.=oo |
| o+.o |
+-----------------+
---> 6056c1cd2a03
Removing intermediate container 88606172045f
Step 9 : RUN sed -i '/pam_loginuid.so/c session optional pam_loginuid.so' /etc/pam.d/sshd
---> Running in 3aaef862133f
---> 482a73ec612f
Removing intermediate container 3aaef862133f
Step 10 : RUN echo -e "Host *\n StrictHostKeyChecking no" >> /etc/ssh/ssh_config
---> Running in b4dc4cf46b0b
---> 1d0c355220f0
Removing intermediate container b4dc4cf46b0b
Step 11 : RUN yum -y install supervisor
---> Running in 627592e9a5d9
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile

base: mirror.mirohost.net
epel: ftp.icm.edu.pl
extras: mirror.mirohost.net
updates: mirror.mirohost.net
Resolving Dependencies
--> Running transaction check
---> Package supervisor.noarch 0:2.1-9.el6 will be installed
--> Processing Dependency: python-meld3 for package: supervisor-2.1-9.el6.noarch
--> Running transaction check
---> Package python-meld3.x86_64 0:0.6.7-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size

Installing:
supervisor noarch 2.1-9.el6 epel 292 k
Installing for dependencies:
python-meld3 x86_64 0.6.7-1.el6 epel 71 k

Transaction Summary

Install 2 Package(s)

Total download size: 364 k
Installed size: 1.4 M
Downloading Packages:

Total 920 kB/s | 364 kB 00:00
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Importing GPG key 0x0608B895:
Userid : EPEL (6) [email protected]
Package: epel-release-6-8.noarch (@Extras)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : python-meld3-0.6.7-1.el6.x86_64 1/2
Installing : supervisor-2.1-9.el6.noarch 2/2
Verifying : python-meld3-0.6.7-1.el6.x86_64 1/2
Verifying : supervisor-2.1-9.el6.noarch 2/2

Installed:
supervisor.noarch 0:2.1-9.el6

Dependency Installed:
python-meld3.x86_64 0:0.6.7-1.el6

Complete!
---> 345047fa3c74
Removing intermediate container 627592e9a5d9
Step 12 : RUN mkdir /etc/supervisor.d/
---> Running in 1773f2c9538b
---> d54708ce5b6a
Removing intermediate container 1773f2c9538b
Step 13 : RUN echo -e "[program:sshd]\ncommand=/sbin/service sshd start" >> /etc/supervisord.conf
---> Running in 284fb87718a7
---> 6183f0057ec2
Removing intermediate container 284fb87718a7
Step 14 : RUN echo -e "[program:ntpd]\ncommand=/sbin/service ntpd start" >> /etc/supervisord.conf
---> Running in 3b0d4f5ec805
---> e838c6190758
Removing intermediate container 3b0d4f5ec805
Step 15 : RUN wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/ambari.repo
---> Running in 0446acbd23d8
--2017-04-14 08:08:33-- http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/ambari.repo
Resolving public-repo-1.hortonworks.com... 54.192.229.78, 54.192.229.18, 54.192.229.20, ...
Connecting to public-repo-1.hortonworks.com|54.192.229.78|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 287 [binary/octet-stream]
Saving to: `/etc/yum.repos.d/ambari.repo'

 0K                                                       100% 27.1M=0s

2017-04-14 08:08:33 (27.1 MB/s) - `/etc/yum.repos.d/ambari.repo' saved [287/287]

---> 10f0e36e684f
Removing intermediate container 0446acbd23d8
Step 16 : RUN yum install ambari-server -y
---> Running in 12b3346676c6
Loaded plugins: fastestmirror, ovl
Setting up Install Process
Loading mirror speeds from cached hostfile

base: mirror.mirohost.net
epel: ftp.icm.edu.pl
extras: mirror.mirohost.net
updates: mirror.mirohost.net
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.x86_64 0:2.2.2.0-460 will be installed
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-2.2.2.0-460.x86_64
--> Running transaction check
---> Package postgresql-server.x86_64 0:8.4.20-7.el6 will be installed
--> Processing Dependency: postgresql-libs(x86-64) = 8.4.20-7.el6 for package: postgresql-server-8.4.20-7.el6.x86_64
--> Processing Dependency: postgresql(x86-64) = 8.4.20-7.el6 for package: postgresql-server-8.4.20-7.el6.x86_64
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.20-7.el6.x86_64
--> Running transaction check
---> Package postgresql.x86_64 0:8.4.20-7.el6 will be installed
---> Package postgresql-libs.x86_64 0:8.4.20-7.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size

Installing:
ambari-server x86_64 2.2.2.0-460 Updates-ambari-2.2.2.0 409 M
Installing for dependencies:
postgresql x86_64 8.4.20-7.el6 base 2.6 M
postgresql-libs x86_64 8.4.20-7.el6 base 202 k
postgresql-server x86_64 8.4.20-7.el6 base 3.4 M

Transaction Summary

Install 4 Package(s)

Total download size: 415 M
Installed size: 473 M
Downloading Packages:

Total 2.2 MB/s | 415 MB 03:07
warning: rpmts_HdrFromFdno: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY
Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Importing GPG key 0x07513CAD:
Userid: "Jenkins (HDP Builds) [email protected]"
From : http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : postgresql-libs-8.4.20-7.el6.x86_64 1/4
Installing : postgresql-8.4.20-7.el6.x86_64 2/4
Installing : postgresql-server-8.4.20-7.el6.x86_64 3/4
Installing : ambari-server-2.2.2.0-460.x86_64 4/4
Verifying : postgresql-8.4.20-7.el6.x86_64 1/4
Verifying : ambari-server-2.2.2.0-460.x86_64 2/4
Verifying : postgresql-server-8.4.20-7.el6.x86_64 3/4
Verifying : postgresql-libs-8.4.20-7.el6.x86_64 4/4

Installed:
ambari-server.x86_64 0:2.2.2.0-460

Dependency Installed:
postgresql.x86_64 0:8.4.20-7.el6 postgresql-libs.x86_64 0:8.4.20-7.el6
postgresql-server.x86_64 0:8.4.20-7.el6

Complete!
---> 7ae20d07b28b
Removing intermediate container 12b3346676c6
Step 17 : RUN ambari-server setup -s
---> Running in e73d99782d78
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
FATAL: Could not load /lib/modules/3.10.0-514.el7.x86_64/modules.dep: No such file or directory
iptables v1.4.7: can't initialize iptables table nat': Permission denied (you must be root) Perhaps iptables or your kernel needs to be upgraded. FATAL: Could not load /lib/modules/3.10.0-514.el7.x86_64/modules.dep: No such file or directory iptables v1.4.7: can't initialize iptables table mangle': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
FATAL: Could not load /lib/modules/3.10.0-514.el7.x86_64/modules.dep: No such file or directory
iptables v1.4.7: can't initialize iptables table security': Permission denied (you must be root) Perhaps iptables or your kernel needs to be upgraded. FATAL: Could not load /lib/modules/3.10.0-514.el7.x86_64/modules.dep: No such file or directory iptables v1.4.7: can't initialize iptables table raw': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
FATAL: Could not load /lib/modules/3.10.0-514.el7.x86_64/modules.dep: No such file or directory
iptables v1.4.7: can't initialize iptables table `filter': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.

WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)?
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK

Enter choice (1):
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)?
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u60-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz
jdk-8u60-linux-x64.tar.gz... 100% (172.8 MB of 172.8 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz
Installing JDK to /usr/jdk64/
Successfully installed JDK to /usr/jdk64/
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to /var/lib/ambari-server/resources/jce_policy-8.zip

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip
Installing JCE policy...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take upto a minute.
Initializing database: [ OK ]

About to start PostgreSQL
Configuring local database...
Connecting to local database...done.
Configuring PostgreSQL...
Restarting PostgreSQL
Extracting system views...
ambari-admin-2.2.2.0.460.jar
......
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
---> cb9fe8e400db
Removing intermediate container e73d99782d78
Step 18 : RUN yum clean all
---> Running in 32e74cdbbe9b
Loaded plugins: fastestmirror, ovl
Cleaning repos: Updates-ambari-2.2.2.0 base epel extras updates
Cleaning up Everything
Cleaning up list of fastest mirrors
---> 08c07090b0d6
Removing intermediate container 32e74cdbbe9b
Step 19 : EXPOSE 22 8080 8081 8082 8083 8084 8085 8086 8087 8088
---> Running in a95552db9a7a
---> e6c7446d196f
Removing intermediate container a95552db9a7a
Step 20 : CMD /usr/bin/supervisord -n
---> Running in 426096c55a57
---> 1375ed19a76c
Removing intermediate container 426096c55a57
Successfully built 1375ed19a76c
914de0b20af99b7e8399abe1bac6c857e5262c759d9e0982679d92b9bd41ba1b
"docker rm" requires at least 1 argument(s).
See 'docker rm --help'.

Usage: docker rm [OPTIONS] CONTAINER [CONTAINER...]

Remove one or more containers
Using the following hostnames:

774043b7fd74
fe3afc8bc984
6ee12cb81172

Copying back the private key...
Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Running initdb: This may take upto a minute.
About to start PostgreSQL
ERROR: Exiting with exit code 1.
REASON: Unable to start PostgreSQL server. Status None. . Exiting

Preparing to Deploy process stuck

When deploying services, the page doesn't move further than step 3. This is what it looks like:

I'm attaching the tail from /var/log/ambari-server/ambari-server.log on the main docker node (n=3, port 8080) for this attempt here:
stacktrace.txt

I've tried this using various versions of Ambari on CentOS 6 and CentOS 7, to similar effect. Using the 2.2.2 it does go through, but give several errors during the process, for instance:

"Consistency check failed on Ambari" (detailed here, checked solutions, error persisted)
"Error in persisting web client state at ambari server: Error 0"

Eventually once everything is installed, when you go back to the Ambari dashboard, you can see the services are installed but the hosts are no longer available (the default group is showing 0 nodes)

It's also worth mentioning that early in the Ambari setup process, when we're registering the hosts, it warns that iptables is running and that ntpd is not. I was not able to make it work after fiddling with iptables, since I wasn't able to disable them all together, but running something like

iptables -I INPUT -j ACCEPT

should theoretically work. I was able to get ntpd running by adding the following to the Dockerfile right after Supervisor is installed:

RUN yum -y install initscripts && yum clean all
ADD ntpdconfig /etc/sysconfig/ntpd
RUN service ntpd restart
RUN ntpdate -u pool.ntp.org

Where ntpdconfig looks like this:

# Drop root to id 'ntp:ntp' by default.
#OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"

# Set to 'yes' to sync hw clock after successful ntpdate
SYNC_HWCLOCK=no

# Additional options for ntpdate
NTPDATE_OPTIONS=""

Any help or direction you could provide would be awesome! Thanks!

run.sh from-source doesn't work

thank you for putting this together! I need to containerize some things and it's nice not to have to start from scratch :-)

it fails at step
mvn -f $HADOOP_SRC_HOME package -DskipTests -Dtar -Pdist -q || exit 1
in run.sh. I'm thinking it's because there's nothing at $HOME/Workspace on my machine, and as far as I can tell, nothing in this script sets that up for me. Also, it might be helpful to remind the user in the README to ensure maven is installed prior to running the script.

I'm thinking I can fix this on my own by grabbing the latest hadoop/spark repos and ensuring run.sh is pointing to the appropriate pom files - posting the issue here to give you a heads up as to some of the minor pain points I'm encountering in using the tool.

Support running docker containers in swarm mode

So that we can run docker containers in a real cluster!

Document the usage by from-ambari and from-source

From ambari can refer to the web page: https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide
From source can use the current usage section, with more details provided, e.g. users has to set the hadoop/spark source directory before running the run.sh.

After using Ambari to deploy the cluster, Ambari dies on restart

Add services stuck at "Preparing to Deploy: 5 of 11 tasks completed" forever

I add services on ambari,everything works fine up to this point,after which it remains at "Preparing to Deploy: 5 of 11 tasks completed."forevery. and ambari-server.log no obvious errors.

I'm sure hosts iptables selinux and the time has been synced.because other services have been installed successful.
By the way,there were error in hosts and some nodes hostname,but it's all right now.

So is such a problem a database?

Add usage documentation to README.md and command --help

So that users know to use it.

container name conflicts

got this issue when i used the script on mac. seems like there is a name conflict. I did stop and remove the containers but it still happens. any ideas on how to handle it ?

Successfully built 09098cd5ea16
Successfully tagged caochong-ambari:latest
"docker rm" requires at least 1 argument.
See 'docker rm --help'.

Usage:  docker rm [OPTIONS] CONTAINER [CONTAINER...]

Remove one or more containers
docker: Error response from daemon: Conflict. The container name "/caochong-ambari-0" is already in use by container "92b3b26333cc105bfee4d6f6b4eddc0a433e774360c1bcbfe797093c6fa531e9". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
Using the following hostnames:
------------------------------
92b3b26333cc
548781d99167

------------------------------
Copying back the private key...
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Running initdb: This may take upto a minute.
About to start PostgreSQL
ERROR: Exiting with exit code 1.
REASON: Unable to start PostgreSQL server. Status None. . Exiting

weiqingy / caochong Goto Github PK

caochong's People

Stargazers

Watchers

Forkers

caochong's Issues

================================================================================ Package Arch Version Repository Size

Transaction Summary

================================================================================ Package Arch Version Repository Size

Transaction Summary

Total download size: 22 M Downloading Packages:

================================================================================ Package Arch Version Repository Size

Transaction Summary

Total download size: 22 M Installed size: 46 M Downloading Packages:

================================================================================ Package Arch Version Repository Size

Transaction Summary

Total download size: 1.1 M Installed size: 2.9 M Downloading Packages:

================================================================================ Package Arch Version Repository Size

Transaction Summary

Total download size: 364 k Installed size: 1.4 M Downloading Packages:

================================================================================ Package Arch Version Repository Size

Transaction Summary

Total download size: 415 M Installed size: 473 M Downloading Packages:

Remove one or more containers Using the following hostnames:

774043b7fd74 fe3afc8bc984 6ee12cb81172

Recommend Projects

Recommend Topics

Recommend Org

================================================================================
Package Arch Version Repository Size

================================================================================
Package Arch Version Repository Size

Total download size: 22 M
Downloading Packages:

================================================================================
Package Arch Version Repository
Size

Total download size: 22 M
Installed size: 46 M
Downloading Packages:

================================================================================
Package Arch Version Repository Size

Total download size: 1.1 M
Installed size: 2.9 M
Downloading Packages:

================================================================================
Package Arch Version Repository Size

Total download size: 364 k
Installed size: 1.4 M
Downloading Packages:

================================================================================
Package Arch Version Repository Size

Total download size: 415 M
Installed size: 473 M
Downloading Packages:

Remove one or more containers
Using the following hostnames:

774043b7fd74
fe3afc8bc984
6ee12cb81172