janeczku / docker-machine-vultr

:m: Docker Machine driver for Vultr Cloud

Docker Machine driver plugin for Vultr

Provision Vultr cloud instances using the docker-machine command line tool.

Installation

You need to have installed Docker Machine v0.5.1 or later.

Download the Vultr driver matching your platform from the release page.

If you are running on macOS:

$ curl -L https://github.com/janeczku/docker-machine-vultr/releases/download/v1.3.0/docker-machine-driver-vultr-`uname -s`-`uname -m` \
  -o /usr/local/bin/docker-machine-driver-vultr && chmod +x /usr/local/bin/docker-machine-driver-vultr

If you are running on Linux:

$ curl -L https://github.com/janeczku/docker-machine-vultr/releases/download/v1.3.0/docker-machine-driver-vultr-`uname -s`-`uname -m` \
  -o /tmp/docker-machine-driver-vultr && chmod +x /tmp/docker-machine-driver-vultr &&
  sudo cp /tmp/docker-machine-driver-vultr /usr/local/bin/docker-machine-driver-vultr

Otherwise, download the binary from the release page directly.

Usage instructions

Grab your API key from the Vultr control panel and pass that to docker-machine create with the --vultr-api-key option.

Example for creating a new machine running RancherOS:

docker-machine create --driver vultr --vultr-api-key=abc123 rancheros-machine

Example for creating a new machine running Ubuntu 16.04:

docker-machine create --driver vultr --vultr-api-key=abc123 --vultr-os-id=215 ubuntu-machine

Command line flags:

• --vultr-api-key: required Your Vultr API key.
• --vultr-ssh-user: SSH username.
• --vultr-region-id: Region the VPS will be created in (DCID). See available Region IDs.
• --vultr-plan-id: Plan to use for this VPS (VPSPLANID). See available Plan IDs.
• --vultr-os-id: Operating system ID to use (OSID). See available OS IDs.
• --vultr-ros-version: RancherOS version to use if an OSID was not specified (e.g. 'v1.0.1', 'latest').
• --vultr-pxe-script: PXE script ID. Requires the 'Custom OS' ('--vultr-os-id=159')
• --vultr-boot-script: Boot script ID. Mutually exclusive of '--vultr-pxe-script'.
• --vultr-ssh-key-id: Use an existing SSH key in your Vultr account instead of generating a new one.
• --vultr-ipv6: Enable IPv6 support for the VPS.
• --vultr-private-networking: Enable private networking support for the VPS.
• --vultr-backups: Enable automatic backups for the VPS.
• --vultr-userdata: Path to file with cloud-init user-data.
• --vultr-snapshot-id: ID of an existing Snapshot in your Vultr account.
• --vultr-reserved-ip: ID of a reserved IP in your Vultr account.
• --vultr-tag: Tag to assign to the VPS.
• --vultr-firewall-group: ID of existing firewall group to assign.
• --vultr-api-endpoint: Override default Vultr API endpoint URL.

If the OS ID is not specified, RancherOS will be used as operating system for the instance. You can select a specific RancherOS version by specifying the --vultr-ros-version flag.

PXE deployment

You can boot a custom OS using a PXE boot script that you created in your Vultr account panel by passing it's ID with the --vultr-pxe-script flag and setting --vultr-os-id to 159. The operating system must support cloud-init and be configured to use the ec2 datasource type.

Environment variables and default values:

CLI option	Environment variable	Default
--vultr-api-key	VULTR_API_KEY	-
--vultr-ssh-user	VULTR_SSH_USER	root
--vultr-region-id	VULTR_REGION	1 (New Jersey)
--vultr-plan-id	VULTR_PLAN	201 (1024 MB, 25 GB SSD)
--vultr-os-id	VULTR_OS	-
--vultr-ros-version	VULTR_ROS_VERSION	v1.0.2
--vultr-pxe-script	VULTR_PXE_SCRIPT	-
--vultr-boot-script	VULTR_BOOT_SCRIPT	-
--vultr-ssh-key-id	VULTR_SSH_KEY	-
--vultr-ipv6	VULTR_IPV6	false
--vultr-private-networking	VULTR_PRIVATE_NETWORKING	false
--vultr-backups	VULTR_BACKUPS	false
--vultr-userdata	VULTR_USERDATA	-
--vultr-snapshot-id	VULTR_SNAPSHOT	-
--vultr-reserved-ip	VULTR_RESERVED_IP	-
--vultr-tag	VULTR_TAG	-
--vultr-firewall-group	VULTR_FIREWALL_GROUP	-
--vultr-api-endpoint	VULTR_API_ENDPOINT	-

Find available plans for all Vultr locations

Check out vultr-status.appspot.com for a live listing of the available plans per region. Get the corresponding --vultr-region-id and --vultr-plan-id parameters with the click of a button.

docker-machine-vultr's Issues

Unable to query docker version: Cannot connect to the docker engine endpoint

When I run

docker-machine create --driver vultr --vultr-api-key=asd --vultr-os-id=215 --vultr-plan-id=29 --vultr-region-id=8 --vultr-ssh-key-id=asd asd
the installation can't seem to finish

I get this prompt ±10 times

Waiting for SSH to be available...
Enter passphrase for key '/Users/martinbottanek/.ssh/id_rsa': 
Detecting the provisioner...
Enter passphrase for key '/Users/martinbottanek/.ssh/id_rsa': 

and then I have to CTRL-C

running docker-machine ls shows a running node with this error message: Unable to query docker version: Cannot connect to the docker engine endpoint

Any ideas? Thank you!

mac binary 1.0.0 is not working

I downloaded a mac binary from release 1.0.0 page , and I followed instructions but it didn't work with this error. (docker-machine version 0.5.6 build 61388e9, El Capitan 10.11.3)

docker-machine create --driver vultr --vultr-api-key=IEnteredRealAPIKey docker-test
Running pre-create checks...
(docker2) Validating Vultr VPS parameters...
(docker2) getting client
Error creating machine: Error with pre-create check: unexpected EOF

So, I built binary with go 1.5.3 and that works for me.

go get -u github.com/janeczku/docker-machine-vultr/bin
mv $GOPATH/bin/bin $GOPATH/bin/docker-machine-driver-vultr

I hope this will help.

I often received "Could not get lock /var/lib/dpkg/lock" error while creating docker host

Here is the log:

ζ docker-machine create -d vultr v1                                                              [0099b16]
Running pre-create checks...
(v1) Validating Vultr VPS parameters...
(v1) getting client
(v1) getting client
(v1) getting client
Creating machine...
(v1) getting client
(v1) Creating Vultr VPS...
(v1) getting client
(v1) Waiting for IP address to become available...
(v1) Created Vultr VPS ID: 6140199, Public IP: 45.32.130.69, Private IP:
Waiting for machine to be running, this may take a few minutes...
(v1) getting client
(v1) getting client
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with ubuntu(systemd)...
Error creating machine: Error running provisioning: ssh command error:
command : sudo apt-get update
err     : exit status 100
output  : Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

As you can see, the sudo apt-get update command cannot execute as there is another apt-get running in the background, maybe because the system setup script running for the first time.

I think we should wait for a little longer or wait until the other apt exit.

Invoking vultr boot scripts

Is there a parameter for vultr-boot scripts? I can't seem to find it. All I'm trying to do is automatically add swap.

Thanks!

CentOS 7 no longer provisions correctly

It gets to Setting Docker configuration on the remote daemon... before:

Error running SSH command: something went wrong running an SSH command
command : netstat -tln
err : exit status 127
output : bash: netstat: command not found

docker-machine create not compatible with VULTR firewall

Hi,

I just set up a machine using this Driver.

docker-machine create \
	--driver vultr \
	--vultr-api-key=$VULTR_API_KEY \
	--vultr-region-id=7 \
	--vultr-plan-id=201 \
	--vultr-backups=0 \
	--vultr-os-id=159 \
	--vultr-ros-version="v0.9.1" \
	pet-machine3

It boots up without any issues.
After that I applied a new firewall configuration through the VULTR web interface.
With the following open ports:

TCP 22
TCP 80
TCP 443
TCP 2376
UDP 2376 as well...

Even with using the default ssh client of my Ubuntu desktop I can not connect when the firewall is enabled.

I've also tried the Ubuntu image (215). It reproduces also with that configuration.

When I boot a machine using a default system image from VULTR (e.g. Ubuntu) I can not reproduce the issue.

I've reported the issue to VULTR, but I did not receive any update yet which allowed me to fix the configuration. I will make them aware of this issue report by referring it from the ticket.

Update:

VULTR support was able to get it working on one specific machine. Other machines are still not working. I'm awaiting their feedback to see how we can get this fixed.

update 29-3-2017:
Issue should be resolved, but I still need to verify this.

Incorrect Usage error when running the example

I am trying to create a create a machine using docker-machine 0.4.1 on OSX 10.10.5, but I get an

Incorrect Usage

message

the driver is on my path and is executable.
I am using the custom os (rancher) example command exactly as is, just substituting my API key.

Does this require docker-machine 0.5 perhaps?

How to build an i386 binary?

Hey,

please build an i386 binary or tell me build instructions für i386.

Thx

greetings sam

How to add an already provisioned machine?

How would I add the already provisioned machine on another computer?

The iPXE boot script is not removed on docker-machine rm

The iPXE boot script that is created for provisioning a rancherOS machine is not removed on machine removal.

Docker machine fails to create multiple servers at the same time

Hi, I'm trying to simultaneously create multiple Vultr servers using docker-machine at the same time, but I get the following error:

(testmachine2) Validating Vultr VPS parameters...
Running pre-create checks...
(testmachine1) Validating Vultr VPS parameters...
Creating machine...
Creating machine...
(testmachine1) Creating Vultr VPS
(testmachine2) Creating Vultr VPS
(testmachine2) Waiting for IP address to become available...
Error creating machine: Error in driver during machine creation: Post https://api.vultr.com/v1/server/create?api_key=API_KEY: http: ContentLength=176 with Body length 0
(testmachine2) Created Vultr VPS ID: XXXXXXX, Public IP: XX.XX.XXX.XXX, Private IP: 
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...

I ran the following bash script

docker-machine create --driver vultr \
    --vultr-api-key API_KEY \
    --vultr-region-id=19 \
    --vultr-plan-id=201 \
    --vultr-os-id=215 \
    --vultr-private-networking=false \
    --vultr-backups=false \
    --vultr-ssh-user=root \
    --vultr-ipv6=false \
    testmachine1 &
docker-machine create --driver vultr \
    --vultr-api-key API_KEY \
    --vultr-region-id=19 \
    --vultr-plan-id=201 \
    --vultr-os-id=215 \
    --vultr-private-networking=false \
    --vultr-backups=false \
    --vultr-ssh-user=root \
    --vultr-ipv6=false \
    testmachine2 &

Could this be related to issue #12 regarding Vultr's API rate limit to 2 requests per second? It seems that v1.0.7 has implemented retry logic for API rate limit errors, but I'm not entirely sure if it's working, or if this problem is related to the API rate limit problem.

Is RancherOS available on Vultr?

It doesn't seem to show up on the os list https://api.vultr.com/v1/os/list

Add to docker/machine list of driver plugins

There's now a list of available driver plugins in the Docker Machine docs: https://github.com/docker/machine/blob/master/docs/AVAILABLE_DRIVER_PLUGINS.md

It'd be great if the Vultr driver were listed there! :)

Vultr API Rate limiting problem

Hi there,

Thanks for the docker machine driver!

I've ran into rate limitation problem as soon as I try to provision more than one machine and maybe some of it can be controlled from Rancher (which I'm using for controlling docker machine and I've posted a feature request about that rancher/rancher#4464) but I've also contacted the Vultr support to ask about their rate limiting policy and I thought you might find their answer useful.

Our recommendations here are to cache the responses of API calls (for example, our plans list rarely changes), and not use your API key unless a method requires authentication.

Unfortunately, we've seen some very poorly coded API clients before, which hammered our API with tons of requests per minute. Due to issues like that, whitelisting hosts is not possible.

We do have customers successfully managing hundreds of servers via the API, so this is definitely something that is doable.

I did take a quick look at that, and it seems to poll every 2 seconds while a server is pending. This is likely the source of the issue you're seeing.

Hope the advice can contribute to manage the rate limit in order to be able to provision several machines simultaneously!

Thanks,

Jun

Vultr SSH key with ID doesn't exist during creation in docker machine

I executed the command below in order to create a new instance. However, it return error "Vultr SSH key with ID {vultr-ssh-key} doesn't exist". I can find the ssh key id in Console (Server -> SSH Keys). Please help to solve this problem.

docker-machine create
--driver vultr
--vultr-api-key={my-api-key}
--vultr-region-id=25
--vultr-plan-id=202
--vultr-os-id=215
--vultr-ssh-key-id={vultr-ssh-key}
--vultr-private-networking
ubuntu-cms

iPXE boot script for rancherOS is hardcoded

First of all, great job, loving it!

It would be really useful if it was possible to specify a boot script ID when doing creating the machine. Another option would be to pass an argument pointing to a boot script file.

Should we try to do a PR with your code to docker-machine to be an official provider?

Just as the title says, if we merge and update a little bit this repo with some of the PRs proposed, maybe it's worth pursing an official merge in docker-machine.

Thumbs up if you believe that this is a good idea!

default os-id does not work

it seems there is a problem with downloading vmlinuz

The default plan `93` is deprecated

The plan list from API https://api.vultr.com/v1/plans/list?type=vc2, shows 93 plan is deprecated:

  "93": {
    "VPSPLANID": "93",
    "name": "1024 MB RAM,20 GB SSD,2.00 TB BW",
    "vcpu_count": "1",
    "ram": "1024",
    "disk": "20",
    "bandwidth": "2.00",
    "bandwidth_gb": "2048",
    "price_per_month": "10.00",
    "plan_type": "SSD",
    "windows": false,
    "available_locations": [1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 19, 24, 25, 39, 40],
    "deprecated": true
  },

It might be replaced with 201:

  "201": {
    "VPSPLANID": "201",
    "name": "1024 MB RAM,25 GB SSD,1.00 TB BW",
    "vcpu_count": "1",
    "ram": "1024",
    "disk": "25",
    "bandwidth": "1.00",
    "bandwidth_gb": "1024",
    "price_per_month": "5.00",
    "plan_type": "SSD",
    "windows": false,
    "available_locations": [1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 19, 24, 25, 39, 40]
  },

can't get rancheros to ipxe boot

It used to work seamlessly last time I tried, but right now I get an ipxe permission error;
It might come from the vultr API, so I tried allowing any IP to access to API, without success.

Here is a dump of the boot console it it can help:

Any idea?
Thanks for any help :)

Make this go get'able

I had a bit of trouble installing this.

Can we make it go get'able?

go get github.com/janeczku/docker-machine-vultr
go install github.com/janeczku/docker-machine-vultr

Ought to be enough.

Failed to create Vultr host as the Public IP of the server became '0.0.0.0'

It used to work, but during my test today, I found docker-machine create -d vultr failed to create a Vultr host.

 docker-machine create -d vultr v1
Running pre-create checks...
(v1) Validating Vultr VPS parameters...
(v1) getting client
(v1) getting client
(v1) getting client
Creating machine...
(v1) getting client
(v1) Creating Vultr VPS...
(v1) getting client
(v1) Waiting for IP address to become available...
(v1) Created Vultr VPS ID: 6472698, Public IP: 0.0.0.0, Private IP:
Waiting for machine to be running, this may take a few minutes...
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
(v1) getting client
Detecting operating system of created instance...
Waiting for SSH to be available...
Password:

It began to ask me the password, which should never happen. The SSH key should has been set up already.

Creating a new machine hangs with rpc warning

I am trying this with docker-machine 0.5.0-rc4 on OSX 10.10.5 and when I try to create a rancherOS machine all I get is a:

WARNING >>> Error attempting heartbeat call to plugin server: rpc: can't find method RpcServerDriver.Heartbeat

... and the process never finishes. The machine is not created.

Debian based distros no longer finish provisioning

I've tested this with Debian 8, 9 and Ubuntu 17.04. None of them will complete provisioning. They each get to:

Setting Docker configuration on the remote daemon...
Error creating machine: Error running provisioning: ssh command error:
command : sudo systemctl -f start docker
err : exit status 1
output : Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.

RancherOS - Doesn't work at all

Been trying to solve this for hours and it just doesn't work.
If you create 10 different nodes, doesn't matter what type of node it is, it keeps installing, destroying and reinstalling all the Vultr servers and it's just hanging there.
If you decide to create a single node and choose all 3 types it doesn't even create a server at all.

Download link in main Readme points to old version

The download instructions, which people usually just copy and paste, points to v1.3.0 but v1.4.0 is the current/latest release since 7 months ago. It would also be better to improve the command given here to fetch from "latest" rather than hard-code the version in the Readme, although I realise that github don't currently make this easy.

Creating docker machine hangs on 'Waiting machine to be running'

Creating docker machine hangs on 'Waiting machine to be running':

Running pre-create checks...
(test-test.new) Validating Vultr VPS parameters...
Creating machine...
(test-test.new) Creating Vultr VPS
(test-test.new) Waiting for IP address to become available...
(test-test.new) Created Vultr VPS ID: 17638002, Public IP: 45.76.62.159, Private IP:
Waiting for machine to be running, this may take a few minutes...
Error creating machine: Error waiting for machine to be running: Maximum number of retries (60) exceeded

Container is created and its status becomes 'running' even before the docker-machine command fails. And in the vultr UI I can see it. Also I can ssh to the container.

Command:
docker-machine create --driver vultr --vultr-tag test-test.new --vultr-api-key=xxxxx --vultr-os-id=215 -vultr-region-id 6 --vultr-plan-id=201 test-test.new
OS: (tried both) Ubuntu 16.04 or Ubuntu 18.04
Driver: (tried both) https://github.com/janeczku/docker-machine-vultr/releases/download/v1.3.0/docker-machine-driver-vultr-Linux-x86_6
or
https://github.com/janeczku/docker-machine-vultr/releases/download/v1.4.0/docker-machine-driver-vultr-Linux-x86_6
Docker-machine: (tried both) v. 0.15.0, build b48dc28d or 0.14.0, build 89b8332

ssh never becomes available

I am using the vulture machine driver, deploying using the rancher ui.

Using region 9, default endpoint, my ssh personal ssh key and ros v1.1.1, rancher-server 1.6.13

The deployment starts, the server gets deployed on vultr, i can connect using SSH personally, but the rancher server still "waits for available ssh connection". Is the whole point that the rancher-server instance is actually trying to ssh into that sever using its own SSH key ( i guess ) so i would need to reference that in the ssh key id section?

thank you for this great driver!

Autoconfiguration of private network interface

Would it be possible to add auto-configuration of private interface in VPS when --vultr-private-networking is enabled?
I'm thinking abount establishing docker swarm using vultr private network only.

Thanks for advance.

How can I deploy a Docker Application on Vultr?

First of all, appreciated that you developed such a great tool to link docker-machine with vultr.

My question is that how can I directly deploy a Docker service as an application instead of init an entire OS (e.g. Rancher OS, and other OSs) because actually, we can deploy Docker applications on vultr directly through their control panel.

the rancheros instance does not mount storage

It seems the only mounted root filesystem is of type "overlay", and sized arounds 350Mo.

Any idea why?

Add SnapshotID please ;-)

Hello,

I don't know GO otherwise I'd attempt to add this request. Simply, Vultr allows for loading a pre-stored snapshot, hostname and label onto the new host. I'd like these to be added in as it is a very handy feature.

In summary, please add;

1. snapshotid
2. hostname
3. label
4. there are many others available here: https://www.vultr.com/api/#server_create

Example:
curl -H 'API-Key: EXAMPLE' https://api.vultr.com/v1/server/create --data 'DCID=1' --data 'VPSPLANID=1' --data 'OSID=127' --data 'SNAPSHOTID=123456asdf456' --data 'hostname=domain.com' --data 'label=domain.com'.

If you can provide some simple guidance to add then I can do so ;-)

Deploying a RancherOS cluster (Docker Swarm Mode)

So you can enable private networking with --vultr-private-network

It's not clear to me whether its possible to create a warm mode cluster that utilizes a private network. My understanding of Vultr's services is you have to do a bit of extra work to setup private interfaces anyway.

Has anyone successfully setup a RancherOS / Docker Swarm Mode cluster that discovered/heartbeats over a private network?

Can't create a new machine with "docker-machine create"

$ docker-machine create --driver vultr --vultr-api-key=foobar --vultr-region-id 25 --vultr-plan-id 94 server-name

Got this reponse

Running pre-create checks...
(server-name) Validating Vultr VPS parameters...
Error with pre-create check: "strconv.ParseFloat: parsing \"<nil>\": invalid syntax"

Cannot run with Rancher Server

See rancher/rancher#4433