eksctl-io / eksctl Goto Github PK

It was observed that it blocked once, needs more testing to verify why.

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/cfn.go#L41

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/cfn.go#L355

I created a cluster and it worked great. I zapped it via the CloudFormation console.

Then the next time I tried to create a cluster I get an error:

2018-07-03T12:07:39+01:00 [ℹ]  importing SSH public key "/Users/jstrachan/.ssh/id_jstrachan_testing_rsa.pub" as "EKS-eks1"
2018-07-03T12:07:40+01:00 [✖]  importing SSH public key: InvalidKeyPair.Duplicate: The keypair 'EKS-eks1' already exists.

I had to manually work around this via:

aws ec2 delete-key-pair --key-name EKS-eks1 --region us-west-2

It'd be nice to detect this error message and log as a warning and continue; or better use the aws CLI to query if the KeyPair exists for the given name first?

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/cfn.go#L73-L87

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/cmd/eksctl/create.go#L125

we're using eksctl in jx create cluster eks and its working well - many thanks!.

However it'd be even more awesome if we could add a CLI option to be able to enable the arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser policy on the worker nodes; so that nodes are able to push to ECR. Then we can use ECR OOTB in Jenkins X on clusters created via eksctl

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-eks-cluster.html

Hello,

It seems the readme/doc is not synced up with the code as the former mentions --name as a flag when the command exposes --cluster-name.

Am I reading it wrong? :)

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/eks.go#L164

In no particular order we should consider the following:

• https://github.com/kubernetes-incubator/external-dns
• https://github.com/zalando-incubator/kube-ingress-aws-controller
• https://github.com/kubernetes-sigs/aws-alb-ingress-controller

Some of these add-ons require instance roles and pre-extisting resource (e.g. Route 53 zone for external DNS) etc, so we could ensure that's set up correctly and make things easy for users.

As it's the intended way with CloudFormation.

it's too easy to forget to specify --cluster-name <name> and type eksctl create cluster <name> instead, we should probably support simple argument mode, because kops and kubicorn support this also.

Hello ,

After creating the cluster using the eksctl , I was not able to get eksadmin service account token to access the api server endpoint for doing the programatic depoyment.

Could you please guide me on how to get the token for the same.
I have used the below command '
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep eks-admin | awk '{print $1}')

I could see only the following service accounts:

aws-node
kube-proxy
kube-dns
default

Regards
Arun

As I was trying to use eksctl, I spent a lot of time debugging whether or not it understood which profile to use mostly by digging through the source code. I was using .aws/credentials + AWS_PROFILE env var. Turns out, it does.

It would be very helpful, given this is meant to be a "guided experience" tool, if it just popped out a message when it started up telling you which profile it is trying to use, ala:

[x] Using aws profile: my-mfa-profile

eksctl create cluster --cluster-name dev --region us-east-1

This has been hung on 2018-06-20T14:16:22-05:00 [ℹ] creating control plane "dev" for over thirty minutes now. If it does take this long, it would be nice to show some additional output periodically to let me know it's doing something

2018-06-20T14:15:01-05:00 [ℹ]  importing SSH public key "/Users/samirpatel/.ssh/id_rsa.pub" as "EKS-dev"
2018-06-20T14:15:01-05:00 [ℹ]  creating EKS cluster "dev" in "us-east-1" region
2018-06-20T14:15:01-05:00 [ℹ]  creating ServiceRole stack "EKS-dev-ServiceRole"
2018-06-20T14:15:01-05:00 [ℹ]  creating VPC stack "EKS-dev-VPC"
2018-06-20T14:15:21-05:00 [✔]  created ServiceRole stack "EKS-dev-ServiceRole"
2018-06-20T14:16:22-05:00 [✔]  created VPC stack "EKS-dev-VPC"
2018-06-20T14:16:22-05:00 [ℹ]  creating control plane "dev"

We can either instruct the user how to install kubectl and heptio-authenticator-aws, or install it for them. However, with OS packages we could probably consider pulling these in that way...
If we decide to install, perhaps we should install them into ~/.eksctl/bin or something like that.
https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/cmd/eksctl/create.go#L155-L157

We've been promoting the project using eksctl.io URL, but the page doesn't actually link to the repo, neither it displays stars. Sharing buttons would be also nice to add, so visitors are encouraged to share with their friends.

Goreleaser actually supports Homebrew, so it should be just a matter of enabling it.

I'm trying to create a cluster with using a specific profile. I've set the AWS_PROFILE environment variable (as per this) but its still creating the cluster using the default cli profile.

We use specific profiles to distinguish between our environments (dev, non-prod, prod) and we're also using MFA.

Looking at #56 it would appear that using profiles can work?

At the moment we create clusters dedicated for single user, we need to extend this.

https://docs.aws.amazon.com/eks/latest/userguide/managing-auth.html

https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html

It'd be nice to have two features:

• add any existing IAM user to a cluster
• add all admin (as in, those with Administrator policy) to a cluster when it's created (may need sync loop)

We would like to implement Cluster API. One of the modes in which one can use Cluster API would be eksctl apply -f cluster.yaml, this would let users create and update clusters using a declarative format of a Cluster API object.
Another mode of using Cluster API would be via a controller running in a management cluster (#20).

Add another debug level (perhaps v=5) where we enable debug logging of the AWS Go SDK (with request tracing).

This will help with issue investigation but also with implementing mocking of services for tests.

We would like to implement Cluster API. One of the modes in which one can use Cluster API is by having a management cluster which runs a controller (actuator), which creates new clusters and nodegroups when it receives Cluster API objects.
Another mode of using it would be via CLI (#19).

It looks like we need to sync wires to disk or wait for it, not so sure what's best.

...
2018-07-05T07:18:44+01:00 [✔]  all EKS cluster "amazing-sheepdog-1530770615" resources has been created
2018-07-05T07:18:44+01:00 [✔]  wrote "test2"
2018-07-05T07:18:46+01:00 [ℹ]  the cluster has 0 nodes
2018-07-05T07:18:46+01:00 [ℹ]  waiting for at least 1 nodes to become ready
2018-07-05T07:19:17+01:00 [ℹ]  the cluster has 1 nodes
2018-07-05T07:19:17+01:00 [ℹ]  node "ip-192-168-85-159.us-west-2.compute.internal" is ready
2018-07-05T07:19:17+01:00 [▶]  kubectl: "/Users/ilya/Library/Local/Homebrew/bin/kubectl"
2018-07-05T07:19:18+01:00 [▶]  clientVersion="v1.10.4" err="The connection to the server localhost:8080 was refused - did you specify the right host or port?\n"
2018-07-05T07:19:18+01:00 [▶]  heptio-authenticator-aws: "/usr/local/bin/heptio-authenticator-aws"
2018-07-05T07:19:18+01:00 [✖]  unable to use kubectl with the EKS cluster (check 'kubectl --kubeconfig="test2" --context="[email protected]" version'): error: stat "test2": no such file or directory
2018-07-05T07:19:18+01:00 [ℹ]  cluster should be functional despite missing client binaries that need to be installed in the PATH
2018-07-05T07:19:18+01:00 [✔]  EKS cluster "amazing-sheepdog-1530770615" in "us-west-2" region is ready
 [0] >> kubectl --kubeconfig="test2" --context="[email protected]" version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-18T14:14:00Z", GoVersion:"go1.9.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:13:43Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
 [0] >>

We are using clientcmd.ModifyConfig, which in-turns calls ioutil.WriteFile.

It will fail if a user has many clusters, so it needs paging.

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/eks.go#L128

It should also use data structure and a formatter instead of using logger.

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/eks.go#L127

Currently, AMI IDs have to copied from the docs, we need to make it easier to automate (e.g. via a script).

Created a cluster:

2018-06-05T16:00:29-07:00 [ℹ]  importing SSH public key "/Users/argu/.ssh/id_rsa.pub" as "EKS-adorable-sculpture-1528239629"
2018-06-05T16:00:29-07:00 [ℹ]  creating EKS cluster "adorable-sculpture-1528239629"
2018-06-05T16:00:29-07:00 [ℹ]  creating VPC stack "EKS-adorable-sculpture-1528239629-VPC"
2018-06-05T16:00:29-07:00 [ℹ]  creating ServiceRole stack "EKS-adorable-sculpture-1528239629-ServiceRole"
2018-06-05T16:01:10-07:00 [✔]  created ServiceRole stack "EKS-adorable-sculpture-1528239629-ServiceRole"
2018-06-05T16:01:50-07:00 [✔]  created VPC stack "EKS-adorable-sculpture-1528239629-VPC"
2018-06-05T16:01:50-07:00 [ℹ]  creating control plane "adorable-sculpture-1528239629"
2018-06-05T16:10:32-07:00 [✔]  created control plane "adorable-sculpture-1528239629"
2018-06-05T16:10:32-07:00 [ℹ]  creating DefaultNodeGroup stack "EKS-adorable-sculpture-1528239629-DefaultNodeGroup"
2018-06-05T16:14:12-07:00 [✔]  created DefaultNodeGroup stack "EKS-adorable-sculpture-1528239629-DefaultNodeGroup"
2018-06-05T16:14:12-07:00 [✔]  all EKS cluster "adorable-sculpture-1528239629" resources has been created
2018-06-05T16:14:12-07:00 [ℹ]  wrote "kubeconfig"
2018-06-05T16:14:13-07:00 [ℹ]  the cluster has 0 nodes
2018-06-05T16:14:13-07:00 [ℹ]  waiting for at least 2 nodes to become ready
2018-06-05T16:14:59-07:00 [ℹ]  the cluster has 2 nodes
2018-06-05T16:14:59-07:00 [ℹ]  node "ip-192-168-236-174.us-west-2.compute.internal" is ready
2018-06-05T16:14:59-07:00 [ℹ]  node "ip-192-168-87-30.us-west-2.compute.internal" is ready
2018-06-05T16:14:59-07:00 [✖]  kubectl version v1.8.0 was found at "kubectl", minimum required version to use EKS is v1.10.0
2018-06-05T16:14:59-07:00 [ℹ]  cluster should be functions despite missing client binaries that need to be installed in the PATH

Upgraded kubectl to 1.10.2
Installed heptio-authenticator-aws in PATH
Had to grab cluster name from the log output
eksctl get cluster --cluster-name adorable-sculpture-1528239629 and eksctl get cluster --cluster-name adorable-sculpture-1528239629 --region us-west-2 is giving an empty output

How do you fix this:

$ EKS_CLUSTER_NAME='dk-cluster-1'
$ eksctl create cluster \  
 --cluster-name ${EKS_CLUSTER_NAME} \
 --kubeconfig ~/.kube/config \
 --nodes 3 \
 --ssh-public-key ~/.ssh/${EKS_CLUSTER_NAME}.id_rsa.pub
2018-06-13T10:05:48-07:00 [ℹ]  importing SSH public key "/Users/jmenchaca/.ssh/dk-cluster-1.id_rsa.pub" as "EKS-dk-cluster-1"
2018-06-13T10:05:49-07:00 [ℹ]  creating EKS cluster "dk-cluster-1"
2018-06-13T10:05:49-07:00 [ℹ]  creating VPC stack "EKS-dk-cluster-1-VPC"
2018-06-13T10:05:49-07:00 [ℹ]  creating ServiceRole stack "EKS-dk-cluster-1-ServiceRole"
2018-06-13T10:06:29-07:00 [✔]  created ServiceRole stack "EKS-dk-cluster-1-ServiceRole"
2018-06-13T10:07:20-07:00 [✔]  created VPC stack "EKS-dk-cluster-1-VPC"
2018-06-13T10:07:20-07:00 [ℹ]  creating control plane "dk-cluster-1"
2018-06-13T10:17:41-07:00 [✔]  created control plane "dk-cluster-1"
2018-06-13T10:17:41-07:00 [ℹ]  creating DefaultNodeGroup stack "EKS-dk-cluster-1-DefaultNodeGroup"
2018-06-13T10:21:22-07:00 [✔]  created DefaultNodeGroup stack "EKS-dk-cluster-1-DefaultNodeGroup"
2018-06-13T10:21:22-07:00 [✔]  all EKS cluster "dk-cluster-1" resources has been created
2018-06-13T10:21:22-07:00 [ℹ]  wrote "/Users/jmenchaca/.kube/config"
2018-06-13T10:21:27-07:00 [ℹ]  the cluster has 0 nodes
2018-06-13T10:21:27-07:00 [ℹ]  waiting for at least 3 nodes to become ready
panic: interface conversion: runtime.Object is nil, not *v1.Node

goroutine 1 [running]:
github.com/weaveworks/eksctl/pkg/eks.(*ClusterConfig).WaitForNodes(0xc4202f3d40, 0xc42057a5a0, 0x0, 0x0)
	/go/src/github.com/weaveworks/eksctl/pkg/eks/nodegroup.go:103 +0x7fa
main.doCreateCluster(0xc4202f3d40, 0xc420171cb0, 0xc420171c80)
	/go/src/github.com/weaveworks/eksctl/cmd/eksctl/create.go:153 +0x485
main.createClusterCmd.func1(0xc4201efb80, 0xc42033c700, 0x0, 0x8)
	/go/src/github.com/weaveworks/eksctl/cmd/eksctl/create.go:57 +0x2a
github.com/weaveworks/eksctl/vendor/github.com/spf13/cobra.(*Command).execute(0xc4201efb80, 0xc42033c680, 0x8, 0x8, 0xc4201efb80, 0xc42033c680)
	/go/src/github.com/weaveworks/eksctl/vendor/github.com/spf13/cobra/command.go:766 +0x2c1
github.com/weaveworks/eksctl/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x2a1cc00, 0xc420171f68, 0x1ca8d86, 0xc420171f78)
	/go/src/github.com/weaveworks/eksctl/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/weaveworks/eksctl/vendor/github.com/spf13/cobra.(*Command).Execute(0x2a1cc00, 0x1ff01a3, 0x1)
	/go/src/github.com/weaveworks/eksctl/vendor/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
	/go/src/github.com/weaveworks/eksctl/cmd/eksctl/main.go:28 +0x2d

As mentioned by @richardcase in #66 (comment).

Authenticator doesn't support this at the moment (kubernetes-sigs/aws-iam-authenticator#100), but it should be easy to fix.

Right now this is really down to Circle CI – whichever job succeeds first. It's a little bit confusing to who who pay attention. We should be able to make it less confusing and show the release we want people to download at the top of the page.

I'm not sure if this is out of scope of this project, but I'd love to be able to convert our existing kops clusters to EKS without the manual work.

Thanks!

• make install instructions simpler with latest_release floating tag
• review --help for all subcommand
• test against internal EKS endpoint (and fix any potential issues)
• squash the history into sensible chunks
• update AMI IDs for each of the GA regions
• re-create the private repo
• push to the new repo and check CircleCI works
• remove TODOs from the code
• tag 0.1.0-alpha.2 (as 0.1.0-alpha.1 was a test release)
• open the repo to the public
• enable github pages

Please allow option to generate new SSH keys, or use specified key, rather than using ~/.ssh/id_rsa.pub

I'm getting this output for the following command:

eksctl create cluster -n default -t t2.large -N 1 -M 1 -m 1 -r us-east-1 --ssh-public-key ~/.ssh/my-key.pub
2018-06-08T11:11:58-07:00 [ℹ]  importing SSH public key "/Users/jamesalbert/.ssh/my-key.pub" as "EKS-default"
2018-06-08T11:11:59-07:00 [ℹ]  creating EKS cluster "default" in "us-east-1" region
2018-06-08T11:11:59-07:00 [ℹ]  creating VPC stack "EKS-default-VPC"
2018-06-08T11:11:59-07:00 [ℹ]  creating ServiceRole stack "EKS-default-ServiceRole"

The only thing that gets created is the key pair. It just hangs here (I left it running for 45 minutes twice). Verbose logs don't tell too much more than this. I'm using a god mode role so I don't think it's a permissions issue. Any ideas?

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/eks.go#L22

If you are using MFA there are 2 errors reported from CheckAllCommands. Specifically when getting the kubectl version and then when checking the server version:

2018-06-19T17:20:17+01:00 [▶]  kubectl: "/usr/local/bin/kubectl"
2018-06-19T17:20:17+01:00 [▶]  clientVersion="v1.10.2" err="Assume Role MFA token code: could not get token: EOF\nUnable to connect to the server: getting token: exec: exit status 1\n"
2018-06-19T17:20:17+01:00 [▶]  heptio-authenticator-aws: "/Users/richardcase/bin/heptio-authenticator-aws"
2018-06-19T17:20:17+01:00 [✖]  unable to use kubectl with the EKS cluster (check ' --kubeconfig /Users/richardcase/.kube/config version'): Assume Role MFA token code: could not get token: EOF
Unable to connect to the server: getting token: exec: exit status 1
2018-06-19T17:20:17+01:00 [ℹ]  cluster should be functions despite missing client binaries that need to be installed in the PATH

Potentially the following should happen:

• Only get the client version in CheckKubectlVersion. This would be by setting --client=true
• Not checking the server version in CheckAllCommands. As it stands eksctl doesn't allow you to specify a Kubernetes version (i.e. it gets the latest version supported by EKS). Should we expose a version flag then we can check its above the minimum version supported.

As seen in #75, there are still a few issues with handing of certain error states.

I just installed eksclt and ran eksctl create cluster --cluster-name=eks-test
The VPC cloudformation stack can't be built. All subnets creation fails with this error:

Template error: Fn::Select cannot select nonexistent value at index 1

It would be great to be able easily add to the node template user-data. While one could get most of the same functionality with custom AMIs, then you have to maintain AMIs. For instance, I hacked the nodegroup template to add support for insecure registries (deploy pods from a registry deployed within the cluster):

"echo {\\\"insecure-registries\\\": [\\\"$DNS_CLUSTER_IP/8\\\"]} > /etc/docker/daemon.json" , "\n",
"systemctl restart docker", "\n",

It can stall forever right now, at least in theory.

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/cfn.go#L56

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/pkg/eks/eks.go#L76

https://github.com/docopt/docopt.go/blob/master/README.md

From the description, it looks much nicer then Cobra.

Installation on RHEL results in message /lib/ld-musl-x86_64.so.1: bad ELF interpreter: No such file or directory

os information

cat /etc/*-release

NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4"
Red Hat Enterprise Linux Server release 7.4 (Maipo)
Red Hat Enterprise Linux Server release 7.4 (Maipo)

need the capability to specify vpc and subnets in create cluster command to reuse the existing direct connect vpc.

We could probably add some unit tests, but the critical code path is reliant on AWS APIs and mocking CloudFormation is going to involve a lot of effort due to its complex nature (unless there is something ready-made we can use). Also, we would also need to mock Kubernetes somehow, there is probably prior art, but that could be very specialised. Personally, I feel that integration test is what we need the most.

Right now it's a bit ad-hoc, especially ASG size, but also instance types and regions are not validated at all.

https://github.com/weaveworks/eksctl/blob/0a39d1a2c1d107e1da90c27889d43bddba42ab3a/cmd/eksctl/create.go#L70

It'd be interesting to experiment with doing the heavy lifting to help ECS users try moving some of their workloads to EKS. We wouldn't be able to do magic here, but it should be possible to generate manifests and put them in a git repo etc.

Following the download instructions from https://eksctl.io/, and got the following error:

~ $ curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.