aws-samples / eks-workshop Goto Github PK
View Code? Open in Web Editor NEWAWS Workshop for Learning EKS
Home Page: https://eksworkshop.com
License: MIT No Attribution
AWS Workshop for Learning EKS
Home Page: https://eksworkshop.com
License: MIT No Attribution
This is the error message I got while creating a cluster with the following command
eksctl create cluster --name=eksworkshop-eksctl --nodes=3 --node-ami=auto --region=${AWS_REGION}
2018-10-23T21:46:08Z [ℹ] using region us-west-2
2018-10-23T21:46:09Z [ℹ] setting availability zones to [us-west-2a us-west-2b us-west-2c]
2018-10-23T21:46:10Z [ℹ] using "ami-0a54c984b9f908c81" for nodes
2018-10-23T21:46:10Z [ℹ] creating EKS cluster "eksworkshop-eksctl" in "us-west-2" region
2018-10-23T21:46:10Z [ℹ] will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
2018-10-23T21:46:10Z [ℹ] if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-west-2 --name=eksworkshop-eksctl'
2018-10-23T21:46:10Z [ℹ] creating cluster stack "eksctl-eksworkshop-eksctl-cluster"
2018-10-23T21:47:06Z [✖] unexpected status "ROLLBACK_IN_PROGRESS" while waiting for CloudFormation stack "eksctl-eksworkshop-eksctl-cluster" to reach "CREATE_COMPLETE" status
2018-10-23T21:47:06Z [ℹ] fetching stack events in attempt to troubleshoot the root cause of the failure
2018-10-23T21:47:06Z [ℹ] AWS::CloudFormation::Stack/eksctl-eksworkshop-eksctl-cluster: ROLLBACK_IN_PROGRESS – "The following resource(s) failed to create: [SubnetUSWEST2A, SubnetUSWEST2B, PublicSubnetRoute, PolicyNLB, SubnetUSWEST2C]. . Rollback requested by user."
2018-10-23T21:47:06Z [✖] AWS::EC2::Subnet/SubnetUSWEST2B: CREATE_FAILED – "Resource creation cancelled"
2018-10-23T21:47:06Z [✖] AWS::EC2::Subnet/SubnetUSWEST2C: CREATE_FAILED – "Resource creation cancelled"
2018-10-23T21:47:06Z [✖] AWS::IAM::Policy/PolicyNLB: CREATE_FAILED – "Resource creation cancelled"
2018-10-23T21:47:06Z [✖] AWS::EC2::Subnet/SubnetUSWEST2A: CREATE_FAILED – "Resource creation cancelled"
2018-10-23T21:47:06Z [✖] AWS::EC2::Route/PublicSubnetRoute: CREATE_FAILED – "route table rtb-0400aa3431520b725 and network gateway igw-0c49354aae0fc78cf belong to different networks (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 2c781c6e-aad3-4510-959b-df0ac89ea59a)"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Route/PublicSubnetRoute: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::SecurityGroup/ControlPlaneSecurityGroup: CREATE_COMPLETE
2018-10-23T21:47:06Z [ℹ] AWS::EC2::SecurityGroup/ControlPlaneSecurityGroup: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::RouteTable/RouteTable: CREATE_COMPLETE
2018-10-23T21:47:06Z [ℹ] AWS::IAM::Policy/PolicyNLB: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Subnet/SubnetUSWEST2A: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Subnet/SubnetUSWEST2C: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::RouteTable/RouteTable: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Subnet/SubnetUSWEST2B: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::IAM::Policy/PolicyNLB: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Subnet/SubnetUSWEST2C: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Subnet/SubnetUSWEST2A: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::RouteTable/RouteTable: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::Subnet/SubnetUSWEST2B: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::SecurityGroup/ControlPlaneSecurityGroup: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::IAM::Role/ServiceRole: CREATE_COMPLETE
2018-10-23T21:47:06Z [ℹ] AWS::EC2::VPC/VPC: CREATE_COMPLETE
2018-10-23T21:47:06Z [ℹ] AWS::EC2::InternetGateway/InternetGateway: CREATE_COMPLETE
2018-10-23T21:47:06Z [ℹ] AWS::IAM::Role/ServiceRole: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::InternetGateway/InternetGateway: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::IAM::Role/ServiceRole: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::VPC/VPC: CREATE_IN_PROGRESS – "Resource creation Initiated"
2018-10-23T21:47:06Z [ℹ] AWS::EC2::InternetGateway/InternetGateway: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::EC2::VPC/VPC: CREATE_IN_PROGRESS
2018-10-23T21:47:06Z [ℹ] AWS::CloudFormation::Stack/eksctl-eksworkshop-eksctl-cluster: CREATE_IN_PROGRESS – "User Initiated"
2018-10-23T21:47:06Z [ℹ] 1 error(s) occurred and cluster hasn't been created properly, you may wish to check CloudFormation console
2018-10-23T21:47:06Z [ℹ] to cleanup resources, run 'eksctl delete cluster --region=us-west-2 --name=eksworkshop-eksctl'
2018-10-23T21:47:06Z [✖] waiting for CloudFormation stack "eksctl-eksworkshop-eksctl-cluster" to reach "CREATE_COMPLETE" status: ResourceNotReady: failed waiting for successful resource state
2018-10-23T21:47:06Z [✖] failed to create cluster "eksworkshop-eksctl"
The issue went away when I deleted the stack and ran the eksctl create cluster command once again
* Push the Docker image to registry (_*_ECR_*_)
* Write k8s manifest file
* Deploy the application
* Update the business logic, push changes to git, update the app using pipeline
* Sidecar Injection
* service mesh? Let's use a super common sidecar example
missing period after:
https://eksworkshop.com/integration/monitoring/prereqs/
'if you don't have it installed'
https://eksworkshop.com/integration/monitoring/deploy-prometheus/
After
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9090
We may want to add something like:
If you get an error similar to error: unable to forward port because pod is not running. Current status=Pending
the Prometheus pods may just not had a chance to come up yet. If you get this error, wait 30s, and retry the command.
In https://eksworkshop.com/integration/monitoring/prereqs/ we should articulate what exactly we're trying to achieve/prove by executing helm ls
.
From https://eksworkshop.com/onug/logging/
When running:
helm install stable/metrics-server --name metrics-server --version 2.0.2 --namespace metrics
I get the error:
Error: release metrics-server failed: namespaces "metrics" is forbidden: User "system:serviceaccount:kube-system:default" cannot get namespaces in the namespace "metrics"
I tried to manually create the ns first and re-ran, but no dice. Any ideas whats going on? Is this dependent on another step in the tutorial that I [apparently] skipped?
Showcase how to...
https://eksworkshop.com/integration/monitoring/deploy-grafana/
abc123-1422578166.us-west-2.elb.amazonaws.comfoo:/tmp $
foo:/tmp $
would be nice to have whitespace between prompt and end of URL to make easier for copy/paste
* Configure authn, authz and RBAC
* Configure network policies
it could be as simple as helm install nginx
Deploy solution for pod based roles. will use kube2iam unless there is other feedback
We need to add that to the cleanup.
At https://eksworkshop.com/onug/scaling/deploy_hpa/#deploy-the-metrics-server if the RBAC for helm is not setup correctly first per https://eksworkshop.com/onug/helm/deploy/ the metrics server install will fail.
helm reset
then helm init
is a way to recover if you installed helm before applying these correct RBAC settings.
Expected Outcomes
Use the same apps deployed via helm
https://eksworkshop.com/integration/monitoring/deploy-prometheus/
We DL and edit it as values.yaml, but call helm install as prometheus-values.yaml.
* Take an existing application, create Docker image
* how to write a dockerfile
Found some bugs and typos in Autoscaling chapter. I sent a PR that is already merged into Integration. This is ready to merge into master
* Monitor traffic between services (Prometheus + Grafana)
The code block on https://eksworkshop.com/codepipeline/role/ should change directory to ~/environment before running the commands. The previous commands causes you to be in ~/environment/fluentd (which is subsequently deleted if you did the clean up logs section - but Linux still has your cwd set as ~/environment/fluentd)
https://eksworkshop.com/integration/monitoring/deploy-prometheus/
Change:
curl -o prometheus-values.yaml https://raw.githubusercontent.com/helm/charts/master/stable/prometheus/values.yaml
Search for # storageClass: “-”, uncomment and change the value to “prometheus”. You will do this twice, under both server & alertmanager manifests
to something like:
To install Prometheus, first we'll download the Prometheus configuration using curl, then edit the storageClass attribute.
To download the configuration as values.yaml, execute the following command:
curl -o prometheus-values.yaml https://raw.githubusercontent.com/helm/charts/master/stable/prometheus/values.yaml
Next, using a text editor, open up values.yaml, search for
# storageClass: “-”
, uncomment and change the value to “prometheus”. You will do this twice, under both server & alertmanager manifests
$ git submodule init git submodule update
error: pathspec 'git' did not match any file(s) known to git.
error: pathspec 'submodule' did not match any file(s) known to git.
error: pathspec 'update' did not match any file(s) known to git.
$ npm install
-bash: npm: command not found
Sorry I have brand new laptop : )
Need to include npm install steps (https://www.npmjs.com/get-npm).
$ npm run server
(truncated for abbreviation)......
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "cleanup/eksctl.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "children" in page "deploy/_index.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "dashboard/dashboard.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "deploy/viewservices.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "eksctl/launcheks.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "children" in page "prerequisites/_index.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "prerequisites/account.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "prerequisites/k8stools.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "prerequisites/sshkey.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "prerequisites/workspace.md"
ERROR 2018/09/26 11:22:31 Unable to locate template for shortcode "notice" in page "prerequisites/workspaceiam.md"
(truncated for abbreviation)......
Error: Error building site: logged 12 error(s)
Expected Outcomes
https://eksworkshop.com/integration/monitoring/deploy-grafana/
To improve the user experience, replace the editing of values.yaml with a sed script or similar that performs the text substitutions from the command line.
Under EKS section in intro, change chapters:
Customer Managed
Managed Control Plane
->
Customer Managed Data Plane
Amazon Managed Control Plane
Use Skaffold?
https://eksworkshop.com/integration/monitoring/deploy-prometheus/
This may not be an intuitive operation for new users, so creating detailed steps will help.
https://eksworkshop.com/integration/prerequisites/iamrole/
For each step, a screenshot should accompany. If IAM is new to a user, screenshots with what should be selected / filled out will help them.
https://eksworkshop.com/integration/monitoring/deploy-prometheus/
To improve the user experience, replace the editing of values.yaml with a sed script or similar that performs the text substitutions from the command line.
Prometheus and Grafana default installation uses ClusterIP to expose its metrics and dashboards. This doesn't work well with Cloud9 when you try to run port-forwarding and try to use the ports where the services are listening.
On this slide, note the EKS channel within the k8s slack workspace.
we should be ready to have a discussion of EKS upgrade process and AMIs
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.