hashicorp / consul-service-mesh-for-developers

Respository containing configuration for KubeCon 2019 workshop

Service Mesh for the Developer Workflow

Abstract

Service mesh is often presented as a solution for network engineering and system operability, increasing security, reliability, and observability. However, service mesh is also an incredibly useful tool for developers, and understanding how to leverage this technology can dramatically simplify your day to day workflow.

By leveraging free and open-source tools and a scenario-based approach, we will illustrate how a service mesh can help with application resilience, observability, and debugging.

By the end of this workshop you will understand: How to use metrics and distributed tracing effectively Reliability patterns like retries, timeouts, and circuit breaking How to leverage Canary deployments How you can effectively debug distributed systems

The cloud-native, open-source technology used in this tutorial include: Envoy Prometheus Gloo shot Consul Service Mesh Loop Squash Open Census

Requirements

Ideally it will be possible to bundle these tools into a Docker Container to account for for different environments and minimum setup for the user.

• Docker
• Kubectl
• VSCode / Vim
• Squash
• Gloo Shot
• Loop
• Consul (CLI)

Lesson Plan (total 1hr30 mins)

• What is a Service Mesh (10 mins)
• Setting up the environment (10 mins)
  • yard up
  • install gloo api gateway
  • expose service through gateway
  • Deploy your first application (10 mins)
• Tracing
  • using tracing
  • make changes to service to apply tracing correctly
• Deploy changes as canary
• Add some resilience using service mesh (retry, timeout, etc)
• Debug service
  • try with squash
  • try with loop

consul-service-mesh-for-developers's Issues

shipyard is failing to start properly

I got an error when executed yard up.

yard up --consul-values ./consul-values.yml

     _______. __    __   __  .______   ____    ____  ___      .______       _______
    /       ||  |  |  | |  | |   _  \  \   \  /   / /   \     |   _  \     |       \
   |   (----`|  |__|  | |  | |  |_)  |  \   \/   / /  ^  \    |  |_)  |    |  .--.  |
    \   \    |   __   | |  | |   ___/    \_    _/ /  /_\  \   |      /     |  |  |  |
.----)   |   |  |  |  | |  | |  |          |  |  /  _____  \  |  |\  \----.|  .--.  |
|_______/    |__|  |__| |__| | _|          |__| /__/     \__\ | _| `._____||_______/


Version: 0.5.4

## Creating K8s cluster in Docker and installing Consul

Unable to find image 'nicholasjackson/consul-k8s-tools:0.5.4' locally
0.5.4: Pulling from nicholasjackson/consul-k8s-tools
7ddbc47eeb70: Pull complete
c1bbdc448b72: Pull complete
8c3b70e39044: Pull complete
45d437916d57: Pull complete
d68133391690: Pull complete
c11bb2d619bd: Pull complete
58e52902e2aa: Pull complete
c0a67f2f55b6: Pull complete
f7c6ea5d21a4: Pull complete
ac78f0266a11: Pull complete
1fb8fd7ffbd4: Pull complete
8bad00d7f0c1: Pull complete
f2e5984e351e: Pull complete
bb7ddfa815bf: Pull complete
43cc05480798: Pull complete
Digest: sha256:1e749c2a3b024f512bf98ca642191ee7625d216a770e76856228bcfeefa0e82a
Status: Downloaded newer image for nicholasjackson/consul-k8s-tools:0.5.4
### Creating Kubernetes cluster, this process will take approximately 2 minutes
#### Create Kubernetes cluster in Docker using K3s


######### ERROR ###########
An error occurred on line 517, check the shipyard log at $HOME/.shipyard/shipyard.log

shipyard.log contains

$ cat $HOME/.shipyard/shipyard.log
time="2019-11-21T23:58:59Z" level=info msg="Created cluster network with ID 8f9e188b40a8ee23619433263565012747ee3b106ed993cce976e7706b514e76"
time="2019-11-21T23:58:59Z" level=info msg="As of v2.0.0 --port will be used for arbitrary port mapping. Please use --api-port/-a instead for configuring the Api Port"
time="2019-11-21T23:58:59Z" level=info msg="Created docker volume  k3d-shipyard-images"
time="2019-11-21T23:58:59Z" level=info msg="Creating cluster [shipyard]"
time="2019-11-21T23:58:59Z" level=info msg="Creating server using docker.io/rancher/k3s:v0.9.1...\n"
time="2019-11-21T23:59:06Z" level=info msg="SUCCESS: created cluster [shipyard]"
time="2019-11-21T23:59:06Z" level=info msg="You can now use the cluster with:\n\nexport KUBECONFIG=\"$(k3d get-kubeconfig --name='shipyard')\"\nkubectl cluster-info"
time="2019-11-21T23:59:06Z" level=info msg="Saving images [k8s.gcr.io/pause:3.1 coredns/coredns:1.6.3 busybox:latest kubernetesui/dashboard:v2.0.0-beta4 kubernetesui/metrics-scraper:v1.0.1 rancher/local-path-provisioner:v0.0.11 hashicorp/consul-k8s:0.9.2 consul:1.6.1 vault:1.3.0 envoyproxy/envoy-alpine:v1.10.0] from local docker daemon..."
time="2019-11-21T23:59:07Z" level=fatal msg="Helper container failed to save images -> Logs from [k3d-shipyard-tools]:\n>>>>>>\n\x02\x00\x00\x00\x00\x00\x01/2019/11/21 23:59:07 ERROR: couldn't save images [k8s.gcr.io/pause:3.1 coredns/coredns:1.6.3 busybox:latest kubernetesui/dashboard:v2.0.0-beta4 kubernetesui/metrics-scraper:v1.0.1 rancher/local-path-provisioner:v0.0.11 hashicorp/consul-k8s:0.9.2 consul:1.6.1 vault:1.3.0 envoyproxy/envoy-alpine:v1.10.0]\n\x02\x00\x00\x00\x00\x00\x005Error response from daemon: reference does not exist\n\n<<<<<<"

Shipyard consul server creation is stuck

Hi! A year late to the party, just stumbled on this repo from https://www.youtube.com/watch?v=gRpgyHfLQBI&list=PLmD8u-IFdrexJ8SlO_NcBm6nr0lQhRwZk&index=8.

When I run Shipyard with just yard up, then everything works perfectly.
However, when I use the consul-values.yaml linked here, it looks like the setup is getting stuck on the Consul server.

This is what the yard startup log looks like ( I bailed after ~15 mins )

And the shipyard.log file goes up to the consul bit then has only these lines No resources found in default namespace.

I've never dealt with consul before, just testing the waters, so any hints on how I can debug this would be awesome!