tak2siva / envoy-pilot

Envoy xDS Server with Consul

Envoy-Pilot

Envoy Pilot or Envoy xDS is a control plane implementation for Envoy written in Golang and uses Consul for persistence by default. It can also run without Consul by loading configuration from file.

This is an extension of go-control-plane

Currently Supports

• CDS
• LDS
• RDS
• EDS
• ADS (for CDS & LDS)

Note: Some infrequent configurations might not be mapped. Feel free to PR

Checkout Envoy XDS PROTOCOL Overview for more detail

Also Checkout Sample Project

• Envoy xDS Example From File
• Envoy xDS Example With Consul

File Config

Checkout the above example to load config from file

Consul Usage

xDS Server will be exposed on port 7777

Run Envoy Proxy with the following configurations or use --service-node && --service-cluster

node:
  id: ride-service-replica-2
  cluster: ride-service

Every *DS requires two keys to be set in consul

• config
• version

And the key template is xDS/app-cluster/ride-service/CDS/(config|version)

For CDS add KV pairs

• xDS/app-cluster/ride-service/CDS/version => "1.0"
• xDS/app-cluster/ride-service/CDS/config => "[{ { "name": "app1", "connect_timeout": "0.250s", "type": "STRICT_DNS", "lb_policy": "RANDOM", "hosts": [{ "socket_address": { "address": "127.0.0.2", "port_value": 1234 } }] } }]"

Pushing new configuration

• Envoy-Pilot will be polling for version change every 10 seconds.
• If there is a version mismatch for any of xDS/app-cluster/ride-service/(CDS|LDS|RDS|EDS)/version then new config xDS/app-cluster/ride-service/(CDS|LDS|RDS|EDS)/config will be pushed to subscriber envoy.
• If update succeed there will be an ACK log for the instance.

Running Docker Compose

From root directory

docker network create envoy-pilot_xds-demo
docker-compose -f docker-compose.consul.yaml up
docker-compose -f docker-compose.server.yaml up --build

Running test

cd test/integration
docker-compose up --build

cd test/rspec
DEVMODE=true rspec basic_spec.rb --order defined --format documentation

Runnnig Docker

Consul url need to be set in .env

env_values.txt

CONSUL_PATH="http://consul-server:8500"
CONSUL_PREFIX=""xDS"

Docker run

docker run -v $(pwd)/env_values.txt:/.env -p 7777:7777 -p 9090:9090 tak2siva/envoy-pilot:latest

Helm Chart

Install using the Helm Chart for Envoy-Pilot.

Debugging

• xDS-Server is running on port 7777
• A http server is running on port 9090 for debugging

localhost:9090/dump/KEY_TEMPLATE will give a json dump of proto mapping

Ex:

http://localhost:9090/dump/xDS/app-cluster/ride-service/(CDS|LDS|RDS|EDS)/config

• To get list of subcsribers hit localhost:9090/dump/subscribers/

Prometheus metrics

Prometheus is running on localhost:8081/metrics and the following stats are available

• xds_active_connections[cluster] (GAUGE)
• xds_active_subscribers[cluster][type] (GAUGE)
• xds_update_counter[cluster][subscribedTo] (Counter)

envoy-pilot's Issues

Migrating from Envoy v2 to v3 services/filters

Migrating from Envoy v2 to v3 services/filters.

DEVMODE=true rspec ./basic_spec.rb --order defined --format documentation

An error occurred while loading ./basic_spec.rb.
Failure/Error: return gem_original_require(path)

LoadError:
cannot load such file -- plissken

./basic_spec.rb:3:in `<top (required)>'

No examples found.

Finished in 0.00108 seconds (files took 0.63475 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

Deployed in the k8s cluster using the pilot, it is found that if the automatic expansion and contraction operation is performed, the pilot will prompt an error and the context is canceled.

Add support for DeltaAggregatedResources in XDS

Add support for each `http_filter` in LDS

Ref the Issue

List of filters

EDS in filemode working correctly?

EDS in filemode working correctly?

Cluster tls_context config not sent to Envoy

Hi,

I'm testing Envoy-Pilot as control-plane for envoys using the File Config method.

All looks good except I'm having problems supplying tls_context config through cds. This results in envoys using http to the backend, when I want them to use https.

The same configuration works just fine when configuring envoys statically.

My CDS.yaml file looks like this:

- name: service1
  connect_timeout: 15.0s
  type: STATIC
  dns_lookup_family: V4_ONLY
  lb_policy: ROUND_ROBIN
  hosts: 
  - socket_address: 
      address: 10.10.10.20 
      port_value: 443
  tls_context:
    sni: www.examplehost.com

Config_dump from enoy does not include tls_context config.

   "@type": "type.googleapis.com/envoy.admin.v2alpha.ClustersConfigDump",
   "version_info": "bha00nbfmjktkn861csg",
   "static_clusters": [
    {
     "cluster": {
      "name": "xds_cluster",
      "type": "STRICT_DNS",
      "connect_timeout": "0.250s",
      "hosts": [
       {
        "socket_address": {
         "address": "10.10.10.10",
         "port_value": 7777
        }
       }
      ],
      "http2_protocol_options": {},
      "dns_refresh_rate": "5s"
     },
     "last_updated": "2019-02-01T08:16:29.946Z"
    }
   ],
   "dynamic_active_clusters": [
    {
     "version_info": "bha00nbfmjktkn861csg",
     "cluster": {
      "name": "service1",
      "connect_timeout": "15s",
      "hosts": [
       {
        "socket_address": {
         "address": "10.10.10.20",
         "port_value": 443
        }
       }
      ]
     },
     "last_updated": "2019-02-01T08:16:29.953Z"
    }
   ]
  },```

Is what I'm trying to archive possible through cds, or is there something wrong with my syntax in the CDS.yaml file?