yuraxdrumz / prometheus-multi-tenant-proxy

Prometheus multi-tenant Proxy. Needed to deploy Prometheus in a multi-tenant way

Prometheus — Multi-tenant proxy

---

This project aims to make it easy to deploy a Prometheus Server in a multi-tenant way.

This project has some reference from the prometheus label injector

The proxy enforces the namespace label in a given PromQL query while providing a basic auth layer.

What is it?

It is a simple golang proxy. It does basic auth, logs the requests, and serves as a Prometheus reverse proxy.

Actually, Prometheus does not check the auth of any request. By itself, it does not provide any multi-tenant mechanism. So, if you have untrusted tenants, you have to ensure a tenant uses its labels and does not use any other tenants' value.

Requirements

To use this project, place the proxy in front of your Prometheus server instance, configure the auth proxy configuration and run it.

Run it

$ prometheus-multi-tenant-proxy run --prometheus-endpoint http://localhost:9090 --port 9091 --auth-config ./my-auth-config.yaml --reload-interval=5 --unprotected-endpoints /-/healthy,/-/ready

Where:

• --port: Port used to expose this proxy.
• --prometheus-endpoint: URL of your Prometheus instance.
• --auth-config: Authentication configuration file path.
• --reload-interval: Interval in minutes to reload the auth config file.
• --unprotected-endpoints: Comma separated list of endpoints that do not require authentication.

Configure the proxy

The auth configuration is straightforward. Just create a YAML file my-auth-config.yaml with the following structure:

// Authn Contains a list of users
type Authn struct {
	Users []User `yaml:"users"`
}

// User Identifies a user including the tenant
type User struct {
	Username   string   `yaml:"username"`
	Password   string   `yaml:"password"`
	Namespace  string   `yaml:"namespace"`
	Namespaces []string `yaml:"namespaces"`
}

An example is available at configs/multiple.user.yaml file:

users:
  - username: User-a
    password: pass-a
    namespace: tenant-a
  - username: User-b
    password: pass-b
    namespace: tenant-b

or if you need to allow multiple namespaces for a single user, an example is available at configs/multiple.namespaces.yaml file:

users:
  - username: Happy
    password: Prometheus
    namespace: default
  - username: Sad
    password: Prometheus
    namespace: kube-system
  - username: Multiple
    password: Namespaces
    namespace: monitoring
    namespaces:
      - default
      - kube-system
      - kube-public
  - username: Multiple
    password: NamespacesWithoutNamespace
    namespaces:
      - default
      - kube-system
      - kube-public

A tenant can contain multiple users. But a user is tied to a single tenant.

Build it

If you want to build it from this repository, follow the instructions bellow:

$ docker run -it --entrypoint /bin/bash --rm golang:1.18.8-buster
root@9b2da74fb4b8:/go# git clone https://github.com/k8spin/prometheus-multi-tenant-proxy.git
Cloning into 'prometheus-multi-tenant-proxy'...
remote: Enumerating objects: 310, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 310 (delta 6), reused 14 (delta 6), pack-reused 295
Receiving objects: 100% (310/310), 255.71 KiB | 1.66 MiB/s, done.
Resolving deltas: 100% (122/122), done.
root@9b2da74fb4b8:/go# cd prometheus-multi-tenant-proxy/cmd/prometheus-multi-tenant-proxy/
root@9b2da74fb4b8:/go/prometheus-multi-tenant-proxy/cmd/prometheus-multi-tenant-proxy# go build
go: downloading github.com/urfave/cli/v2 v2.24.3
go: downloading github.com/prometheus/prometheus v0.42.0
go: downloading github.com/prometheus-community/prom-label-proxy v0.6.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/efficientgo/tools/core v0.0.0-20220225185207-fe763185946b
go: downloading github.com/go-openapi/runtime v0.25.0
go: downloading github.com/go-openapi/strfmt v0.21.3
go: downloading github.com/metalmatze/signal v0.0.0-20210307161603-1c9aa721a97a
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/prometheus/alertmanager v0.25.0
go: downloading github.com/prometheus/client_golang v1.14.0
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2
go: downloading github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673
go: downloading github.com/opentracing/opentracing-go v1.2.0
go: downloading go.opentelemetry.io/otel v1.11.2
go: downloading go.opentelemetry.io/otel/trace v1.11.2
go: downloading github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d
go: downloading github.com/go-openapi/errors v0.20.3
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading github.com/oklog/ulid v1.3.1
go: downloading go.mongodb.org/mongo-driver v1.11.0
go: downloading github.com/cespare/xxhash/v2 v2.2.0
go: downloading github.com/grafana/regexp v0.0.0-20221122212121-6b5c0a4cb7fd
go: downloading github.com/prometheus/common v0.39.0
go: downloading github.com/go-openapi/swag v0.22.3
go: downloading github.com/go-openapi/validate v0.22.0
go: downloading github.com/go-openapi/analysis v0.21.4
go: downloading github.com/go-openapi/loads v0.21.2
go: downloading github.com/go-openapi/spec v0.20.7
go: downloading github.com/russross/blackfriday/v2 v2.1.0
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/golang/protobuf v1.5.2
go: downloading github.com/prometheus/client_model v0.3.0
go: downloading github.com/prometheus/procfs v0.8.0
go: downloading google.golang.org/protobuf v1.28.1
go: downloading github.com/go-logr/logr v1.2.3
go: downloading github.com/go-kit/log v0.2.1
go: downloading golang.org/x/exp v0.0.0-20230124195608-d38c7dcee874
go: downloading github.com/mailru/easyjson v0.7.7
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/go-openapi/jsonpointer v0.19.5
go: downloading github.com/go-openapi/jsonreference v0.20.0
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4
go: downloading github.com/go-logr/stdr v1.2.2
go: downloading golang.org/x/sys v0.4.0
go: downloading github.com/dennwc/varint v1.0.0
go: downloading go.uber.org/atomic v1.10.0
go: downloading github.com/stretchr/testify v1.8.1
go: downloading github.com/go-logfmt/logfmt v0.5.1
go: downloading github.com/josharian/intern v1.0.0
go: downloading github.com/gogo/protobuf v1.3.2
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading go.uber.org/goleak v1.2.0
go: downloading github.com/pmezard/go-difflib v1.0.0
root@9b2da74fb4b8:/go/prometheus-multi-tenant-proxy/cmd/prometheus-multi-tenant-proxy# ./prometheus-multi-tenant-proxy
NAME:
   Prometheus multi-tenant proxy - Makes your Prometheus server multi tenant

USAGE:
   Prometheus multi-tenant proxy [global options] command [command options] [arguments...]

VERSION:
   dev

AUTHORS:
   Angel Barrera <[email protected]>
   Pau Rosello <[email protected]>

COMMANDS:
   run      Runs the Prometheus multi-tenant proxy
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

Build the container image

If you want to build a container image with this proxy, run:

$ docker build -t prometheus-multi-tenant-proxy:local -f build/package/Dockerfile .

After built, just run it:

$ docker run --rm prometheus-multi-tenant-proxy:local

Using this project at work or in production?

See ADOPTERS.md for what companies are doing with this project today.

License

The scripts and documentation in this project are released under the GNU GPLv3