After configuring the queriers to connect to the frontend using this config

frontend_worker:
      frontend_address: cortex-query-frontend-headless.<namespace>.svc.cluster.local:9095

The queriers are not connecting and the query frontend gets into a crashloop because no queriers has connected

level=info ts=2020-07-03T14:55:35.497218784Z caller=frontend.go:454 msg="not ready: number of queriers connected to query-frontend is 0"

The queriers are not logging any errors or information about why it is not connecting.

When deploying from the latest commit, I'm getting these errors:

Error: template: cortex/templates/table-manager-servicemonitor.yaml:1:14: executing "cortex/templates/table-manager-servicemonitor.yaml" at <.Values.table_manager.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/store-gateway-servicemonitor.yaml:1:14: executing "cortex/templates/store-gateway-servicemonitor.yaml" at <.Values.store_gateway.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/ruler-servicemonitor.yaml:1:14: executing "cortex/templates/ruler-servicemonitor.yaml" at <.Values.ruler.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/query-frontend-servicemonitor.yaml:1:14: executing "cortex/templates/query-frontend-servicemonitor.yaml" at <.Values.query_frontend.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/querier-servicemonitor.yaml:1:14: executing "cortex/templates/querier-servicemonitor.yaml" at <.Values.querier.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/ingester-servicemonitor.yaml:1:14: executing "cortex/templates/ingester-servicemonitor.yaml" at <.Values.ingester.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/distributor-servicemonitor.yaml:1:14: executing "cortex/templates/distributor-servicemonitor.yaml" at <.Values.distributor.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/configs-servicemonitor.yaml:1:14: executing "cortex/templates/configs-servicemonitor.yaml" at <.Values.configs.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/compactor-servicemonitor.yaml:1:14: executing "cortex/templates/compactor-servicemonitor.yaml" at <.Values.compactor.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled
Error: template: cortex/templates/alertmanager-servicemonitor.yaml:1:14: executing "cortex/templates/alertmanager-servicemonitor.yaml" at <.Values.alertmanager.serviceMonitor.enabled>: nil pointer evaluating interface {}.enabled

The top-level values.yaml doesn't contain any default values for these serviceMonitor entries.

I was able to work around the issue by adding the following to my custom values file:

table_manager:
  serviceMonitor:
    enabled: false

store_gateway:
  serviceMonitor:
    enabled: false

ruler:
  serviceMonitor:
    enabled: false

query_frontend:
  serviceMonitor:
    enabled: false

querier:
  serviceMonitor:
    enabled: false

ingester:
  serviceMonitor:
    enabled: false

distributor:
  serviceMonitor:
    enabled: false

configs:
  serviceMonitor:
    enabled: false

compactor:
  serviceMonitor:
    enabled: false

alertmanager:
  serviceMonitor:
    enabled: false

I'd offer a PR, but I'm not fully certain what the author had in mind for these serviceMonitor entries, and it's not something that I'm using right now.

Thanks!

In order to better support running Cortex with WAL the option to run the ingesters as a statefulset is preferred.

I've been looking for some sort of direction for running cortex on GKE with bigtable as a backend store, and I just can't find anything. Since I couldn't find anything in the documentation, I was hoping that the Helm chart might have the dials I needed to turn, but I don't see them here either.

How hard would it be to get the helm chart to support GKE+Helm?

Hi:

I'm trying to put my data to aws s3 bucket

but it always shows no s3 endpoint in config file this error message event I gave config in the values.yaml

here is the config :

  storage:
    engine: blocks
    aws:
      s3: my-testbucket
      endpoint: 's3.ap-southeast-1.amazonaws.com'
      # bucketname: 
      s3forcepathstyle: false
      access_key_id: xxxxxxxxxxxxx
      secret_access_key: xxxxxxxxxxxx

Is that something missing? from the README，it's hard to know what value I should use

Add checks to compactor service to avoid creating the service if the compactor is not enabled

Problem: The chart was initially created for just AWS workloads. The values are scattered throughout the values file making it difficult to follow and make for clunky future enhancements.

Potential Solution: Centralize cloud provider settings. This should also only include templates that are applicable to the selected cloud provider and settings.

Example:

cloud:
  provider: aws
  aws:
    datastore:
      type: dynamodb
      dynamodb:
        ...
      s3:
        ...
    ...
  gcp:
    ...

Problem: The current default tolerations and affinities are too environment-specific.

It seems for any templates using Values.clusterDomain are ignored and are using cluster.local as the default value. I found this true on the follow template files:

• cortex/templates/alertmanager-dep.yaml
• cortex/templates/querier-dep.yaml
• cortex/templates/ruler-dep.yaml
• cortex/templates/nginx-config.yaml

configuration value

clusterDomain: clarson

output from chart on: cortex/templates/nginx-config.yaml

proxy_pass http://cortex-distributor.cortex.svc.cluster.local:8080$request_uri;

Using helm template generation (ignore my bash variables)

helm template cortex cortex-helm/cortex --version $CHART_VERSION -f config.yaml --create-namespace -n cortex > $APP_VERSION/install.yaml

Doesn't matter if I point the querier at frontend or frontend headless...

level=error ts=2021-01-20T23:36:23.059978229Z caller=frontend_processor.go:61 msg="error processing requests" address=172.20.19
4.35:9095 err="rpc error: code = Unavailable desc = transport is closing"

The nginx config doesn't allow an override of the DNS resolver you would prefer it to use.

https://github.com/cortexproject/cortex-helm-chart/blob/master/templates/nginx-config.yaml#L32

In our case, we're using CoreDNS. It would be nice to be able to change this. I can make a quick PR.

Problem: Log level is currently hardcoded to 'debug' and is not configurable.

The deployed resources via the helm chart are failing to start, throwing following error message:
error validating config: invalid schema config: schema config for chunks is missing the 'prefix' setting

It appears the values.yaml file is missing the chunk section inside the schema config [https://cortexmetrics.io/docs/chunks-storage/schema-configuration/#schema-config-example]

Hey,

I managed to deploy cortex with consul to the kubernetes cluster. but the injesters are failing:

consul-consul-js6d2 1/1 Running 0 19m
consul-consul-server-0 1/1 Running 0 19m
consul-consul-server-1 1/1 Running 0 19m
consul-consul-wcfgw 1/1 Running 0 19m
cortex-alertmanager-755c9b4b89-xcpcc 1/1 Running 0 18m
cortex-configs-5d5697cd86-5f2kp 1/1 Running 0 18m
cortex-distributor-59d79c45d-r6gbb 1/1 Running 0 18m
cortex-distributor-59d79c45d-sxg7j 1/1 Running 0 18m
cortex-ingester-5bddf84c86-5bvdq 0/1 CrashLoopBackOff 8 18m
cortex-ingester-5bddf84c86-jm4pj 0/1 CrashLoopBackOff 8 18m
cortex-ingester-5bddf84c86-wc66p 0/1 CrashLoopBackOff 8 18m
cortex-nginx-7777b94c7c-dsk9s 1/1 Running 0 18m
cortex-nginx-7777b94c7c-v6xvl 1/1 Running 0 18m
cortex-querier-dbdc7877-djhfp 1/1 Running 0 18m
cortex-querier-dbdc7877-qkt4n 1/1 Running 0 18m
cortex-query-frontend-5b6d7458f6-ddqm9 1/1 Running 0 18m
cortex-query-frontend-5b6d7458f6-m2qrl 1/1 Running 0 18m
cortex-ruler-6d5d87967c-855kq 1/1 Running 0 18m
cortex-table-manager-5dd895987c-6nsk7 1/1 Running 0 18m

The injester logs are:

level=info ts=2020-08-19T08:08:48.086424152Z caller=server.go:179 http=[::]:8080 grpc=[::]:9095 msg="server listening on addresses"
level=info ts=2020-08-19T08:08:48.109296185Z caller=main.go:119 msg="Starting Cortex" version="(version=1.1.0, branch=HEAD, revision=9db28c16)"
level=info ts=2020-08-19T08:08:48.109674551Z caller=module_service.go:58 msg=initialising module=runtime-config
level=info ts=2020-08-19T08:08:48.109713991Z caller=manager.go:109 msg="runtime config disabled: file not specified"
level=info ts=2020-08-19T08:08:48.109789562Z caller=module_service.go:58 msg=initialising module=memberlist-kv
level=info ts=2020-08-19T08:08:48.109892824Z caller=module_service.go:58 msg=initialising module=store
level=info ts=2020-08-19T08:08:48.110027045Z caller=module_service.go:58 msg=initialising module=ingester
level=info ts=2020-08-19T08:08:48.110459112Z caller=lifecycler.go:485 msg="not loading tokens from file, tokens file path is empty"
level=info ts=2020-08-19T08:08:48.110722767Z caller=cortex.go:336 msg="Cortex started"
level=error ts=2020-08-19T08:08:48.113373408Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.115753676Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.118156474Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.120795706Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.123158083Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.125487879Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.128252664Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.130838254Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.133505106Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=error ts=2020-08-19T08:08:48.135682611Z caller=client.go:131 msg="error getting key" key=collectors/ring err="Get http://consul:8500/v1/kv/collectors/ring?consistent=: dial tcp: lookup consul on 100.64.0.10:53: no such host"
level=warn ts=2020-08-19T08:08:48.135900014Z caller=module_service.go:88 msg="module failed with error" module=ingester err="ingester subservice failed: service &{0xc00060b0e0 {{{consul collectors/ {{consul:8500 20000000000 true 1 1} {[] 10000000000 10} { false 2000000000 0x29d3870} 0x29d5e80} } 60000000000 1} 512 5000000000 0 0 60000000000 [eth0 en0] 0 0 cortex-ingester-5bddf84c86-wc66p false 9095} 0xc000b1a000 {0xc0008abd00} 0xc00009de60 cortex-ingester-5bddf84c86-wc66p 100.123.71.104:9095 ingester ring true {{0 0} 0 0 0 0} 2 [] {0 0} {13819107869685958768 85609582 0x564a1a0} false {{0 0} 0 0 0 0} 0} failed: failed to join the ring ingester: failed to CAS collectors/ring"
level=error ts=2020-08-19T08:08:48.135959875Z caller=cortex.go:348 msg="module failed" module=ingester error="ingester subservice failed: service &{0xc00060b0e0 {{{consul collectors/ {{consul:8500 20000000000 true 1 1} {[] 10000000000 10} { false 2000000000 0x29d3870} 0x29d5e80} } 60000000000 1} 512 5000000000 0 0 60000000000 [eth0 en0] 0 0 cortex-ingester-5bddf84c86-wc66p false 9095} 0xc000b1a000 {0xc0008abd00} 0xc00009de60 cortex-ingester-5bddf84c86-wc66p 100.123.71.104:9095 ingester ring true {{0 0} 0 0 0 0} 2 [] {0 0} {13819107869685958768 85609582 0x564a1a0} false {{0 0} 0 0 0 0} 0} failed: failed to join the ring ingester: failed to CAS collectors/ring"
level=info ts=2020-08-19T08:08:48.136058047Z caller=module_service.go:90 msg="module stopped" module=runtime-config
level=info ts=2020-08-19T08:08:48.136096157Z caller=module_service.go:90 msg="module stopped" module=memberlist-kv
level=info ts=2020-08-19T08:08:48.136140288Z caller=module_service.go:90 msg="module stopped" module=store
level=info ts=2020-08-19T08:08:48.13625325Z caller=server_service.go:48 msg="server stopped"
level=info ts=2020-08-19T08:08:48.136296721Z caller=cortex.go:337 msg="Cortex stopped"
level=error ts=2020-08-19T08:08:48.136353082Z caller=log.go:140 msg="error running cortex" err="failed services\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).Run\n\t/go/src/github.com/cortexproject/cortex/pkg/cortex/cortex.go:388\nmain.main\n\t/go/src/github.com/cortexproject/cortex/cmd/cortex/main.go:121\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"

Are there any changes that need to be made to the consul helm chart or the cortex helm chart to make it work?
Thanks !

Thanks for the efforts and contributions on this repo and it is going to help us a lot.
How can I specify ruler storage configuration using this helm chart?
https://cortexmetrics.io/docs/configuration/configuration-file/#ruler_config

Hey,

I am trying to install cortex using helm, running the command:
helm install cortex --name cortex --namespace cortex <path-to-cortex-helm-chart>
returns error:
Error: unknown flag: --name
this is because since helm v3 the name flag was removed.

Removing --name and running:
helm install cortex --namespace cortex /local-path/cortex-helm-chart
returns another error:
Error: found in Chart.yaml, but missing in charts/ directory: memcached, memcached, memcached

Any help with this is much appreciated!
Thanks.

### UPDATE:
After running:
helm dependency update <path-to-cortex-helm-chart>

and then trying to run helm install cortex I now get a new error:
Error: template: cortex/templates/table-manager-dep.yaml:102:7: executing "cortex/templates/table-manager-dep.yaml" at <eq .Values.useExternalConfig>: error calling eq: missing argument for comparison

https://github.com/cortexproject/cortex-helm-chart/releases/tag/v0.3.0
The release is called "0.2.0" while the tag for the relase is "0.3.0"

Plain Helm install with the chart is failing on the dependencies in the chart.yaml file:
My-MacBook-Pro % helm install cortex --namespace cortex -f values.yaml /Users/vikramyerneni/Desktop/repos/cortex-helm-chart Error: found in Chart.yaml, but missing in charts/ directory: memcached, memcached, memcached My-MacBook-Pro%

And the path "https://kubernetes-charts.storage.googleapis.com" in the file don't have any memcached .tgz files or associated path.

Lots of changes are currently in the master branch, which shows a chart version of 0.2.1, but this helm repo only offers version 0.2.0

❯ helm search repo cortex
NAME            CHART VERSION   APP VERSION     DESCRIPTION
cortex/cortex   0.2.0           v1.4.0          Cortex

Please publish the latest version of this chart to the helm chart repo: https://cortexproject.github.io/cortex-helm-chart

I'm trying to setup blocks_storage with the helm chart. I have the blocks_storage config block setup for s3 based on some other examples I came across here. And I configured etcd as the kvstore.

The issue I'm facing is:
level=error ts=2021-01-29T13:19:07.934930043Z caller=log.go:150 msg="error running cortex" err="failed to initialize querier for engine 'blocks': no store-gateway address configured

I have tried setting the store_gateway_addresses to cortex-store-gateway-headless:9095 (I have simplified the name here). This brings the pods up but they fail to actually connect:

level=error ts=2021-01-29T15:35:51.351902357Z caller=frontend_processor.go:61 msg="error processing requests" address=10.27.74.198:9095 err="rpc error: code = Unavailable desc = transport is closing"

What else might I be missing?

The new config can be passed in via the "config-yaml" argument. When set, the legacy arguments will be ignored. Review schema_config.go for further details.

Greetings,

Thanks for creating this chart! It helped our team go from knowing Cortex was available, to standing it up in about a day's work, but I think aligning this chart with other official Cortex docs would have got us up faster - I'm curious on your ideas to make it better.

Naming conventions in values.yaml don't correlate 1:1 with the official Cortex architecture diagram and make it hard to parse what is optional for an MVP Cortex stack, and what is required. There's also similar names for top-level keys (config vs configs). Based on that diagram, I expect values.yaml to have the following required top-level keys:

• config_file (renamed from config, with a link in the values file to this page, and a blurb about how anything specified here is rawly interpreted)
• nginx
• distributor
• ingester
• query_frontend
• querier
• table_manager

Additionally, the following keys should exist, with a sub-key of enabled and a default of false:

• configs_api (renamed from configs)
• ruler
• alertmanager

What do you think?

Cortex has switched to using a single binary ( quay.io/cortexproject/cortex ) and uses a target flag to determine the service (e.g. -target=ingester ).

Problem: Currently image tags are duplicated throughout the values file.

Potential Solution: Provide a top-level image tag for defining the Cortex version.

Problem: This chart is out of date and should be updated to the latest version of Cortex before adding new options and features.

https://github.com/cortexproject/cortex/releases/tag/v0.3.0

Greetings,

The current chart does not support all ingress controller schemas, including AWS Load Balancer Controller. The main limitation currently is InstanceIP mode, where Ingress objects must be backed by a NodePort type Service. It'd be a good idea to also support additional paths in the Ingress object so SSL redirection can happen.

Might I suggest adding the following values:

1. nginx.service.type: sets the type on the Service fronting nginx
2. ingress.extraPaths: raw YAML for additional paths to be added to the Ingress - here's how grafana did this

It would be good to have some documentation on the values on what they do etc. For example https://github.com/helm/charts/tree/master/stable/prometheus#configuration this is a really nice way of documenting values showing how to set each value (YAML path), it's function & the default values.

If using local file storage for alertmanager the silences are stored on disk and lost if the pod is recreated, by having the ability to mount a pvc volume these silences can persist even if the pod is recreated.

By running alertmanager as a statefulset also makes it easier to run alertmanager as a cluster since each cluster peer needs to be explicitly specified.

Problem: Istio specific pod annotations are present on all pods and should be removed to make it clear that the deployments do not rely on Istio.

The helm chart currently does not allow usage of externally created/managed secrets for retrieving the config.yaml.

Configured this chart for blocks storage engine with S3 backend and was seeing failed to open TSDB: tsdb/fake: mkdir tsdb: read-only file system on ingester Pods.
A short investigation led me to believe that the PodSecurityPolicy does not allow ingester to create the /tsdb dir due to readOnlyRootFs being set.

My workaround was adding an emptyDir volume to the ingester Deployment and mounting it on /tsdb which appears to have fixed the problem. Not sure if this is the correct way, but it's broken "out of the box" as of 0.2.0 of this chart.

Hi,

Can you please help how cortex helm template can be used for connecting to cassandra. I wanted to connect all the cortex components whichever needs a Cassandra connect to pull data or push data into cassandra.

As nginx uses http 1.0 by default, which is very old and istio service mesh wont support 1.0 without making changes to istio config. Instead of changing istio config we can set 1.1 by default in nginx config file.

   server { # simple reverse-proxy
     listen 80;
     proxy_set_header X-Scope-OrgID 0;
    proxy_http_version 1.1;

I'm trying to install cortex with block storage and I'm getting the error below, please let me know what I'm doing wrong. thanks

level=info ts=2020-10-07T04:51:48.651140802Z caller=server.go:179 http=[::]:8080 grpc=[::]:9095 msg="server listening on addresses"
level=error ts=2020-10-07T04:51:48.651791474Z caller=log.go:140 msg="error initializing cortex" err="Must set -dynamodb.url in aws mode\nerror creating index client\ngithub.com/cortexproject/cortex/pkg/chunk/storage.NewStore\n\t/go/src/github.com/cortexproject/cortex/pkg/chunk/storage/factory.go:153\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).initStore\n\t/go/src/github.com/cortexproject/cortex/pkg/cortex/modules.go:288\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).initModuleServices\n\t/go/src/github.com/cortexproject/cortex/pkg/cortex/cortex.go:298\ngithub.com/cortexproject/cortex/pkg/cortex.New\n\t/go/src/github.com/cortexproject/cortex/pkg/cortex/cortex.go:237\nmain.main\n\t/go/src/github.com/cortexproject/cortex/cmd/cortex/main.go:116\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357\nerror initialising module: store\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).initModuleServices\n\t/go/src/github.com/cortexproject/cortex/pkg/cortex/cortex.go:300\ngithub.com/cortexproject/cortex/pkg/cortex.New\n\t/go/src/github.com/cortexproject/cortex/pkg/cortex/cortex.go:237\nmain.main\n\t/go/src/github.com/cortexproject/cortex/cmd/cortex/main.go:116\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"

-- values-dev.yaml--
storage:
engine: chunks
aws:
s3: "https://cortext-storage.s3.amazonaws.com"
bucketnames: "cortex-storage"

-- install command --
helm upgrade --install cortex --namespace cortex eks/helm/clop/cortex --values eks/helm/clop/cortex/values-dev.yaml

It would be nice to have a bit more flexibility in the nginx config for example to define additional directives in the http and/or server context. This could be achieved by adding something like .Values.nginx.config.additionalHttpConfig / .Values.nginx.config.additionalServerConfig to pass additional directives or by defining a value .Values.nginx.configRaw (not sure about the naming of the values) which lets the user overwrite the entire nginx config. Is this something you'd be interested to have in the chart or should we better deploy nginx separately in such a case?

Thanks!

Greetings,

I'm a new user of the chart and also new to Cortex, it took me a bit of digging to figure out memberlist was a standalone option for kvstore, and I wanted to offer some ideas about it:

memberlist isn't mentioned at the top of the README - which states an external kvstore is "required"; this appears false when using memberlist. I might suggest:
- In addition to naming specific kvstores in the README, it might be best to link people to the cortex docs on the issue. It's worth mentioning in the README that memberlist (while it has limitations) does not require additional architecture outside this chart for an MVP.
- Making memberlist (instead of consul) the default kvstore in the chart's values means this chart stands on its own; while a breaking change, I think that's pretty valuable to get new people to use the chart. This gets people to MVP with the least amount of configuration possible (all they should need is a configuration for storage) - if they want all the bells and whistles (from what I can read, HA/scalability improvements?), they can continue forward from something that already works.

What are your ideas on this?

The repository in the Chart.yaml is deprecated and removed a few days ago.
Helm suggested pointing to the new https://charts.helm.sh/stable

After editing the file and running the helm dependency update everything seems to work fine, and deployment works.

I opened a pull request to fix it for everyone.

Can I create serviceaccount for dynamodb and s3 for giving it IAM role? Just need an annotation

In our setup we don't need the nginx component since we have our own proxy components in front of Cortex.

I removed schema and table manager config from values.yaml file but i am seeing below errors.

error validating config: invalid schema config: schema config for chunks is missing the 'prefix' setting

Why cortex is looking for schema config when it is not required when using block storage engine ?

Seems like 93baa61 introduced syntax errors in Go template expressions.

When cloning the repo, then invoking

$ helm dependency update
$ helm install cortex --namespace cortex /path/to/working/copy

I receive the following output:

Error: template: cortex/templates/table-manager-dep.yaml:102:7: executing "cortex/templates/table-manager-dep.yaml" at <eq .Values.useExternalConfig>: error calling eq: missing argument for comparison

The named code location looks as follows:

{{- if eq .Values.useExternalConfig }}
            secretName: {{ .Values.externalConfigSecretName }}
{{- else }}
            secretName: {{ template "cortex.fullname" . }}
{{- end }}

According to text/template documentation, eq does indeed expect a second argument, and I guess it should simply be {{- if .Values.useExternalConfig }}.

Alertmanager logs says
"Failed to resolve cortex-alertmanager-alertmanager-1.cortex-alertmanager-alertmanager.alertmanager.svc.cluster.local:9094: lookup no such host"
Root cause,
The servicename used in alertmanager-statefulset.yaml is of cluster endpoint service.
To fix the issue it should use headless service which will resolve the dns of individual pods by hostname.

Will raise a PR for the same.

There are 3 places where limit is used instead of limits in values.yaml causing helm to fail to deploy this chart.

Currently, the helm chart only supports the Chunks storage mode for cortex. The helm chart should also support configurations and components for the use of blocks storage.

I forked this chart and prepared an updated chart compatible with cortex v1.0.0 and k8s 1.16, I am using it to deploy cortex with cassandra and azure blob storage backend, it should work also on aws (dynamodb and/or s3).
If interested, I can submit a pull request.

I'm attempting to use Azure Blob Storage for a deployment - using this config:

ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: nginx
  hosts:
    - host: cortex.dev.foo.com
      paths:
        - /
  tls: []
config:
  ingester:
    lifecycler:
      ring:
        kvstore:
          consul:
            host: "cortex-consul.cortex:8500"
  schema:
     configs: []
  storage:
    engine: blocks
    azure:
      container_name: cortex
      account_name: mystorageaccount
      account_key: REDACTED

It's apparently using s3 by default, and I don't see anywhere to alter that

Problem: Query Frontend is currently not utilized in this chart.

Setting the tag to true does not create the memcached-blocks charts:

tags:
  blocks-storage-memcached: true

Can we just use the condition bool instead?

❯ helm version
version.BuildInfo{Version:"v3.5.1", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"dirty", GoVersion:"go1.15.7"}

We are currently trying to run cortex in eks using this helm chart but we are hitting an issue we can't get past.
Our ingesters won't move to a healthy state. They keep receiving a 503 from the /ready endpoint.
We tried both deployment and stateful set versions. We use consul as kv store. The logs indicate no errors.

level=debug ts=2020-10-01T14:25:34.57219078Z caller=api.go:93 msg="api: registering route" methods= path=/config auth=false
level=debug ts=2020-10-01T14:25:34.572454703Z caller=api.go:93 msg="api: registering route" methods= path=/ auth=false
level=debug ts=2020-10-01T14:25:34.573142542Z caller=api.go:93 msg="api: registering route" methods= path=/ingester/flush auth=false
level=debug ts=2020-10-01T14:25:34.573655461Z caller=api.go:93 msg="api: registering route" methods= path=/ingester/shutdown auth=false
level=debug ts=2020-10-01T14:25:34.573728608Z caller=api.go:93 msg="api: registering route" methods= path=/ingester/push auth=true
level=debug ts=2020-10-01T14:25:34.573770878Z caller=api.go:93 msg="api: registering route" methods= path=/flush auth=false
level=debug ts=2020-10-01T14:25:34.573798376Z caller=api.go:93 msg="api: registering route" methods= path=/shutdown auth=false
level=debug ts=2020-10-01T14:25:34.573845679Z caller=api.go:93 msg="api: registering route" methods= path=/push auth=true
level=debug ts=2020-10-01T14:25:34.573877311Z caller=api.go:93 msg="api: registering route" methods= path=/services auth=false
level=info ts=2020-10-01T14:25:34.573899803Z caller=main.go:119 msg="Starting Cortex" version="(version=1.1.0, branch=HEAD, revision=9db28c16)"
level=debug ts=2020-10-01T14:25:34.574059574Z caller=module_service.go:48 msg="module waiting for initialization" module=ingester waiting_for=memberlist-kv
level=info ts=2020-10-01T14:25:34.589200835Z caller=module_service.go:58 msg=initialising module=runtime-config
level=info ts=2020-10-01T14:25:34.58939623Z caller=manager.go:109 msg="runtime config disabled: file not specified"
level=info ts=2020-10-01T14:25:34.589766741Z caller=module_service.go:58 msg=initialising module=memberlist-kv
level=info ts=2020-10-01T14:25:34.589801774Z caller=module_service.go:58 msg=initialising module=store
level=debug ts=2020-10-01T14:25:34.589825403Z caller=module_service.go:48 msg="module waiting for initialization" module=ingester waiting_for=store
level=debug ts=2020-10-01T14:25:34.589837603Z caller=module_service.go:48 msg="module waiting for initialization" module=ingester waiting_for=runtime-config
level=info ts=2020-10-01T14:25:34.58984705Z caller=module_service.go:58 msg=initialising module=ingester
level=info ts=2020-10-01T14:25:34.589923423Z caller=lifecycler.go:485 msg="not loading tokens from file, tokens file path is empty"
level=info ts=2020-10-01T14:25:34.590464682Z caller=cortex.go:336 msg="Cortex started"
level=info ts=2020-10-01T14:25:34.610024694Z caller=lifecycler.go:509 msg="instance not found in ring, adding with no tokens" ring=ingester
level=debug ts=2020-10-01T14:25:34.680082221Z caller=lifecycler.go:357 msg="JoinAfter expired" ring=ingester
level=info ts=2020-10-01T14:25:34.680133566Z caller=lifecycler.go:361 msg="auto-joining cluster after timeout" ring=ingester
level=debug ts=2020-10-01T14:26:43.877990081Z caller=logging.go:44 traceID=1eb1ec8aa289a159 msg="GET /ready (503) 4.166397ms"

all other pods start and reach their ready state however in their logs they have errors about the ingestors.

for example the logs from querier.

level=error ts=2020-10-01T14:32:48.382107979Z caller=pool.go:161 msg="error removing stale clients" err="too many failed ingesters"
level=error ts=2020-10-01T14:33:03.382028125Z caller=pool.go:161 msg="error removing stale clients" err="too many failed ingesters"
level=error ts=2020-10-01T14:33:18.381983644Z caller=pool.go:161 msg="error removing stale clients" err="too many failed ingesters"
level=error ts=2020-10-01T14:33:33.382013304Z caller=pool.go:161 msg="error removing stale clients" err="too many failed ingesters"
level=error ts=2020-10-01T14:33:48.382010272Z caller=pool.go:161 msg="error removing stale clients" err="too many failed ingesters"