Comments (23)
thanks for opening this @jacobtomlinson. Just want to say i'm in favor of this and would be happy to dedicate some time for generalizing and testing to give the chart a new home! For starters, would this essentially be defined under a subfolder in this repository and installed as helm install --name my-release dask/dask-jupyterhub
?
from helm-chart.
from helm-chart.
Thanks @jacobtomlinson for opening this discussion. Just chiming in to say that I would support this such a move, if it finds support here and that, if a Dask JupyterHub/Gateway chart existed, we would gladly focus our development and maintenance energies here.
from helm-chart.
ping @dask/maintenance
from helm-chart.
I am generally in favour, but have to trust your assessment on how much of a maintenance burden this would be - I assume more than updating dask/distributed versions.
from helm-chart.
@martindurant it is effectively a meta chart so will require updating the versions of the two charts it depends on, one of them being Dask Gateway so we maintain that anyway. I'm happy that this can also be automated.
from helm-chart.
I am convinced
from helm-chart.
I think I'm also onboard with the general idea of us maintaining a multi-user Dask + Jupyter helm chart, powered by Dask-Gateway & Jupyterhub.
The helm chart itself probably needs some updating. Over time it's become fairly opinionated and tailored to pangeo's needs & deployments. For example, I think the tolerations at https://github.com/pangeo-data/helm-chart/blob/b1a230a88d2587713eb440c9de402770f6ae32e6/pangeo/values.yaml#L66-L87 effectively require a kubernetes cluster with node pools that are able to handle those.
from helm-chart.
@TomAugspurger - small detail but this:
For example, I think the tolerations at https://github.com/pangeo-data/helm-chart/blob/b1a230a88d2587713eb440c9de402770f6ae32e6/pangeo/values.yaml#L66-L87 effectively require a kubernetes cluster with node pools that are able to handle those.
is not quite right. This just set's the worker/scheduler pod tolerations to match those of dask-kubernetes (ref). We may actually want to discuss this as a fixture in dask-gateway but it doesn't require cluster nodes to have a matching taint.
from helm-chart.
I looked into this a bit yesterday. I think the biggest things to decide on are
- How to structure the repository (one chart, with flags to enable / disable jupyterhub / dask-gateway, or two separate charts)
- If we have two charts, what do we name them?
helm install dask
gives you the same thing as today (Dask with dask-kubernetes and a single jupyterlab / notebook server).helm install ...
gives you JupyterHub + Dask Gateway for multiple users and clusters. Should we continue to call that "pangeo"? "dask-multiuser"? "daskhub"?
from helm-chart.
A couple additional things I'm struggling with, to make a helm install dask-multiuser
thing work out of the box.
- Assuming we want to piggy back on JupyterHub auth, it's difficult to get this working out of the box. We need to set the value
jupyterhub.hub.services.dask-gateway.url
to the rendered value ofhttp://traefik-{{ .Release.Name }}-dask-gateway.{{ .Release.Namespace }}
. However, helm doesn't evaluate values in the chart'svalues.yaml
. Does anyone know if there's an easy way to set a rendered value in a subchart? Everything I've tried has come up empty.
Alternatively, we don't proxy through a JupyterHub service, and instead expose the gateway publicly? Pangeo might be moving to that to open up some non-JupyterHub-based workflows. Then things are fine.
- (Less critical than 1): it'd be nice to configure the singleuser environment to have the right address & proxy address. I've had trouble with that for the same reason, they depend on values that I don't think are known the first time we do a
helm install
.
from helm-chart.
My preference would be to not call it pangeo
. One of the reasons we are moving it here is because it's pretty generic and will be useful to other communities. I like dask-multiuser
and daskhub
or dask-hub
.
I would aim for one chart with configurable options for gateway or no gateway.
I think one way to do things would be to create a config map which gets rendered with the same template and then mount the config map as environment variables in order to configure things. Happy to help/pair on it.
from helm-chart.
create a config map which gets rendered with the same template and then mount the config map as environment variables in order to configure things.
I think I tried this earlier and failed. The rendering does work, but then I had trouble with mounting the configmap as a volume. I'll give it one more shot today and reach out if I have issues.
from helm-chart.
You may have a better experience mounting the config options as environment variables rather than a volume.
from helm-chart.
Indeed, I was just looking at that page and this looks much better. Trying it out now. JupyterHub didn't seem to like my first attempt:
File "/usr/local/lib/python3.6/dist-packages/traitlets/traitlets.py", line 1524, in error
raise TraitError(e)
traitlets.traitlets.TraitError: The 'environment' trait of a PatchedKubeSpawner instance must be a dict, but a value of class 'list' (i.e. [{'name': 'DASK_GATEWAY__AUTH__TYPE', 'valueFrom': {'configMapKeyRef': {'key': 'DASK_GATEWAY__AUTH__TYPE', 'name': 'daskhub-config'}}}, {'name': 'DASK_GATEWAY__ADDRESS', 'valueFrom': {'configMapKeyRef': {'key': 'DASK_GATEWAY__ADDRESS', 'name': 'daskhub-config'}}}]) was specified.
jupyterhub:
singleuser:
extraEnv:
- name: DASK_GATEWAY__AUTH__TYPE
valueFrom:
configMapKeyRef:
name: daskhub-config
key: DASK_GATEWAY__AUTH__TYPE
- name: DASK_GATEWAY__ADDRESS
valueFrom:
configMapKeyRef:
name: daskhub-config
key: DASK_GATEWAY__ADDRESS
from helm-chart.
Hmm that may be a limitation of KubeSpawner
.
from helm-chart.
hub.extraEnv
does note that it can take a list of EnvVar objects https://zero-to-jupyterhub.readthedocs.io/en/latest/reference/reference.html#hub-extraenv. I think that's at https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/08c13609c1d0c6cb07d45d49d0a876100cf941eb/jupyterhub/templates/hub/deployment.yaml#L208-L210.
singleuser.extraEnv
doesn't have any similar note, and I can't find where it would be handled.
from helm-chart.
@consideRatio does #68 (comment) sound right to you? That there isn't a simple way to pass singleuser environment variables as a list of EnvVar
objects?
I think one possible solution would be for jupyterhub to run the user-provided extraEnv
through the tpl
function to render templates, as discussed in https://hackernoon.com/the-art-of-the-helm-chart-patterns-from-the-official-kubernetes-charts-8a7cafa86d12. I think that would allow us to provide a default values.yaml
like
jupyterhub:
singleuser:
extraEnv:
DASK_GATEWAY__ADDRESS: "traefik-{{ .Release.Name }}-dask-gateway"
That would ensure that the singleuser pod gets the rendered form, rather than the template placeholders.
from helm-chart.
@TomAugspurger ah, I see nice that it was possible to use at all! There is an issue with using lists as they will get overridden rather then merged when you have two different configuration files specifying hub.extraEnv. And, since this is a Helm chart to be distributed for use by others, its very likely that users will end up thinking that they can configure hub.extraEnv themselves, but end up overriding your configuration.
I suggest making a inline comment of that issue for now, and awaiting an improvement in z2jh that allows hub.extraEnv to be a dict that can hold k8s EnvVar
objects.
from helm-chart.
Ah, I didn't realize that it would be overwritten, but that does make sense.
I'll give an alternative method one more shot, and if that doesn't work then I'll add notes saying how to add additional hub.extraEnv
values.
from helm-chart.
I read through the article and considered various options, and I think the way to go for a chart to be distributed for other users etc is to rely on a extraSomething being an object which keys represents the extraSomething elements to be added. While z2jh supports this, it assumes that you pass a string as a value rather than an entire EnvVar object which should also be made supported.
To conclude, fixing jupyterhub/zero-to-jupyterhub-k8s#1103 is what I would recommend. Hmm.. I'll go ahead and spend time doing that now.
from helm-chart.
Thanks for looking into jupyterhub/zero-to-jupyterhub-k8s#1103. I do think fixing that would generally be useful, but we may have a workaround in #94 in this case :)
from helm-chart.
Thanks @TomAugspurger for pushing this forward!
from helm-chart.
Related Issues (20)
- Daskhub `additional_worker_groups` configuration HOT 2
- Additional worker group quirks HOT 4
- [BUG] Using a hostname to expose Jupyter causes Helm Render to fail
- Release 2022.5.0 HOT 13
- Hourly check of upstream Docker image version failing
- Remove generated files from version control HOT 5
- Add `envFrom` in dask/dask workers deployment HOT 1
- Automatic updates have broken HOT 6
- Chart README not being packaged
- SSL error when having https enabled with manual certificates in jupyterhub side. HOT 2
- is it possible to change the scheduler.service.name ? HOT 2
- Chart is using depracted `bokeh-port` argument HOT 8
- Add RELEASE.md HOT 1
- Add Liveness and Readiness Probes to Pods
- OverrideNames failure
- DaskHub cannot be placed in a subdirectory URL
- jupyterhub helm chart update HOT 4
- client = cluster.get_client() : Timed out HOT 1
- ARM support HOT 4
- Network policies break daskhub HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from helm-chart.