Kubernetes no longer allows RW mounting of config maps about osprey

[jsravn]

Hi @philosifer. It should work even with the k8s change. I don't think dex requires it to be rw. However I believe subpath mounts were temporarily broken in some k8s versions (1.8.8 iirc) which would cause the dex yaml to fail to work.

[philosifer]

I put the subpaths back in but still with the initcontainer intermediate step and it still works so i think its more to do with the config map than using subpaths. I'm on 1.10.2 btw.

[totahuanocotl]

What error do you get with the Config map @philosifer?
I submitted a PR for Dex itself a while ago because it has an issue resolving the templates when mounting them as volumes: dexidp/dex#1206.
It was rejected though, and I did not submit a second one. See if it the description of the issue matches what you are seeing.

[philosifer]

This is the error I get from kubectl describe pod if i don't do the initcontainer trick

Error: failed to start container "dex": Error response from daemon: error setting label on mount source '/var/lib/kubelet/pods/b7f8be8b-65a8-11e8-a1a1-005056a6113c/volume-subpaths/dex-web-templates/dex/5': read-only file system

That's repeated for a few of the other subpath mounts before it gives up and fails

I've done some more investigating. I have two clusters, development which uses default settings for security and production on which i've set it to use certificates everywhere, https for etcd etc. I only see the read-only issue on the production cluster, it works fine as you wrote it on the dev cluster but i've no idea what exactly is causing that difference. Both are built with virtually identical kubeadm config files apart from the certificate lines added to the production setup.

[jsravn]

@totahuanocotl ping

[totahuanocotl]

@philosifer, sorry for the late response.
Do you still have this issue?
If so, I'll try to replicate it and try to find the issue.
If you could provide the configuration you used that would be helpful.

[philosifer]

Yes I still have the issue but not much time to troubleshoot it further currently. It is also now affecting my attempt to redeploy harbor with helm (their issue #4496) and i might end up having to use the ReadOnlyAPIDataVolumes feature gate for now as i have too many places to put in workarounds for everything.