Comments (9)
Ah, I see now. Please rename to "Master can't pivot behind the proxy". OKD-specific machine-config-daemon-pull
service doesn't yet use proxy env vars
from okd.
Fix merged in 4.4.0-0.okd-2020-06-02-224917 (thanks @evertmulder!), also backported to fcos-4.5 branch
from okd.
I saw this during sshing into the server:
Fedora 30.20191014.1 (CoreOS preview)
Tracker: https://github.com/coreos/fedora-coreos-tracker
Preview release: breaking changes may occur
Last login: Sat Nov 30 10:07:06 2019 from 172.23.240.46
[systemd]
Failed Units: 3
machine-config-daemon-host.service
machine-config-daemon-pull.service
rpc-statd.service
[core@localhost ~]$
from okd.
systemctl status machine-config-daemon-host
shows that:
[core@localhost ~]$ sudo systemctl status machine-config-daemon-host
● machine-config-daemon-host.service - Machine Config Daemon Initial
Loaded: loaded (/etc/systemd/system/machine-config-daemon-host.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/machine-config-daemon-host.service.d
└─10-default-env.conf
Active: failed (Result: exit-code) since Sat 2019-11-30 09:32:02 UTC; 40min ago
Process: 2078 ExecStart=/usr/local/bin/machine-config-daemon pivot (code=exited, status=203/EXEC)
Main PID: 2078 (code=exited, status=203/EXEC)
CPU: 954us
Nov 30 09:32:02 localhost systemd[1]: Starting Machine Config Daemon Initial...
Nov 30 09:32:02 localhost systemd[2078]: machine-config-daemon-host.service: Failed to execute command: Permission denied
Nov 30 09:32:02 localhost systemd[2078]: machine-config-daemon-host.service: Failed at step EXEC spawning /usr/local/bin/machine-config-daemon: Permission denied
Nov 30 09:32:02 localhost systemd[1]: machine-config-daemon-host.service: Main process exited, code=exited, status=203/EXEC
Nov 30 09:32:02 localhost systemd[1]: machine-config-daemon-host.service: Failed with result 'exit-code'.
Nov 30 09:32:02 localhost systemd[1]: Failed to start Machine Config Daemon Initial.
Nov 30 09:32:02 localhost systemd[1]: machine-config-daemon-host.service: Consumed 954us CPU time.
from okd.
It seems as if the machine-config-daemon is an empty binary:
[core@localhost ~]$ ls -alh /usr/local/bin
total 40K
drwxr-xr-x. 2 root root 270 Nov 30 10:44 .
drwxr-xr-x. 11 root root 114 Nov 30 10:44 ..
-rwxr-xr-x. 1 root root 1.4K Nov 30 10:44 etcd-member-add.sh
-rwxr-xr-x. 1 root root 2.5K Nov 30 10:44 etcd-member-recover.sh
-rwxr-xr-x. 1 root root 642 Nov 30 10:44 etcd-member-remove.sh
-rwxr-xr-x. 1 root root 931 Nov 30 10:44 etcd-snapshot-backup.sh
-rwxr-xr-x. 1 root root 1.7K Nov 30 10:44 etcd-snapshot-restore.sh
-rw-r--r--. 1 root root 0 Nov 30 10:44 machine-config-daemon
-rw-r--r--. 1 root root 12K Nov 30 10:44 openshift-recovery-tools
-rwxr-xr-x. 1 root root 1.2K Nov 30 10:44 recover-kubeconfig.sh
-rwxr-xr-x. 1 root root 1.1K Nov 30 10:44 tokenize-signer.sh
from okd.
journalctl --no-pager | grep machine-config-dameon
shows:
ine-config-daemon-host.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(2d): [finished] processing unit "machine-config-daemon-host.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(2f): [started] enabling unit "machine-config-daemon-host.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(2f): [finished] enabling unit "machine-config-daemon-host.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(30): [started] processing unit "machine-config-daemon-pull.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(30): op(31): [started] writing unit "machine-config-daemon-pull.service" at "/sysroot/etc/systemd/system/machine-config-daemon-pull.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(30): op(31): [finished] writing unit "machine-config-daemon-pull.service" at "/sysroot/etc/systemd/system/machine-config-daemon-pull.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(30): [finished] processing unit "machine-config-daemon-pull.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(32): [started] enabling unit "machine-config-daemon-pull.service"
Nov 30 10:51:08 localhost ignition[818]: INFO : files: op(32): [finished] enabling unit "machine-config-daemon-pull.service"
Nov 30 10:51:10 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=machine-config-daemon-pull comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Nov 30 10:51:10 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=machine-config-daemon-host comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Nov 30 10:51:10 localhost systemd[1]: machine-config-daemon-pull.service: Main process exited, code=exited, status=125/n/a
Nov 30 10:51:10 localhost systemd[1]: machine-config-daemon-pull.service: Failed with result 'exit-code'.
Nov 30 10:51:10 localhost systemd[1]: machine-config-daemon-pull.service: Consumed 178ms CPU time.
Nov 30 10:51:10 localhost systemd[1263]: machine-config-daemon-host.service: Failed to execute command: Permission denied
Nov 30 10:51:10 localhost systemd[1263]: machine-config-daemon-host.service: Failed at step EXEC spawning /usr/local/bin/machine-config-daemon: Permission denied
Nov 30 10:51:10 localhost systemd[1]: machine-config-daemon-host.service: Main process exited, code=exited, status=203/EXEC
Nov 30 10:51:10 localhost systemd[1]: machine-config-daemon-host.service: Failed with result 'exit-code'.
Nov 30 10:51:10 localhost systemd[1]: machine-config-daemon-host.service: Consumed 767us CPU time.
The service machine-config-daemon-pull
fails. It can't download the initial docker image.
If I set the proxy manually in the systemd file for it, and restart the both services:
sudo systemctl restart machine-config-daemon-pull
sudo systemctl restart machine-config-daemon-host
The downloads work until the master restarts.
So in my opinion the proxy credentials should be rendered in the ignition files which are served from the bootstrap server to the masters and workers.
from okd.
Dupe of #14?
from okd.
@vrutkovs:
It's not related to VMWARE. If a proxy is configured in the install-config.yaml it should also be rendered in the master.ign and worker.ign served by the machine-config-server.
from okd.
A workaround I used to get the installation (OKD beta5) working behind a firewall is to execute the following in every master and worker on first boot:
sudo -i
mkdir /etc/systemd/system/machine-config-daemon-firstboot.service.d
cp /etc/systemd/system/machine-config-daemon-host.service.d/10-default-env.conf /etc/systemd/system/machine-config-daemon-firstboot.service.d
systemctl daemon-reload
systemctl stop machine-config-daemon-firstboot
systemctl start machine-config-daemon-firstboot
from okd.
Related Issues (20)
- Mount a storageclass with a minio external storage server
- [Libvrit] Mirroring of images with imageContentSourcePolicy not working ?
- CI config for latest 4-stable OKD release picked wrong base release HOT 4
- prometheus-k8s route is returning 404 after OKD cluster upgrade to 4.13
- Connot pull from OKD cluster via pod or oc command with error x509: certificate is valid for ***Clusterdomain *** , not *** registry-1.docker.io ***
- Disconnected 4.12 install fails pivot due to missing podman credentials HOT 1
- [OKD 4.13][vSphere 4.13] IPI Install broken. Bootstrap doesn't clean up due to https://github.com/openshift/installer/pull/6770
- [AWS] Cluster bootstrap fails with IngressStateEndpoints_MissingSubsets + operators not initialized HOT 3
- Release Artifacts for 4.13.0-0.okd-2023-09-03-082426 are named 4.13.0-0.okd-2023-08-18-135805 HOT 2
- image listed in release.txt file for 4.13.0-0.okd-2023-08-18-135805 is not preset HOT 2
- What's the deal with 4.13.0-0.okd-2023-09-03-112738 HOT 10
- ClusterOperator network hung with 4.13.0-0.okd-2023-09-03-082426 update HOT 2
- get nightly release after upgrading to 4.13.0-0.okd-2023-09-03-082426 HOT 1
- Bootstrap node does not finish, installation does not progress
- Release not accepted HOT 2
- Install OKD 4.13.11 on Openstack
- OKD4 : Error: unhealthy cluster
- Master Node not using cluster Proxy for 1 Url ignoring noProxy setting
- Possible Missing Release Image in quay.io Image Registry HOT 6
- 4.13.0-0.okd-2023-09-03-082426 "Shutdown Guest OS" stuck on workers HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from okd.