Comments (7)
I can take a look at addressing this when I bump the embedded spegel version. I will note that this currently only affects servers, as agents do not support the --bind-address flag, so the listener address cannot be configured. Might want to promote that to an agent flag as well I guess.
from k3s.
OK, I've modified that PR to always bind the supervisor on the loopback addresses, in addition to the requested address
from k3s.
Might want to promote that to an agent flag as well I guess.
Having the bind flag on agents would definitely be desirable (and make the overall config options more consistent).
from k3s.
there are already some changes staged in this space in
from k3s.
One of the problems here is that spegel needs to be able to identify which requests are from the local containerd instance and should proxy to other nodes, as opposed to those from other nodes that should not be reproxied. Right now this is done by identifying requests to localhost, and I can't think of another really good way to do that without opening more ports. We may just need to set it up so that it binds to localhost plus the configured bind address.
from k3s.
Note that the embedded registry still will not work on nodes where IPv6 is the primary address family due to #9897
from k3s.
Validated on master branch with version v1.30.2-rc2+k3s1
Environment Details
Infrastructure
- Cloud
- Hosted
Node(s) CPU architecture, OS, and Version:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"
$ uname -m
x86_64
Cluster Configuration:
HA: 3 server/ 1 agent
Config.yaml:
token: xxxx
cluster-init: true
write-kubeconfig-mode: "0644"
node-external-ip: 1.1.1.1
node-label:
- k3s-upgrade=server
embedded-registry: true
bind-address: x.x.x.x
registries.yaml:
mirrors:
private.registry.com:
endpoint:
- private.registry.com
docker.io:
endpoint:
- private.registry.com
k8s.gcr.io:
endpoint:
- private.registry.com
configs:
private.registry.com:
auth:
username: <username>
password: <password>
tls:
ca_file: /home/ubuntu/ca.pem
Testing Steps
- Copy config.yaml
$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s && sudo cp registries.yaml /etc/rancher/k3s
- Install k3s
curl -sfL https://get.k3s.io | sudo INSTALL_K3S_VERSION='v1.30.2-rc2+k3s1' sh -s - server
- Verify Cluster Status:
kubectl get nodes -o wide
kubectl get pods -A
Replication Results:
- k3s version used for replication:
$ k3s -v
k3s version v1.30.1+k3s1 (80978b5b)
go version go1.22.2
$ journalctl -xeu k3s-agent | grep 'received image event'
Validation Results:
- k3s version used for validation:
$ k3s -v
k3s version v1.30.2-rc2+k3s1 (b4d4ed8f)
go version go1.22.4
Sample spegel logs from the journal logs on the agent node:
$ journalctl -xeu k3s-agent | grep 'received image event'
Jun 14 17:50:51 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:51Z" level=info msg="spegel 2024/06/14 17:50:51 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"docker.io/rancher/mirrored-pause:3.6@sha256:74bf6fc6be13c4ec53a86a5acf9fdbc6787b176db0693659ad6ac89f115e182c\" \"type\"=\"CREATE\""
Jun 14 17:50:51 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:51Z" level=info msg="spegel 2024/06/14 17:50:51 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"docker.io/rancher/mirrored-pause@sha256:74bf6fc6be13c4ec53a86a5acf9fdbc6787b176db0693659ad6ac89f115e182c\" \"type\"=\"CREATE\""
Jun 14 17:50:55 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:55Z" level=info msg="spegel 2024/06/14 17:50:55 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/mytestcontainer:unprivileged@sha256:7e418465981575a9abef4ee16a80c562a2d2d171e591c1475c38347ef3ec2a72\" \"type\"=\"CREATE\""
Jun 14 17:50:55 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:55Z" level=info msg="spegel 2024/06/14 17:50:55 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/mytestcontainer@sha256:7e418465981575a9abef4ee16a80c562a2d2d171e591c1475c38347ef3ec2a72\" \"type\"=\"CREATE\""
Jun 14 17:51:29 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:51:29Z" level=info msg="spegel 2024/06/14 17:51:29 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/nginx:latest@sha256:80550935209dd7f6b2d7e8401b9365837e3edd4b047f5a1a7d393e9f04d34498\" \"type\"=\"CREATE\""
Jun 14 17:51:29 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:51:29Z" level=info msg="spegel 2024/06/14 17:51:29 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/nginx@sha256:80550935209dd7f6b2d7e8401b9365837e3edd4b047f5a1a7d393e9f04d34498\" \"type\"=\"CREATE\""
from k3s.
Related Issues (20)
- Validate SUSE Liberty Linux 8.9
- Enhance install script to support Suse Liberty Linux HOT 1
- [v1.29.3+k3s1] Certificate rotation on k3s-agent fails to include sans addressing, causing api requests via kubectl to potentially fail if kube-vip is in use HOT 1
- Add etcd snapshot s3 config secret support
- Multiple simultaneous snapshots result in silent failure and/or corruption of at least one snapshot HOT 2
- [Release-1.29] - Multiple simultaneous snapshots result in silent failure and/or corruption of at least one snapshot HOT 1
- [Release-1.28] - Multiple simultaneous snapshots result in silent failure and/or corruption of at least one snapshot HOT 1
- [Release-1.27] - Multiple simultaneous snapshots result in silent failure and/or corruption of at least one snapshot HOT 1
- kubectl logs selectively timing out based on the host CIDR range HOT 4
- v1.30.2-rc3 Images missing HOT 4
- rpm.rancher.io RHEL 9 packages missing/removed HOT 3
- k3s crashes completely with "Observed a panic: "integer divide by zero"" HOT 9
- Allow configuration of Rootlesskit's CopyUpDirs through an environment variable
- Embedded registry mirror pulling failed
- Snapshot option discrepancy with --etcd-snapshot-name
- Support for dynamic ports all the way up to 65535.
- How can I stop k3s apiserver listening {IP}:6443/static/charts directory? HOT 2
- Extreme CPU Usage on small cluster HOT 1
- kubelet service is missing on single node cluster HOT 3
- Cannot override k3s default kubelet-args via user-provided kubelet config file HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from k3s.