alexellis / k8s-on-raspbian Goto Github PK
View Code? Open in Web Editor NEWKubernetes on Raspbian (Raspberry Pi)
Home Page: https://blog.alexellis.io/build-your-own-bare-metal-arm-cluster/
License: MIT License
Kubernetes on Raspbian (Raspberry Pi)
Home Page: https://blog.alexellis.io/build-your-own-bare-metal-arm-cluster/
License: MIT License
kubernetes master node starts and kubectl get pods shows Ready status.
kubernetes master node starts, and kubectl get pods shows NotReady status
kubectl describe nodes shows this error in the event log:
Failed to update Node Allocatable Limits ["kubepods"]: failed to set supported cgroup subsystems for cgroup [kubepods]: Failed to find subsystem mount for required subsystem: pids
not sure - disable whatever requires this cgroup? is it something new in 1.14? or enable that cgroup in rasberian lite somewhere? (I'm not cgroup expert, so I don't know how to even start)
(follow the guide in this repo, I get these results at the "Check everything worked:" step of the guide)
Can't schedule pods / nodes not ready.
Docker version docker version
(e.g. Docker 17.0.05 ):
Docker version 18.09.0, build 4d60db4
What version of Kubernetes are you using? kubectl version
:
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:53:57Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/arm"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/arm"}
Operating System and version (e.g. Linux, Windows, MacOS):
2018-11-13-raspbian-stretch-lite.img
4.14.79-v7+
What ARM or Raspberry Pi board are you using?
Rasberry Pi 3 B+
seems to be a missing configuration, Could you please add it in the tutorial as a comment or a note please.
on raspbian, current version the /proc/sys/net/bridge/bridge-nf-call-iptables is not available
sudo modprobe br_netfilter
with persistence
echo "br_netfilter" >> /etc/modules
ARM PI 3 b+
Docker version docker version
(e.g. Docker 17.0.05 ):
What version of Kubernetes are you using? kubectl version
:
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.1", GitCommit:"eec55b9ba98609a46fee712359c7b5b365bdd920", GitTreeState:"clean", BuildDate:"2018-12-13T10:39:04Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/arm"}
Operating System and version (e.g. Linux, Windows, MacOS): Linux
Get a dashboard up and running
The instruction to get a dashboard in pi cluster does not work.
The URL for alternate dashboard returns 404.
Doing this for learning purposes. Wanted to see if the dashboard can be setup in an arm hardware.
Raspberry pi model 3 b+
Docker version docker version
(e.g. Docker 17.0.05 ):
Docker version 18.09.0, build 4d60db4
What version of Kubernetes are you using? kubectl version
:
kubectl version 1.13.1
kubedm version 1.13.1
kubelet version 1.13.1
Operating System and version (e.g. Linux, Windows, MacOS):
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
Hello!
Thanks for sharing this nice guide. I have an idea, and I have some questions, maybe you can help me.
There is a project called YunoHost and it is aimed at helping home users self host services like WordPress or Nextcloud on an arm computer at home.
It is nice, but based on bash let's say, and as I love k8s api, I'm wondering about k8s mem footprint on such small devices.
In a context of a one node cluster, what is the mem footprint of:
Another question, do we need also a network plugin? Or as there is only one node, it is enough?
And then, what about optimization, I think there are many rooms for improvements in this context:
I'm just thinking out loud here, if you have any ideas, please share them here, and do not hesitate to close the issue once you answered!
Thanks again and have a nice day :)
When trying to follow the manual steps, I get a few errors, I hope these can be addressed.
When running this:
curl -sSL get.docker.com | sh && \
sudo usermod pi -aG docker
newgrp docker
I get this: newgrp: group 'docker' does not exist
Should it be??:
curl -sSL get.docker.com | sh && \
sudo usermod pi -aG docker \
newgrp docker
Anyway, if I try running the lines above, I get this: E: The repository 'https://download.docker.com/linux/raspbian 10 Release' does not have a Release file.
And when moving on in the steps, I get the same error later on when running:
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \
sudo apt-get update -q && \
sudo apt-get install -qy kubeadm
Any idea what this is? The is my first venture into kubernetes on Raspberry Pi, so it might just be me that's clueless here.
In GUIDE.md, in the "Start up the Kubernetes dashboard" section, the kubectl apply ...
command is referencing a resource that no longer exists. Here's the kubectl apply ...
command from the guide along with the response:
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/alternative/kubernetes-dashboard-arm-head.yaml
error: unable to read URL "https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/alternative/kubernetes-dashboard-arm-head.yaml", server reported 404 Not Found, status code=404
I poked around in the Kubernetes Dashboard repo and it appears that an ARM version is no longer available. I did find some deployment specs referencing older versions of the ARM dashboard image, but those images use the now retired Kubernetes Hipster instead of the Kubernetes Metrics Server. The dashboard pod log shows:
2020/03/07 17:00:00 Metric client health check failed: the server could not find the requested resource (get services heapster). Retrying in 30 seconds.
I expected that the kubectl apply ...
would successfully download the deployment spec and create the required resources.
See above, the kubectl apply ...
returns a 404 not found
Find a current arm version of the Kubernetes Dashboard.
kubectl apply ...
command referenced in GUIDE.md, i.e.,kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/alternative/kubernetes-dashboard-arm-head.yaml
I was hoping to install the dashboard to visually inspect my cluster.
docker version
(e.g. Docker 17.0.05 ):$ d version
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:22:34 2019
OS/Arch: darwin/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:29:19 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
kubectl version
:k version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:08:14Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/arm"}
Operating System and version (e.g. Linux, Windows, MacOS):
The client is a Mac
What ARM or Raspberry Pi board are you using?
Pi 4B
Please comment here here with your name and URL to your cluster, plus any useful notes you have.
I'll then add it to the Readers' RPi clusters in the main README.
Alex
Join the #arm-and-pi channel on OpenFaaS Slack to chat with our community - https://docs.openfaas.com/community/
This isn't an issue so much as a heads-up. Looks like this Medium article pretty much plagiarized the guide in this repo.
https://medium.com/developingnodes/setting-up-kubernetes-cluster-on-raspberry-pi-15cc44f404b5
Following the guide in k8s-on-raspbian/GUIDE.md.
When I run:
pi@raspberrypi2:~ $ curl -4 http://127.0.0.1:31118 -d "test"
The HTTP request is sent but it is forever awaiting a response.
All pods are running
pi@raspberrypi2:~ $ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default markdownrender-f9744b577-gsmk9 1/1 Running 0 14m
kube-system coredns-6955765f44-hsbhx 1/1 Running 0 23m
kube-system coredns-6955765f44-zzf9w 1/1 Running 0 23m
kube-system etcd-raspberrypi2 1/1 Running 0 24m
kube-system kube-apiserver-raspberrypi2 1/1 Running 0 24m
kube-system kube-controller-manager-raspberrypi2 1/1 Running 0 24m
kube-system kube-proxy-9zx54 1/1 Running 0 11m
kube-system kube-proxy-st75l 1/1 Running 0 23m
kube-system kube-scheduler-raspberrypi2 1/1 Running 0 24m
kube-system weave-net-qhmps 2/2 Running 2 11m
kube-system weave-net-vhq6j 2/2 Running 1 16m
Node (raspberrypi3) successfully joins
pi@raspberrypi2:~ $ kubectl get nodes
NAME STATUS ROLES AGE VERSION
raspberrypi2 Ready master 26m v1.17.3
raspberrypi3 Ready <none> 13m v1.17.3
I'm using the right port:
pi@raspberrypi2:~ $ kubectl get service markdownrender
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
markdownrender NodePort 10.106.161.23 <none> 8080:31118/TCP 17m
Routes show the 31118 port open
pi@raspberrypi2:~ $ netstat -tulpn
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN
tcp 0 0 192.168.0.23:2379 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN
tcp 0 0 192.168.0.23:2380 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:2381 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:45009 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:10257 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:10259 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:6784 0.0.0.0:* LISTEN
tcp6 0 0 :::10250 :::* LISTEN
tcp6 0 0 :::10251 :::* LISTEN
tcp6 0 0 :::6443 :::* LISTEN
tcp6 0 0 :::10252 :::* LISTEN
tcp6 2 0 :::31118 :::* LISTEN
tcp6 0 0 :::10256 :::* LISTEN
tcp6 0 0 :::22 :::* LISTEN
tcp6 0 0 :::6781 :::* LISTEN
tcp6 0 0 :::6782 :::* LISTEN
tcp6 0 0 :::6783 :::* LISTEN
I believe I am missing something very simple. I already tried reset and init and the problem persists.
Follow @davidcollom's comment in #16.
swap would be enabled after each boot:
sudo systemctl disable dphys-swapfile
followed by a reboot, fixed this.
sudo usermod pi -aG docker
should be sudo usermod pi -aG docker \
Docker version docker version
(e.g. Docker 17.0.05 ):
What version of Kubernetes are you using? kubectl version
:
Operating System and version (e.g. Linux, Windows, MacOS):
What ARM or Raspberry Pi board are you using?
The current flannel yaml linked to by the guide (https://raw.githubusercontent.com/coreos/flannel/c5d10c8/Documentation/kube-flannel.yml
) no longer starts the flannel pods due to updates since 1.10. Tolerations were added to the yaml in flannel-io/flannel#1044.
Flannel pods should automatically start with the kubectl apply
command
Flannel daemonsets are created but no pods are started
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Nodes were stuck in NotReady
state until I used the updated link.
Docker version docker version
(e.g. Docker 17.0.05 ):
What version of Kubernetes are you using? kubectl version
: Server - 1.13.3 Client - 1.13.2
Operating System and version (e.g. Linux, Windows, MacOS): Linux
Cluster Description:
Marcus Smallman' DIY Raspberry Pi Kubernetes Cluster
Related URL (Blog Post):
https://marcussmallman.io/2018/02/18/diy-rasberry-pi-kubernetes-cluster/
A while ago I was inspired to create my own cluster from reading a bunch of online resources. The prep.sh script in this repo was very helpful. I decided to write up my experience so that other can use/learn from.
When running sudo kubeadm init --token-ttl=0 --pod-network-cidr=10.244.0.0/16
it always fatally errors out
This should complete the init
sudo kubeadm init --token-ttl=0 --pod-network-cidr=10.244.0.0/16
This results in a timeout and fatal errors.
Applying the timeout API patch does not help at all:
sudo sed -i 's/failureThreshold: 8/failureThreshold: 20/g' /etc/kubernetes/manifests/kube-apiserver.yaml && \ sudo sed -i 's/initialDelaySeconds: [0-9]\+/initialDelaySeconds: 360/' /etc/kubernetes/manifests/kube-apiserver.yaml
I am setting up a new Raspberry Pi Master node.
This is repeatable, following this guide and always 100% results in this error.
R-Pi 4
SanDisk Extreme Pro 128GB
All steps in the guide - no need to list that here.
What version of Kubernetes are you using? kubectl version
:
All steps in the guide - no need to list that here.
Operating System and version (e.g. Linux, Windows, MacOS):
Raspian Lite
What ARM or Raspberry Pi board are you using?
Raspberry Pi 4
Hi, I follow the the instruction until the init step. At that point I am getting:
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
Thanks for your help.
docker version
(e.g. Docker 17.0.05 ):Server:
Version: 1.8.3
API version: 1.20
Go version: go1.4.3
Git commit: f4bf5c7
Built:
OS/Arch: linux/arm
What version of Kubernetes are you using? kubectl version
:
sudo kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/arm"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Operating System and version (e.g. Linux, Windows, MacOS):
Mac OS High Sierra
Raspberry Pi 3 B+
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.