mrsteve81 / k3s-virtualbox

Create multi-node K3s cluster in VirtualBox

k3s-virtualbox

This project makes it easy to create single- and multi-node k3s cluster in VirtualBox, e.g. for local development on Kubernetes.

My inspirations for this project:

• k3d (the idea is pretty similar, but it uses containers which are much more lightweight compared to virtual machines)
• khuedoan/homelab (uses physical servers as nodes)

Overview

flowchart LR
  M["make"]
  M -- vagrant up --> V
  M -- ansible-playbook site.yml --> A
  V -- create VMs --> VB
  A -- provision VMs --> VB
  
  subgraph Architecture Overview
    direction TB
    M
    subgraph V["Vagrant"]
        direction RL
        V1[Vagrantfile]
    end
    subgraph A["Ansible"]
        direction RL
        A1[site.yml]
        A2[inventory.yml]
    end
    subgraph VB["[VirtualBox] Ubuntu Server VMs"]
        direction BT
        master0[master0 - 192.168.0.200]
        worker0[worker0 - 192.168.0.201]
        worker1[worker1 - 192.168.0.202]
    end
  end

Loading

Demo
Virtual machines created via Vagrant
Homepage with Ingress discovery powered by Hajimari
Continuous deployment with ArgoCD

Setup VMs & k3s cluster

• Clone this repository
• Replace default_interface in Vagrantfile with the name of your host's main network interface
• Edit inventory.yml. If your home network subnet is different than 192.168.0.0/24 then you have to update IPs. You can also change the number of worker nodes (only 1 master node is supported for now).
• Linux (not tested)
  • Install VirtualBox, kubectl, vagrant, ansible
• Windows
  • Install VirtualBox, WSL2
  • Create Ubuntu distribution: wsl --install -d ubuntu (because you can't install Ansible on Windows)
  • Convert distribution to use WSL2: wsl --set-version ubuntu 2
  • Enter the machine wsl -d Ubuntu
  • Install ansible, kubectl
  • Install vagrant and configure it to work with VirtualBox that is installed on the host. Helpful resources:
    • vagrant up – Running Vagrant under WSL2
    • Vagrant and Windows Subsystem for Linux
• ssh-keygen - generate ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub keys. Vagrant will append id_rsa.pub to ~/.ssh/authorized_keys of each VM so that VMs will be accessible over SSH.
• vagrant plugin install vagrant-timezone
• make - this command will create declared virtual machines (via Vagrant) and then provision them (via Ansible)
• after a while, the cluster should be set up

$ kubectl get nodes
NAME      STATUS   VERSION        INTERNAL-IP        OS-IMAGE            CONTAINER-RUNTIME
master0   Ready    v1.25.0+k3s1   192.168.0.200    Ubuntu 22.04.1 LTS    containerd://1.6.8-k3s1
worker0   Ready    v1.25.0+k3s1   192.168.0.201    Ubuntu 22.04.1 LTS    containerd://1.6.8-k3s1
worker1   Ready    v1.25.0+k3s1   192.168.0.202    Ubuntu 22.04.1 LTS    containerd://1.6.8-k3s1

Usage

The cluster is ready. Now you can deploy some apps.

• make argocd - deploy ArgoCD, wait for it be ready and then configure it to deploy apps declared inside the cluster/apps folder
• wait until argocd deploys metallb and ingress
• when make ingressip returns 192.168.0.200 then you should be able to access:
  • The cluster homepage
  • ArgoCD dashboard (admin/admin)

Tech stack

Logo	Name	Description
	WSL2	Windows Subsystem for Linux
	VirtualBox	Virtualization
	Vagrant	Automate VM creation in Virtualbox
	Ansible	Automate VM provisioning and configuration
	Ubuntu Server	Base OS for VMs
	K3s	Lightweight distribution of Kubernetes
	k3sup	Bootstrap K3s over SSH
	Helm	The package manager for Kubernetes
	ArgoCD	GitOps continuous delivery tool for Kubernetes
	MetalLB	Bare metal load-balancer for Kubernetes
	NGINX Ingress Controller	Kubernetes Ingress Controller
	mermaid	JavaScript-based diagramming and charting tool