weaveworks / wks-quickstart-firekube

Example configuration to create Kubernetes clusters powered by ignite and gitops

Deprecated

This repository is no longer maintained. For a more up-to-date way to manage microVMs, please take a look at Flintlock.

Firekube

Firekube is a Kubernetes cluster working on top of ignite and firecracker. Firekube clusters are operated with GitOps.

ignite and firecracker only work on Linux as they need KVM. Fortunately we will also work on macOS using footloose: the Kubernetes nodes are then running inside containers.

Creating a Firekube cluster

Prerequisites: docker, git, kubectl 1.14+.

1. Fork this repository.

2. Clone your fork and cd into it. Use the SSH git URL as the script will push an initial commit to your fork:

   export user="" # Your GitHub handle or org
   
   git clone [email protected]:$user/wks-quickstart-firekube.git
   cd wks-quickstart-firekube

3. Start the cluster:

   ./setup.sh

   This step will take several minutes.

   Please note that the setup.sh script will detect which OS you're on. If it's macOS, the script will automatically set backend: docker inside config.yaml. This config is to start the cluster in containers as there is no KVM on macOS. However, Firekube on Linux will run either with backend: ignite (ignite VMs) or backend: docker (containers). So the setup.sh script won't touch the backend value inside config.yaml if you're on Linux. In case you started Firekube (in containers) on macOS then switched to try it on Linux, please make sure that the backend value is set to backend: ignite so that you can start the cluster on ignite / firecracker VMs.

4. Export the KUBECONFIG environment variable as indicated at the end of the installation:

   export KUBECONFIG=/home/damien/.wks/weavek8sops/example/kubeconfig

Enjoy your Kubernetes cluster!

$ kubectl get nodes
NAME               STATUS   ROLES    AGE     VERSION
67bb6c4812b19ce4   Ready    master   3m42s   v1.14.1
a5cf619fa058882d   Ready    <none>   75s     v1.14.1

Watch GitOps in action

Now that we have a cluster installed, we can commit Kubernetes objects to the git repository and have them appear in the cluster. Let's add podinfo, an example Go microservice, to the cluster.

kubectl apply --dry-run -k github.com/stefanprodan/podinfo//kustomize -o yaml > podinfo.yaml
git add podinfo.yaml
git commit -a -m 'Add podinfo Deployment'
git push

A few seconds later, you should witness the apparition of a podinfo pod in the cluster:

$ kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
podinfo-677768c755-z76xk   1/1     Running   0          30s

To view podinfo web UI:

1. Expose podinfo locally:

   kubectl port-forward deploy/podinfo 9898:9898
   

2. Point your browser to http://127.0.0.1:9898:

   

Deleting a Firekube cluster

Run:

./cleanup.sh

Using a private git repository with firekube

To use a private git repository instead of a fork of wks-quickstart-firekube:

1. Create a private repository and push the wks-quickstart-firekube master branch there. Use the SSH git URL when cloning the private repository:

   git clone [email protected]:$user/$repository.git
   cd $repository
   git remote add quickstart [email protected]:weaveworks/wks-quickstart-firekube.git
   git fetch quickstart
   git merge quickstart/master
   git push
   

2. Create an SSH key pair:

   ssh-keygen -t rsa -b 4096 -C "[email protected]" -f deploy-firekube  -N ""

3. Upload the deploy key to your private repository (with read/write access):

   

4. Start the cluster:

   ./setup.sh --git-deploy-key  ./deploy-firekube

Getting Help

If you have any questions about, feedback for or problems with wks-quickstart-firekube:

• Invite yourself to the Weave Users Slack.
• Ask a question on the #general slack channel.
• File an issue.
• Join the discussion
  • Invite yourself to the Weave community Slack
  • Ask a question on the #firekube Slack channel

Weaveworks follows the CNCF Code of Conduct. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting a Weaveworks project maintainer, or Alexis Richardson ([email protected]).

Your feedback is always welcome!

License

Apache 2.0

wks-quickstart-firekube's Issues

Add a step that shows gitops in action

Once the cluster is up and running we should be able to demo gitops by telling the user to add a manifest to their fork and observe it being deployed to the cluster.

runtime error due to need for cleanup

observe:

$ ./setup.sh 
• Found jk 0.3.0
• Found footloose 0.6.1
• Found ignite 0.5.3
• Found wksctl 0.8.0-beta.1-89-g86cdb09
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:latest present locally 
INFO[0000] Creating machine: firekube-node0 ...         
INFO[0000] Machine firekube-node0 is already created... 
INFO[0000] Creating machine: firekube-node1 ...         
INFO[0000] Machine firekube-node1 is already created... 
• Creating Cluster API manifests
/home/dimitri/src/github.com/weaveworks/wks-quickstart-firekube/setup.js:114
      privateIP: machine.runtimeNetworks[0].ip,
                                        ^
TypeError: Cannot read property '0' of undefined
    at /home/dimitri/src/github.com/weaveworks/wks-quickstart-firekube/setup.js:114:41
Module (setup.js) has not been loaded

Which is a non obvious error message in the context of the actual situation: which was that the VMs aren't actually running (and need to therefore be cleaned up with cleanup.sh).

The containers firekube-node[01] still exist, they are just not running.

Ignite version unnecessary blocks setup.sh

Ignite version requires root permission and it unnecessary blocks setup.sh,
https://github.com/weaveworks/wks-quickstart-firekube/blob/master/setup.sh#L221

The backport PR is in weaveworks/ignite#430
and v0.5.3 will contain this fix.

Add an end to end CI pass

We need to validate PRs! This is best done with creating a cluster and ensuring the whole flow works by, say, adding a new workload to the configured git repository and ensuring it appears in the cluster.

failed to start a cluster (error probably caused by yum / mirrors)

I could not up and running the cluster with ./setup.sh command. It seems to have a failure in between. Probably caused by yum trying to access mirrors.

I tweaked ./setup.sh to add --verbose after wksctl apply per @dlespiau suggestion and attached the logs here:

logs.txt

Add a confirmation step before pushing to users github repo

As a user
I'd like the ability to confirm the push to my github repo
so that I'm not accidentally surprised if I'm working on the wrong repo or have my origin set incorrectly.

The prompt should show where the push would go.
If the user chooses not to push changes, the command should terminate informing the user to perform the push.

How to reassign ports for a server? bind: address already in use

I get an error during installation, another cluster is running on the same server.
How to reassign ports for a server?

FATA[0196] command "ignite run weaveworks/ignite-centos:firekube-pre3 --name=firekube-node0 --cpus=2 --memory=1GB --size=5GB --kernel-image=weaveworks/ignite-kernel:4.19.47 --ssh=/root/wks-quickstart-firekube/cluster-key.pub --ports=2222:22 --ports=6443:6443 --ports=30443:30443 --ports=30080:30080" exited with "time=\"2020-03-08T18:11:18+03:00\" level=info msg=\"Starting image import...\"\ntime=\"2020-03-08T18:12:23+03:00\" level=info msg=\"Imported OCI image \\\"weaveworks/ignite-centos:firekube-pre3\\\" (1.6 GB) to base image with UID \\\"91856a7cd06ce2d8\\\"\"\ntime=\"2020-03-08T18:12:23+03:00\" level=info msg=\"Docker image \\\"weaveworks/ignite-kernel:4.19.47\\\" not found locally, pulling...\"\ntime=\"2020-03-08T18:12:30+03:00\" level=info msg=\"Imported OCI image \\\"weaveworks/ignite-kernel:4.19.47\\\" (49.1 MB) to kernel image with UID \\\"2c3ed95150e84fcf\\\"\"\ntime=\"2020-03-08T18:12:32+03:00\" level=info msg=\"Created VM with ID \\\"21e267d932f6ca4f\\\" and name \\\"firekube-node0\\\"\"\ntime=\"2020-03-08T18:12:32+03:00\" level=info msg=\"Pulling image \\\"weaveworks/ignite:v0.5.5\\\"...\"\ntime=\"2020-03-08T18:12:48+03:00\" level=fatal msg=\"failed to start container for VM \\\"21e267d932f6ca4f\\\": Error response from daemon: driver failed programming external connectivity on endpoint ignite-21e267d932f6ca4f (fe04ef14dd6ae62d91fea283952b8e04230e812e42b634cad1a3df1c03d743e5): Error starting userland proxy: listen tcp 0.0.0.0:6443: bind: address already in use\"\n": exit status 1

Move flux.yaml to apps/v1?

flux.yaml in the working folder after running setup.sh could be updated to apps/v1

- apiVersion: apps/v1beta1
  kind: Deployment

The same would apply for memcached

https://github.com/weaveworks/wks-quickstart-firekube/blob/master/flux.yaml#L91

setup.sh should verify that ignite dependencies are installed

This is what happens if I run ./setup.sh with everything but cni (which is a dependency of ignite) installed (see the last line of the output):

💵 michal@piec (master) ~/go/src/gith/weav/wks-quickstart-firekube> ./setup.sh
• Found jk 0.3.0
• Found footloose 0.6.2
• Found ignite 0.6.1
• Found wksctl 1.1.6-862-g1bf26b84
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:firekube-pre3 present locally 
INFO[0000] Creating machine: firekube-node0 ...         
FATA[0002] command "ignite run weaveworks/ignite-centos:firekube-pre3 --name=firekube-node0 --cpus=2 --memory=1GB --size=5GB --kernel-image=weaveworks/ignite-kernel:4.19.47 --ssh=/home/michal/go/src/github.com/weaveworks/wks-quickstart-firekube/cluster-key.pub --ports=2222:22 --ports=6443:6443 --ports=30443:30443 --ports=30080:30080" exited with "time=\"2019-10-07T22:58:44+02:00\" level=info msg=\"Created VM with ID \\\"7ffb36851667998e\\\" and name \\\"firekube-node0\\\"\"\ntime=\"2019-10-07T22:58:44+02:00\" level=fatal msg=\"[ERROR ExistingFile--opt-cni-bin-loopback]: File /opt/cni/bin/loopback, does not exist\\n[ERROR ExistingFile--opt-cni-bin-bridge]: File /opt/cni/bin/bridge, does not exist\\n\"\n": exit status 1 
🧨 [1] michal@piec (master) ~/go/src/gith/weav/wks-quickstart-firekube> 

In my case this happened because I upgraded ignite without installing cni which is a new dependency added in the version I upgraded to.

I think that we should be just as dilligent at checking deeper dependencies, or implement dependency checking in ignite.

Add a CI step that ensure shellcheck passes

This depends on #66

We don't actually use the wks-controller manifest from the repository

Well, at least we can't update it.

The flux logs show:

system:configmap/docker: running kubectl: Error from server (NotFound): error when creating "STDIN": namespaces "system" not found
system:configmap/repo: running kubectl: Error from server (NotFound): error when creating "STDIN": namespaces "system" not found
system:deployment/wks-controller: running kubectl: Error from server (NotFound): error when creating "STDIN": namespaces "system" not found

That's because the manifests in the git repository are defined to run in namespace: system but should really be in weavek8sops.

Unfortunately, just changing this doesn't work either. The controller does start but can't spin up any other nodes:

diff --git a/docker-config.yaml b/docker-config.yaml
index 2f0f164..7c8529c 100644
--- a/docker-config.yaml
+++ b/docker-config.yaml
@@ -2,7 +2,7 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: docker
-  namespace: system
+  namespace: weavek8sops
 data:
   daemon.json: |
     {
diff --git a/repo-config.yaml b/repo-config.yaml
index 0e14c8e..5ca9aa9 100644
--- a/repo-config.yaml
+++ b/repo-config.yaml
@@ -2,7 +2,7 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: repo
-  namespace: system
+  namespace: weavek8sops
 data:
   kubernetes.repo: |
     [kubernetes]
diff --git a/wks-controller.yaml b/wks-controller.yaml
index 108bc10..9777cb4 100644
--- a/wks-controller.yaml
+++ b/wks-controller.yaml
@@ -3,7 +3,7 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: wks-controller
-  namespace: system
+  namespace: weavek8sops
   labels:
     name: wks-controller
     control-plane: wks-controller

E0913 16:07:20.998038       1 controller.go:160] Error checking existence of machine instance for machine object worker-0; failed to establish connection to machine worker-0: failed to read SSH key: failed to get WKS' secret: secrets "wks-controller-secrets" not found

Provide an easy way to delete the cluster

We currently have no better way than:

export PATH=~/.wks/bin:$PATH
sudo env "PATH=$PATH" footloose delete

to delete the cluster. We should make it easy to do for users.

./setup fails due to cant read property '0' of undefined

hi,
I cloned the latest from master earlier today. the setup files fails due to

• Creating Cluster API manifests
/home/simon/wks-quickstart-firekube/setup.js:114
      privateIP: machine.runtimeNetworks[0].ip,
                                        ^
TypeError: Cannot read property '0' of undefined
    at /home/simon/wks-quickstart-firekube/setup.js:114:41
Module (setup.js) has not been loaded

any insights in what i am missing would be greatly appreciated.

machine info below

~/wks-quickstart-firekube$ ignite version
Ignite version: version.Info{Major:"0", Minor:"7", GitVersion:"v0.7.1", GitCommit:"e80a75a4a451395deeb03e37dde7057b9fc29606", GitTreeState:"clean", BuildDate:"2020-07-13T15:52:07Z", GoVersion:"go1.14.2", Compiler:"gc", Platform:"linux/amd64", SandboxImage:version.Image{Name:"weaveworks/ignite", Tag:"v0.7.1", Delimeter:":"}, KernelImage:version.Image{Name:"weaveworks/ignite-kernel", Tag:"4.19.125", Delimeter:":"}}
Firecracker version: v0.21.1
Runtime: containerd


kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:12:48Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?

docker version
Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:45:44 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.11
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.12
  Git commit:       77e06fd
  Built:            Mon Jun  8 20:24:59 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Deploy to local Gitlab

Followed the Private repo instructions and added Deploy Key but getting the following:

% ./setup.sh --git-deploy-key ./deploy-firekube
• Using git branch: master
• Using git deploy key: ./deploy-firekube
• Using git remote: origin

• Found jk 0.3.0
• Found footloose 0.6.2
• Found wksctl 0.8.1
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: quay.io/footloose/centos7:0.6.0 present locally
INFO[0000] Creating machine: firekube-node0 ...
INFO[0000] Machine firekube-node0 is already created...
INFO[0000] Creating machine: firekube-node1 ...
INFO[0000] Machine firekube-node1 is already created...
• Creating Cluster API manifests
• Updating container images and git parameters
• Pushing initial cluster configuration
Everything up-to-date
• Installing Kubernetes cluster
Error: CloneClusterAPIRepo: failed to clone repository: https://gitlab.domain.com/cdenneen/wks-quickstart-firekube.git: invalid auth method

State of Resource 'kubeadm:config:set-ownership' is Invalid

I have problem when installing firekube, and i'm getting confuse to read the error info, the error like this :

INFO[2020-05-19T05:06:02Z] State of Resource 'kubeadm:config:set-ownership' is Invalid.
Explanation:
{
 "resource": "kubeadm:config:set-ownership",
 "status": "Invalid",
 "reason": "DependencyInvalid",
 "dependencies": [
  {
   "resource": "kubeadm:config:copy",
   "status": "Invalid",
   "reason": "DependencyInvalid",
   "dependencies": [
    {
     "resource": "kubeadm:run-init",
     "status": "Invalid",
     "reason": "DependencyInvalid",
     "dependencies": [
      {
       "resource": "kubeadm:config:images",
       "status": "Invalid",
       "reason": "ApplyError",
       "error": "command exited with 1"
      }
     ]
    }
   ]
  }
 ]
} 

Failure during Installing Kubernetes cluster - no kind "Cluster" is registered for version "cluster.k8s.io/v1alpha1"

Please help. setup.sh run is failing consistently at:

• Installing Kubernetes cluster
FATA[2022-03-25T17:34:39-07:00] Error parsing manifest: failed to parse cluster manifest: for scheme unrecognized API group: cluster.k8s.io. Cause: UnknownGroup. gvk: cluster.k8s.io/v1alpha1, Kind=Cluster. error: no kind "Cluster" is registered for version "cluster.k8s.io/v1alpha1" in scheme "pkg/runtime/scheme.go:100"

Are there more logs that can be of use? If so, where can I find them?

Error when installing Kubernetes clusters

`• Using git branch: master
• Using git remote: origin

• Found jk 0.3.0
• Found footloose 0.6.3
• Found ignite 0.7.1
• Found wksctl 0.10.2
• Creating footloose manifest
• Creating SSH key
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:firekube-pre3 present locally
INFO[0000] Creating machine: firekube-node0 ...
INFO[0000] Machine firekube-node0 is already created...
INFO[0000] Creating machine: firekube-node1 ...
INFO[0000] Machine firekube-node1 is already created...
• Creating Cluster API manifests
• Updating container images and git parameters
• Pushing initial cluster configuration
Everything up-to-date
• Installing Kubernetes cluster
FATA[2021-05-23T14:03:39+08:00] Error parsing manifest: failed to parse cluster manifest: for scheme unrecognized API group: cluster.k8s.io. Cause: UnknownGroup. gvk: cluster.k8s.io/v1alpha1, Kind=Cluster. error: no kind "Cluster" is registered for version "cluster.k8s.io/v1alpha1" in scheme "pkg/runtime/scheme.go:100" `

Spurious ignite version error when run as non-root

setup.sh fails to run ignite version if not run as root and reports it as a version error (which it isn't).

We could (for example):

• update the documentation to put sudo before ./setup.sh
• detect mistaken execution as non-root and fail early.

Alternatively we could sudo internally, but this way we'd limit our compatibility with machines that don't have sudo configured the way we want it to.

💵 michal@piec (master) ~/go/src/gith/weav/wks-quickstart-firekube> ./setup.sh 
• Found jk 0.3.0
• footloose: detected git build, continuing
• error: ignite: error running 'ignite version'.

firekube with the ignite backend requires ignite to spawn VMs that will be used as Kubernetes nodes.

Please install ignite version 0.5.4 or later:

  • GitHub project  : https://github.com/weaveworks/ignite
  • Latest release  : https://github.com/weaveworks/ignite/releases
  • Installation    : https://github.com/weaveworks/ignite#installing
  • Required version: 0.5.4
🧨 [1] michal@piec (master) ~/go/src/gith/weav/wks-quickstart-firekube> ignite version
FATA[0000] This program needs to run as root.           
🧨 [1] michal@piec (master) ~/go/src/gith/weav/wks-quickstart-firekube> sudo ignite version
Ignite version: version.Info{Major:"0", Minor:"5", GitVersion:"v0.5.2", GitCommit:"be96924f7860992f348cd15ed46ee2c064313153", GitTreeState:"clean", BuildDate:"2019-08-26T14:36:21Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"linux/amd64"}
Firecracker version: v0.17.0
💵 michal@piec (master) ~/go/src/gith/weav/wks-quickstart-firekube> 

Kubernetes services not starting up after reboot

Hi,

I installed on a clean Ubuntu Server 18.04 on bare metal. It worked fine until I rebooted. The KUBECONFIG variable is set properly. Then I get the dreadful:
The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?

I reinstalled. First cleanup.sh, removed .wks folder and then ran setup.sh again. Working fine, but does not survive a reboot! Same as before

Why?

do_curl_tarball() not using URL passed in and fails downloads

Downloading https://github.com/weaveworks/wksctl/releases/download/0.6.1/wksctl-0.6.1-darwin-x86_64.tar.gz

curl: (22) The requested URL returned error: 404 Not Found
This is the download message for footloose - notice how it is trying to download wksctl.

cluster key created by root cannot be read by wksctl

When using ignite, footloose has to run as root. The issue is that footloose will then create a cluster private key that is 600, owned by root and that cannot be read by the wksctl process.

Address ignite's CNI dependency for Linux

#83 introduces an installer issue for linux hosts that are either missing or have an out-of-date installation of containernetworking/plugins.

Some potential actions are:

• point users to https://ignite.readthedocs.io/en/stable/installation/#cni-plugins
• detect Linux hosts with KVM enabled -- assume they're using ignite; auto-install cni-plugins for them alongside ignite
• don't touch existing cni-plugin installs (even despite older version)
• always overwrite cni-plugins with the firekube version
• revert #83

This is sticky because firekube doesn't manage host-global deps like docker.
CNI is effectively a host-global dependency. (ignite's CNI plugin dir is currently hard-coded)

Firekube only manages its own binaries in an isolated PATH.
This change involves modifying the host.

[ERROR BinaryInPath-strings]: None of the following binaries [strings] seem to be in your PATH

OS: ubuntu 20.04 with latest updates

See below for path:

ubuntu 5| 4:53 [~] echo $PATH
/opt/cni/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Basically I first ran a ./setup.sh, it failed, then I ran './cleanup.sh', it looks ok, then I ran ./setup.sh again, this time it came up with a different error message. See below for the output.

I've tried cleanup.sh then run the setup.sh doesn't work.

ubuntu 6|13:07 [~] git clone [email protected]:$user/wks-quickstart-firekube.git
Cloning into 'wks-quickstart-firekube'...
Warning: Permanently added the RSA host key for IP address '140.82.112.3' to the list of known hosts.
remote: Enumerating objects: 404, done.
remote: Total 404 (delta 0), reused 0 (delta 0), pack-reused 404
Receiving objects: 100% (404/404), 296.97 KiB | 1.87 MiB/s, done.
Resolving deltas: 100% (234/234), done.
ubuntu 6|13:07 [~] cd wks-quickstart-firekube

ubuntu 6|13:07 [wks-quickstart-firekube] ./setup.sh
• Using git branch: master
• Using git remote: origin

• Found jk 0.3.0
• Found footloose 0.6.3
• Found ignite 0.7.1
• Found wksctl 0.8.4
• Creating footloose manifest
• Creating SSH key
• Creating virtual machines
INFO[0000] Pulling image: weaveworks/ignite-centos:firekube-pre3 ...
INFO[0019] Creating machine: firekube-node0 ...
INFO[0019] Machine firekube-node0 is already created...
INFO[0019] Creating machine: firekube-node1 ...
INFO[0019] Machine firekube-node1 is already created...
• Creating Cluster API manifests
/home/david/wks-quickstart-firekube/setup.js:114
      privateIP: machine.runtimeNetworks[0].ip,
                                        ^
TypeError: Cannot read property '0' of undefined
    at /home/david/wks-quickstart-firekube/setup.js:114:41
Module (setup.js) has not been loaded

ubuntu 6|13:08 [wks-quickstart-firekube] ./cleanup.sh
• Deleting virtual machines
INFO[0000] Deleting machine: firekube-node0 ...
INFO[0000] Deleting machine: firekube-node1 ...

ubuntu 6|13:08 [wks-quickstart-firekube] ./setup.sh
• Using git branch: master
• Using git remote: origin

• Found jk 0.3.0
• Found footloose 0.6.3
• Found ignite 0.7.1
• Found wksctl 0.8.4
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:firekube-pre3 present locally
INFO[0000] Creating machine: firekube-node0 ...
FATA[0000] command "ignite run weaveworks/ignite-centos:firekube-pre3 --name=firekube-node0 --cpus=2 --memory=1GB --size=5GB --kernel-image=weaveworks/ignite-kernel:4.19.47 --ssh=/home/david/wks-quickstart-firekube/cluster-key.pub --ports=2222:22 --ports=6443:6443 --ports=30443:30443 --ports=30080:30080" exited with "time=\"2020-11-06T13:08:56Z\" level=info msg=\"Created VM with ID \\\"2c1e2ab16e34c2f6\\\" and name \\\"firekube-node0\\\"\"\ntime=\"2020-11-06T13:08:56Z\" level=fatal msg=\"[ERROR BinaryInPath-strings]: None of the following binaries [strings] seem to be in your PATH. Please install this tool before continuing.\\n\"\n": exit status 1
ubuntu 6|13:08 [wks-quickstart-firekube]

Mismatched namespace for docker and repo ConfigMaps

Setup appears to create the docker and repo ConfigMap resources in the weavek8sops namespace.

NAMESPACE     NAME                                 DATA   AGE
kube-public   cluster-info                         2      38m
kube-system   coredns                              1      38m
kube-system   extension-apiserver-authentication   6      38m
kube-system   kube-proxy                           2      38m
kube-system   kubeadm-config                       2      38m
kube-system   kubelet-config-1.14                  1      38m
weavek8sops   docker                               1      38m
weavek8sops   repo                                 2      38m
weavek8sops   seed-node-standard-plan              0      38m

But the docker-config.yaml and repo-config.yaml files in wks-quickstart-firekube specify a system namespace, which does not appear to exist in the cluster upon initial creation.

NAME              STATUS   AGE
default           Active   40m
kube-node-lease   Active   40m
kube-public       Active   40m
kube-system       Active   40m
weavek8sops       Active   40m

The flux agent logs errors when applying new changes pushed to the GitOps example repo.

ts=2019-09-27T21:16:32.2766965Z caller=loop.go:126 component=sync-loop event=refreshed url=https://github.com/sbernheim/wks-quickstart-firekube.git branch=master HEAD=8d34fd80bed1dd8d212256e780e53538644077e0
ts=2019-09-27T21:17:02.3934435Z caller=loop.go:126 component=sync-loop event=refreshed url=https://github.com/sbernheim/wks-quickstart-firekube.git branch=master HEAD=8d34fd80bed1dd8d212256e780e53538644077e0
ts=2019-09-27T21:17:21.666008Z caller=sync.go:479 method=Sync cmd=apply args= count=16
ts=2019-09-27T21:17:21.9648657Z caller=sync.go:545 method=Sync cmd="kubectl apply -f -" took=298.7554ms err=null output="namespace/weavek8sops unchanged\nserviceaccount/flux unchanged\nclusterrole.rbac.authorization.k8s.io/flux unchanged\nservice/memcached unchanged\nservice/podinfo unchanged\nclusterrolebinding.rbac.authorization.k8s.io/flux unchanged\ndeployment.apps/flux unchanged\ndeployment.apps/memcached unchanged\ndeployment.apps/podinfo configured\ndeployment.apps/wks-controller unchanged\ncluster.cluster.k8s.io/example unchanged\nmachine.cluster.k8s.io/master-0 unchanged\nhorizontalpodautoscaler.autoscaling/podinfo unchanged\nmachine.cluster.k8s.io/worker-0 unchanged"
ts=2019-09-27T21:17:22.1854842Z caller=sync.go:545 method=Sync cmd="kubectl apply -f -" took=220.4678ms err="running kubectl: Error from server (NotFound): error when creating \"STDIN\": namespaces \"system\" not found" output=
ts=2019-09-27T21:17:22.370605Z caller=sync.go:545 method=Sync cmd="kubectl apply -f -" took=185.0183ms err="running kubectl: Error from server (NotFound): error when creating \"STDIN\": namespaces \"system\" not found" output=
ts=2019-09-27T21:17:27.1637404Z caller=sync.go:150 component=daemon err="system:configmap/docker: running kubectl: Error from server (NotFound): error when creating \"STDIN\": namespaces \"system\" not found; system:configmap/repo: running kubectl: Error from server (NotFound): error when creating \"STDIN\": namespaces \"system\" not found"
ts=2019-09-27T21:17:32.4680012Z caller=loop.go:126 component=sync-loop event=refreshed url=https://github.com/sbernheim/wks-quickstart-firekube.git branch=master HEAD=8d34fd80bed1dd8d212256e780e53538644077e0
ts=2019-09-27T21:18:02.5635086Z caller=loop.go:126 component=sync-loop event=refreshed url=https://github.com/sbernheim/wks-quickstart-firekube.git branch=master HEAD=8d34fd80bed1dd8d212256e780e53538644077e0
ts=2019-09-27T21:18:32.6764407Z caller=loop.go:126 component=sync-loop event=refreshed url=https://github.com/sbernheim/wks-quickstart-firekube.git branch=master HEAD=8d34fd80bed1dd8d212256e780e53538644077e0

Finish cleaning up the last shellcheck issues

We should be able to either fix the few remaining linting issues (or ignore some).

Doesnt' work on my Mac or Linux

/Users/bbarclay/wks-quickstart-firekube/setup.js:105
const sshPort = machine => machine.ports.find(p => p.guest == 22).host;
^
TypeError: Cannot read property 'find' of null
at sshPort (/Users/bbarclay/wks-quickstart-firekube/setup.js:105:42)
at /Users/bbarclay/wks-quickstart-firekube/setup.js:115:16
Module (setup.js) has not been loaded

My docker ps on the mac doesn't work now.

In linux the error was

command "ignite run weaveworks/ignite-centos:firekube-pre3 --name=firekube-node0 --cpus=2 --memory=1GB --size=5GB --kernel-image=weaveworks/ignite-kernel:4.19.47 --ssh=/var/www/laradock/wks-quickstart-firekube/cluster-key.pub --ports=2222:22 --ports=6443:6443 --ports=30443:30443 --ports=30080:30080" exited with "time="2020-12-23T08:13:28Z" level=info msg="containerd image \"weaveworks/ignite-centos:firekube-pre3\" not found locally, pulling..."\ntime="2020-12-23T08:14:16Z" level=info msg="Starting image import..."\ntime="2020-12-23T08:14:41Z" level=info msg="Imported OCI image \"weaveworks/ignite-centos:firekube-pre3\" (1.6 GB) to base image with UID \"756cc53de5a0c781\""\ntime="2020-12-23T08:14:41Z" level=info msg="containerd image \"weaveworks/ignite-kernel:4.19.47\" not found locally, pulling..."\ntime="2020-12-23T08:14:45Z" level=info msg="Imported OCI image \"weaveworks/ignite-kernel:4.19.47\" (49.6 MB) to kernel image with UID \"9414cdaba181972c\""\ntime="2020-12-23T08:14:46Z" level=info msg="Created VM with ID \"f265e346fe62fdf7\" and name \"firekube-node0\""\ntime="2020-12-23T08:14:46Z" level=fatal msg="[ERROR ExistingFile--dev-kvm]: File /dev/kvm, does not exist\n[ERROR ExistingFile--opt-cni-bin-bridge]: File /opt/cni/bin/bridge, does not exist\n"\n": exit status 1

WKS-quickstart-firekube setup should run a footloose server

Currently, the set of machines created via footloose create is disjoint from the set created via ReST calls to the footloose server. Either we need to create our initial setup via the server or footloose needs to support merging the two namespaces.

The requested URL returned error: 400

Just cloned the repo and ran setup.sh. Note: I'm on an M1 mac

% ./setup.sh
• Using git branch: master
• Using git remote: origin

• Downloading https://github.com/jkcfg/jk/releases/download/0.3.0/jk-darwin-• error: uknown arch: arm64
-=O=#   #    #     #                                                                                          
curl: (22) The requested URL returned error: 400

Failed to create SSH client: failed to connect to 127.0.0.1:2222

[daniel@reef wks-quickstart-firekube ]$ ./setup.sh 
• Found jk 0.3.0
• Found footloose 0.6.1
• Found ignite 0.5.2
• Found wksctl 0.8.0-beta.1
• Creating footloose manifest
• Creating SSH key
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:latest present locally 
INFO[0000] Creating machine: 9213fe1ac2ab ...           
INFO[0000] Machine 9213fe1ac2ab is already created...   
INFO[0000] Creating machine: b77ff909c2a8 ...           
INFO[0000] Machine b77ff909c2a8 is already created...   
• Creating Cluster API manifests
• Updating container images and git parameters
• Pushing initial cluster configuration
Everything up-to-date
• Installing Kubernetes cluster
FATA[2019-09-17T10:55:54+02:00] Failed to create SSH client: failed to connect to 127.0.0.1:2222: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain 
[daniel@reef wks-quickstart-firekube ]$ 

Fails to start with wksctl 0.8.1 or 0.8.2-alpha1

I'm running on this system:

uname -a
Linux rouquette-Precision-7530 5.0.0-31-generic #33-Ubuntu SMP Mon Sep 30 18:51:59 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-07T14:30:40Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

With wksctl 0.8.1, I get this behavior:

./setup.sh 
• Using git branch: master
• Using git remote: origin

• Found jk 0.3.0
• Found footloose 0.6.2
• Found ignite 0.6.2
• Found wksctl 0.8.1
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:firekube-pre3 present locally 
INFO[0000] Creating machine: firekube-node0 ...         
INFO[0000] Machine firekube-node0 is already created... 
INFO[0000] Creating machine: firekube-node1 ...         
INFO[0000] Machine firekube-node1 is already created... 
• Creating Cluster API manifests
• Updating container images and git parameters
• Pushing initial cluster configuration
[master 20cf4a5] Initial cluster configuration
 4 files changed, 66 insertions(+), 2 deletions(-)
 create mode 100644 footloose.yaml
 create mode 100644 machines.yaml
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 12 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.62 KiB | 1.62 MiB/s, done.
Total 6 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To github.com:NicolasRouquette/wks-quickstart-firekube.git
   0d7364e..20cf4a5  HEAD -> master
• Installing Kubernetes cluster
Error: failed to create SSH client: failed to connect to 127.0.0.1:2222: ssh: handshake failed: ssh: host key mismatch
Usage:
  wksctl apply [flags]

Flags:
      --cluster string              Location of cluster manifest (default "cluster.yaml")
      --config-directory string     Directory containing configuration information for the cluster (default ".")
      --git-branch string           Git branch WKS should use to sync with your cluster (default "master")
      --git-deploy-key string       Path to the Git deploy key
      --git-path string             Relative path to files in Git (default ".")
      --git-url string              Git repo containing your cluster and machine information
  -h, --help                        help for apply
      --machines string             Location of machines manifest (default "machines.yaml")
      --namespace string            namespace override for WKS components (default "weavek8sops")
      --sealed-secret-cert string   Path to a certificate used to encrypt sealed secrets
      --sealed-secret-key string    Path to a key used to decrypt sealed secrets
      --use-manifest-namespace      use namespaces from supplied manifests (overriding any --namespace argument)

Global Flags:
  -v, --verbose   Enable verbose output

failed to create SSH client: failed to connect to 127.0.0.1:2222: ssh: handshake failed: ssh: host key mismatch

For wksctl 0.8.2-alpha1, I modified cluster.yaml to remove spec.providerSpec.sshKeyPath
Instead, I added to the setup.sh script:

wksctl apply --git-url="$(git_http_url "$(git_remote_fetchurl "${git_remote}")")" --git-branch="$(git_current_branch)" --ssh-key ${SCRIPT_DIR}/$cluster_key ${git_deploy_key} 

I get the same error:

...
• Installing Kubernetes cluster
Error: failed to create SSH client: failed to connect to 127.0.0.1:2222: ssh: handshake failed: ssh: host key mismatch

Error: containerd-shim-runc-v1: file does not exist: unknown

System info

alex@nuc7:~/wks-quickstart-firekube$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
alex@nuc7:~/wks-quickstart-firekube$ docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false
alex@nuc7:~/wks-quickstart-firekube$ ignite version
FATA[0000] This program needs to run as root.           
alex@nuc7:~/wks-quickstart-firekube$ sudo ignite version
Ignite version: version.Info{Major:"0", Minor:"6", GitVersion:"v0.6.0", GitCommit:"a8220f7b4a07ed17935fa909287286406b58106e", GitTreeState:"clean", BuildDate:"2019-08-30T22:30:57Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Firecracker version: v0.17.0
alex@nuc7:~/wks-quickstart-firekube$ 

The error when setting up:

./setup.sh --git-deploy-key  ./deploy-firekube
• Using git deploy key: ./deploy-firekube
• Found jk 0.3.0
• Found footloose 0.6.1
• Found ignite 0.6.0
• Downloading https://github.com/weaveworks/wksctl/releases/download/0.7.0/wksctl-0.7.0-linux-x86_64.tar.gz
################################################################################################################################################ 100.0%
examples/footloose
examples/footloose/README.md
examples/footloose/centos7
examples/footloose/centos7/docker
examples/footloose/centos7/docker/multimaster.yaml
examples/footloose/centos7/docker/singlemaster.yaml
examples/footloose/centos7/ignite
examples/footloose/centos7/ignite/multimaster.yaml
examples/footloose/centos7/ignite/singlemaster.yaml
examples/footloose/cluster.yaml
examples/footloose/docker-config.yaml
examples/footloose/machines-multimaster.yaml
examples/footloose/machines.yaml
examples/footloose/repo-config.yaml
examples/footloose/ubuntu1804
examples/footloose/ubuntu1804/docker
examples/footloose/ubuntu1804/docker/multimaster.yaml
examples/footloose/ubuntu1804/docker/singlemaster.yaml
examples/footloose/ubuntu1804/ignite
examples/footloose/ubuntu1804/ignite/multimaster.yaml
examples/footloose/ubuntu1804/ignite/singlemaster.yaml
examples/footloose/upload-controller-image.sh
examples/footloose/upload-image.sh
examples/footloose/user.sudoers
examples/gce
examples/gce/.gitignore
examples/gce/README.md
examples/gce/cluster.yaml
examples/gce/create-instances.sh
examples/gce/create-network.sh
examples/gce/delete-instances.sh
examples/gce/delete-network.sh
examples/gce/generate-machines-manifest.js
examples/gce/generate-machines-manifest.sh
examples/gce/repo-config.yaml
examples/vagrant
examples/vagrant/.gitignore
examples/vagrant/README.md
examples/vagrant/Vagrantfile
examples/vagrant/cluster.yaml
examples/vagrant/docker-config.yaml
examples/vagrant/machines.yaml
examples/vagrant/machines2.yaml
examples/vagrant/repo-config.yaml
wksctl
• Found wksctl 0.7.0
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Creating SSH key: cluster-key ...            
INFO[0001] Pulling image: weaveworks/ignite-centos:latest ... 
INFO[0006] Creating machine: firekube-node0 ...         
FATA[0025] command "ignite run weaveworks/ignite-centos:latest --name=firekube-node0 --cpus=2 --memory=1GB --size=5GB --kernel-image=weaveworks/ignite-kernel:4.19.47 --ssh=/home/alex/wks-quickstart-firekube/cluster-key.pub --ports=2222:22 --ports=6443:6443 --ports=30443:30443 --ports=30080:30080" exited with "time=\"2019-09-14T07:41:30Z\" level=info msg=\"Docker image \\\"weaveworks/ignite-centos:latest\\\" not found locally, pulling...\"\ntime=\"2019-09-14T07:41:42Z\" level=info msg=\"Starting image import...\"\ntime=\"2019-09-14T07:41:44Z\" level=info msg=\"Imported OCI image \\\"weaveworks/ignite-centos:latest\\\" (226.8 MB) to base image with UID \\\"65cb7f0e65d1a599\\\"\"\ntime=\"2019-09-14T07:41:45Z\" level=info msg=\"Created VM with ID \\\"7a71487cbbce16e1\\\" and name \\\"firekube-node0\\\"\"\ntime=\"2019-09-14T07:41:45Z\" level=info msg=\"Pulling image \\\"weaveworks/ignite:v0.6.0\\\"...\"\ntime=\"2019-09-14T07:41:48Z\" level=fatal msg=\"failed to start container for VM \\\"7a71487cbbce16e1\\\": runtime \\\"io.containerd.runc.v1\\\" binary not installed \\\"containerd-shim-runc-v1\\\": file does not exist: unknown\"\n": exit status 1 

I did have ignite working on its own on this computer, it does have KVM etc.

Remove the need for a flux deploy key

We still require a deploy key for flux and that's a step it'd be good to remove for a more user-friendly install. This should be possible:

• flux now has a read-only mode
• public repositories can be cloned without authentication!

Fails to install Kubernetes cluster

Hi, I am running setup on my system:
Linux shawtao-PC 5.8.0-53-generic #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

kubelet version:
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:18:45Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}

An error: Failed to install :kubelet when installing Kubernetes cluster.

• Installing Kubernetes cluster
INFO[2021-05-23T16:39:23+08:00] Applying resource="install:base"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:device-mapper-persistent-data"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:lvm2"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:yum-utils"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:yum-versionlock"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:config"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:config-file-1"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:config-file-2"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:config-file-0"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:cri"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:docker"
INFO[2021-05-23T16:39:24+08:00] Applying resource="lock-package:docker"
INFO[2021-05-23T16:39:24+08:00] Applying resource="systemd:daemon-reload"
INFO[2021-05-23T16:39:24+08:00] Applying resource="service-init:docker-service"
INFO[2021-05-23T16:39:24+08:00] Applying resource="install:k8s"
INFO[2021-05-23T16:39:25+08:00] Applying resource="install:kubelet"
ERRO[2021-05-23T16:44:26+08:00] Failed resource="install:kubelet"
command exited with 1
INFO[2021-05-23T16:44:26+08:00] State of Resource 'top' is Invalid.
Explanation:
{
"resource": "top",
"status": "Invalid"
}
ERRO[2021-05-23T16:44:26+08:00] Failed resource="install:k8s"
Apply failed because a child failed
INFO[2021-05-23T16:44:26+08:00] State of Resource 'top' is Invalid.
Explanation:
{
"resource": "top",
"status": "Invalid"
}
ERRO[2021-05-23T16:44:26+08:00] Apply of Plan failed:
Apply failed because a child failed
Error: failed to set up seed node (127.0.0.1): Apply failed because a child failed

Typo in README?

kubectl apply --dry-run -k github.com/stefanprodan/podinfo//kustomize -o yaml > podinfo.yaml
git add podinfo.yaml
git commit -a -m 'Add podinfo Deployment'
git push

Is there meant to be a double / in the URL?

Automatically use the footloose docker backend on macOS

ignite/firecracker require KVM and so won't work on macOS

Is this meant to be run on individual hosts?

What i mean is, if I have 3 VMs in aws and I want to avoid single point of failure (I presume in the demo, both master and node(s) are on the same actual host so if the host goes down you lose everything), how does that work? Is there a way to tell firekube the information of the various hosts whether they be baremetal and/or cloud VMs? or would you go on each host and run the cli?

also is this still being maintained? It seems like activity in this repo is from years ago, and the documentation still seems to assume docker and I don't see direct containerd or crio mentioned too much.

wksctl is now versioned with a 'v' prefix which breaks the curl download

This was identified and filed against wksctl - see weaveworks/wksctl#169

"wksctl init" failure

Expectation:
Flux.yaml and wks-controller.yaml get updated correctly with git-repo url and wks-controller image version.

Outcome:

1. Neither file is updated and wksctl init fails silently.
2. wksctl apply points to https://github.com/weaveworks/wks-quickstart-firekube rather than my fork
3. Outdated wksctl-controller gets loaded into cluster based on outdated flux_yaml file, resulting in a perpetual failure with ImagePullBackOff:

container "controller" in pod "wks-controller-849c449bf4-csbnc" is waiting to start: trying and failing to pull image 

Updates to script required to make it work

@jrryjcksn @palemtnrider here was the sequence of actions necessary to get the quickstart running:

# 1. fork wks-quickstart-firekube, either with hub or in the GitHub UI

GITHUB_ORG=dimitropoulos
REPO_NAME=wks-quickstart-firekube
DEPLOY_KEY_NAME=$REPO_NAME-deploykey

git clone http://github.com/$GITHUB_ORG/$REPO_NAME
cd $REPO_NAME

sudo rm $(which wk)
sudo rm $(which ignite)
sudo rm $(which footloose)
sudo rm $(which wksctl)

echo "$DEPLOY_KEY_NAME*" > ./.gitignore
ssh-keygen -N "" -q -f $DEPLOY_KEY_NAME

# 1. remove the `sshKeyPath` line from `cluster.yaml`

hub api \
--method POST \
/repos/$GITHUB_ORG/$REPO_NAME/keys \
--field title=$DEPLOY_KEY_NAME \
--field key="$(cat ~/src/github.com/$GITHUB_ORG/$REPO_NAME/$DEPLOY_KEY_NAME.pub)" \
--field readOnly=false


# 1. update the image
# ```diff
# containers:
# - name: controller
# image: docker.io/wksctl/controller:master-8e831136
# ```

./setup.sh

export KUBECONFIG=/home/dimitri/.wks/weavek8sops/example/kubeconfig

We can spin off issues from here, e.g. updating the image.

Error: unknown command "init" for "wksctl"

• Found wksctl 0.7.0
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Creating SSH key: cluster-key ...            
INFO[0003] Pulling image: weaveworks/ignite-centos:latest ... 
INFO[0050] Creating machine: firekube-node0 ...         
INFO[0071] Creating machine: firekube-node1 ...         
• Creating Cluster API manifests
• Updating container images and git parameters
Error: unknown command "init" for "wksctl"
Run 'wksctl --help' for usage.
unknown command "init" for "wksctl"

Only Master Node is Up but NotReady

Environment
macOS Catalina - V 10.15.7
docker desktop - v 3.3.3(64133) disabled Kubenetes
kubernetes

k version --short
Client Version: v1.19.7
Server Version: v1.16.11

./setup.sh had a good progress. but it provides only one kubernetes node (master) and that's also not in Ready state.

Progress of Setup.sh

footloose server and container status

Kubernetes Nodes

kubectl get all -A

Starting setup.sh with git-deploy-key fails

Creating a firekube cluster with the --git-deploy-key as described in README fails, with an error indicating that the git URL was not authenticatable via the provided authentication.

This seems to be because of an issue, where there are git_http_url and git_ssh_url helpers defined, but only one is ever used in setup.sh:

 log "Updating container images and git parameters"
-wksctl init --git-url=$(git_http_url $(git config --get remote.origin.url)) --git-branch=$(git rev-parse --abbrev-ref HEAD)
+wksctl init --git-url=$(git_ssh_url $(git config --get remote.origin.url)) --git-branch=$(git rev-parse --abbrev-ref HEAD)

 log "Installing Kubernetes cluster"
-wksctl apply --git-url=$(git_http_url $(git config --get remote.origin.url)) --git-branch=$(git rev-parse --abbrev-ref HEAD) $git_deploy_key
+wksctl apply --git-url=$(git_ssh_url $(git config --get remote.origin.url)) --git-branch=$(git rev-parse --abbrev-ref HEAD) $git_deploy_key
 wksctl kubeconfig

These simple changes appear to have solved it for me, and my cluster comes online successfully, but ideally the PR that resolves this will have some logic so the user does not need to update setup.sh and the process described in README will be all that is needed.

error misdirection when origin matches `weaveworks` rather than the fork

when running ./setup.sh I got the following output:

$ ./setup.sh 
• Found jk 0.3.0
• Found footloose 0.6.1
• Found ignite 0.5.3
• Found wksctl 0.8.0-beta.2
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:latest present locally 
INFO[0000] Creating machine: 0ce12e0e2ec3 ...           
INFO[0000] Machine 0ce12e0e2ec3 is already created...   
INFO[0000] Creating machine: 7407ed83c1c5 ...           
INFO[0000] Machine 7407ed83c1c5 is already created...   
• Creating Cluster API manifests
• Updating container images and git parameters
• Pushing initial cluster configuration
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 32 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 322 bytes | 322.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To github.com:dimitropoulos/wks-quickstart-firekube.git
   8905010..fb82c4d  master -> master
• Installing Kubernetes cluster
Error: MachinesManifestPath: machines manifest not found
Usage:
  wksctl apply [flags]

Flags:
      --cluster string              Location of cluster manifest (default "cluster.yaml")
      --config-directory string     Directory containing configuration information for the cluster (default ".")
      --git-branch string           Git branch WKS should use to sync with your cluster (default "master")
      --git-deploy-key string       Path to the Git deploy key
      --git-path string             Relative path to files in Git (default ".")
      --git-url string              Git repo containing your cluster and machine information
  -h, --help                        help for apply
      --machines string             Location of machines manifest (default "machines.yaml")
      --namespace string            namespace override for WKS components (default "weavek8sops")
      --sealed-secret-cert string   Path to a certificate used to encrypt sealed secrets
      --sealed-secret-key string    Path to a key used to decrypt sealed secrets
      --use-manifest-namespace      use namespaces from supplied manifests (overriding any --namespace argument)

Global Flags:
  -v, --verbose   Enable verbose output

MachinesManifestPath: machines manifest not found

The only modification to the script in this case was to add --verbose to wksctl apply

setup.sh is stuck on "INFO[0002] Creating machine: firekube-node1 ..."

OS: ubuntu 20.04

It is stuck on "INFO[0002] Creating machine: firekube-node1 ..."
I can CTRL+C to stop it.
Is there any logs I can post here for troubleshooting?

firecracker 6|22:04 [wks-quickstart-firekube] ./setup.sh
• Using git branch: master
• Using git remote: origin

• Found jk 0.3.0
• Found footloose 0.6.3
• Found ignite 0.7.1
• Found wksctl 0.8.4
• Creating footloose manifest
• Creating virtual machines
INFO[0000] Docker Image: weaveworks/ignite-centos:firekube-pre3 present locally
INFO[0000] Creating machine: firekube-node0 ...
INFO[0002] Creating machine: firekube-node1 ...

Use git+ssh url to push and hand off the https url to flux

We want the best of both worlds:

• Be able to push to git from setup.sh -> git URL
• Have flux pull from the git repo without auth (without deploy key) -> https URL

So we need to:

• git clone the repo with the git+ssh URL
• hand off the git+https URL to wksctl apply

It worked on my machine because I had 🤦‍♂️ :

[url "[email protected]:"]
        insteadOf = https://github.com/

Dependencies aren't automatically upgraded when needed

When bumping the required version for one of our dependencies, such a footloose, setup.sh will just bail out instead of trying to upgrade (download) the new dependency.

$ ./setup.sh 
• Found jk 0.3.0
• Found footloose 0.6.1
• error: footloose: Found version 0.6.1 but 0.6.2 is the minimum required version.

firekube requires footloose to spawn VMs that will be used as Kubernetes nodes.

Please install footloose version 0.6.2 or later:

  • GitHub project  : https://github.com/weaveworks/footloose
  • Latest release  : https://github.com/weaveworks/footloose/releases
  • Installation    : https://github.com/weaveworks/footloose#install
  • Required version: 0.6.2

We should automatically remove the old version and download the new version!