panzerstadt / zmk-pw-builder

backend API for bt60 zmk.uf2 builds

init, pull, deploy

# at repo root direcotry
git pull
git submodule update --init --recursive
gcloud auth login
# login to gcloud using [email protected]
gcloud config set run/region europe-north1
gcloud run deploy
# service name is zmk-pw-builder
# source is root directory

what you'll see is:

Deploying from source. To deploy a container use [--image]. See https://cloud.google.com/run/docs/deploying-source-code for more details.
Source code location (/Users/tliqun/Documents/Github-work/zmk-pw-builder):
Next time, use `gcloud run deploy --source .` to deploy the current directory.

Service name (zmk-pw-builder):
This command is equivalent to running `gcloud builds submit --tag [IMAGE] /Users/tliqun/Documents/Github-work/zmk-pw-builder` and `gcloud run deploy zmk-pw-builder --image [IMAGE]`

Building using Dockerfile and deploying container to Cloud Run service [zmk-pw-builder] in project [polarityworks] region [europe-north1]
⠛ Building and deploying... Building Container.
  ✓ Uploading sources...
  ⠛ Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/
  builds/baabc825-0672-4185-bac1-959cfaa78505?project=152784376009].
  . Creating Revision...
  . Routing traffic...

local test run

docker build -t zmknode . docker run -p 8080:8080 --name zmkbuilder zmknode

local enter docker and see see

docker exec -it zmkbuilder bash

local stop

docker ps docker stop zmkbuilder docker rm zmkbuilder

run the above commands go to localhost:8080 you should be able to see your docker building corne

deploy

gcloud run deploy pick europe-north1

default configs for gcloud

gcloud config set run/region europe-north1

ref

basically what i need to run is to initialize west

• a container with zmk-build-arm -> save to cloud registry
• CMD will pass it all the required arguments and just run west build

while running

• run CMD to update
• rebuild container an version up

FROM zmkfirmware/zmk-build-arm:2.5 COPY . . RUN ["west", "init", "-l", "config"] --> check if this config only requires west.yml or other stuff (most probably just west.yml) RUN ["west", "update"] RUN ["west", "zephyr-export"]

and then: https://docs.zephyrproject.org/latest/guides/west/build-flash-debug.html#west-building

FROM docker-zmk

these settings are per-device

CMD ["west", "build", "-d", "/build/output", "-s", "zmk/app", "-b", "nice_nano_v2", "--", "-DSHIELD=corne_left", "-DZMK_CONFIG=/config"]

ref

this is for bt60 only so

the below builds bt60 to build/output folder, using /config's files.

config files will be supplied by the frontend

west build -d /build/output -s zmk/app -b bt60 -- -DZMK_CONFIG=/config

volumes: - ./zmk-build:/build/output/zephyr

zmk-pw-builder's Issues

Make ZMK builder easily upgradeable

Updating the cached version of zmk the builder uses to the latest version, we can sort the zmk version just need some guidance on how to tell the builder to use a new version

How To Build:

• refer to README.md

ZMK upgrade Steps:

• update dockerfile base image. its currently on zmkfirmware/zmk-build-arm:stable
• build

Adding boards:

• add zmk config repo into board-config directory
• add the directory into mapper.js
• build

Guide on how to support for ROTR, ALU40, 3x3 macropad and 3x2 macropad

Steps:

• prep zmk config for rotr
• got to zmk-pw-builder
• pull the zmk config repo into board-config
• update .gitmodules to point to latest branch
• update mapper.js to include the new board-config
• make sure all the board-configs are pulled (check README.md in board-config)
• deploy with gcloud run deploy (check main README.md)

Adding support for building from multiple zmk branches

this sounds like it needs a rework of the dockerfile and builder hmm

what do we want to achieve?

• being able to build differnet uf2 using different zmk builds? e.g. zmk-armv1 for bt60, zmk-armvXX for rotr etc

how do we want to set this up? options:

A: different zmk branches for different board configs?

this requires a bit of rework on the nodejs implementation, as well as builds in the dockerfile. it doesn't sound too hard, but might not scale too well if we end up with 2- different zmks for 20 different boards (the single dockerfile would be like a thousand lines long with different things all in the same place, making it brittle / hard to debug)

B. different dockerfiles for different zmk branches?

this keeps the dockerfiles small at least, and we would use the same code as the one we have now. but this requires us to add another layer to the builder, either:
B1: a script to build all the separate dockerfiles, run seperate gcloud deployments, and end up with separate endpoints that each board must call (increase complexity on the client size (pw-lp) OR
B2 a dockerfile/container management layer to deploy multiple containers together, and serve them all behind a unified endpoint, where the load balancer would punt each request to different instances.

i personally am leaning toward option B1 (multiple dockerfiles, multiple endpoints) as there is a lot of complexity in keeping state on different connections through a load balancer, and it doesn't seem like we need it yet

B1 diagram

Client A           Client B     Client C          Client D
    | bt60v1          |  bt65      |  bt75        | rotr
    v                 v            v              v
Server zmk-armv1      Server zmk-armv3       Server zmk-armv88

B2 diagram

Client A   Client B   Client C
    |         |         |
    v         v         v
+----------------------------+
|        Load Balancer       |
+----------------------------+
    |         |         |
    v         v         v
Server 1   Server 2   Server 3