Right now we are just using println! for all of our logging, what we want to do is support a structured logging format. To do this we will leverage tracing and tracing_subscriber.

1. Replace all println! with an equivalent tracing event
2. Create a custom tracing subscriber that emits logs like we currently have
   • Trace/Debug should not log
   • Info should be unformatted
     • It should be prefixed by the job if in a job, to know if it is in a job use spans
   • Warn/Error should be prefixed with that level
   • We want the events and spans to have metadata, we can create this in a strongly typed manor using the valuable API
3. Add support to the CLI for json logging using the default Json formatter
   • This can either be with a cli flag like -f json or an env var

I imagine in code this will look something like this, the metadata should prob be more structured though via the valuable trait.

info!("Just a log line", type%="pipeline_start");
{
	let _job_span = span!("Job Name", type=%"job_span");
	info!("Log from job", type=%"job_log", stream=%"stdout");
}
info!("All done!", type=%"pipeline_end");

Just a log line
Job Name: Log from job
All done

I've installed the cicada CLI, and generated a file named Astro.ts, but this is the error i get every time.

Cicada has been authorized in github.

cicada run Astro

 ◥◣ ▲ ◢◤  Cicada is in alpha, it may not work as expected
  ◸ ▽ ◹   Please report any issues here: https://github.com/cicadahq/cicada

✅ Downloaded deno v1.32.5
[error] specified file not found in archive

With 0.1.x we will prob not follow semver very closely as thing might break, once we go to 0.2.x and up we will want to check that the pipeline is compatible with the current CLI.

I imagine this is just a regex that grabs the version from the deno import then semver compares it with the crate's version.

The permission screen to sign in with Cicada

Can access basically everything on my account!

As comaprison, here is another service that needs access to code: Deno deploy

And similar services like cloudflare pages, etc will allow you to select what repositories to give access to.

Thanks for the early waitlist invite, but I can not test Cicada with these permissions.

When package managed, e.g. from AUR, brew, etc the binary should not self update but still have warning when there is an update.

I have been trying out Cicada in one of my projects, but if I go to the dashboard I can only view the logs for the very first pipeline that I ran. I can only view the results for later pipeline runs in the links that are posted by the Cicada Bot in my PRs, but not on the dashboard.

In the browser console (Firefox) there are a few errors:

Object { input: {…}, result: TRPCClientError, elapsedMs: 384, context: {} }
​
context: Object {  }
​
elapsedMs: 384
​
input: Object { namespace: undefined, repo: undefined }
​
result: TRPCClientError: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "namespace"
    ],
    "message": "Required"
  }
]
​
<prototype>: Object { … }

Firefox can’t establish a connection to the server at wss://cicada-ws.fly.dev/. 4 [_app-5ea6b5860d5394e9.js:5:66099](https://cicada.build/_next/static/chunks/pages/_app-5ea6b5860d5394e9.js)
Firefox can’t establish a connection to the server at wss://cicada-ws.fly.dev/. [_app-5ea6b5860d5394e9.js:5:66099](https://cicada.build/_next/static/chunks/pages/_app-5ea6b5860d5394e9.js)
Firefox can’t establish a connection to the server at wss://cicada-ws.fly.dev/.

This is my pipeline file:

import { Job, Pipeline } from "https://deno.land/x/[email protected]/mod.ts";

const job = new Job({
  name: "My First Job",
  image: "ubuntu:22.04",
  steps: [
    {
      name: "Run bash",
      run: "echo hello from bash!",
    },
    {
      name: "Run deno/typescript",
      run: () => {
        console.log("Hello from deno typescript");
        console.log("I can read the local file system");
        console.log("e.g. here are all the directories in my project");
        console.log(Array.from(Deno.readDirSync("/app")));
        console.log("Or I can see my environment variables");
        console.log(Deno.env.toObject());
      },
    },
  ],
});

export default new Pipeline([job], {
  on: {
    pullRequest: ["main"],
    push: ["main"],
  },
});

I just followed the quickstart and installed cicada with npm, but cicada init throws an error after answering the prompts:

cicada init
✔ What should we call your pipeline · my-pipeline
✔ Select a template · Default
✔ Would you like to setup autocomplete for VSCode? · yes
[error] program not found

There is a .cicada folder made with a pipeline however.

deno --version

deno 1.32.5 (release, x86_64-pc-windows-msvc)
v8 11.2.214.9
typescript 5.0.3

cicada --version
cicada 0.1.41

node --version
v20.0.0

npm --version
9.6.4

when running cicada init it would be great to update the deno path so the repo knows the cicada file is a deno one

  "deno.enablePaths": [
    ".cicada",
  ],

Modules

This is an RFC for how Cicada should integrate with 3rd-party code. First we propose what we think is best, then we ask some questions. We’d love to hear your thoughts!

Proposal

A Cicada module is a custom application for the Cicada platform that performs a complex but frequently repeated jobs/steps/tasks. Think of them just CircleCI Orbs, GitHub Actions actions, Buildkite Plugins, but for Cicada. Our modules are modelled heavily on how Deno does modules.

A module can be an abstraction over a Job, an abstraction over a Step, or a utility function that can be used inside a step.

Module Registry

A module is just deno/typescript. So theoretically, the code can be hosted anywhere and just imported into your pipeline. However, rather than having hundreds of Cicada modules strewn throughout the internet, we want to have a registry that makes it easy to find all the different Cicada modules. This is similar to other popular CI/CD providers like the GitHub Actions marketplace or CircleCI orbs. We want our registry to be just like deno.land, but only for Cicada modules.

However, the registry isn't built yet. In the meantime, we have have created github.com/cicadahq/modules. We will have a folder for every module and each module will have a mod.ts file. e.g. git/mod.ts, npm/mod.ts, github/mod.ts. Once we build the registry, we will likely split each module up into different repos.

Module Usage and Examples

Below are a few examples of how you would use a module inside a Cicada pipeline. Our aim is to make it as simple and idiomatic as possible. We'd love to hear your thoughts!

Replacing Steps

Running tests with NPM

import { Job, Pipeline, Secret } from "https://deno.land/x/cicada/lib.ts";
import * as npm from "https://deno.land/x/cicada-modules/npm/mod.ts"

const test = new Job({
	image: "ubuntu:22.04",
	steps: [
		npm.installNode({version: "16.4"}), 
		npm.installDependencies(), 
		npm.lint(), 
		npm.test(), 
	]
})

export new Pipeline([ test ])

Publish to NPM package registry

import { Job, Pipeline, Secret } from "https://deno.land/x/cicada/lib.ts";
import * as npm from "https://deno.land/x/cicada-modules/npm/mod.ts"

const publish = new Job({
	image: "ubuntu:22.04",
	steps: [
		npm.installNode({version: "16.4"}), 
		npm.installDependencies(), 
		npm.publish(token: new Secret.value("npm-token") ), 
	]
})

export new Pipeline([ publish ])

Deploy using terraform

import { Job, Pipeline, Secret } from "https://deno.land/x/cicada/lib.ts";
import * as terraform from "https://deno.land/x/cicada-modules/terraform/mod.ts"

const deploy = new Job({
	image: "ubuntu:22.04",
	steps: [
		terraform.install({token: new Secret.value("terraform-token")), 
		terraform.init(),
		terraform.format(),
		terraform.plan(),
		terraform.apply(),
	]
})

export new Pipeline([ deploy ])

Run security checks using Snyk

import { Job, Pipeline, Secret } from "https://deno.land/x/cicada/lib.ts";
import * as snyk from "https://deno.land/x/cicada-modules/snyk/mod.ts"

const security = new Job({
	image: "ubuntu:22.04",
	steps: [
		snky.install({token: new Secret.value("snyk-token")), 
		snyk.codeTest(),
		snyk.iacTest(),
		snyk.containerTest(),
		snyk.publishResultsToGitHub({token: Secret.value("github-token"))
	]
})

export new Pipeline([ security ])

Replacing Jobs

Each of the above jobs, could be replaced using a job module like this

import { Job, Pipeline, Secret } from "https://deno.land/x/cicada/lib.ts";
import * as npm from "https://deno.land/x/cicada-modules/npm/mod.ts"
import * as terraform from "https://deno.land/x/cicada-modules/terraform/mod.ts"
import * as snyk from "https://deno.land/x/cicada-modules/snyk/mod.ts"

export new Pipeline([
		new npm.TestSuite({ nodeVersion: "16.4"}),
		new npm.Publish({ token: new Secret.value("npm-token") }),
		new terraform.Deploy({ token: new Secret.value("terraform-token") }),
		new snyk.SecurityChecks({ token: new Secret.value("snyk-token") }),
	])

Questions

1. “Module”: Do you like the name “modules”. We’ve also been considering package, plugin, extension, wing. We want it to be as easy as possible to understand. 3rd parties. We find CircleCI “orbs” fear it could be a barrier to Cicada adoption
2. Where should we start: What modules would you like to see? Rather than just saying “git” or “npm”, please give some concrete examples of workflows that you use regularly in CI/CD. e.g. “I always check out the existing git repo” or “I bump the NPM package.json version”. We would like to focus on building modules for these first!
3. Module naming convention: A module can output Jobs, Steps, or general utility functions. How should we make it clear which exports are which? As you can see above, the only difference between a step and a job is the job uses title case. However, this doesn’t account for utility functions. Some other options are:
   • Prefix each with the type: npm.Job.Test, npm.step.test, npm.fn.test
   • Only prefix jobs like npm.Job.test. Donnt
   • Title case jobs, leave the rest as camel case
   • Don’t prefix anything or have any standard! Users should just rely on deno + typescript + good docs to work out which is which

Edits

• [April 19, 12:30pm] Updated Module Registry description
• [April 19, 12:38pm] Added required "image" field for jobs

The image field should support support a local Dockerfile like docker compose

Proposed syntax:

import { Job, Pipeline } from "https://deno.land/x/cicada/lib.ts";

const job = new Job({
  image: {
    build: {
      context: ".",
      dockerfile: "Dockerfile",
    }
  },
  steps: [
    "echo 'Hello World!"
  ]
});

export default new Pipeline([job]);

Currently due to Deno requiring glibc alpine support is not very good. This is tracked upstream at denoland/deno#3711

One solution is using the frolvlad/alpine-glibc image which sets up glibc which is what the official Deno alpine image does: https://github.com/denoland/deno_docker/blob/54c9c57fccf473d7eb2c69d707d5a0c451b75b79/alpine.dockerfile#L16

Hi, this project looks great !

I followed the quickstart instructions. After running cicada init and installing deno, I'm stuck at cicada run :

[error] Cicada requires buildctl >=0.11 to run.

I'm not sure what to do next. I'm on ubuntu linux, and docker cli is intalled on my machine. But buildctl is indeed not installed, but I don't know how to fix it