chewkeanho / automataci

An open-source, redistributable, template-guided, and semi-autonomous CI infrastructure readily available your next project.

AutomataCI

A multi-purpose Git template repository with built-in natively operable semi-autonomous CI infrastructure.

Run the CI with full-force anywhere anytime without solely depending on any 3rd-party provider.

Why It Matters

Some good business reasons why using AutomataCI:

1. Starts off with a solid & tested foundation - All CI jobs' recipes for all supported technologies are thoroughly tested and equipped with proper default configurations - rapid prototyping at will.
2. Avoiding threats by mercenaric suppliers - CI is a nerve system for your product(s)' production; handle it in-house and natively rather than leaving it nakedly vulnerable for any kind of 3rd-party extortions.
3. Steadily improvable - Designed to be continuously adaptive from market changes.
4. Simple and scalable - Built completely using only POSIX Shell and Power Shell; Start operating natively without relying on additional installations.

Cross-compiling Confidently

Build confidently not just for the common CPUs but other hardware as well with your selected supportive technologies.

Maximizing Reaches

Packaging your software for as many standardized distribution channels as possible; Makes customers' user experience easier and seamless.

Focusing Business Objectives

Deploy multiple technologies for your product development via integrations (within sensible needs of course).

Securing Point of Origin

GPG cryptographically signed packages alongside common shasum checking - end-to-end verifiable solution for all security level needs; whenever, wherever.

Optimally Containerized

Package small-sized Open-Source Image Container (for Docker OR Podman); AND compile cross-platform images whenever possible.

Supported Technologies

• C Programming Language
  1. i386 & amd64 GCC Compilers
  2. ARM (-64, -el, -hf) Cross-Compilers
  3. MIPS (-64 -64el, -64r6, -64r6el, -el, -r6, -r6el) Cross-Compilers
  4. RISCV Cross-Compiler
  5. S390X Cross-Compiler
  6. PowerPC & ppc64el Cross-Compilers
  7. AVR Cross-Compiler
  8. Clang
  9. MinGW
  10. Emscripten WASM
• Go
  1. Standard Go
• Javascript & Typescript Web Frameworks
  1. Angular
• Nim Programming Language
• Python Programming Language
• Rust Programming Language
• Some Manual MISC stuffs

Deployable Distributions

• Microsoft Windows Ecosystems
  1. Chocolatey ecosystem (for choco install capability)
  2. MSI packages (for conventional installation)
  3. WinGet repo (for winget install capability - COMING SOON)
• Fleet Deployments
  1. OCI compatible containers (for docker run or podman run capability)
  2. PyPi repo (for pip install capability)
  3. cargo repo (for cargo install capability)
• UNIX Ecosystems (Linux & BSD)
  1. .deb repo (with apt-get install capability)
  2. .flatpak repo (for flatpak install capability)
  3. Homebrew ecosystem (for brew install capability)
  4. .opkg repo (with opkg install capability - COMING SOON?!)
  5. .rpm repo (with yum install capability)
• Security
  1. GitHub Security Advisory (for CVE management)
• Static Website
  1. Cloudflare Pages
  2. GitHub Pages

3rd-Party CI Integrations

AutomataCI works seamlessly out of the box by integration with:

1. GitHub Actions
2. GitLab CI (COMING SOON)

How-tos, Documentations & Specifications

Well, the instructions are available as a PDF Handbooks in case of offline needs. Please take your time and download a copy:

• International English

To Contribute

AutomataCI cannot be made successfully without contributions from (Holloway) Chew, Kean Ho, his teams, and supports from external folks. If you had been using AutomataCI and wish to contribute back, there are 2 ways to do so:

Financial

To financially support the project, please head over to Holloway's sponorship store here:

A small token purchase would helps a lot.

Craftmanship

If you wish to bring in codes contribution, bug report, and ideas, please feel free to refer the PDF Handbook and execute accordingly.

License

AutomataCI is licensed under OSI compatible Apache 2.0 License.

automataci's Issues

Release Job Can Reset Git Root Repo to Nothing

Description

Apparently, if the releases directory is missing or malformed, Release job will hard reset the root repository to its first commit and official nuke the project. This is not desirable and safety check must me added here to prevent this from happening again.

Expected Behavior

Safety check is added to GIT::hard_reset_to_init function to cross-check its root against the root repository before actually performing the reset.

Current Behavior

No safety check is added.

Steps to Reproduce [COMPULSORY]

1. Execute everything.
2. At ./ci release stage, observe that it shall erase the project and attempting to push to remote.

Associated Data Files

No response

Develop PURGE Job Recipe

Description

Basically nuke the entire repo back to initial state.

Expected Behavior

Purge is available.

Current Behavior

Purge is not available.

Associated Data Files

No response

Add Sha256 File Release Job Recipe

Description

Generate Sha256 checksum file at the release job for temperament checksum purposes.

Expected Behavior

a sha256.txt checksum file is auto-generated for the uploadable pkg/ directory.

Current Behavior

sha256 is practically not available.

Associated Data Files

No response

Port Flatpak Private Repo Export

Description

It appears Flatpak allows private repo export for Release job. Let's do it.

Expected Behavior

Flatpak is exported and import struction is generated for customer to easily import the app.

Current Behavior

Flatpak repo export is not available.

Associated Data Files

No response

Add Chcolatey Release Recipe for Closing the Archive Ecosystem Loop

Description

Since we use Chocolatey extensively to setup the Windows environment, it's time to create its release recipe and close the loop.

Doc: https://docs.chocolatey.org/en-us/create/create-packages

Expected Behavior

Chocolatey is supported natively.

Current Behavior

Chocolatey is not supported.

Associated Data Files

No response

Add Homebrew Release Recipe for Closing the Archive Ecosystem Loop

Description

Since we use Homebrew heavily for environment setup, it's time to close the loop and allow any product built from this system to be usable on Homebrew itself. Let's support it.

Spec: https://docs.brew.sh/Formula-Cookbook

Blog: https://devopstales.github.io/linux/brew-repo/

Expected Behavior

Release job now release HomeBrew to a designated tap (a GitHub Homebrew-[REPO])

Current Behavior

HomeBrew is not supported.

Associated Data Files

No response

Add ipk Package Recipe

Description

It seems like the the OpenWRT's OPKG package manager is similar to the .deb package format but much simple due to embedded usage.

Source: https://git.yoctoproject.org/opkg/

Release repo example: https://openwrt.pkgs.org/

Blogs: https://raymii.org/s/tutorials/Building_IPK_packages_by_hand.html

Expected Behavior

.ipk is packaged alongside the rest.

Current Behavior

.ipk is not available.

Associated Data Files

No response

Port Docker (GitHub Packages) CI Recipe

Description

Since I have this recipe in the past, I should port it in.

Expected Behavior

Packaging and releasing docker to GitHub Packages is seamless.

Current Behavior

Packaging and releasing docker to GitHub Packages requires manual efforts.

Associated Data Files

No response

Port PyPi Release Job API

Description

Ensures the PyPi packages are exported to targeted upstream.

Expected Behavior

PyPi packages can be seamlessly upstream to a targeted remote server.

Current Behavior

PyPi upstream is unavailable.

Associated Data Files

No response

Update PyPi Compilation

Description

Current PyPi Compilation yielded a deprecation notice:

copying Sample/Entity.py -> build/lib/Sample
/home/u0/Documents/holloway/AutomataCI/tools/python-engine/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
installing to build/bdist.linux-x86_64/wheel

Please comply.

Expected Behavior

PyPi is using up-to-date compilation method.

Current Behavior

PyPi isn ot using up-to-date compilation method.

Steps to Reproduce [COMPULSORY]

1. Set to Python Project.
2. Run full piplines until package.
3. Observe the log with deprecation notice. Package will be successful.

Associated Data Files

No response

Allow GitHub Action to Support the Project Natively

Description

Since Native CI is callable externally, let' set GitHub Action to call all those default commands then.

Expected Behavior

GitHub Actions are implemented.

Current Behavior

GitHub Actions are not supported (for now).

Associated Data Files

No response

Port Debian & TinyDebian Build CI Recipes

Description

Debian and TinyDebian OS are staple in my tech world. Hence, let's port the CI in for a proper build. Data source: refer private ZORALab's Actualizer.

Expected Behavior

Template can easily initiate a Debian OS build workspace immediately.

Current Behavior

To initialize a Debian OS build workspace, manual work are needed.

Associated Data Files

No response

Port Changelog Generator Support

Description

Appearently, DEB requires its own changelog generator. Let's port it from ZORALab's Monteur.

Expected Behavior

Changelog is automatically built and generated.

Current Behavior

Changelog is not available.

Associated Data Files

No response

Port Docker Release Job CI Recipe

Description

Docker can be released on certain platforms like Docker repository or GitHub's packages repository. The process should be ported here.

Expected Behavior

Docker can be seamlessly up-streamed via Release job.

Current Behavior

Docker has to be manually upstreamed.

Associated Data Files

No response

Port GitHub Pages's Git Root Reset Functionalities

Description

Since I already got the GitHub Pages CI recipes at hand, I should port it in for seamless execution.

Expected Behavior

Publishing to GitHub Pages is seamless.

Current Behavior

Publishing to GitHub Pages requires manual work.

Associated Data Files

No response

Port Angular Entire CI Recipes

Description

Since the Native CI is entirely in shell scripts, I have to port the existing job recipes here. Data source: see private repo contract 2022-12-25.

Expected Behavior

Template can initiate a NodeJS workspace immediately.

Current Behavior

To initiate NodeJS workspace, manual efforts are needed.

Associated Data Files

1. https://stackoverflow.com/questions/3653495/is-there-a-virtual-environment-for-node-js

Add Update AutomataCI instruction

Description

At the moment, update instruction is missing out. Please remember to add it in.

Page URL Address

No response

Expected Presentation

There is an update instructions in the engineering specifications.

Current Behavior

update instructions are missing.

Associated Data Files

No response

Port RPM Entire CI Recipe

Description

Don't leave out Red Hat's ecosystem's rpm distributions.

Expected Behavior

NativeCI can package valid GPG-signed RPM.

Current Behavior

No possible.

Associated Data Files

No response

Port DEB Entire CI Recipes

Description

Since I got Debian deb distribution ecosystem and its build specs, I should port it in so that the packaging and releasing processes are simply seamless.

Expected Behavior

packing .deb for Debian-based system are seamless.

Current Behavior

manual efforts are needed.

Associated Data Files

No response

Develop CLEAN Job recipe

Description

A customizable clean job recipe.

Expected Behavior

customizable clean job is available.

Current Behavior

clean job is not available.

Associated Data Files

No response

Merge resources/ directory into src/ directory

Description

They both serve the same purposes: working on MISC stuffs. Let's merge them to simplify things.

Expected Behavior

resources/ is merged completely into src/ directory.

Current Behavior

resources/ and src/ are 2 separate directories.

Associated Data Files

No response

Puis-je utiliser ceci ?

Question

Please state your question in a brief and ended with question mark (?). You
may include sub-questions with "if cases" to communicate more effectively
instead of many 'ping-pong' passing.

Comme indiqué dans le titre.

Background

Please provide a short description of what you are trying to ask.

Current Research Artifacts (COMPULSORY)

Please list out all your research artifacts that you did (e.g. link, blog post,
documentations, and etc) for us to gauge your context and current efforts. We
may not reply and will close the ticket with perjuidice if this section remains
empty and unconvincing.

Attachment

Please drag and drop the necessary data files (e.g. screenshot, logs, etc)
below.

Où se trouve le fichier de licence ?

Question

Please state your question in a brief and ended with question mark (?). You
may include sub-questions with "if cases" to communicate more effectively
instead of many 'ping-pong' passing.

zzzz

Background

Please provide a short description of what you are trying to ask.

Current Research Artifacts (COMPULSORY)

Please list out all your research artifacts that you did (e.g. link, blog post,
documentations, and etc) for us to gauge your context and current efforts. We
may not reply and will close the ticket with perjuidice if this section remains
empty and unconvincing.

Attachment

Please drag and drop the necessary data files (e.g. screenshot, logs, etc)
below.

Develop Python's STOP recipe

Description

The Stop recipe should issue the deactivate user guide. Right now, it's still in placeholder mode.

Expected Behavior

Stop recipe for Python issues deactivate user guide.

Current Behavior

Stop recipe is not doing anything.

Associated Data Files

No response

Port Go Entire CI Recipes from ZORALab's Monteur

Description

Since the Native CI is entirely in shell scripts, I have to port the existing job recipes here. Data source: https://monteur.zoralab.com/en

Expected Behavior

Template can initiate a Go workspace immediately.

Current Behavior

To initiate Go workspace, manual efforts are needed.

Associated Data Files

No response

Port C Entire CI Recipes from CWR

Description

Since the Native CI is entirely in shell scripts, I have to port the existing job recipes here. Data source: https://github.com/hollowaykeanho/cwr

Expected Behavior

Template can initiate a C workspace immediately.

Current Behavior

To initiate C workspace, manual efforts are needed.

Associated Data Files

No response

Add WASM support for C Programming Language

Description

WASM will be the next generation technology for rendering web interface. It should be supported by default.

Expected Behavior

WASM is natively supported for C Programming Language.

Current Behavior

WASM for C Programming Language is not available.

Associated Data Files

1. https://emscripten.org/
2. https://formulae.brew.sh/formula/emscripten
3. https://community.chocolatey.org/packages/emscripten
4. https://developer.mozilla.org/en-US/docs/WebAssembly/C_to_wasm

Add a PDF Document Generator for Offline Needs

Description

To make sure developers can operate the repo during offline-mode without requiring an external software like Hugo.

Expected Behavior

A developer spec document template is available.

Current Behavior

No such facility.

Associated Data Files

No response

Add Native CI into Template

Description

It's safe to say I can deploy the native CI (not relying on external CI services) to develop a product regardless whether the template will be used here or elsewhere. So, let's integrate it in.

Expected Behavior

Next new spin of repository has native CI included in. Target are:

start.cmd - initiate the project development
prepare.cmd - update the project to a stated version
test.cmd - initiate project test sequences.
build.cmd - initiate project build sequences.
package.cmd - initiate project package sequences.
release.cmd - initiate project release sequences.
stop.cmd - initiate project stop sequences.

Current Behavior

CI is not available and has to be manually included.

Associated Data Files

No response

Add "Artwork" Issue Template for GitHub

Description

Since there is an offering of "Artwork" tag, I should add its issue template to facilitate easier case filing.

Expected Behavior

A dedicated "Artwork" related issue template is available.

Current Behavior

No "Artwork" related issue template available for use.

Associated Data Files

No response

Initiate Nim Entire CI Recipe

Description

Nim programming language is where I want to go next. Let's initialize it.

Expected Behavior

Template can initiate a Nim workspace immediately.

Current Behavior

To initiate Nim workspace, manual efforts are needed.

Associated Data Files

No response

Add Contributing Guidelines into Engineering Spec

Description

So that others know how to go about that automataCI directory.

Page URL Address

No response

Expected Presentation

There is a clear storyline flow of contributing guideline.

Current Behavior

Contributing specs are missing.

Associated Data Files

No response

Port Createrepo Release Job Recipe for .RPM

Description

As titled.

Expected Behavior

RPM now has a repo for yum install.

Current Behavior

RPM does not have any repo.

Associated Data Files

No response

Fix Changelog Assembling Function bug

Description

The changelogs (md and deb) are badly compiled when dealing with multiple data entries. It's a bug so please fix it.

Expected Behavior

changelogs are assembled correctly when multiple data entries are available.

Current Behavior

changelogs are badly assembled.

Steps to Reproduce [COMPULSORY]

1. fake some entries.
2. run ./ci.cmd package.
3. decompress tmp/deb/changelog.gz
4. observe pkg/CHANGELOG.md and tmp/deb/changelog.

Associated Data Files

No response

Add WASM support for Go Programming Language

Description

WASM will be the next generation technology for rendering web interface. It should be supported by default.

Expected Behavior

WASM is natively supported for Go Programming Language.

Current Behavior

WASM for Go Programming Language is not available.

Associated Data Files

No response

Add Static Repo Yum Source File and GPG Keyring for Closing the .rpm Ecosystem Loop

Description

At the moment, the .rpm package does not close the apt loop due to the missing supply of source.list file and the required GPG public key. This can be done using the Google Chrome way where each package setup these ecosystem files and close the loop.

Expected Behavior

All .rpm package now supplies source.list and keyring in itself.

Current Behavior

.rpm is missing of source.list and keyring file.

Associated Data Files

No response

Add Baseline `src/` Directory Recipe

Description

In case someone wants to use it outside of supported technologies.

Expected Behavior

A default baseline src/ directory is supported with empty .ci jobs for development.

Current Behavior

src/ is support missing. How can you forget? haha. Everyone can only use it for Python tech.

Associated Data Files

No response

Add Changelog Latest Entries Sealing Function into Release Job

Description

The latest changelog entries should be sealed into the version number autonomously during the Release job.

Expected Behavior

latest changelog entries are sealed autonomously in Release job.

Current Behavior

latest changelog entries had to be sealed manually.

Associated Data Files

No response

Add Auto-Generative Instruction for Static Repo in Release Job

Description

Add an auto-generative instruction documentation (in Markdown) for static repo in the release job.

Expected Behavior

a Markdown document is always available.

Current Behavior

a Markdown is missing.

Associated Data Files

No response

Add "Communities" Issue Template

Description

Since by default the label are offering communities related contents, I should add the GitHub Issue filing template in.

Expected Behavior

A dedicated "Communities" related issue content is filed.

Current Behavior

Unavailable.

Associated Data Files

No response

Add Static Repo Apt Source File and GPG Keyring for Closing the .deb Ecosystem Loop

Description

At the moment, the .deb package does not close the apt loop due to the missing supply of source.list file and the required GPG public key. This can be done using the Google Chrome way where each package setup these ecosystem files and close the loop.

Expected Behavior

All .deb package now supplies source.list and keyring in itself.

Current Behavior

.deb is missing of source.list and keyring file.

Associated Data Files

No response

Add Sha512 File Release Job Recipe

Description

Generate Sha512 checksum file at the release job for temperament checksum purposes.

Expected Behavior

a sha512.txt checksum file is auto-generated for the uploadable pkg/ directory.

Current Behavior

sha512 is practically not available.

Associated Data Files

No response

Port Rust Entire CI Recipes from ZORALab's Monteur

Description

Since the Native CI is entirely in shell scripts, I have to port the existing job recipes here. Data source: https://monteur.zoralab.com/en

Expected Behavior

Template can initiate a Rust workspace immediately.

Current Behavior

To initiate Rust workspace, manual efforts are needed.

Associated Data Files

No response

Port Reprepro Release Job Recipe for .deb

Description

Reprepro is the one responsible for assembling a private .deb repo. Let's enable it.

Expected Behavior

User can source .deb repo using apt.

Current Behavior

User is unable to source .deb repo using apt.

Associated Data Files

No response

Port PyPi Python Packaging Job Recipe

Description

Python can be packaged and upstream as an open-source module via PyPi channel. Hence, let's port its packaging job recipe here.

Expected Behavior

Python project is able to publish to PyPi without manual efforts.

Current Behavior

Publishing to PyPi requires full manual efforts.

Associated Data Files

No response

Port Python Entire CI Recipes

Description

Since the Native CI is entirely in shell scripts, I have to port the existing job recipes here. Data source:
https://github.com/ChewKeanHo/Template-Python

Expected Behavior

Template can initiate a Python workspace immediately.

Current Behavior

To initiate Python workspace, manual efforts are needed.

Associated Data Files

No response

Port .tar.gz and .zip Packaging Recipe

Description

The conventional CI used .tar.gz / .tar.xz and zip pacakges.

Expected Behavior

Package recipes automatically package .tar.XX and .zip for all outputs.

Current Behavior

Non is available.

Associated Data Files

No response

Re-enable Choco's Emscripten package installer Once The Maintainer Fixes its Bug

Description

Apparently, the choco emscripten package installer has a bug where it fails to report its installed dependencies (git-LFS install).

See:

No workaround is available unless the fixes come from the package itself.

Expected Behavior

choco install emscripten -y should not yield any error.

AutomataCI re-enables its automatic installation back again while the documented manual steps serve as a redundancy.

Current Behavior

choco install emscripten -y is yielding error due to its internal dependency (likely false alarm mishandling).

Currently, AutomataCI disabled its automatic installation and document the manual steps.

Associated Data Files

1. aminya/chocolatey-emscripten#2
2. https://github.com/corygalyna/AutomataCI/actions/runs/6387879358/job/17337116795
3. https://community.chocolatey.org/packages/emscripten#discussion

Add GPG Detach Signature File Release Job Recipe

Description

Generate a gpg detach signature checksum file at the release job for temperament checksum purposes.

Expected Behavior

gpg detach signature checksum files are auto-generated for the uploadable pkg/ directory.

Current Behavior

gpg detach signature feature is practically not available.

Associated Data Files

No response

Port Flatpak Packaging Job Recipe

Description

Right now, Flatpak is not available for build. We should support it as it is gaining traction in Linux ecosystems.

Expected Behavior

Flatpak creation is seamlessly included.

Current Behavior

Flatpak is not available.

Associated Data Files

No response