con / job Goto Github PK

NB WiP -- yet to finish dumping ideas...

A somewhat in spirit of automations like con/tinuous, datalad/git-annex, datalad-extensions testing, etc.

An automation to provide "tuned" forks of multiple git repos, possibly without offering "tune ups" back to original locations (since they might not want them). Use cases:

• NIDM improved openneuro datasets. ref: https://github.com/OpenNeuroDatasets-NIDM/nidm-openneuro-doc/issues has an outline of some initial original ideas/TODOs

• neurobagel - related but different ;) annotation of some of the openneuro datasets

Following high level configuration structure I see

• sources: list of original locations where to clone/fork from
  • organization, e.g. https://github.com/OpenNeuroDatasets (starting with github but later might be extended to gitlab)
  • repos-regex (optional), to what repos within organization to limit, .* by default
  • name-tuneup (optional, later): how to possibly rename fork in case of multiple sources possibly colliding, e.g. "s,^,openneuro-,g" to add an openneuro- prefix
• destination: e.g. https://github.com/OpenNeuroDatasets-NIDM (starting with github but who knows - might later want to support gitlab)
• transformations: list of commands to do

conjob init https://github.com/OpenNeuroDatasets https://github.com/OpenNeuroDatasets-NIDM/ BIDS2NIDM

which would

• initiate OpenNeuroDatasets-NIDM organization
• populate it with forks of all (default) repos from OpenNeuroDatasets
• run BIDS2NIDM on the default branch
• save the results in its own default branch

Possible additional features:

• initiation/update of PRs against original repos to "offer" changes introduced
• rerender command to update all templated produced

Aspects which come to mind

• to scale up we better make every repo monitor original location in its own CI but that would require .github/workflows change too in that repo in some branch. Could be done and just operate "out of branch" (e.g. some action to run in conjob/ branch while working on main branch of the repo)
• so it would b