learncv / ck

Collective Knowledge framework (CK) helps to organize any software project as a database of reusable components with common automation actions and extensible meta descriptions based on FAIR principles (findability, accessibility, interoperability, and reusability). See real-world use cases from Arm, General Motors, IBM, RPi, ACM, and MLPerf:

Collective Knowledge framework (CK)

Linux/MacOS: Windows:

Overview

Collective Knowledge framework (CK) helps to organize software projects as a database of reusable components with common automation actions and extensible meta descriptions based on FAIR principles (findability, accessibility, interoperability, and reusability).

We want to help everyone share, reuse, and extend their knowledge in the form of reusable artifacts and portable workflows with a common API, CLI, and meta description. See how CK supports collaborative and reproducible AI, ML, and Systems R&D:

• Project overview (accepted for Philosophical Transactions of the Royal Society A)
• Real-world use-cases from Arm, General Motors, IBM, MLPerf, the Raspberry Pi foundation, and ACM
• Reddit discussion about reproducing 150 papers

Please take this quick questionnaire to help us shape the future of the CK framework!

News

• 2020 August: We have completed the prototyping phase and successfully validated CK in several projects to automate the design space exploration of efficient ML/SW/HW stacks, simplify MLPerf benchmark submissions, and quickly deploy emerging technologies (AI, ML, quantum) in production in the most efficient way (speed, accuracy, energy, costs) across diverse platforms from data centers to mobile phones and edge devices. See this white paper and the Reddit discussion for more details. We are now preparing the CK2 and new projects - don't hesitate to get in touch with Grigori Fursin for further details!

Documentation

• News
• Online CK documentation
  • Why CK?
  • CK Basics
  • Try CK
• Publications

CK-powered workflows, automation actions, and reusable artifacts

• Real-world use-cases
• Reproducibility initiatives: [methodology], [events]
• Showroom (public projects powered by CK):
  • MLPerf automation
  • Student Cluster Competition automation: SCC18, digital artifacts
  • Jupyter notebooks: ML/SW/HW DSE for edge devices
  • ML-based autotuning project: reproducible paper demo, MILEPOST
  • Stable Docker containers with CK workflows: MLPerf example, cKnowledge.io, Docker Hub
  • Quantum hackathons
  • ACM SW/HW co-design tournaments for Pareto-efficient deep learning
  • Portable CK workflows and components for:
    • TensorFlow
    • PyTorch
    • TensorRT
    • OpenVino
    • individual NN operators
    • object detection
  • GUI to automate ML/SW/HW benchmarking with MLPerf example (under development)
  • Reproduced papers
  • Live scoreboards for reproduced papers
• Examples of CK components (automations, API, meta descriptions):
  • program : image-classification-tflite-loadgen [cKnowledge.io] [GitHub]
  • program : image-classification-tflite [GitHub]
  • soft : lib.mlperf.loadgen.static [GitHub]
  • package : lib-mlperf-loadgen-static [GitHub]
  • package : model-onnx-mlperf-mobilenet [GitHub]
  • package : lib-tflite [cKnowledge.io] [GitHub]
  • docker : object-detection-tf-py.tensorrt.ubuntu-18.04 [cKnowledge.io]
  • docker [GitHub]
  • docker : speech-recognition.rnnt [GitHub]
  • package : model-tf-* [GitHub]
  • script : mlperf-inference-v0.7.image-classification [cKnowledge.io]
  • jnotebook : object-detection [GitHub]

Installation

Follow this guide to install CK framework on your platform.

CK supports the following platforms:

	As a host platform	As a target platform
Generic Linux	✓	✓
Linux (Arm)	✓	✓
Raspberry Pi	✓	✓
MacOS	✓	±
Windows	✓	✓
Android	±	✓
iOS	TBD	TBD
Bare-metal (edge devices)	-	±

Example

Here we show how to pull a GitHub repo in the CK format and use a unified CK interface to compile and run any program (image corner detection in our case) with any compatible data set on any compatible platform:

python3 -m pip install ck

ck pull repo --url=https://github.com/ctuning/ck-crowdtuning

ck ls program:*susan*

ck search dataset --tags=jpeg

ck compile program:cbench-automotive-susan --speed

ck run program:cbench-automotive-susan --cmd_key=corners --repeat=1 --env.MY_ENV=123 --env.TEST=xyz

You can check output of this program in the following directory:

cd `ck find program:cbench-automotive-susan`/tmp
ls -l

tmp-output.tmp - image with detected corners (rename to ppm to view it)

Check CK docs for further details.

Open CK portal

cKnowledge.io: organizing ML and Systems knowledge in the form of portable CK workflows, automation actions, and reusable components:

• All CK ML&systems components
• CK compatible repositories
• CK portable solutions (including MLPerf inference on edge)
• CK-based adaptive containers
• CK modules
• Automation actions
• Portable program workflows
• Meta packages
• Software detection (code, data, models)
• Platform detection
• Shared experiments
• Reproduced results from papers

Author

• Grigori Fursin (cTuning foundation and cKnowledge SAS)

Acknowledgments

We would like to thank all contributors and collaborators for their support, fruitful discussions, and useful feedback! See more acknowledgments in the CK journal article.