vvazrod / shipyard-server

Containerized real-time tasks deployment and management tool.

Shipyard

This system provides a simpler way of orchestrating real-time tasks in distributed environments. Manage nodes, tasks and trigger deployments using containers through a well-defined REST API.

Installation

The recommended way of running this software is by using the varrrro/shipyard-server Docker image. The server needs to connect to a Mondo database whose URL is provided through an environment variable called DB_HOST.

You can also do a test deployment of the system with docker-compose using the following example configuration. Please note that this example uses your user's SSH configuration and keys.

version: "3.0"

services:
  management:
    image: varrrro/shipyard-server
    depends_on:
    - db
    environment:
      DB_HOST: ${DB_HOST}
    volumes:
    - ~/.ssh:/root/.ssh
    ports:
    - 8000:8000

  db:
    image: mongo

Usage

Although you could use the server's REST API directly, it is recommended to use the official CLI client to interact with the system.

shipyard-server's Issues

Server deployment with containers

The server should be deployed using a Docker container. The image for that container must be provided and built automatically from this repo. An example deployment using docker-compose should also be provided.

• Add image Dockerfile
• Set up automatic image build on Docker Hub
• Provide docker-compose example

Deploy task to device

Before deploying a task to a device, the feasibility of the whole task set that we want to run on the device needs to be checked. This is done by computing the total CPU utilization and depends on the number of cores of the target device. Removal of tasks from a device should be possible too.

• Check task set feasibility
• Send files to device
• Build image on device
• Run container on device
• Remove task from device

Update tasks and nodes

The user should be able to update a task or node after it has been created.

• Update task specs
• Update task source files
• Update node values

Improve error handling

Controllers should handle service errors better, which means raising custom errors for different situations that give enough information to the controller.

Task CRUD

Basic management of tasks registered on the system. Only fetching, adding and deleting tasks is going to be supported for the time being. The task files (sources, image and specs) are going to be sent as a tar file to the API.

• Task reception and processing (endpoint controller)
• Add task to database
• Task info endpoints and controllers (list and individuals)
• Retrieve task info from database
• Task deletion endpoint and controller
• Remove task from database

Add continuous integration

It would be nice to add some GitHub Actions for running tests and building the Docker image automatically in a unified way to ensure quality control.

Node CRUD

Basic device management. For the time being, only fetching, addition and deletion are going to be supported.

• Device information endpoints and handlers (list and individual)
• Device creation endpoint and handler
• Device deletion endpoint and handler
• Retrieve device(s) from database
• Insert new device into database
• Remove device from database
• Controllers unit tests
• Service unit tests