Coder Social home page Coder Social logo

Comments (2)

vincent-zurczak avatar vincent-zurczak commented on June 2, 2024

OK...
A first implementation has been done. It works the following way.

  • The DM has an implementation to store its state (currently, it can either use a storage based on properties files, or a storage that stores nothing).
  • The DM stores the application names, their resource directory and the state of the root instances that are deployed (with their IP addresses and machine IDs).
  • The state of root instances that are not deployed is not persisted.
  • When the DM restarts, it restores the applications and their states. For VMs that are supposed to be running, it sends a message to the agent so that this one can send its model back to the DM.

This solution was though for cases where the DM crashes.
It was supposed to allow restoration from almost any machine, provided we have used a valid storage (properties file, or use a database storage). However, this solution implies the restoration can take up to few minutes. Besides, if new instances have been defined at runtime, and that their root instances were not running, then these instance definitions are lost (they are not persisted). In fact, this solution does not address one common use case and reality.

The DM does not need to run all the time.

So, it should be possible to start the DM, perform some actions, stop it, restart it later, perform another action, stop it again, and so on. Reducing the amount of time the DM runs means reducing security issues and vulnerabilities. So, we need another implementation to cover this usage. The solution proposed below also takes other issues into account.

The idea is that the DM should have a directory to persist things.

  • This directory would contain a sub-directory called applications. This is where the DM should unzip all the uploaded applications. Currently, they can be anywhere in the disk.
  • This directory would contain a sub-directory called state. It would contain instances files, one per application. Such a file would contain all the instance definitions.
  • Eventually, there should be a conf directory that would store the configuration of the DM. This includes the IP address of the messaging server, as well as the credentials for the messaging.

There will be only one implementation for state storage: a file based one.
People in charge of production environments will have to perform backups of this directory. Since this directory contains everything required, backup will be quite simple.

Backup and restoration should work as follows:

  1. When the DM starts, it finds its directory, configures the messaging, loads applications and restores states.
  2. It sends a message to all the running agents to get the current states of their instances. Messages will be light. So, it should go fast.
  3. When a root instance changes its state, a backup of the state should be performed.
  4. When instances are created or deleted, a backup of the state should be performed.
  5. When the DM is shutdown, a backup of the state should be performed.

Other instance changes do not require a backup.
They do not contain information that cannot be retrieved dynamically.

Root instances should be able to perform their machine ID and their IP address.
So, we should add a way in instances files to predefine them.

from roboconf-platform.

vincent-zurczak avatar vincent-zurczak commented on June 2, 2024

Last, but not least...
There should be a default location for this directory. But it should be customizable through an environment variable ROBOCONF_DM_DIR. If this variable points to an inexisting directory, then an error should be thrown, or we should migrate to the default location.

Eventually, the method tryToChangeMessageServerIp should be deleted from the DM.
For tests and use of the Debug Sample, we should have utility methods to create a default configuration for the DM.

from roboconf-platform.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.