Coder Social home page Coder Social logo

componolit / gneiss Goto Github PK

View Code? Open in Web Editor NEW
22.0 7.0 2.0 1.51 MB

Framework for platform-independent SPARK components

License: GNU Affero General Public License v3.0

Ada 81.55% C++ 12.48% C 3.66% Shell 0.48% Python 1.83%
ada spark component-based formal-verification formal-methods embedded

gneiss's Introduction

Gneiss

Many applications still follow a monolithic design pattern today. Often, their size and complexity precludes thorough verification and increases the likelihood of errors. The lack of isolation allows an errors in an uncritical part of a software to impact other security critical parts.

A well-known solution to this problem are systems comprised of components which only interact through well-defined communication channels. In such systems functionality is split into complex untrusted components and simple trusted components. While untrusted parts realize sophisticated application logic, trusted components are typically small, implement mandatory policies, and enforce security properties. An open question is how to implement such trusted components correctly.

Gneiss is a SPARK library providing a component-based systems abstraction for trusted components. Its main design goals are portability, performance and verifiability. Components built against the library can be compiled for the Genode OS Framework and Linux without modification. Only a minimal runtime such as our ada-runtime is required. To enable high-performance implementations, Gneiss imposes a fully asynchronous programming style where all work is done inside event handlers. All language constructs used in Gneiss can be analyzed by the SPARK proof tools to facilitate formally verified components.

Architecture

The library is comprised of two parts: a platform independent interface and a platform implementation. The interface consists mostly of specifications that define the API and the SPARK contracts. The platform implementation is required to implement those specs with platform specific mechanisms. This can either be a native Ada platform or a binding to a different language.

Interfaces are implemented as regular or generic packages. Since components built with Gneiss use an asynchronous approach most interfaces trigger callbacks or events. To avoid access procedures that are forbidden in SPARK, event handlers and callbacks are provided as formal generic parameters to interface packages.

Instances of interfaces are called sessions. There are three session types: client, server and dispatcher. The client uses a server session of its own type. The specific implementation of the server is not visible for the client but determined by the platform. The dispatcher is like a special purpose client that does not connect to any server but the platform itself. It is responsible for registering a server implementation on the platform and thereby making it available for clients.

Building a component

A component is a collection of SPARK packages that contain all its state and implementation. To interact with the system the state needs to hold a system capability and interface instance objects. To announce the component to the system it needs to instantiate a component package with an initialization procedure. This must only be done once per component. An empty component looks as follows:

with Gneiss;
with Gneiss.Component;

package Component is

   procedure Construct (Cap : Gneiss.Capability);
   procedure Destruct;

   package Main is new Gneiss.Component (Construct, Destruct);

end Component;

The procedure Construct is the first procedure of the component that is called (except for elaboration code) and receives a valid system capability. This capability is required to initialize clients or register servers. The procedure Destruct is called when the platform decides to stop the component and allows it to finalize component state. As a convention the components main package is always called Component and it must contain an instance of the generic package Gneiss.Component that is named Main.

The simplest interfaces are used like regular libraries. An example is the Log client that provides standard logging facilities. A hello world with the component described above looks as follows

with Gneiss.Log;
with Gneiss.Log.Client;

package body Component is

   package Log is new Gneiss.Log;
   package Log_Client is new Log.Client;

   Client : Gneiss.Log.Client_Session;

   procedure Construct (Cap : Gneiss.Capability)
   is
   begin
      Log_Client.Initialize (Client, Cap, "channel1");
      if Log.Initialized (Client) then
         Log_Client.Info (Client, "Hello World!");
         Log_Client.Warning (Client, "Hello World!");
         Log_Client.Error (Client, "Hello World!");
         Main.Vacate (Cap, Main.Success);
      else
         Main.Vacate (Cap, Main.Failure);
      end if;
   end Construct;

   procedure Destruct
   is
   begin
      Log_Client.Finalize (Client);
   end Destruct;

end Component;

In Construct the component initializes the log session. Since the initialization can fail it checks again if it succeeded. If this is the case it prints "Hello World!" as an info, warning and error message.

The component then calls Vacate to tell the platform that it has finished its work and can be terminated. Vacate does not immediately kill the component but tell the platform that it can safely be stopped now. The current method will still return normally. There is no guarantee that a subprogram that called Vacate is not called again.

When the platform decides to terminate the component at some point it will call Destruct. This procedure only checks if the Log session has been initialized and finalizes it if this is the case. In more complex scenarios the Destruct procedure can be used to safely shut down hardware devices or write data to disk.

POSIX:

[Hello_World] Info: Hello World!
[Hello_World] Warning: Hello World!
[Hello_World] Error: Hello World!

Genode:

[init -> test-hello_world -> Hello_World] Hello World!
[init -> test-hello_world -> Hello_World] Warning: Hello World!
[init -> test-hello_world -> Hello_World] Error: Hello World!

Since Gneiss is an asynchronous framework it often requires callbacks to be implemented. The Timer interface is a good example and easy to show. It only requires a single callback procedure that is called after a previously specified time. The package spec is the same as in the previous example.

with Gneiss.Log;
with Gneiss.Log.Client;
with Gneiss.Timer;
with Gneiss.Timer.Client;

package body Component is

   Gneiss_Log   : Gneiss.Log.Client_Session;
   Gneiss_Timer : Gneiss.Timer.Client_Session;
   Capability   : Gneiss.Capability;

   procedure Event;

   package Log_Client is new Gneiss_Log.Client;
   package Timer_Client is new Gneiss_Timer.Client (Event);

   procedure Construct (Cap : Gneiss.Capability)
   is
   begin
      Capability := Cap;
      Log_Client.Initialize (Log, Cap, "Timer");
      Timer_Client.Initialize (Timer, Cap);
      if
         Gneiss_Log.Initialized (Log)
         and then Gneiss.Timer.Initialized (Timer)
      then
         Log_Client.Info (Log, "Start!");
         Timer_Client.Set_Timeout (Timer, 60.0);
      else
         Main.Vacate (Cap, Main.Failure);
      end if;
   end Construct;

   procedure Event
   is
   begin
      if
         Gneiss_Log.Initialized (Log)
         and then Gneiss_Timer.Initialized (Timer)
      then
         Log_Client.Info ("60s passed!");
         Main.Vacate (Capability, Main.Success);
      else
         Main.Vacate (Capability, Main.Failure);
      end if;
   end Event;

   procedure Destruct
   is
   begin
      Componolit.Gneiss.Log.Client.Finalize (Log);
      Timer_Client.Finalize (Timer);
   end Destruct;

end Component;

The usage of the log session here is equivalent to the first example. Also the initialization of the timer session is similar. When Set_Timeout is called on the timer a timeout is set on the platform. The platform will then call the Event procedure when this timeout triggers. Since the Event procedure has no arguments and can be used in multiple contexts no preconditions can be set. This requires an initialization check of the timer session.

Some interfaces need more than a simple event callback and require additional interface specific procedures or functions as generic arguments. These more specialized callbacks can provide preconditions that provide certain guarantees about initialized arguments.

Furthermore the capability needs to be copied to be available in the Event for component termination. The capability is a special object that can be copied but not created from scratch. Only construct is called with a valid capability object which then must be kept somewhere to be used in other contexts.

Implementing a new platform

The generic approach to implement a new platform is to create a new directory in platform and provide bodies for all specs in the src directory. Some of those specs have private parts that include Gneiss_Internal packages and rename their types. Those are platform-specific types and their declaration together with the according Gneiss_Internal package spec need to be provided. Platform specific types can be anything, as they're private to all components.

The log client for example consists of two (in this example simplified) specs:

private with Gneiss_Internal.Log;

generic
package Gneiss.Log is

   type Client_Session is limited private;

   function Initialized (C : Client_Session) return Boolean;

private

   type Client_Session is new Gneiss_Internal.Log.Client_Session;

end Gneiss.Log;
generic
package Gneiss.Log.Client is

   procedure Initialize (C     : in out Client_Session;
                         Cap   :        Capability;
                         Label :        String) with
      Pre => not Initialized (C);

   procedure Info (C : in out Client_Session;
                   M :        String) with
      Pre  => Initialized (C),
      Post => Initialized (C);

end Gneiss.Log.Client;

The client session is a limited private type that can neither be assigned nor copied. Its state functions, such as the initialization in this case are provided by the Log package while all modifying procedures are provided by Log.Client. In case of a generic package this allows the use of state functions as function contracts for formal generic parameters.

An exemplary POSIX implementation consists of three parts: the internal type package, a client body and a C implementation. Since the label should be printed as a prefix in front of each message it needs to be saved in the Client_Session type. As it can have any length it only is a record containing a pointer to the actual string object:

with System;

package Gneiss_Internal.Log is

   type Client_Session is limited record
      Label : System.Address := System.Null_Address;
   end record;

end Gneiss_Internal.Log;

As the session type is limited and has no initialization operation it requires a default initializer. The default value should be a state that marks the session as not initialized. Before the package body can be implemented a C implementation must be present. Since the session type is limited Ada will always pass it by reference, so pointers have to be used in the language binding. This roughly represents the subprograms defined in the package spec:

#include <string.h>
#include <stdlib.h>
#include <stdio.h>

typedef struct session
{
    char *label;
} session_t;

void initialize(session_t *session, char *label)
{
    session->label = malloc(strlen(label) + 1);
    if(session->label){
        memcpy(session->label, label, strlen(label) + 1);
    }
}

void info(session_t *session, char *msg)
{
    fputs("[", stderr);
    fputs(session->label, stderr);
    fputs("] ", stderr);
    fputs(msg, stderr);
    fputs("\n", stderr);
}

The struct session is equal to the Ada record. A pointer in C is what a System.Address is in Ada. Also by using in out as passing mode or having a limited type Ada will use pointers so session_t will always be passed as a pointer. Procedures in Ada have no return type so they all are void functions in C.

with System;

package body Gneiss.Log.Client is

   procedure Initialize (C     : in out Client_Session;
                         Cap   :        Capability;
                         Label :        String)
   is
      procedure C_Initialize (C_Client : in out Client_Session;
                              C_Label  :        System.Address) with
         Import,
         Convention => C,
         External_Name => "initialize";
      C_String : String := Label & Character'Val (0);
   begin
      C_Initialize (C, C_String'Address);
   end Initialize;

   procedure Info (C : in out Client_Session;
                   M :        String)
   is
      procedure C_Info (C_Client  : in out Client_Session;
                        C_Message :        System.Address) with
         Import,
         Convention => C,
         External_Name => "info";
      C_Msg : String := M & Character'Val (0);
   begin
      C_Info (C, C_Msg'Address);
   end Info;

end Gneiss.Log.Client;

In the package body the C functions are imported. The Client_Session can be in out as it is passed as a pointer and might be modified by C. The message string is more complicated. While C expects a pointer with a null terminated string, Ada uses an array of characters and passes meta data about the length of the string. To convert an Ada string to a C string it is put on the stack and a NULL character is appended as a terminator. Then the address of the first string element is passed to C.

The last part is the package body for Log which only needs to implement Initialized. Since this functions properties are likely required in the proof context it should not be implemented in C. Also since its contract is fixed it needs to be a expression function to get into the proof context:

with System;

package body Gneiss.Log is

   use type System.Address;

   function Initialized (C : Client_Session) return Boolean is
      (C.Label /= System.Null_Address);

end Gneiss.Log;

The initialization checks if Label is a valid address. This ensures that all procedures that have an Initialized precondition can safely use the label. It also makes sure that if malloc fails in C the session will not be initialized.

Buildsystem

Gneiss aims to integrate into the existing build systems of the supported platforms. On Genode Gneiss components can be built with the native build system. On Linux we chose a custom approach that fits our use case.

Cement

The Cement build system allows designing and building Gneiss systems that run on Linux either in a GNU userspace or as init process. A system consists of a core component that is executed with a configuration. The configuration declares the components and their communication channels. A component is compiled into a shared object that is loaded by the core component and then forks into its own process.

The build configuration is done in XML. The example for a hello world system looks as follows:

<config>
    <component name="log_server" file="libcomponent_linux_log_server.so"/>
    <component name="hello_world" file="libcomponent_hello_world.so">
        <service name="Log" server="log_server"/>
    </component>
</config>

The hello_world component that is implemented by libcomponent_hello_world.so is allowed to communicate via a Log session to the log_server which then prints its outputs to the terminal. To compile this system cement can be called with

$ cd /path/to/gneiss
$ ./cement build -b build test/hello_world/hello_world.xml . test init lib

The arguments after the configuration file are the directory where the Gneiss repository is located and the directories that contain the project files for the components and libraries.

Core is built in build/bin and the components are built in build/lib. To run the system add the components that are shared libraries to the preload path and run core with the build configuration.

$ export LD_LIBRARY_PATH=build/lib
$ ./build/bin/core test/hello_world/hello_world.xml

The resulting output will be:

I: Loading config from test/hello_world/hello_world.xml
I: Started log_server with PID 19294
I: Started hello_world with PID 19295
[hello_world:log_hello_world] Info: Hello World!
[hello_world:log_hello_world] Warning: Hello World!
[hello_world:log_hello_world] Error: Hello World!
[hello_world:log_hello_world] Info: Destructing...
I: Component hello_world exited with status 0

gneiss's People

Contributors

jklmnn avatar senier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

gneiss's Issues

Fix aliasing issues

With the current implementation we run into many aliasing issues that are hard to fix. Due to the inadequacy of the aliasing checker (e.g. that globals are treated as atomics and that accesses are not distinguished between read/write) the approach of further splitting the data path does not work.

The only other option with the current implementation seems to be disabling some aliasing checks. But this is cumbersome as it has to be done by the user and furthermore dangerous as it might happen that an actual aliasing problem is hidden in disabled false positives.

The best option seems to be to remove the identifier/instance types completely and always pass the whole session object. This requires that only the passed session object is used but never the global.

To still be able to identify the caller of a callback the session should be tagged by the application. A tag can then be retrieved from the passed session in the callback and any decision (e.g. different return values for different callers) must be based solely on that tag but not on the global session object.

Further it might happen that multiple sessions of different types are used that all can be called in callbacks. To use a session of another type in a callback all sessions must be in separate constructs (separated by session type). If a session is in the same compound type as the session passed as argument aliasing will happen. This shouldn't be a problem as tags can be provided freely by the application (and therefore multiple times over multiple different session types).

The tag type is intended to fit into a 32bit unsigned value (similar to the request id of a block request).

Dispatcher error handling

Currently when a dispatcher gets a session request it checks if a free session is available and then calls accept with that session. Accept will then initialize it. This is fine as long as all initialization steps inside the server implementation work.

In the case of a failing initialization the client would then talk to an incomplete server (the server cannot initialize some of its parts, e.g. hardware or another server connection it is proxying to). To solve this problem The dispatcher and server interface should be changed:

The server interface should get a generic parameter function Initialize (S : Server_Instance) return Boolean which can be implemented. This represents the internal state of the server implementation.
Furthermore the Session_Accept procedure should only take sessions that are already initialized and there should be a procedure Session_Initialize that does all the initialization work (the platform part still needs to be initialized before the user defined part).
This makes sure that no incomplete sessions can exist.

Also this could lead to a session being accepted multiple times (since it is initialized in both cases). This could either be fixed by adding this property to the server interface or by checking that only sessions can be accepted that have been initialized in the same context (the session is uninitialized -> initialization -> accept, all in the same procedure).

Support generic callbacks in block client

The client interfaces requires a generic operation that is called whenever an event happened on the platform side. A user of the client can instantiate the generic client package with that operation and the type of the state to be passed to the callback.

Generic parameters are:

  • State type
  • Callback function

In addition, the client package has an operation to update the state.

Proof of tests and platform code

To consolidate the current implementation all tests and the platform code written in SPARK should be proven for the absence of runtime errors.

  • test/hello_world
  • test/rom
  • test/timer
  • test/block_client
  • test/block_server
  • test/block_proxy
  • src/common/*strings
  • src/common/*strings_generic
  • src/log/client/genode/*log-client
  • src/log/client/linux/*log-client
  • src/log/client/muen/*log-client
  • src/rom/client/genode/*rom-client
  • src/rom/client/linux/*rom-client
  • src/rom/client/muen/*rom-client
  • src/timer/timer*
  • src/timer/client/genode/*timer-client
  • src/timer/client/linux/*timer-client
  • src/timer/client/muen/*timer-client
  • src/block/*block
  • src/block/client/genode/*block-client
  • src/block/client/linux/*block-client
  • src/block/client/muen/*block-client
  • src/block/server/genode/*block-server
  • src/block/server/genode/*block-dispatcher
  • src/block/server/genode/cxx-block-server
  • src/block/server/genode/cxx-block-dispatcher
  • src/block/server/muen/*block-server
  • src/block/server/muen/*block-dispatcher
  • src/platform/genode/*component
  • src/platform/linux/*component
  • src/platform/muen/*component
  • src/platform/muen/*main
  • src/platform/muen/*muen
  • src/platform/muen/*muen
  • src/platform/muen/*muen_registry
  • src/platform/muen/*muchannel_reader
  • src/platform/muen/*muchannel_writer

This list lists all units that should be proven. The * is only used to shorten componolit-interfaces-. The units are selected by all packages that have bodies or that implement expression functions if they don't have a body. Also all generic packages should be proven via the tests that use them.

Make request type private.

The request type stores partially platform state and shouldn't be changed at the wrong place. To solve this issue it will be made private and only accessible via specific functions/procedures.

Source directory restructuring

Since the current directory structure requires building all interfaces for all platforms and furthermore makes it hard to add code that is shared by some but not all platforms a new structure is required:

src/
├─cai.ads
├─cai-types.ads
├─<cai-generic.ads>
├─log/
  ├─cai-log.ads
  ├─generic
    ├─cai-log.adb
  ├─<platform>
    ├─cai-log.adb

src will now contain only the specs that are generic to all interfaces. Each interface has its own subdirectory containing the specs and the platform independent bodies. For each platform a separate subdirectory exists in each interface. Futhermore interfaces can supply a generic platform that contains code shared by multiple platforms.

Block client interface improvements

To prevent copying data multiple times the platform should make the required buffer available to the application by calling the application with a buffer argument.
The client will gain an additional formal parameter which is a procedure that receives the write buffer as out parameter. This out parameter directly points to the buffer.
Due to the ceasing requirement of a write buffer for the write request all enqueue procedures can be merged.
The same will apply to the read procedure. Instead of providing a buffer a callback is triggered that calls the read with a buffer as in parameter.
Furthermore the supported function will only check for the request type but not the whole request.

  • async write via callback
  • async read via callback
  • supported only for request type

Create interface for read-only configuration

Interface to retrieve a component-specific configuration which is a string of arbitrary format. The platform implementation has a well-known location for the configuration (Genode: ROM named "config", Muen: memory region named "config" (or similar), POSIX: environment variable named "CAI_CONFIG" ointing to file).

The config interface is realize as generic package with a procedure with a single in-parameter as formal parameter:

generic
   type Element is private;
   type Index is range <>;
   type Buffer is array (Index range <>) of Element;
   with procedure Parse (Data : Buffer);
package CAI.Configuration.Client
   function Create return CAI.Configuration.Client_Session;
   procedure Initialize (C : in out CAI.Configuration.Client_Session);
   function Initialized (C : CAI.Configuration.Client_Session) return Boolean;
   procedure Load (C : in out CAI.Configuration.Client_Session);
   procedure Finalize (C : in out CAI.Configuration.Client_Session);
end CAI.Configuration.Client;

Component startup

Provide CAI user with defined way to start a component.

Idea:

Provide a generic package CAI.Component which the user has to instantiate with a Construct procedure. This procedure is called during component startup to set up the component. The generic package defines a symbol that is the entry point of the component.

Implement POSIX platform

Implement sessions for the POSIX platform:

  • Block client -> aio
  • Log -> stderr
  • Timer -> clock_gettime

Requires #32

Fix SYNC implementation for Genode

In the Genode platform implementation we use Block::Packet_descriptor::END to signal a SYNC request. Not sure whether this is right - check and remove if this does not denote a sync operation.

If we make SYNC a no-op on Genode, are we going to deliver a SYNC event with ERROR set? Or a syncable flag for the device?

Check supported on operations instead of requests

Currently we pass requests to the Supported function. This does not make too much sense, as operations are either supported or they aren't and this does not change. Make Supported operate on operations instead.

Memory pool allocator

To allocate large memory regions that do not fit into the BSS an allocator is required. The allocation mechanism itself is platform dependent and is not required to be valid SPARK. The public platform independent interface looks as follows:

generic
   type T is private;
package Cai.Memory_Pool is

   pragma Elaborate_Body;

   Instance : T;

private

   Instance_Address : System.Address;

   for Instance'Address use Instance_Address;

end Cai.Memory_Pool;

An implementation on Genode would use the new keyword and implement __gnat_malloc on Genode:

package body Cai.Memory_Pool is

   type T_Access is access T;
   Instance_Access : T_Access := new T;
   use type System.Address;

begin
   if Instance_Address = System.Null_Address then
      Instance_Address := Instance_Access.all'Address;
   end if;
end Cai.Memory_Pool;

Make buffers generic

Right now, a generic buffer type defined in Block is used for the Block.Client.Submit_Read and Block.Client.Submit_Write operations. We should make such functions generic in their buffer type to allow the user to chose which buffer type to use. The same is true for the to-be-implemented server interface.

Packet allocation failure

When submitting larger amounts of packets the allocation fails with

Error: Uncaught exception of type 'Genode::Packet_stream_source<Genode::Packet_stream_policy<Block::Packet_descriptor, 256u, 256u, char> >::Packet_alloc_failed'

This doesn't seem to be taken into account by the ready function which checks if the submit queue can take more requests. Aside from the check I'm not sure if a signal handler is in place for "packets can be allocated again".

Create documentation

Document the following aspects of the interfaces:

  1. How to use a client interfaces
  2. How to implement a server interface
    • How to create a custom types package for server interface
  3. How to support a new platform

CI test on all platforms

The CAI should be tested by the CI on all supported platforms. The test should as far as possible support the three steps:

  • build
  • prove
  • run

Generic session interface

For a completely asynchronous interface all sessions follow an approach using an event handler. There is no difference between a server or a client interface since both generate and handle events.

A session interface has the following minimal interface:

private with Cai.Internal.Session;

generic
   type State is limited private;
   with procedure Event (S : in out State);
package Cai.Session

   type Session is private;

   function Create return Session;

   procedure Initialize (S : in out Session; C : in out State);

   procedure Finalize (S : in out Session);

private

   type Session is limited record
      Cai.Internal.Session;
   end record;

end Cai.Session;

Each session state is implemented as a private record with an internal type. The internal type can be anything and is defined by the platform. Furthermore each session can (and should) implement additional functions that can be called with the Session type. This includes client function to call other sessions and polling functions to handle all events in the Event procedure.

Update CAI to Genode 19.05

When trying to build CAI on Genode to test the renaming of the Configuration interface I had to update Genode to 19.05 to use our latest runtime. This caused the block interface to break when building. We should check what work is required to make it work again (especially if we can fix it short time or if we have to use Genodes new block client interface from now on).

Interface clean up

There are currently some parts of the interface design that have redundant arguments, etc:

  • In the block client interface an incoming response is retrieved by the Next function. if it is a read response the data can be retrieved by calling Read on this request. It then has to be acknowledged by calling Release after which the platform discards it. Both Read and Release require the request as argument although no other request than the one returned by Next is available for handling. Therefor it doen't need to be passed as an argument. (obsoleted by new interface, cf. #69)

  • A similar problem exists for the block server. While the Discard procedure already doesn't take the request as argument the Read, Write and Acknowledge procedures do. They also depend on the request returned by Head which also implements a FIFO Queue so only the first element is available. Either the response path needs to be separated from the request path to provide the ability to acknowledge out of order (which is desirable) or only first request in the queue can be responded at a time. (obsoleted by new interface, cf. #69)

  • The block dispatcher provides a session label with the Session_Request procedure. This label can be used to identify the requesting client. It is then passed to the Session_Accept procedure. Inbetween it could have been changed. Since Session_Accept uses the label to identify the actual session to connect the label shouldn't be changed as any other session is probably not available or cannot be safely connected to. This information should be instead passed via the dispatcher capability (moved to #50).

Startup of event-driven components

So far, we mainly supported the notion of programs with one Ada main loop that would do arbitrary things, sometimes block, sometimes poll for events. However, experience show that fully event-driven programs are much cleaner and performant. More importantly, when done right, such components avoid unnecessary state and back-pressure from clients. This is how components are supposed to be structured on Genode these days.

For Ada components following this pattern we need a couple of things:

  • An interrupt-like mechanism to signal progress from the platform
  • Interface to initialize a component and set up signal sources
  • Interfaces that
    1. work in polling-mode upon reception of a signal
    2. allow for detecting the readiness of downstream components
    3. allow for rejecting a request if the component is not immediately ready to process it
  • A strategy to implement that concept in an Ada main loop on platforms that do not directly support signals (e.g. Muen)

We should review the existing block-device interface to meet this pattern. For Genode, we have to create a component (akin to libc/component.h) that allows for easy setup of such event-driven Ada components.

(This topic was migrated from the last comment of Componolit/ada-runtime#22 where it didn't belong).

Implement NBD server

The NBD server implements a block server session and exports if over a socket.

Timer event support

Implement a session that triggers an event after a specific amount of time.

  • events are relative (in X seconds)
  • data type Duration
  • One single oneshot timer is supported
  • Timers can be overwritten

Configurable block buffer size

The initialization of the block buffer should accept a requested buffer size in bytes to determine its internal packet buffer. The maximal transfer size will then be the min(buffer size, device maximal transfer size from the server).

Dispatcher session capability

The procedures Session_Request, Session_Accept, Session_Cleanup must only be called in the Dispatch procedure of the dispatcher. This is currently not enforced. To enforce this the Dispatch procedure will receive a limited private in paramter that is required to be passed to each of this three methods. As it is a limited private type it cannot be copied or stored outside the Dispatch procedure.

Platform specification Linux/Posix

The implementations of block and configuration client on posix raised obscurities about the distinction of the posix/Linux platforms. The current posix implementation is a C implementation based on libc that uses standard posix mechanisms. Although parts of the code are Linux specific and will most probably not compile on other platforms. This raises the issue if posix is the correct platform definition.

I see three ways to solve this issue:

  1. rename posix to linux
  2. implement platform generic mechanisms
  3. divide posix into more platforms with a large shared code base

Some ideas:

  1. This is probably the easiest option. But it would collide with an eventual in kernel implementation which then also would be called linux.
  2. Since all the code runs on posix conform platforms with posix conform build systems/compilers abstraction layers that provide implementations for different platforms could be embedded without changing the public interface of the posix platform for the CAI build system. In detail this means wrappers around the problematic code (in this case inotify and block size detection) that provided different implementations for different posix platforms which are selected via macros. These implementations could then be extended easily for new posix compatible platforms.
  3. Basically like 2. but instead of macros the distinction is done via the CAI build system. Each posix directory would have a subdirectory for Linux/BSD etc. if necessary.

Personally I prefer 2.

Capability support

To provide better support for capability based system all sessions require an opaque capability object that is provided by the platform. This object resides in the package state and cannot be initialized there. The construct method receives an in out parameter with an initial value it can copy to its local state.

Block error semantics

Currently procedures the procedures Allocate_Request, Enqueue and Acknowledge are allowed to fail. Failing means that the request status does not change.

For Acknowledge this is sufficient since any request can be acknowledged and the server cannot create or destroy requests by its own.

On the other hand Allocate_Request and Enqueue are called with requests created by the application that can be erroneous.

Allocate_Request needs at least a temporary/permanent error semantic. A temporary indicates that there are currently insignificant resources to allocate that request and that the application should try again later. A permanent error indicates that this set of arguments will never succeed in allocating the request, e.g. because the request type is not supported by the platform. In this case the application can stop retrying, or in case this operation is required, detect that it cannot proceed.

Enqueue is more difficult to decide. Either there is the guarantee that a request that can be allocated can also be enqueued. In this case the only possible error would be a full enqueue queue which is temporary. I currently cannot think of a scenario where enqueue would fail permanently (especially of one that cannot be catched by Allocate_Request.

An idea would be to extend Allocate_Request by one parameter showing the result:

procedure Allocate_Request (C : in out Client_Session;
                            R : in out Request;
                            ...
                            S :    out Result);

Where the result could be:

  • Success: in case of success
  • Retry: in case of a temporary error
  • Unsupported: in case of an unsupported operation
  • Too_Large: in case the request length is to large

The last one could also replace the Maximum_Transfer_Size function.

Task scheduler

To run multiple consecutive tasks in a asynchronous environment a scheduling approach is required.
Each task is executed in an event handler that is called by the platform. When it has been called it will return eventually and be finished or not. If it is finished the scheduler should call the next task, if not it should return to the platform and wait for the next event. (Basically cooperative scheduling)

To realize this in SPARK generic packages can be used. Each task is an instance of the Task package that is instantiated with the procedure it executes and the next consecutive task (the last task is a special implementation that does nothing). To prevent overly nested packages instances are only implementation but not data-dependent. For a single implementation an array of instance objects is passed. Each task will then first execute the implementation for each instance before it will call the next task.

Build Block.Client.Job interface

Implement a state machine approach on top of our current (low-level) block client interface.

Migrate all tests an applications to the new interface.

Remove `Update_Response_Queue`

Instead of emulating a queue we should completely remove Update_Response_Queue and call Update_Request on all requests. Platforms that use a queue internally can implement this via a cache.

Implement Linux and Muen

The latest interfaces changes were only implemented on Genode. They also need to be implemented on Linux and Muen.

Implement timer interface

This interface allows users to request the current monotonic time (similar to Clock in Ada.Real_Time). We should also think about function to retrieve precision and bounds of the timer.

Support multiple LOG instances

The new client log interface assumes that we have one single connection to a log server that is already initialized when we start using it. We would not be able to implement, e.g. a log splitter with the current interface.

Our LOG session should get a Create function and an Initialize procedure as the block session. In Initialize we could then configure e.g. different log backends or target facilities.

Move all CBE tests into CAI

Call tests in the CBE repo actually should become CAI platform tests.

Once #24 is done, we can enable client tests in the CI. For server tests, we need to implement NBD and check whether this can be used on TravisCI.

Block server interface improvements

The block server interface currently wakes up client implicitly after the callback returns on Genode. Yet this only works if the callback has been called from the servers own signal handler. If the callback is called from another signal handler (e.g. in a proxy component) the client won't be woken up.
To fix this a procedure Unblock_Client (S : Server_Session) will be introduced that has to be called by the application to wake up the client.

Furthermore the Initialize procedure of the server will additionally get the requested transfer size of the client. It is free reject a connection based on this information.

Add Trim operation

The TRIM operation is beneficial for optimizing SSD storage. For Trim the request type is equivalent to Read/Write. The request is:

procedure Trim (D : in out Component.Block_Device; R : in out Request);

Restructuring for better proof properties

The generic packages get procedures as formal parameters that have implicit contracts. Currently these contracts cannot be annotated since the required functions are inside the generic package.
To overcome this problem all functions of the client and server packages should reside inside the parent package that also defines all required types and their property functions.

Furthermore the generics need to get a pre and postcondition function as formal parameter for each procedure and function they receive.

Also requests should be bound to a specific session on allocation and unbound on release.

Steps:

  • move functions to parent package
  • pre and post condition for generics
  • bind request to specific session

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.