microsoft / microsoft-bonsai-api

A collection of libraries for interfacing simulators with the Bonsai platform.

Bonsai Platform API

This folder contains support libraries for connecting to the Bonsai Azure Service.

Microsoft Open Source Code of Conduct

This repository is subject to the Microsoft Open Source Code of Conduct.

Contributing Samples

Please read the Microsoft contributing guidelines.

Submit a PR adding a brief description of the code you wish to be included and add an appropriate hyperlink to the sample code section. If the sample you wish to code is small and self contained it can be added to the samples directory for the language it is written. For example Python samples can be found in Python/samples.

Sample Code

Samples that show how to use microsoft-bonsai-api and connect with the Bonsai platform.

Python

cartpole - Classic CartPole simulator in Python. Balancing an inverted pendulum on a moving cart.

gym-highway - Make decisions to speed up, slow down, change lanes, etc with a multi-lane highway. This sample uses arrays in Inkling for states and demonstrates usage of using an element of the array in the reward.

house-energy - Control the temperature inside of a house according to the desired set temperatures just like a thermostat.

microgrid - Control a local energy grid to meet the load by charging or discharging the battery, importing or exporting energy from the traditional grid, and consuming generated PV power.

plastic-extrusion - Control a plastic extruder to optimize production of PVC rods with a specified length and tolerance.

quanser-qube - Simulation of the real hardware, academic learning tool: Quanser Qube. Swing Up and Balancing an inverted pendulum with a rotary base.

simple-adder - A minimal working example of a Python simulator that can be connected to Bonsai and used to train a brain. You can think of it as Project Bonsai's "Hello World".

C#

cartpole - Classic CartPole simulator in C#. Balancing an inverted pendulum on a moving cart.

Java

cartpole - Classic CartPole simulator in Java. Balancing an inverted pendulum on a moving cart.

Typescript

adder - Simple simulator that adds numbers and computes a reward based on the result.

Other Bonsai repositories

bonsai-common - A Python framework for interfacing with the bonsai platform.

bonsai-sim-connector-template - A template for creating Bonsai Connectors. It can serve as a starting point for making a simulation platform work with Bonsai.

moabsim-py - Simulator for balancing a ball on a plate, based on hardware. Tutorial 2 is meant for deployment. Additional Inkling samples can be found in /Machine-Teaching-Examples.

Simulator Integration Checklist & Best Practices

• Make sure workspace is registered & correct.
• Error handling: Any cloud service can have transient errors and our SDK have Retry policy in place, but not all errors can be retried, so a good error handling logic will make your sim sessions more robust. Please check cartpole sim sample, for the hints.
• Make sure there is a Re-Register (Re create sim session), in the error handling logic, when you are unregistering or deleting sim. To make sure, sim will keep running.
• Enable verbose SDK logging: If you are hitting issues. Enable logging in BonsaiClientConfig. And set the right LogLevel to get more information.
• For hosted sim, You can also enable Sim logging for Episodes, iterations and system log, using our CLI commands (1, 2) for start-training and start-logging. These logs will go to Log Analytics workspace.

microsoft-bonsai-api's Issues

Cartpole example has type issue when trying to run it as-is.

The cartpole states need to be cast as floats, as inkling type mismatch occurs.

Failed to build C# SDK

Build project "Microsoft.Bonsai.Api.Simulator.csproj" failed with following error, seems some partial class file is missing in the source.

1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(42,13,42,18): error CS0103: The name 'Clent' does not exist in the current context 1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(205,13,205,29): error CS0103: The name 'cacellationToken' does not exist in the current context 1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(213,13,213,38): error CS0103: The name 'HttpStatusCode_statusCode' does not exist in the current context 1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(217,22,217,33): error CS0103: The name '_statusCode' does not exist in the current context 1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(219,119,219,130): error CS0103: The name '_statusCode' does not exist in the current context 1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(238,17,238,26): error CS0103: The name 'exRequest' does not exist in the current context 1>D:\GithubRoot\microsoft-bonsai-api\CSharp\Microsoft.Bonsai.Api\Simulator\Generated\Session.cs(265,22,265,33): error CS0103: The name '_statusCode' does not exist in the current context

BuildTool: VS2019 16.7.1

Local inference test

Hi, I have trained and exported a "brain" from Bonsai UI using the Moab example.

I'd like to run inference on a local machine but I'm not sure how to do it as there is no samples in this regard.

With docker I can run the exported brain which opens a server at port 5000. Then I guess I have to communicate with the brain using the bonsai api, but not sure exactly how to send an state and get a resulting action.

My two main questions are:

• How to set up config to connect the client to the local docker instance?
• To get a new event, seems that I need to specify workspace and session, but running locally I don't have the first and I don't see how to create a session without a simulator (I don't need a simulator right now, just to test inference).

Following the cartpole sample this is what I have for now:

config = BonsaiClientConfig(["--api-host", "http://localhost:5000"])
client = BonsaiClient(config)

# TODO create a session

try:
    event = client.session.advance(
        workspace_name=config.workspace,
        session_id=registered_session.session_id,
        body=sim_state,
    )
    sequence_id = event.sequence_id
    print("[{}] Last Event: {}".format(time.strftime("%H:%M:%S"), event.type))
except HttpResponseError as ex:
    ....

Could you please indicate how to do local inference with an exported brain? A complete sample using Moab demo would be amazing :)
Thanks!

bonsai-cli and microsoft-bonsai-api are out of date

bonsai-cli is now at 1.0.4 and microsoft-bonsai-api is 0.1.2. Can we test and upgrade the package here?

improve edge cases with the way get_state is called in simulator samples

Three related issues:

1. Some simulators retrieve state from data structures that have not been initialized until an episode has begun. In this case, it is inconvenient to have to implement a get_state method that is safe to call right after the sim object has been created before it has been initialized in a config. Sim developers shouldn't be required to handle the zen-like edge case of "What is the state of the environment when the environment doesn't yet exist?"
2. Although get_state shouldn't, in theory, have side effects sometimes this can happen. For example, if get_state is performing bookkeeping operations such as counting iterations or taking deltas to compute velocity from a previous state value. Although, ideally that should be done in the step function instead of the get_state function, it would be nice to only call get_state once per iteration in order to avoid such issues.
3. The behavior of the main() function should match the behavior of the test_policy() function. Currently they differ with respect to the previous two points.

To sum up, in samples such as [https://github.com/microsoft/microsoft-bonsai-api/blob/main/Python/samples/cartpole/main.py](Python Cartpole), for both the test path and the main path:

• get_state should only be called when an episode is active. (EpisodeStart has been called and EpisodeStop hasn't been called to end that episode.)
• get_state should only be called once per iteration

csproj.user file should not be checked in

It appears taht the user settings file Microsoft.Bonsai.Api.Samples.Cartpole.csproj.user seems like it should be ignored and shouldn't be checked in, based on https://github.com/github/gitignore/blob/master/VisualStudio.gitignore.

I recommend that we delete the file and make sure it's on the ignore list. Don't have time to test that right now, so I'm opening an issue...

clone with submodules fails with permission denied error

I received an error when attempting to clone this repository with submodules. This can occur cloning from the command line with --recurse-submodules or when using GUI tools such as Visual Studio's GitHub clone feature that attempts to clone submodules by default.

To reproduce, run the following:
git clone --recurse-submodules https://github.com/microsoft/microsoft-bonsai-api.git

Result:

Cloning into 'microsoft-bonsai-api'...
remote: Enumerating objects: 1338, done.
remote: Counting objects: 100% (1338/1338), done.
remote: Compressing objects: 100% (699/699), done.
remote: Total 1338 (delta 751), reused 1096 (delta 536), pack-reused 0
Receiving objects: 100% (1338/1338), 1.52 MiB | 3.73 MiB/s, done.
Resolving deltas: 100% (751/751), done.
Submodule 'Java/samples/cartpole-java' ([email protected]:BonsaiAI/cartpole-java.git) registered for path 'Java/samples/cartpole-java'
Cloning into 'C:/t/CloneTest/microsoft-bonsai-api/Java/samples/cartpole-java'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:BonsaiAI/cartpole-java.git' into submodule path 'C:/t/CloneTest/microsoft-bonsai-api/Java/samples/cartpole-java' failed
Failed to clone 'Java/samples/cartpole-java'. Retry scheduled
Cloning into 'C:/t/CloneTest/microsoft-bonsai-api/Java/samples/cartpole-java'...
[email protected]: Permission denied (publickey).

'error' is not surfaced in API.

In the spec for the API there is an error string that the simulator can return with the state. This is not surfaced in the API and it does not appear to be a way to send it.

Should we remove the submodules from this repository?

1. Not all users will clone the repository with submodules, leaving them missing some of the content.
2. Submodules are an advanced Git feature that can be confusing to use if you don't work with them all the time.
3. Every time new commits are made to one of the submodule repositories, the submodules of microsoft-bonsai-api will become out of date and we need to make extra pull requests to update them.

Proposal:

• Java/samples/cartpole-java: This is our Java sample. We should copy it into microsoft-bonsai-api instead of referencing it as a submodule. Then we can archive/delete the cartpole-java repository.
• Python/samples/moabsim-py: This is the Moab simulator. We should copy it into the moabian repository so everything moab-related is together. Then we can reference it with links in the microsoft-bonsai-api markdown so that folks know to go use it in the other repository.