mathiswellmann / gym-rs

OpenAI's Gym written in pure Rust for blazingly fast performance

OpenAI Gym (Unofficial Rust Implementation)

This library aims be be close to the original OpenAI Gym library written in Python.

If you don't mind Python and would like to use the original implementation from Rust, check out a OpenAI Gym wrapper.

Prerequisites

This library use's SDL2 to enable various forms of rendering. Even when an SDL2 window is not explictly shown, renders can be saved to files making it a mandatory dependency if any form of rendering is to be done.

• SDL2
• SDL2_gfx

On Ubuntu you can install the dependency as such:

sudo apt-get install libsdl2-dev

On Arch:

sudo pacman -S sdl2 sdl2_gfx

If your using nix, you can get into the reproducible build environment as simple as:

nix-shell

Usage

To use this crate in your project, put this in your Cargo.toml:

[dependencies]
gym_rs = "0.3.0"

Usage on Windows:

As per #6, here are some instructions for the windows folks:

0. clone the repo & cd to the root dir of the repo
1. modify Cargo.toml, remove dependency sdl2 line and add following code:

[dependencies.sdl2]
version = "0.35.2"
default-features = false
features = ["static-link", "use-vcpkg", "gfx"]

[package.metadata.vcpkg]
dependencies = ["sdl2", "sdl2-gfx"]
git = "https://github.com/microsoft/vcpkg"
rev = "16ee2ec"

[package.metadata.vcpkg.target]
x86_64-pc-windows-msvc = { triplet = "x64-windows-static-md" }

2. install cargo-vcpkg using cargo install cargo-vcpkg
3. under the root dir of the repo, cargo vcpkg build
4. now build and run, such as cargo run --example=mountain_car

Examples

cargo run --example=cartpole

cargo run --example=mountain_car

Contributions

Contributions are welcome. For the contribution guidelines, please take a look at CONTRIBUTING.md.

Donations

If you would like to support the development of this crate, feel free to send over a donation:

Monero:

47xMvxNKsCKMt2owkDuN1Bci2KMiqGrAFCQFSLijWLs49ua67222Wu3LZryyopDVPYgYmAnYkSZSz9ZW2buaDwdyKTWGwwb

gym-rs's Issues

[Linux] Error in Build in Cartpole Example due to sdl2-gfx

Hi, I encountered a few errors in building the cartpole example.

I was able to get them sorted out, but still making this issue to document. Things I had to do to solve:

• Install cmake (sudo apt install cmake)
  • This was a fresh installation that didn't happen to have cmake installed, which is unlikely, yet possible for beginners/students
• Add the following lines to cargo.toml (not sure if this is strictly required... but nonetheless resolved an error)
  • and run cargo vcpkg build before cargo build

[package]
name = "gym-rs"
...

[dependencies]
...

+ [package.metadata.vcpkg]
+ dependencies = ["sdl2", "sdl2-gfx"]
+ git = "https://github.com/microsoft/vcpkg"
+ rev = "501db0f" # or change based on the latest release in GitHub

• Install libsdl2-gfx-dev in addition to libsdl2-dev
  • I feel that other users may encounter the same issue; perhaps would be good to update readme

Thanks for making this awesome library! Feel free to investigate more or close the issue.

Cartpole enviorment can start in an unrecoverable state.

Hi, i am using gym-rs to learn rust by making a genetic algorithm to balance a stick. Your stick ;)

A lot of simulations seems to end after the first action. After some debugging i found that the init state can be already beyond what makes it "done". I think the init state needs to be changed to land inside the "not done space". I am very new to rust, so i dont have a solution to post. But i think it should be an easy fix. ... i hope.

Debugging code and result:

    let mut init_observation: CartPoleObservation = env.reset(None, false, None).0;
    println!("---------------------");
    println!("env.x_threshold {}", env.x_threshold);
    println!("env.theta_threshold_radians {}", env.theta_threshold_radians);
    println!("init_observation {:?}", Vec::from(init_observation));
    println!("---------------------");

Prints the following:

env.x_threshold 2.4
env.theta_threshold_radians 0.20943951023931953
init_observation [0.012382612957275718, 0.014833725476475657, -0.3528077663467408, 0.3304920176194186]

PS/Note: the print might be impacted by me changing a small detail that i suspect was a bug.

        self.state = CartPoleObservation {
            x,
            x_dot,
            theta_dot,
            theta,
        };

to

        self.state = CartPoleObservation {
            x,
            x_dot,
            theta,
            theta_dot,
        };
        

Update readme to make it easier to use gym-rs on windows

Hi, i had a bit of problems with SDL2 install, but found out that cargo can bring it in for us. I belive this will make gym-rs easier to get started with and use.

Not sure if it should be added in toml, or just as a hint in README.md

[dependencies.sdl2]
version = "0.35.2"
default-features = false
features = ["static-link", "use-vcpkg", "gfx"]

[package.metadata.vcpkg]
dependencies = ["sdl2", "sdl2-gfx"]
git = "https://github.com/microsoft/vcpkg"
rev = "501db0f"

[package.metadata.vcpkg.target]
x86_64-pc-windows-msvc = { triplet = "x64-windows-static-md" }

[Question] Is this still an active project?

Latest version is not published on Crates.io

Rendering doesn't work on MacOS

Running both examples on MacOS Monterey (version 12.1). The training was successful but no window was shown when "rendering champion..." is outputted.

$ RUST_LOG=info cargo run --example cart_pole --release --features="cosyne"
 INFO  cosyne::cosyne > gen 0, champion fitness: 129.0000
 INFO  cosyne::cosyne > gen 1, champion fitness: 129.0000
 INFO  cosyne::cosyne > gen 2, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 3, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 4, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 5, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 6, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 7, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 8, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 9, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 10, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 11, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 12, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 13, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 14, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 15, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 16, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 17, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 18, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 19, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 20, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 21, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 22, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 23, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 24, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 25, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 26, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 27, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 28, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 29, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 30, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 31, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 32, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 33, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 34, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 35, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 36, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 37, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 38, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 39, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 40, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 41, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 42, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 43, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 44, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 45, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 46, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 47, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 48, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 49, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 50, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 51, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 52, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 53, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 54, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 55, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 56, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 57, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 58, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 59, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 60, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 61, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 62, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 63, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 64, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 65, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 66, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 67, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 68, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 69, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 70, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 71, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 72, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 73, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 74, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 75, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 76, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 77, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 78, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 79, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 80, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 81, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 82, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 83, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 84, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 85, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 86, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 87, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 88, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 89, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 90, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 91, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 92, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 93, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 94, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 95, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 96, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 97, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 98, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 99, champion fitness: 400.0000
champion: (ANN { num_inputs: 4, num_outputs: 1, layers: [Layer { input_len: 4, output_len: 1, gene_len: 5, activation: Relu, weights: Matrix { data: VecStorage { data: [0.07007277006795642, -0.23631177830558103, 0.44346640185648933, 0.13902369122718516], nrows: Dynamic { value: 1 }, ncols: Dynamic { value: 4 } } }, biases: Matrix { data: VecStorage { data: [0.5129485592119563], nrows: Dynamic { value: 1 }, ncols: Dynamic { value: 1 } } }, act: Matrix { data: VecStorage { data: [0.0, 0.0, 0.0, 0.0], nrows: Dynamic { value: 4 }, ncols: Dynamic { value: 1 } } }, act_func: 0x10c1714a0, net: Matrix { data: VecStorage { data: [0.0], nrows: Dynamic { value: 1 }, ncols: Dynamic { value: 1 } } } }], num_genes: 5 }, 400.0)
training time: 0ms
rendering champion...            <--- Nothing happens