mathiswellmann / gym-rs Goto Github PK

View Code? Open in Web Editor NEW

64.0 64.0 8.0 3.31 MB

OpenAI's Gym written in pure Rust for blazingly fast performance

License: MIT License

Rust 98.03% Nix 1.97%

ai ml openai-gym reinforcement-learning rl

gym-rs's Issues

[Question] Is this still an active project?

Update readme to make it easier to use gym-rs on windows

Hi, i had a bit of problems with SDL2 install, but found out that cargo can bring it in for us. I belive this will make gym-rs easier to get started with and use.

Not sure if it should be added in toml, or just as a hint in README.md

[dependencies.sdl2]
version = "0.35.2"
default-features = false
features = ["static-link", "use-vcpkg", "gfx"]

[package.metadata.vcpkg]
dependencies = ["sdl2", "sdl2-gfx"]
git = "https://github.com/microsoft/vcpkg"
rev = "501db0f"

[package.metadata.vcpkg.target]
x86_64-pc-windows-msvc = { triplet = "x64-windows-static-md" }

Latest version is not published on Crates.io

Cartpole enviorment can start in an unrecoverable state.

Hi, i am using gym-rs to learn rust by making a genetic algorithm to balance a stick. Your stick ;)

A lot of simulations seems to end after the first action. After some debugging i found that the init state can be already beyond what makes it "done". I think the init state needs to be changed to land inside the "not done space". I am very new to rust, so i dont have a solution to post. But i think it should be an easy fix. ... i hope.

Debugging code and result:


    let mut init_observation: CartPoleObservation = env.reset(None, false, None).0;
    println!("---------------------");
    println!("env.x_threshold {}", env.x_threshold);
    println!("env.theta_threshold_radians {}", env.theta_threshold_radians);
    println!("init_observation {:?}", Vec::from(init_observation));
    println!("---------------------");

Prints the following:

env.x_threshold 2.4
env.theta_threshold_radians 0.20943951023931953
init_observation [0.012382612957275718, 0.014833725476475657, -0.3528077663467408, 0.3304920176194186]

PS/Note: the print might be impacted by me changing a small detail that i suspect was a bug.

        self.state = CartPoleObservation {
            x,
            x_dot,
            theta_dot,
            theta,
        };

        self.state = CartPoleObservation {
            x,
            x_dot,
            theta,
            theta_dot,
        };

Rendering doesn't work on MacOS

Running both examples on MacOS Monterey (version 12.1). The training was successful but no window was shown when "rendering champion..." is outputted.

$ RUST_LOG=info cargo run --example cart_pole --release --features="cosyne"
 INFO  cosyne::cosyne > gen 0, champion fitness: 129.0000
 INFO  cosyne::cosyne > gen 1, champion fitness: 129.0000
 INFO  cosyne::cosyne > gen 2, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 3, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 4, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 5, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 6, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 7, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 8, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 9, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 10, champion fitness: 261.0000
 INFO  cosyne::cosyne > gen 11, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 12, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 13, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 14, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 15, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 16, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 17, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 18, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 19, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 20, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 21, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 22, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 23, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 24, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 25, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 26, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 27, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 28, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 29, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 30, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 31, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 32, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 33, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 34, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 35, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 36, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 37, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 38, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 39, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 40, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 41, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 42, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 43, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 44, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 45, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 46, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 47, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 48, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 49, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 50, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 51, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 52, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 53, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 54, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 55, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 56, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 57, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 58, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 59, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 60, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 61, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 62, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 63, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 64, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 65, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 66, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 67, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 68, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 69, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 70, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 71, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 72, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 73, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 74, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 75, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 76, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 77, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 78, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 79, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 80, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 81, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 82, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 83, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 84, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 85, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 86, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 87, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 88, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 89, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 90, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 91, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 92, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 93, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 94, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 95, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 96, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 97, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 98, champion fitness: 400.0000
 INFO  cosyne::cosyne > gen 99, champion fitness: 400.0000
champion: (ANN { num_inputs: 4, num_outputs: 1, layers: [Layer { input_len: 4, output_len: 1, gene_len: 5, activation: Relu, weights: Matrix { data: VecStorage { data: [0.07007277006795642, -0.23631177830558103, 0.44346640185648933, 0.13902369122718516], nrows: Dynamic { value: 1 }, ncols: Dynamic { value: 4 } } }, biases: Matrix { data: VecStorage { data: [0.5129485592119563], nrows: Dynamic { value: 1 }, ncols: Dynamic { value: 1 } } }, act: Matrix { data: VecStorage { data: [0.0, 0.0, 0.0, 0.0], nrows: Dynamic { value: 4 }, ncols: Dynamic { value: 1 } } }, act_func: 0x10c1714a0, net: Matrix { data: VecStorage { data: [0.0], nrows: Dynamic { value: 1 }, ncols: Dynamic { value: 1 } } } }], num_genes: 5 }, 400.0)
training time: 0ms
rendering champion...            <--- Nothing happens

[Linux] Error in Build in Cartpole Example due to sdl2-gfx

Hi, I encountered a few errors in building the cartpole example.

I was able to get them sorted out, but still making this issue to document. Things I had to do to solve:

Install cmake (sudo apt install cmake)
- This was a fresh installation that didn't happen to have cmake installed, which is unlikely, yet possible for beginners/students
Add the following lines to cargo.toml (not sure if this is strictly required... but nonetheless resolved an error)
- and run cargo vcpkg build before cargo build

[package]
name = "gym-rs"
...

[dependencies]
...

+ [package.metadata.vcpkg]
+ dependencies = ["sdl2", "sdl2-gfx"]
+ git = "https://github.com/microsoft/vcpkg"
+ rev = "501db0f" # or change based on the latest release in GitHub

Install libsdl2-gfx-dev in addition to libsdl2-dev
- I feel that other users may encounter the same issue; perhaps would be good to update readme

Thanks for making this awesome library! Feel free to investigate more or close the issue.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.