aunum / gold Goto Github PK
View Code? Open in Web Editor NEWReinforcement Learning in Go
License: Apache License 2.0
Reinforcement Learning in Go
License: Apache License 2.0
Hi,
I'm trying to create my own agent, following the example here: https://github.com/aunum/gold/tree/master/pkg/v1/agent/reinforce It's an example for CartPole-v0
, while I'm playing around with LunarLander-v2
.
I've come now to implementing the "step" function. And I realized that Env.Step()
function takes int
as an argument. Maybe that's what is needed for the CartPole-v0
, but LunarLander-v2
's step is float
: https://github.com/openai/gym/blob/master/gym/envs/box2d/lunar_lander.py#L111
# Action is two floats [main engine, left-right engines].
# Main engine: -1..0 off, 0..+1 throttle from 50% to 100% power. Engine can't work with less than 50% power.
# Left-right: -1.0..-0.5 fire left engine, +0.5..+1.0 fire right engine, -0.5..0.5 off
Pretty much everywhere in the gold
library where I meet anything action-related - it's int
.
How should it be mapped to the continuous environments? Or is it not possible at the moment? If so, would it be difficult to implement it?
#5 was caused by a local build of the sphere container that hadn't been pushed and highlighted the need for build pipelines
currently if you refresh the page on the agent ui it will fail to reload the charts
It seems that the only working environment is CartPole-v1
. If I try to use LunarLander-v2
or something else from Box2D, I get an error, for example:
server, err := env.FindOrCreate(env.GymServerConfig)
if err != nil {
log.Fatal(err)
}
defer server.Close()
environment, err = server.Make("LunarLander-v2", env.WithRecorder())
if err != nil {
log.Fatal(err)
}
Error:
2020/04/10 18:15:13 rpc error: code = Unknown desc = Exception calling application: module 'gym.envs.box2d' has no attribute 'LunarLander'
I wonder whether it's expected behavior or not? Should I perform some extra steps to enable additional environments?
Does it support GPU training?
Hi,
I'm running a lot of episodes of LunarLander in a loop (maybe hundreds). When I call environment.End()
in the end, I get all the episodes information printed, like this:
...
"15":
episode_id: 15
episode_length: 80
reward: 20.416142
timestamp:
seconds: 1588666897
...
But, for some reason, it records only a few videos:
Videos:
"0":
content_type: video/mp4
"1":
content_type: video/mp4
episode_id: 1
"8":
content_type: video/mp4
episode_id: 8
"27":
content_type: video/mp4
episode_id: 27
I'm having a hard time trying to find the parameter that can turn on saving video for each and every episode. Is it even possible?
C:\Users\<...>\go\pkg\mod\github.com\aunum\[email protected]\pkg\v1\agent\q\experiments\taxi>go run main.go
# github.com/ory/dockertest/docker/pkg/system
..\..\..\..\..\..\..\..\ory\[email protected]+incompatible\docker\pkg\system\filesys_windows.go:113:24: cannot use uintptr(unsafe.Pointer(&sd[0])) (type uintptr) as type *"golang.org/x/sys/windows".SECURITY_DESCRIPTOR in assignment
go version go1.16.6 windows/amd64
Using go1.14.1
with 8dedf68 I get a panic in taxi experiment:
go run -trimpath ./pkg/v1/agent/q/experiments/taxi/main.go
ℹ tracking data in /var/folders/zy/lzyp__pd7vx6762s5jcgkkl80000gn/T/stats.104998610.json
ℹ connecting to existing container "stupefied_taussig"
✔ connected to server "gym"
✔ created env: f07159db-890d-45ca-a295-aa9e383507de
ℹ serving agent api/ui on http://localhost:39421
ℹ running for 5000 episodes
✖ rpc error: code = Unknown desc = Exception calling application: 'int' object has no attribute 'ravel'
goroutine 1 [running]:
runtime/debug.Stack(0x75, 0x0, 0x0)
runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
runtime/debug/stack.go:16 +0x22
github.com/aunum/log.fatalf(0xc0007e0100, 0x1c76e46, 0x3, 0xc0007e1f28, 0x1, 0x1)
github.com/aunum/[email protected]/log.go:77 +0x11d
github.com/aunum/log.Fatalf(...)
github.com/aunum/[email protected]/log.go:64
github.com/aunum/log.Fatal(0xc0007e1f28, 0x1, 0x1)
github.com/aunum/[email protected]/log.go:109 +0xa3
github.com/aunum/gold/pkg/v1/common/require.NoError(...)
github.com/aunum/gold/pkg/v1/common/require/require.go:18
main.main()
command-line-arguments/main.go:36 +0x303
exit status 1
I installed and tried running the demo:
C:\Users\0\Desktop\gold-master>go run ./pkg/v1/agent/deepq/experiments/cartpole/main.go
github.com/ory/dockertest/docker/pkg/system
....\go\pkg\mod\github.com\ory\[email protected]+incompatible\docker\pkg\system\filesys_windows.go:113:26: cannot use uintptr(unsafe.Pointer(&sd[0])) (value of type uintptr) as *"golang.org/x/sys/windows".SECURITY_DESCRIPTOR value in assignment
Currently toy text problems don't have visualization because the only way they are rendered is with ansi color codes. Should provide a means of rendering that locally in the terminal.
Cloned repository and using the quick start instructions.
$ go run ./pkg/v1/agent/deepq/experiments/cartpole/main.go
# github.com/aunum/gold/pkg/v1/dense
pkg/v1/dense/value.go:57:19: cannot convert FauxZero (type float64) to type complex64
pkg/v1/dense/value.go:59:20: cannot convert FauxZero (type float64) to type complex128
Using go v1.14 on linux/amd64
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.