aunum / gold Goto Github PK

Hi,

I'm trying to create my own agent, following the example here: https://github.com/aunum/gold/tree/master/pkg/v1/agent/reinforce It's an example for CartPole-v0, while I'm playing around with LunarLander-v2.

I've come now to implementing the "step" function. And I realized that Env.Step() function takes int as an argument. Maybe that's what is needed for the CartPole-v0, but LunarLander-v2's step is float: https://github.com/openai/gym/blob/master/gym/envs/box2d/lunar_lander.py#L111

# Action is two floats [main engine, left-right engines].
# Main engine: -1..0 off, 0..+1 throttle from 50% to 100% power. Engine can't work with less than 50% power.
# Left-right:  -1.0..-0.5 fire left engine, +0.5..+1.0 fire right engine, -0.5..0.5 off

Pretty much everywhere in the gold library where I meet anything action-related - it's int.

How should it be mapped to the continuous environments? Or is it not possible at the moment? If so, would it be difficult to implement it?

#5 was caused by a local build of the sphere container that hadn't been pushed and highlighted the need for build pipelines

currently if you refresh the page on the agent ui it will fail to reload the charts

It seems that the only working environment is CartPole-v1. If I try to use LunarLander-v2 or something else from Box2D, I get an error, for example:

	server, err := env.FindOrCreate(env.GymServerConfig)
	if err != nil {
		log.Fatal(err)
	}
	defer server.Close()

	environment, err = server.Make("LunarLander-v2", env.WithRecorder())
	if err != nil {
		log.Fatal(err)
	}

Error:

2020/04/10 18:15:13 rpc error: code = Unknown desc = Exception calling application: module 'gym.envs.box2d' has no attribute 'LunarLander'

I wonder whether it's expected behavior or not? Should I perform some extra steps to enable additional environments?

Does it support GPU training?

Hi,

I'm running a lot of episodes of LunarLander in a loop (maybe hundreds). When I call environment.End() in the end, I get all the episodes information printed, like this:

...

  "15":
    episode_id: 15
    episode_length: 80
    reward: 20.416142
    timestamp:
      seconds: 1588666897

...

But, for some reason, it records only a few videos:

Videos:
  "0":
    content_type: video/mp4
  "1":
    content_type: video/mp4
    episode_id: 1
  "8":
    content_type: video/mp4
    episode_id: 8
  "27":
    content_type: video/mp4
    episode_id: 27

I'm having a hard time trying to find the parameter that can turn on saving video for each and every episode. Is it even possible?

C:\Users\<...>\go\pkg\mod\github.com\aunum\[email protected]\pkg\v1\agent\q\experiments\taxi>go run main.go
# github.com/ory/dockertest/docker/pkg/system
..\..\..\..\..\..\..\..\ory\[email protected]+incompatible\docker\pkg\system\filesys_windows.go:113:24: cannot use uintptr(unsafe.Pointer(&sd[0])) (type uintptr) as type *"golang.org/x/sys/windows".SECURITY_DESCRIPTOR in assignment

go version go1.16.6 windows/amd64

Using go1.14.1 with 8dedf68 I get a panic in taxi experiment:

go run -trimpath ./pkg/v1/agent/q/experiments/taxi/main.go
ℹ  tracking data in /var/folders/zy/lzyp__pd7vx6762s5jcgkkl80000gn/T/stats.104998610.json
ℹ  connecting to existing container "stupefied_taussig"
✔  connected to server "gym"
✔  created env: f07159db-890d-45ca-a295-aa9e383507de
ℹ  serving agent api/ui on http://localhost:39421
ℹ  running for 5000 episodes
✖  rpc error: code = Unknown desc = Exception calling application: 'int' object has no attribute 'ravel'
goroutine 1 [running]:
runtime/debug.Stack(0x75, 0x0, 0x0)
        runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
        runtime/debug/stack.go:16 +0x22
github.com/aunum/log.fatalf(0xc0007e0100, 0x1c76e46, 0x3, 0xc0007e1f28, 0x1, 0x1)
        github.com/aunum/[email protected]/log.go:77 +0x11d
github.com/aunum/log.Fatalf(...)
        github.com/aunum/[email protected]/log.go:64
github.com/aunum/log.Fatal(0xc0007e1f28, 0x1, 0x1)
        github.com/aunum/[email protected]/log.go:109 +0xa3
github.com/aunum/gold/pkg/v1/common/require.NoError(...)
        github.com/aunum/gold/pkg/v1/common/require/require.go:18
main.main()
        command-line-arguments/main.go:36 +0x303
exit status 1

I installed and tried running the demo:

C:\Users\0\Desktop\gold-master>go run ./pkg/v1/agent/deepq/experiments/cartpole/main.go

github.com/ory/dockertest/docker/pkg/system
....\go\pkg\mod\github.com\ory\[email protected]+incompatible\docker\pkg\system\filesys_windows.go:113:26: cannot use uintptr(unsafe.Pointer(&sd[0])) (value of type uintptr) as *"golang.org/x/sys/windows".SECURITY_DESCRIPTOR value in assignment

Currently toy text problems don't have visualization because the only way they are rendered is with ansi color codes. Should provide a means of rendering that locally in the terminal.

Cloned repository and using the quick start instructions.

$ go run ./pkg/v1/agent/deepq/experiments/cartpole/main.go
# github.com/aunum/gold/pkg/v1/dense
pkg/v1/dense/value.go:57:19: cannot convert FauxZero (type float64) to type complex64
pkg/v1/dense/value.go:59:20: cannot convert FauxZero (type float64) to type complex128

Using go v1.14 on linux/amd64