Reason for Space wrapper? about commonrlspaces.jl

[findmyway]

Hello @zsunberg ,

But I am wondering what the purpose of the Space wrapper is.

It is to support some complicated state/action spaces.

Let's say we have an environment of action space of ((:left, :top), (false, true)). Now we want to sample one random action from it. But what's the result? Is it one of (:left, :top) and (false, true), or is it one of (:left, false), (:left, true), (:top, false), (:top, true)?

To avoid such confusion, the Space wrapper is added to explicitly distinguish between such two cases.

For the former, we use Space(((:left, :top), (false, true))), for the later we use (Space((:left, :top)), Space((false, true))).

Another way to address such confusion is to define dedicated space types, like TupleSpace, VectorSpace or DictSpace. Personally I feel this approach is a bit redundant. However, that's just based on my limited programming experience.

Why don't we just allow people to use normal Vectors or Tuples and then define the traits for those objects directly?

I think we can add the support for some common element types (Number, String, Symbol, etc) like you proposed above. So that for some common spcaes, users don't need create the Space wrapper at all. What do you think?

[zsunberg]

What if we get rid of Space and define the elements of a space to be all objects x for which x in space returns true. This eliminates the ambiguity. union and rand would also work nicely with this.

We could also introduce product (or setproduct or cartesian_product) to accomplish what you were thinking with (Space((:left, :top)), Space((false, true))).

What do you think? I am going to start a PR with this so we can discuss.

[findmyway]

What if we get rid of Space and define the elements of a space to be all objects x for which x in space returns true. This eliminates the ambiguity. union and rand would also work nicely with this.

Yes, But then we may need to figure out how to handle continuous spaces.

What do you think? I am going to start a PR with this so we can discuss.

That'd be great!

[zsunberg]

closed with #9