juliareinforcementlearning / commonrlspaces.jl Goto Github PK

Iteration needs to be implemented for discrete RepeatedSpace. Currently the test is broken:

Currently, we use Base.eltype and Base.clamp in our interface. However this can be problematic since people tend to interpret these differently. For instance, eltype(1..2) returns Int. (and hence the broken test at

Should we introduce CommonRLSpaces.element_type and CommonRLInterface.clamp to deal with these?

Hi @findmyway, I noticed you added clamp! to the readme. If I am implementing an agent, when should I expect clamp! to be implemented in addition to clamp. For instance, obviously I cannot expect clamp!(3, 1..1) to work, but I would expect clamp!(::Vector, ::Box) to work. In my code, how can I check and be confident that it will work?

• Make sure the concept of Space is applicable to all the environments defined in RLEnvs
• Check whether it is applicable to some other packages:
  • POMDPs.jl
  • AlphaZero.jl
  • Dojo.jl
• Test on GPU
• Move this into CommonRLInterface

Hi,

It seems that this package cannot be added correctly...
Any hints for the solution?
Thank you.

pkg> add CommonRLSpaces
Resolving package versions...
ERROR: Unsatisfiable requirements detected for package ReinforcementLearningExperiments [6bd458e5]:
ReinforcementLearningExperiments [6bd458e5] log:
├─possible versions are: 0.1.0-0.3.5 or uninstalled
├─restricted to versions * by an explicit requirement, leaving only versions 0.1.0-0.3.5
├─restricted by compatibility requirements with ReinforcementLearningCore [de1b191a] to versions: 0.1.0-0.1.4 or uninstalled, leaving only versions: 0.1.0-0.1.4
│ └─ReinforcementLearningCore [de1b191a] log:
│ ├─possible versions are: 0.8.13 or uninstalled
│ └─ReinforcementLearningCore [de1b191a] is fixed to version 0.8.13
└─restricted by compatibility requirements with IntervalSets [8197267c] to versions: 0.2.0-0.3.5 or uninstalled — no versions left
└─IntervalSets [8197267c] log:
├─possible versions are: 0.2.1-0.7.8 or uninstalled
├─restricted to versions * by an explicit requirement, leaving only versions 0.2.1-0.7.8
└─restricted by compatibility requirements with CommonRLSpaces [408f5b3e] to versions: 0.7.0-0.7.8
└─CommonRLSpaces [408f5b3e] log:
├─possible versions are: 0.1.0-0.2.1 or uninstalled
└─restricted to versions * by an explicit requirement, leaving only versions 0.1.0-0.2.1

This would be nice:

julia> space = NamedTupleProduct(fruit=(:litchi, :longan, :mango), velocity=-1..1)

julia> rand(space)
(fruit = :mango, velocity = 0.142)

Hey @findmyway, thanks for getting started on this package.

@jamgochiana and I are trying to improve the continuous action space support in POMDPs.jl and we're hoping to use some version of this package.

I like the idea of defining some traits to communicate about properties of the spaces. But I am wondering what the purpose of the Space wrapper is. Why don't we just allow people to use normal Vectors or Tuples and then define the traits for those objects directly? i.e.

SpaceStyle(::AbstractVector) = DiscreteSpaceStyle{()}()

I recently discovered DomainSets.jl. It contains some of the sets we need, for example cartesian products, etc. What should this package's relationship to DomainSets be? I see a few possibilities.

1. No relationship - we act like DomainSets doesn't exist (but perhaps learn from it)
2. We maintain compatibility in case someone wants to use things from DomainSets, but also re-implement the functionality that we want here.
3. We use them as a base package for our interface and encourage using their sets in most cases, and only implement a few things that they don't provide (i.e. we would probably get rid of our product and use theirs.)
4. We re-export some of their sets like we do with IntervalSets (this seems bad)
5. We archive this package because DomainSets provides everything we need.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!