Comments (9)
@shmily326 Thank you for opening an issue! I will look into it and get back to you.
from deepbots.
@shmily326 Just a reminder, you can pip install git+https://github.com/aidudezzz/deepbots.git
for general use before we publish the next version of deepbots on PyPI.
We would like to close this issue. Feel free to open another issue or reopen it if needed. Also, we will be glad if you share your work or experience with us. 😄
from deepbots.
Thanks @shmily326! You seem to be correct in your comment, we had several issues regarding emitters and receivers since the beginning, and there were some bugs in webots too (see cyberbotics/webots#1384, where multiple issues were fixed).
As @KelvinYang0320 mentioned, we will look into it and incorporate required changes to make it work as close as possible to what is expected.
Meanwhile, i would suggest using the RobotSupervisor scheme which uses the same controller both to control the robot and act as supervisor. Its usage is much more efficient and straightforward in cases where you don't specifically require separation between robot and supervisor. If you want you can share any additional information about your use-case, so we can discuss it further.
from deepbots.
Hi @shmily326
With the following modifications,
1.
def step(self, action):
"""
The basic step method that steps the controller,
calls the method that sends the action through the emitter
and returns the (observations, reward, done, info) object.
:param action: Whatever the use-case uses as an action, e.g.
an integer representing discrete actions
:type action: Defined by the implementation of handle_emitter
:return: (observations, reward, done, info) as provided by the
corresponding methods as implemented for the use-case
"""
print(self.getFromDef("ROBOT").getPosition()[0], "step-1")
if super(Supervisor, self).step(self.timestep) == -1:
exit()
print(self.getFromDef("ROBOT").getPosition()[0], "step-2")
self.handle_emitter(action)
print(self.getFromDef("ROBOT").getPosition()[0], "step-3")
return (
self.get_observations(),
self.get_reward(action),
self.is_done(),
self.get_info(),
)
def handle_emitter(self):
"""
This emitter uses the user-implemented create_message() method to get
whatever data the robot gathered, convert it to a string if needed and
then use the emitter to send the data in a string utf-8 encoding to the
supervisor.
"""
print("handle_emitter")
data = self.create_message()
...
def handle_receiver(self):
"""
This receiver uses the basic Webots receiver-handling code. The
use_message_data() method should be implemented to actually use the
data received from the supervisor.
"""
print("handle_receiver")
if self.receiver.getQueueLength() > 0:
...
you will get the following in cartPoleWorldEmitterReceiver on Webots 2023a:
0:00:00:000~0:00:00:032
RESET
0.0 step-1
handle_receiver
handle_emitter
0:00:00:032~0:00:00:064
-1.546550598149922e-22 step-2
-1.546550598149922e-22 step-3
-1.546550598149922e-22 step-1
handle_receiver
handle_emitter
0:00:00:064~0:00:00:096
1.1115304692030285e-08 step-2
1.1115304692030285e-08 step-3
1.1115304692030285e-08 step-1
handle_receiver
handle_emitter
From my perspective, you will not get the next state in
from deepbots.
@shmily326 I have opened a PR to address that.
Could you check if the problem is solved?
git clone https://github.com/aidudezzz/deepbots.git
git checkout step_function
pip install -e .
from deepbots.
@KelvinYang0320 Thank you for all of your time, I'm working on multi-agent RL (specifically a multi-UAV navigation scenario and Actor-Critic algorithms), thus I think the emitter-receiver scheme would be more appropriate, and I will check the "
step the controller after applying the action" method and get back to you as soon as possible.
from deepbots.
@shmily326 You can take a look at this PR for a multi-robot example.
Also, we have several examples in deepworlds.
from deepbots.
I'm working on multi-agent RL
That sounds great! For multi-agent scenarios indeed it can be better to have a centralized supervisor that communicates with multiple robots, so you need to use the emitter-receiver scheme. When completed, if you want, we will be happy to include your scenario as an example on our deepworlds repository! 😄
from deepbots.
@shmily326 You can get updated deepbots by
git clone https://github.com/aidudezzz/deepbots.git
cd ./deepbots
pip install -e .
We have merged the PR.
from deepbots.
Related Issues (20)
- There seems to be a bug in the step function of the robot_supervisor.py HOT 1
- Extend deepbots to support stable-baselines and implement gym-style default enviroments HOT 3
- Extend deepbots to support Evolutionary Algorithms HOT 14
- Question: how to get kinect camera information HOT 7
- [Tracker] Extension to Evolutionary Algorithms
- How to use ray to train in deepbots environment. HOT 3
- Create documentation site
- Modify Docker
- get_timestep deprecation warning in supervisor_emitter_receiver
- Is it possible to run several simulations in parallel? HOT 1
- RobotEmitterReceiver class should inherit from Webots Robot class
- Usage of snake_case
- A bug of robot never moving HOT 1
- CSV handle_emitter data types
- Error in gym setup command: 'extras_require' HOT 5
- Migrating to gymnasium
- Unable to install deepbots HOT 2
- Gym Environment and using stable baselines HOT 2
- Request for Assistance with Sim2Real Transformation Using Emitter Receiver in DeepBots HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepbots.