scazlab / human_robot_collaboration Goto Github PK

If the arm Controller is tasked with acting upon a list of object, the choice happens right inside the service, but it would be more robust if the list of objects to act upon is transferred out of the service call and directly inside the class so that we the controller can choose which one to act on with more flexibility. That would improve the success rate of the service during action cleanup for example.

As per 52c9dbc , now the ArmCtrl class accepts an array of objects. The SW then chooses randomly one of those objects among those available, but it should also choose among those that are set in the launch file.

• Object detections
• Gesture with human
• 3D gesture recognition

The force interaction module seems to not be working when the robot passes the screwdriver. It basically always open the hand even without any interaction. To be investigated. Cc @sarah-widder

Ignore this. I created it to be able to upload a picture to use in the README.md file of this repository.

Theoretically, svox-tts can be added to the repository. Also, it would be nice to surface the text that has been spoken to the Baxter's display.

After the deadline, the repository will be restructured, to improve modularity, dependencies and ease of use. Similarly to aruco_ros, we will have three different packages:

• baxter_collaboration_lib → with the libraries
• baxter_collaboration_msgs → with messages and services
• baxter_collaboration → with the source code used in our experiments

Compilation would be faster as well, at least in some cases.

It would be nice to be able to trigger a robot-less mode for testing purposes.

Right now, we have a semi-working situation with the vacuum gripper (i.e. it works, but we didn't have time to code it properly). It would be nice to fully port the sw to a robust scenario.

Checklist

• /baxter_collaboration_lib/src/robot_utils

  • utils.cpp
  • baxter_trac_ik.cpp
  • ros_thread_image.cpp

• /baxter_collaboration_lib/src/robot_interface

  • robot_interface.cpp
  • arm_ctrl.cpp
  • gripper.cpp

• /baxter_collaboration_lib/src/robot_perception

  • aruco_client.cpp
  • cartesian_estimator.cpp
  • cartesian_estimator_client.cpp
  • cartesian_estimator_hsv.cpp

As per title

For no reason at all, sometimes the hsv_detector segfaults. I traced the issue back to some ROS internals. Here is some gdb information:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd9d63700 (LWP 8265)]
0x00007ffff78a4366 in ros::Subscription::pubUpdate(std::vector<std::string, std::allocator<std::string> > const&) ()
   from /opt/ros/indigo/lib/libroscpp.so
(gdb) where
#0  0x00007ffff78a4366 in ros::Subscription::pubUpdate(std::vector<std::string, std::allocator<std::string> > const&) ()
   from /opt/ros/indigo/lib/libroscpp.so
#1  0x00007ffff7849126 in ros::TopicManager::pubUpdate(std::string const&, std::vector<std::string, std::allocator<std::string> > const&) ()
   from /opt/ros/indigo/lib/libroscpp.so
#2  0x00007ffff7850b44 in ros::TopicManager::pubUpdateCallback(XmlRpc::XmlRpcValue&, XmlRpc::XmlRpcValue&) () from /opt/ros/indigo/lib/libroscpp.so
#3  0x00007ffff783f7ba in ros::XMLRPCCallWrapper::execute(XmlRpc::XmlRpcValue&, XmlRpc::XmlRpcValue&) () from /opt/ros/indigo/lib/libroscpp.so
#4  0x00007ffff540987f in XmlRpc::XmlRpcServerConnection::executeMethod(std::string const&, XmlRpc::XmlRpcValue&, XmlRpc::XmlRpcValue&) ()
   from /opt/ros/indigo/lib/libxmlrpcpp.so
#5  0x00007ffff540c22c in XmlRpc::XmlRpcServerConnection::executeRequest() () from /opt/ros/indigo/lib/libxmlrpcpp.so
#6  0x00007ffff540950c in XmlRpc::XmlRpcServerConnection::writeResponse() () from /opt/ros/indigo/lib/libxmlrpcpp.so
#7  0x00007ffff54096d0 in XmlRpc::XmlRpcServerConnection::handleEvent(unsigned int) () from /opt/ros/indigo/lib/libxmlrpcpp.so
#8  0x00007ffff540761e in XmlRpc::XmlRpcDispatch::work(double) () from /opt/ros/indigo/lib/libxmlrpcpp.so
#9  0x00007ffff783c5aa in ros::XMLRPCManager::serverThreadFunc() () from /opt/ros/indigo/lib/libroscpp.so
#10 0x00007ffff421ca4a in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
#11 0x00007ffff4db9184 in start_thread (arg=0x7fffd9d63700) at pthread_create.c:312
#12 0x00007ffff6b7337d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) bt full
#0  0x00007ffff78a4366 in ros::Subscription::pubUpdate(std::vector<std::string, std::allocator<std::string> > const&) ()
   from /opt/ros/indigo/lib/libroscpp.so
No symbol table info available.
#1  0x00007ffff7849126 in ros::TopicManager::pubUpdate(std::string const&, std::vector<std::string, std::allocator<std::string> > const&) ()
   from /opt/ros/indigo/lib/libroscpp.so
No symbol table info available.
#2  0x00007ffff7850b44 in ros::TopicManager::pubUpdateCallback(XmlRpc::XmlRpcValue&, XmlRpc::XmlRpcValue&) () from /opt/ros/indigo/lib/libroscpp.so
No symbol table info available.
#3  0x00007ffff783f7ba in ros::XMLRPCCallWrapper::execute(XmlRpc::XmlRpcValue&, XmlRpc::XmlRpcValue&) () from /opt/ros/indigo/lib/libroscpp.so
No symbol table info available.
#4  0x00007ffff540987f in XmlRpc::XmlRpcServerConnection::executeMethod(std::string const&, XmlRpc::XmlRpcValue&, XmlRpc::XmlRpcValue&) ()
   from /opt/ros/indigo/lib/libxmlrpcpp.so
No symbol table info available.
#5  0x00007ffff540c22c in XmlRpc::XmlRpcServerConnection::executeRequest() () from /opt/ros/indigo/lib/libxmlrpcpp.so
No symbol table info available.
#6  0x00007ffff540950c in XmlRpc::XmlRpcServerConnection::writeResponse() () from /opt/ros/indigo/lib/libxmlrpcpp.so
No symbol table info available.
#7  0x00007ffff54096d0 in XmlRpc::XmlRpcServerConnection::handleEvent(unsigned int) () from /opt/ros/indigo/lib/libxmlrpcpp.so
No symbol table info available.
#8  0x00007ffff540761e in XmlRpc::XmlRpcDispatch::work(double) () from /opt/ros/indigo/lib/libxmlrpcpp.so
No symbol table info available.
#9  0x00007ffff783c5aa in ros::XMLRPCManager::serverThreadFunc() () from /opt/ros/indigo/lib/libroscpp.so
No symbol table info available.
#10 0x00007ffff421ca4a in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
No symbol table info available.
#11 0x00007ffff4db9184 in start_thread (arg=0x7fffd9d63700) at pthread_create.c:312
        __res = <optimized out>
        pd = 0x7fffd9d63700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736848082688, 5626924391969783832, 1, 0, 140736848083392, 140736848082688, -5626849403075100648, 
                -5626913084039954408}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#12 0x00007ffff6b7337d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.
(gdb) list
1	#include <stdio.h>
2	
3	#include <ros/ros.h>
4	#include "robot_perception/cartesian_estimator_hsv.h"
5	
6	int main(int argc, char ** argv)
7	{
8	    ros::init(argc, argv, "hsv_detector");
9	    ros::NodeHandle _n("hsv_detector");
10	

Unfortunately, the only thing I found online talking about this is here. I would need to manually compile ROS and debug it in order to fix it, but for now time is not enough.

@brahmgardner what is the status of the velocity_ctrl branch? What should we do with that branch?

As per title, the force interaction parameters (and possibly others) should be present into modular_action_provider.launch as well.

Cc @sarah-widder

Fake issue to list all of our options:

1. Google TTS → It's the best but it is very difficult to implement without a proper Google Cloud subscription. The best semi-legal way I found is here https://gist.github.com/alotaiba/1728771 and specifically in oaeide's comment: wget -q -U Mozilla -O output.mp3 "http://translate.google.com/translate_tts?ie=UTF-8&total=1&idx=0&textlen=32&client=tw-ob&q=Test&tl=En-gb". For now it works, but it may break any minute.
2. SVOX PICO → It seems the best choice because it is open source, available in C++, and I know a repo that already uses it (this one https://github.com/robotology/speech/tree/master/svox-speech from my previous lab). It should be pretty easy to port to ROS.
3. https://github.com/UbiquityRobotics/speech_commands has a pretty basic TTS (included with the STT). It is more complex, it is web based, but also this one might work.
4. Festival is already there, it's free to use, but it is pretty bad.

The force interaction seems to behave inconsistently. @omangin are you sure that the approach you suggested to @sarah-widder is the way to go?

She fixed the right arm for passing the screwdriver, but now it does not work anymore for holding. Same thing to the left arm. I am starting to wonder if it is worth to revert back to the previous version of the SW.

Add citation in the README.md after April, 30th that references our ICRA paper.

As per title. Ref #42

As per title. Right now ROSThreadImage works only with bgr8 images (ie three channels, BGR). It would be nice to expand the class to work also with other types, most notably mono8 (ie one channel only).

The class shoulud be extended to allow for a new member that encapsulates the type of encoding. This should be passed in the constructor (bgr8 will still be the default), and used in the class. A new test should be added that tests the class also with a mono image.

Right now the object database is a rosparam encoded as an array of arrays of arrays, which is far from being the best solution.

In the branch feature/new_object_db I already changed the launch file to call them as YAML structures, but the backend still needs to be coded.

when i run "Compile tests and run them" step . i meet wiith a problem as ImportError: cannot import name speech. i can not resolve this problem,can you help me?

@JakeBrawer I think it is time for you to either delete the branch or keep only what is needed.