lsw9021 / mass

MASS(Muscle-Actuated Skeletal System)

Abstract

This code implements a basic simulation and control for full-body Musculoskeletal system. Skeletal movements are driven by the actuation of the muscles, coordinated by activation levels. Interfacing with python and pytorch, it is available to use Deep Reinforcement Learning(DRL) algorithm such as Proximal Policy Optimization(PPO).

Publications

Seunghwan Lee, Kyoungmin Lee, Moonseok Park, and Jehee Lee Scalable Muscle-actuated Human Simulation and Control, ACM Transactions on Graphics (SIGGRAPH 2019), Volume 37, Article 73.

Project Page : http://mrl.snu.ac.kr/research/ProjectScalable/Page.htm

Youtube : https://youtu.be/a3jfyJ9JVeM

Paper : http://mrl.snu.ac.kr/research/ProjectScalable/Paper.pdf

How to install

Install TinyXML, Eigen, OpenGL, assimp, Python3, etc...

sudo apt-get install libtinyxml-dev libeigen3-dev libxi-dev libxmu-dev freeglut3-dev libassimp-dev libpython3-dev python3-tk python3-numpy virtualenv ipython3 cmake-curses-gui libbullet-dev pybind11-dev

Install DART 6.8

Please refer to http://dartsim.github.io/ (Install version 6.8)

Manual from DART(http://dartsim.github.io/install_dart_on_ubuntu.html)

Install PIP things

You should first activate virtualenv.

virtualenv /path/to/venv --python=python3
source /path/to/venv/bin/activate

• pytorch(https://pytorch.org/)

• numpy, matplotlib

pip3 install numpy matplotlib ipython

How to compile and run

Resource

Our system require a reference motion to imitate. We provide sample references such as walking, running, and etc...

To learn and simulate, we should provide such a meta data. We provide default meta data in /data/metadata.txt. We parse the text and set the environment. Please note that the learning settings and the test settings should be equal.(metadata.txt should not be changed.)

Compile and Run

mkdir build
cd build
cmake .. 
make -j8

• Run Training

cd python
source /path/to/virtualenv/
python3 main.py -d ../data/metadata.txt

All the training networks are saved in /nn folder.

• Run UI

source /path/to/virtualenv/
./render/render ../data/metadata.txt

• Run Trained data

source /path/to/virtualenv/
./render/render ../data/metadata.txt ../nn/xxx.pt ../nn/xxx_muscle.pt

If you are simulating with the torque-actuated model,

source /path/to/virtualenv/
./render/render ../data/metadata.txt ../nn/xxx.pt

Model Creation & Retargeting (This module is ongoing project.)

This requires Maya and MotionBuilder.

There is a sample model in data/maya folder that I generally use. Currently if you are trying to edit the model, you have to make your own export maya-python code and xml writer so that the simulation code correctly read the musculoskeletal structure. There is also a rig model that is useful to retarget a new motion.

mass's Issues

build error [OSX]

Failing when running make -j4:

[ 50%] Built target mss
[ 58%] Linking CXX shared library ../../python/pymss.dylib
[ 66%] Linking CXX executable render
Undefined symbols for architecture x86_64:
"___kmpc_for_static_fini", referenced from:
_.omp_outlined. in EnvManager.cpp.o
_.omp_outlined..2 in EnvManager.cpp.o
_.omp_outlined..3 in EnvManager.cpp.o
_.omp_outlined..4 in EnvManager.cpp.o
"___kmpc_for_static_init_4", referenced from:
_.omp_outlined. in EnvManager.cpp.o
_.omp_outlined..2 in EnvManager.cpp.o
_.omp_outlined..3 in EnvManager.cpp.o
_.omp_outlined..4 in EnvManager.cpp.o
"___kmpc_fork_call", referenced from:
EnvManager::Steps(int) in EnvManager.cpp.o
EnvManager::StepsAtOnce() in EnvManager.cpp.o
EnvManager::GetMuscleTorques() in EnvManager.cpp.o
EnvManager::GetDesiredTorques() in EnvManager.cpp.o
........
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [../python/pymss.dylib] Error 1
make[1]: *** [python/CMakeFiles/pymss.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Undefined symbols for architecture x86_64:
"___kmpc_fork_call", referenced from:

Any clues?

Building Error

Hi, I think I installed successfully the boost 1.66 and Dart 6.3
and then I tried the Compile and Run part. After executing the make -j8, I got the below error.
Does it seem that it's related to boost?
Thank you.

[ 50%] Built target mss
[ 58%] Linking CXX executable render
[ 83%] Built target pymss
CMakeFiles/render.dir/Window.cpp.o: In function toNumPyArray(Eigen::Matrix<double, -1, 1, 0, -1, 1> const&)': Window.cpp:(.text+0x15d3): undefined reference to boost::python::numpy::dtype boost::python::numpy::detail::get_float_dtype<32>()'
Window.cpp:(.text+0x15e1): undefined reference to boost::python::numpy::empty(boost::python::tuple const&, boost::python::numpy::dtype const&)' CMakeFiles/render.dir/Window.cpp.o: In function MASS::Window::GetActivationFromNN(Eigen::Matrix<double, -1, 1, 0, -1, 1> const&)':
Window.cpp:(.text+0x17ba): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)' Window.cpp:(.text+0x1839): undefined reference to boost::python::numpy::from_object(boost::python::api::object const&, int, int, boost::python::numpy::ndarray::bitflag)'
Window.cpp:(.text+0x1b01): undefined reference to boost::python::throw_error_already_set()' CMakeFiles/render.dir/Window.cpp.o: In function MASS::Window::GetActionFromNN()':
Window.cpp:(.text+0x1c6c): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)' Window.cpp:(.text+0x1ce3): undefined reference to boost::python::numpy::dtype boost::python::numpy::detail::get_float_dtype<32>()'
Window.cpp:(.text+0x1cf3): undefined reference to boost::python::numpy::empty(boost::python::tuple const&, boost::python::numpy::dtype const&)' Window.cpp:(.text+0x1e34): undefined reference to boost::python::numpy::from_object(boost::python::api::object const&, int, int, boost::python::numpy::ndarray::bitflag)'
Window.cpp:(.text+0x2031): undefined reference to boost::python::throw_error_already_set()' CMakeFiles/render.dir/Window.cpp.o: In function MASS::Window::Window(MASS::Environment*)':
Window.cpp:(.text+0x2952): undefined reference to boost::python::detail::str_base::str_base(char const*)' Window.cpp:(.text+0x2962): undefined reference to boost::python::import(boost::python::str)'
Window.cpp:(.text+0x29d4): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)' Window.cpp:(.text+0x2a2e): undefined reference to boost::python::detail::str_base::str_base(char const*)'
Window.cpp:(.text+0x2a39): undefined reference to boost::python::import(boost::python::str)' Window.cpp:(.text+0x2b8c): undefined reference to boost::python::detail::str_base::str_base(char const*)'
Window.cpp:(.text+0x2be1): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)' Window.cpp:(.text+0x2c1b): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)'
Window.cpp:(.text+0x2cf1): undefined reference to boost::python::exec(char const*, boost::python::api::object, boost::python::api::object)' Window.cpp:(.text+0x2d58): undefined reference to boost::python::exec(char const*, boost::python::api::object, boost::python::api::object)'
Window.cpp:(.text+0x2dbf): undefined reference to boost::python::exec(char const*, boost::python::api::object, boost::python::api::object)' Window.cpp:(.text+0x2e26): undefined reference to boost::python::exec(char const*, boost::python::api::object, boost::python::api::object)'
Window.cpp:(.text+0x2e8d): undefined reference to boost::python::exec(char const*, boost::python::api::object, boost::python::api::object)' CMakeFiles/render.dir/Window.cpp.o:Window.cpp:(.text+0x2ef4): more undefined references to boost::python::exec(char const*, boost::python::api::object, boost::python::api::object)' follow
CMakeFiles/render.dir/Window.cpp.o: In function MASS::Window::Window(MASS::Environment*)': Window.cpp:(.text+0x2fe9): undefined reference to boost::python::throw_error_already_set()'
Window.cpp:(.text+0x2ff9): undefined reference to boost::python::throw_error_already_set()' CMakeFiles/render.dir/Window.cpp.o: In function MASS::Window::Window(MASS::Environment*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)':
Window.cpp:(.text+0x340d): undefined reference to boost::python::detail::str_base::str_base(char const*)' Window.cpp:(.text+0x3489): undefined reference to boost::python::exec(boost::python::str, boost::python::api::object, boost::python::api::object)'
Window.cpp:(.text+0x355e): undefined reference to boost::python::detail::str_base::str_base(char const*)' Window.cpp:(.text+0x35f5): undefined reference to boost::python::exec(boost::python::str, boost::python::api::object, boost::python::api::object)'
Window.cpp:(.text+0x366d): undefined reference to boost::python::eval(char const*, boost::python::api::object, boost::python::api::object)' Window.cpp:(.text+0x36f1): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)'
CMakeFiles/render.dir/Window.cpp.o: In function MASS::Window::Window(MASS::Environment*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)': Window.cpp:(.text+0x3a30): undefined reference to boost::python::detail::str_base::str_base(char const*)'
Window.cpp:(.text+0x3aac): undefined reference to boost::python::exec(boost::python::str, boost::python::api::object, boost::python::api::object)' Window.cpp:(.text+0x3b81): undefined reference to boost::python::detail::str_base::str_base(char const*)'
Window.cpp:(.text+0x3c19): undefined reference to boost::python::exec(boost::python::str, boost::python::api::object, boost::python::api::object)' Window.cpp:(.text+0x3d00): undefined reference to boost::python::detail::str_base::str_base(char const*)'
Window.cpp:(.text+0x3d97): undefined reference to boost::python::exec(boost::python::str, boost::python::api::object, boost::python::api::object)' Window.cpp:(.text+0x3e0f): undefined reference to boost::python::eval(char const*, boost::python::api::object, boost::python::api::object)'
Window.cpp:(.text+0x3e93): undefined reference to boost::python::api::getattr(boost::python::api::object const&, char const*)' CMakeFiles/render.dir/Window.cpp.o: In function boost::python::tuple boost::python::make_tuple(int const&)':
Window.cpp:(.text.ZN5boost6python10make_tupleIiEENS0_5tupleERKT[ZN5boost6python10make_tupleIiEENS0_5tupleERKT]+0x76): undefined reference to boost::python::throw_error_already_set()' Window.cpp:(.text._ZN5boost6python10make_tupleIiEENS0_5tupleERKT_[_ZN5boost6python10make_tupleIiEENS0_5tupleERKT_]+0x89): undefined reference to boost::python::throw_error_already_set()'
CMakeFiles/render.dir/Window.cpp.o: In function boost::python::tuple boost::python::make_tuple<long>(long const&)': Window.cpp:(.text._ZN5boost6python10make_tupleIlEENS0_5tupleERKT_[_ZN5boost6python10make_tupleIlEENS0_5tupleERKT_]+0x76): undefined reference to boost::python::throw_error_already_set()'
Window.cpp:(.text.ZN5boost6python10make_tupleIlEENS0_5tupleERKT[ZN5boost6python10make_tupleIlEENS0_5tupleERKT]+0x89): undefined reference to boost::python::throw_error_already_set()' CMakeFiles/render.dir/Window.cpp.o: In function boost::python::detail::returnableboost::python::api::object::type boost::python::call<boost::python::api::object, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >(_object*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, boost::typeboost::python::api::object*)':
Window.cpp:(.text._ZN5boost6python4callINS0_3api6objectENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS0_6detail10returnableIT_E4typeEP7_objectRKT0_PNS_4typeISC_EE[_ZN5boost6python4callINS0_3api6objectENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS0_6detail10returnableIT_E4typeEP7_objectRKT0_PNS_4typeISC_EE]+0x61): undefined reference to boost::python::throw_error_already_set()' CMakeFiles/render.dir/Window.cpp.o:Window.cpp:(.text._ZN5boost6python4callINS0_3api6objectENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS0_6detail10returnableIT_E4typeEP7_objectRKT0_PNS_4typeISC_EE[_ZN5boost6python4callINS0_3api6objectENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS0_6detail10returnableIT_E4typeEP7_objectRKT0_PNS_4typeISC_EE]+0x71): more undefined references to boost::python::throw_error_already_set()' follow
CMakeFiles/render.dir/main.cpp.o: In function main': main.cpp:(.text.startup+0x89): undefined reference to boost::python::numpy::initialize(bool)'
collect2: error: ld returned 1 exit status
render/CMakeFiles/render.dir/build.make:126: recipe for target 'render/render' failed
make[2]: *** [render/render] Error 1
CMakeFiles/Makefile2:145: recipe for target 'render/CMakeFiles/render.dir/all' failed
make[1]: *** [render/CMakeFiles/render.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Get nan from ConstraintSolver in training with backflip.bvh

Hi! Thanks for the great repo!

When I tried to train an agent with backflip.bvh, I got an error like this.

python3 python/main.py -d data/metadata.txt
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
Human(DOFs : 56)
Ground(DOFs : 0)
save simulation nn ../nn/current.pt
save muscle nn ../nn/current_muscle.pt
save simulation nn ../nn/0.pt
save muscle nn ../nn/0_muscle.pt
num states: 130, num actions: 50
Error [BoxedLcpConstraintSolver.cpp:291] [BoxedLcpConstraintSolver] The solution of LCP includes NAN values: Error [BoxedLcpConstraintSolver.cppnan nan:291] nan [BoxedLcpConstraintSolver] The solution of LCP includes NAN values: nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan. We're setting it zero for safety. Consider using more robust solver such as PGS as a secondary solver. If this happens even with PGS solver, please report this as a bug.
nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan. We're setting it zero for safety. Consider using more robust solver such as PGS as a secondary solver. If this happens even with PGS solver, please report this as a bug.

I just modified the metadate.txt as follows
bvh_file /data/motion/walk.bvh true
->
bvh_file /data/motion/backflip.bvh true

I think I have the same environment as yours because I followed your docker instruction.
docker build . -t mass
bash docker.sh

Thanks!!

dulicated np.isnan(states[j]) in main.py?

Hi! Thank you for sharing the excellent work!
I think this is a minor issue,
but there's duplicated np.isnan(states[j]) in main.py #205.
I guess one of them should be rewards?
cheers,

How to create some cube-shaped obstacles in the environment

or how to throw a ball at a character？ has anyone succeeded？

How to visualize the model like fig1 in your paper

Thank you for your amazing work. According to your guidance, I can visualize the results as follows:

How can I replace the cuboid in the figure above with a human skeleton to visualize the effect as shown in Figure 1?

Thank you very much.

Unable to install Compile MASS. Error: Could NOT find DART (missing: gui)

-- Found DART: /usr/local/include found components: collision-bullet dart
CMake Error at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find DART (missing: gui)
Call Stack (most recent call first):
/usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/dart/cmake/DARTConfig.cmake:174 (find_package_handle_standard_args)
render/CMakeLists.txt:7 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/MASS/build/CMakeFiles/CMakeOutput.log".

Forward projection in PD control (GetSPDForces)

I'm comparing equation 10 in the paper and the function GetSPDForces in Character.cpp.
I've found that p_diff is from a comparison between the p_desired and the linear estimation of q to the next time step (q + dq*dt).
I also assume that the term v_diff - dt*mKv.cwiseProduct(ddq) is using a forward projection of dq to the next time step (dq + ddq*dt). This projection seems to cause some significant complication to the calculation of ddq.

Main question:
These forward projection of q and dq are uncommon in PD control. Is there any specific reason not to use just q and dq of the current time step without the projection?

Additional questions:
Why dt*mKv.asDiagonal() is needed to calculate M_inv ?
Why p_diff+v_diff is used in the calculation of ddq ?

Regarding inputs to trajectory minicking network : Reference motion missing ?

Hi,

I was looking into the code and realized that GetState() outputs COM position, velocity and gait phase. This is then used as input during NN training in main.py.

However, Fig. 5 in the paper indicates reference motion as input, which is not the case in the code.

Is it the case that the figure does not represent actual inputs to the network, instead it just represents the components used in the trajectory mimicking phase (reference motion is integral part of reward function) ?

Can you please help clarify ?

Thanks,
Chinmay Swami

Incorrect state calculating?

In function Envirement::GetState, I assumes it gathers relative position and velocity data for human body. Without position of pelvis node (index node 0), in your case, you have 23 node in your model. There should be 22 nodes left recorded index [1, 22].

Here I supposed you excluded pelvis and counting number of node need to processed:

MASS/core/Environment.cpp

Line 294 in 8a18433

int num_body_nodes = skel->getNumBodyNodes() - 1;

But in real computing step, the code tries to iterate over 21 nodes in middle without very first and last node. (Should the upper bound be nodes.size() instead of nodes.size()-1 ?). And then you appended velocity of pelvis in the end?

MASS/core/Environment.cpp

Line 300 in 8a18433

for(int i = 1;i<num_body_nodes;i++)

As there's no obvious reason for excluding the last node, and not be mentioned anywhere in your project, is this behavior incorrect or intended?

Error during building

After following successfully all installation steps, I reach the building phase and I have an error during execution of make -j8 what it seems a c++ parser error:

[ 50%] Built target mss
[ 58%] Building CXX object render/CMakeFiles/render.dir/main.cpp.o
[ 66%] Building CXX object render/CMakeFiles/render.dir/Window.cpp.o
[ 91%] Built target pymss
In file included from /home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/main.cpp:1:0:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:16:1: error: expected class-name before ‘{’ token
 {
 ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:22:7: error: ‘void MASS::Window::draw()’ marked ‘override’, but does not override
  void draw() override;
       ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:23:7: error: ‘void MASS::Window::keyboard(unsigned char, int, int)’ marked ‘override’, but does not override
  void keyboard(unsigned char _key, int _x, int _y) override;
       ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:24:7: error: ‘void MASS::Window::displayTimer(int)’ marked ‘override’, but does not override
  void displayTimer(int _val) override;
       ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/main.cpp: In function ‘int main(int, char**)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/main.cpp:75:10: error: ‘class MASS::Window’ has no member named ‘initWindow’
  window->initWindow(1920,1080,"gui");
          ^
In file included from /home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:1:0:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:16:1: error: expected class-name before ‘{’ token
 {
 ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:22:7: error: ‘void MASS::Window::draw()’ marked ‘override’, but does not override
  void draw() override;
       ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:23:7: error: ‘void MASS::Window::keyboard(unsigned char, int, int)’ marked ‘override’, but does not override
  void keyboard(unsigned char _key, int _x, int _y) override;
       ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.h:24:7: error: ‘void MASS::Window::displayTimer(int)’ marked ‘override’, but does not override
  void displayTimer(int _val) override;
       ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In constructor ‘MASS::Window::Window(MASS::Environment*)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:17:2: error: ‘mBackground’ was not declared in this scope
  mBackground[0] = 1.0;
  ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:22:2: error: ‘mZoom’ was not declared in this scope
  mZoom = 0.25; 
  ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::keyboard(unsigned char, int, int)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:116:3: error: ‘Win3D’ has not been declared
   Win3D::keyboard(_key,_x,_y);break;
   ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::displayTimer(int)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:127:16: error: ‘mDisplayTimeout’ was not declared in this scope
  glutTimerFunc(mDisplayTimeout, refreshTimer, _val);
                ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:127:33: error: ‘refreshTimer’ was not declared in this scope
  glutTimerFunc(mDisplayTimeout, refreshTimer, _val);
                                 ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::SetFocusing()’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:170:3: error: ‘mTrans’ was not declared in this scope
   mTrans = -mEnv->GetWorld()->getSkeleton("Human")->getRootBodyNode()->getCOM();
   ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::DrawBodyNode(const dart::dynamics::BodyNode*)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:273:6: error: ‘mRI’ was not declared in this scope
  if(!mRI)
      ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:276:2: error: ‘mRI’ was not declared in this scope
  mRI->pushMatrix();
  ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::DrawShapeFrame(const dart::dynamics::ShapeFrame*)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:302:6: error: ‘mRI’ was not declared in this scope
  if(!mRI)
      ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:310:2: error: ‘mRI’ was not declared in this scope
  mRI->pushMatrix();
  ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::DrawShape(const dart::dynamics::Shape*, const Vector4d&)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:322:6: error: ‘mRI’ was not declared in this scope
  if(!mRI)
      ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:328:2: error: ‘mRI’ was not declared in this scope
  mRI->setPenColor(color);
  ^
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp: In member function ‘void MASS::Window::DrawMuscles(const std::vector<MASS::Muscle*>&)’:
/home/rudy/proyects/MASS-Muscle-Actuated-Skeletal-System/render/Window.cpp:381:3: error: ‘mRI’ was not declared in this scope
   mRI->setPenColor(color);
   ^
render/CMakeFiles/render.dir/build.make:62: recipe for target 'render/CMakeFiles/render.dir/main.cpp.o' failed
make[2]: *** [render/CMakeFiles/render.dir/main.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
render/CMakeFiles/render.dir/build.make:86: recipe for target 'render/CMakeFiles/render.dir/Window.cpp.o' failed
make[2]: *** [render/CMakeFiles/render.dir/Window.cpp.o] Error 1
CMakeFiles/Makefile2:140: recipe for target 'render/CMakeFiles/render.dir/all' failed
make[1]: *** [render/CMakeFiles/render.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 
2

---

ENVIRONMENT

$ make -v
GNU Make 4.1
Built for x86_64-pc-linux-gnu
$ hostnamectl
Operating System: Linux Mint 18.2
Kernel: Linux 4.8.0-53-generic
Architecture: x86-64
$ sudo pip3 --version
pip 19.1.1 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
$ virtualenv --version
15.2.0

Doesn't render trained networks when CUDA is not available

I think this may be helpful if someone can't use CUDA like me.

I have pytorch-cpu and torchvision-cpu. Everything was working until I tried to render the trained networks. Python gave an error: "Error in Python: <class 'AssertionError'>: Torch not compiled with CUDA enabled".

I fixed it by removing ".cuda()" in Window.cpp line 69:
p::eval("MuscleNN(num_total_muscle_related_dofs,num_actions,num_muscles).cuda()",mns);

now I have:
p::eval("MuscleNN(num_total_muscle_related_dofs,num_actions,num_muscles)",mns);

How should i get a muscle model and get it moving?

could i have Tutorials？

Limited insight given on implementation (trying to reproduce results on other tasks etc.).

• You mention that there is a possibility of training a torque-only actuated model. Could you please walk me through the process of doing this as well as pointing me to the necessary data for this?
• How where the .xml files (human and muscle) and .obj files created? Were they produced from physical/empirical measurements or did you infer them in some way?
• Are the offsets included in the .xml files person-specific or generic (i.e. did you use only a single specific human for them?)
• Could you also explain to me what is the purpose of the reward_param field in the metadata.txt file? Are these values fine-tuned in some way? Running the code for the "running" task while keeping the reward_param constant produced negative results.

who anyone run Mass on google colab?

Has anyone run Mass on Google Colab?

On a YouTube channel; two-minute papers, I came across a mass and looked so cool. I use colab because I don't have the computing power to run deep learning.

although I followed "readme", I met the error "no module named pymss" when I ran main.py; after build the DART6
Can anyone please tell me what's the problem in this case?

How to Train for a successful backflip？

My backflip only works as a jump

How to change the character's height, joint length, and so on?

Thank you very much for your code. May I ask if I want to modify some shape parameters of the model

pybind11 embed.h issue

Hi, I am using Ubuntu 18 and the default pybind11 installation does not have embed.h (https://packages.ubuntu.com/bionic/all/pybind11-dev/filelist) which caused compilation issues. Any suggestions or quick solutions?
Thank you!

Alex

How to convert the trained result back to bvh or other 3d format?

I have trained the walk.bvh, and also can run the trained data.
Now Only can see the result in a rendered viewport.
But how to convert the trained result back to bvh or other 3d formats?

where is the PPO.py file

Hi,
Thanks for sharing your excellent work!
I find the code is not complete.
Could you please provide the PPO.py file.

Regards,
Feng

What are your thoughts on licensing?

Would you mind including a LICENSE.md file?

Issues Installing

Hello! I followed the instructions as listed on the ReadMe but installation and set up fails on the step

cmake -j8

Is there any reason for this?

I am on Ubuntu 20.4 and have all the dependencies downloaded as described

Thank you for any and all help

Can't training using metadata. ValueError: stof

I didn't use a virtualenv, but all requirement wers installed. Cmake install. But when try to start a traning this error shows up

python3 main.py -d ../data/metadata.txt
HumanTraceback (most recent call last):
File "main.py", line 391, in
ppo = PPO(args.meta)
File "main.py", line 65, in init
self.env = EnvManager(meta_file,self.num_slaves)
ValueError: stof

How to produce custom motions using MotionBuilder?

Hi! Thanks for sharing the great work!

Could you share the .fbx file you used with us or tell us the right way to retarget custom motions to your musculoskeletal model using MotionBuilder?
Thanks!

Release the sargent jump & deadlift bvh and xmls?

Hi, thanks for this excellent codebase. I was able to reproduce the results and train a walking/kick policy on Ubuntu 20 with CUDA 11 and latest version of Torch. I was wondering if you could also release the Sargent Jump / Deadlift BVH files, along with the xml files for the additional objects? I'd like to try training those policies as well.

Why are positions from BVH required after learning ?

Hi,

After learning a walking gait for several days and observing the results the gait seems ok.
However I noticed that the BVH file is still needed to run the network after optimization.
I am a bit confused because in the paper there is a claim that says :

Note that PD targets serve as an intermediary between the two network policies in the learning process, but they are not used in actual simulations. Our simulation and control system at runtime is solely muscle-actuated requiring neither PD targets nor PD control at all.

When looking further I noticed that the call to GetActivationFromNN in the Window::Step() methods
makes use of the target positions. Indeed in this method there is a call to
Environment::GetDesiredTorques(). When looking at this method we can see that
the mTargetPositions is used and the output of the mimicking network mAction is added
to the desired positions provided by mTargetPositions.

The mTargetPositions is set in the Environment::SetAction method which looks like this :

void 
Environment::
SetAction(const Eigen::VectorXd& a)
{
	mAction = a*0.1;

	double t = mWorld->getTime();

	std::pair<Eigen::VectorXd,Eigen::VectorXd> pv = mCharacter->GetTargetPosAndVel(t,1.0/mControlHz);
	mTargetPositions = pv.first;
	mTargetVelocities = pv.second;

	mSimCount = 0;
	mRandomSampleIndex = rand()%(mSimulationHz/mControlHz);
	mAverageActivationLevels.setZero();
}

I noticed two things :

1. The mTargetPositions comes from the Character::GetTargetPosAndVel which
   explicitly use the BVH file.
2. the 0.1 factor is multiplying the output of the mimicking controller.

Now my question is, how do you proceed to get rid of this TargetPosition after learning ? Is there a missing part of the code that implements a sort of shrinkage of mTargetPositions in favor of the of output of the mimicking controller. Or did you use an other technique to achieve this ? Or did I mistunderstood some parts of the code ?

Thanks !

Pybind11/embed.h not found

Hello everyone

I have complete all steps before compiling. But when I run cmake .. I get the output :

`-- Found DART: /usr/include found components: collision-bullet dart
-- Found DART: /usr/include found components: gui collision-bullet dart
-- Found DART: /usr/include found components: collision-bullet dart
-- Configuring done
CMake Warning (dev) at render/CMakeLists.txt:20 (add_executable):
Policy CMP0028 is not set: Double colon in target name means ALIAS or
IMPORTED target. Run "cmake --help-policy CMP0028" for policy details.
Use the cmake_policy command to set the policy and suppress this warning.

Target "render" links to target "pybind11::embed" but the target was not
found. Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done

After that I run make -j8 and this is the error I get:

In file included from /home/mohammad/MASS/python/EnvManager.cpp:1:0:
/home/mohammad/MASS/python/EnvManager.h:4:10: fatal error: pybind11/embed.h: No such file or directory
#include <pybind11/embed.h>
^~~~~~~~~~~~~~~~~~
compilation terminated.
python/CMakeFiles/pymss.dir/build.make:62: recipe for target 'python/CMakeFiles/pymss.dir/EnvManager.cpp.o' failed
make[2]: *** [python/CMakeFiles/pymss.dir/EnvManager.cpp.o] Error 1
CMakeFiles/Makefile2:195: recipe for target 'python/CMakeFiles/pymss.dir/all' failed
make[1]: *** [python/CMakeFiles/pymss.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
In file included from /home/mohammad/MASS/render/Window.cpp:1:0:
/home/mohammad/MASS/render/Window.h:9:10: fatal error: pybind11/embed.h: No such file or directory
#include <pybind11/embed.h>
^~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/mohammad/MASS/render/main.cpp:1:0:
/home/mohammad/MASS/render/Window.h:9:10: fatal error: pybind11/embed.h: No such file or directory
#include <pybind11/embed.h>
^~~~~~~~~~~~~~~~~~
compilation terminated.
render/CMakeFiles/render.dir/build.make:62: recipe for target 'render/CMakeFiles/render.dir/Window.cpp.o' failed
make[2]: *** [render/CMakeFiles/render.dir/Window.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
render/CMakeFiles/render.dir/build.make:86: recipe for target 'render/CMakeFiles/render.dir/main.cpp.o' failed
make[2]: *** [render/CMakeFiles/render.dir/main.cpp.o] Error 1
CMakeFiles/Makefile2:140: recipe for target 'render/CMakeFiles/render.dir/all' failed
make[1]: *** [render/CMakeFiles/render.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

I have installed "pybind11[global]" in virtual env but did not help.

Any solution?

Could you describe the process of model creation?

Hi, Lee Seung Hwan!
Thanks for a cool paper and code! Could you please describe a process of model and muscles creation a bit more? E.g. how did you create human.xml and muscle284.xml files? Is there a GUI helper?
Thanks!

relocation R_X86_64_32S against `_Py_NotImplementedStruct` can not be used when making a shared object;

when running the make -j8 command, i'm getting following error messages:

XML reference on how human.xml was constructed

Hello! Is there any documentation on the XML reference for human.xml? I would like to see exactly what all the tags refer to since I am trying to replicate the model in my own way. Thank you for any help

How to reproduce Figure 6 (muscle activation)?

Hi there,
Thanks for sharing the code!

I've been working with your paper and I'd like to reconstruct Figure 6 in your paper.

After training, the model can walk and I somehow obtained 10 x 284 muscle activation for each step.

I tried to reproduce Figure 6 (muscle activation), but then I'm kinda lost.
There are 284 muscles, and I don't know how to use/choose them to have a plot like Figure 6.

Could you give me some tips?
Thanks!

How to connect the model to an exoskeleton?

Hi,
I want to know the method for using this environment for exoskeleton training with DRL. First how can I import exoskeleton to the environment and then how to connect it to human model?
Thanks

numpy version

Hi, thank you for providing the code.
I could "Run Training". But, I cannot "Run UI" or "Run Trained data" etc...
Here is the error message. Maybe this is a numpy version problem...
Could you tell me what version do you use?

./render/render ../data/metadata.txt
Human(DOFs : 56)
Ground(DOFs : 0)
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa
terminate called after throwing an instance of 'boost::python::error_already_set'

Thank you

What Python version

You specify boost version and dart version, but not python version.
I've tried 3.7m, 3.7.4, 3.6.9, 3.3.6, but no luck.
I found out that I had to install python --without-pymalloc. Boost is using _PyUnicode_AsString() which is removed in a version of 3.3, I think, but virtualenv is problematic in these early versions of python3.

Error compiling the renderer

Hello,

Thank you for sharing the project. Very impressed by your results.

Could you help us setting up the code ?

I followed the instruction

1. Installed boost 1.66
2. Installed python dependencies
3. Installed dart (the ubuntu package from here:https://launchpad.net/~dartsim/+archive/ubuntu/ppa, version 6.8.4

After cloning the code and running cmake ../, the compilation goes well until 91%

Here is the end of the output :

[ 50%] Built target mss
[ 58%] Building CXX object render/CMakeFiles/render.dir/Window.cpp.o
[ 83%] Built target pymss
[ 91%] Building CXX object render/CMakeFiles/render.dir/main.cpp.o
In file included from /home/efx/Development/PHD/AiriNew/MASS/render/main.cpp:1:0:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:16:1: error: expected class-name before ‘{’ token
 {
 ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:22:7: error: ‘void MASS::Window::draw()’ marked ‘override’, but does not override
  void draw() override;
       ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:23:7: error: ‘void MASS::Window::keyboard(unsigned char, int, int)’ marked ‘override’, but does not override
  void keyboard(unsigned char _key, int _x, int _y) override;
       ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:24:7: error: ‘void MASS::Window::displayTimer(int)’ marked ‘override’, but does not override
  void displayTimer(int _val) override;
       ^
/home/efx/Development/PHD/AiriNew/MASS/render/main.cpp: In function ‘int main(int, char**)’:
/home/efx/Development/PHD/AiriNew/MASS/render/main.cpp:45:10: error: ‘class MASS::Window’ has no member named ‘initWindow’
  window->initWindow(1920,1080,"gui");
          ^
In file included from /home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:1:0:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:16:1: error: expected class-name before ‘{’ token
 {
 ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:22:7: error: ‘void MASS::Window::draw()’ marked ‘override’, but does not override
  void draw() override;
       ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:23:7: error: ‘void MASS::Window::keyboard(unsigned char, int, int)’ marked ‘override’, but does not override
  void keyboard(unsigned char _key, int _x, int _y) override;
       ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.h:24:7: error: ‘void MASS::Window::displayTimer(int)’ marked ‘override’, but does not override
  void displayTimer(int _val) override;
       ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In constructor ‘MASS::Window::Window(MASS::Environment*)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:17:2: error: ‘mBackground’ was not declared in this scope
  mBackground[0] = 1.0;
  ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:22:2: error: ‘mZoom’ was not declared in this scope
  mZoom = 0.25; 
  ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::keyboard(unsigned char, int, int)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:116:3: error: ‘Win3D’ has not been declared
   Win3D::keyboard(_key,_x,_y);break;
   ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::displayTimer(int)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:127:16: error: ‘mDisplayTimeout’ was not declared in this scope
  glutTimerFunc(mDisplayTimeout, refreshTimer, _val);
                ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:127:33: error: ‘refreshTimer’ was not declared in this scope
  glutTimerFunc(mDisplayTimeout, refreshTimer, _val);
                                 ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::SetFocusing()’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:170:3: error: ‘mTrans’ was not declared in this scope
   mTrans = -mEnv->GetWorld()->getSkeleton("Human")->getRootBodyNode()->getCOM();
   ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::DrawBodyNode(const dart::dynamics::BodyNode*)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:273:6: error: ‘mRI’ was not declared in this scope
  if(!mRI)
      ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:276:2: error: ‘mRI’ was not declared in this scope
  mRI->pushMatrix();
  ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::DrawShapeFrame(const dart::dynamics::ShapeFrame*)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:302:6: error: ‘mRI’ was not declared in this scope
  if(!mRI)
      ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:310:2: error: ‘mRI’ was not declared in this scope
  mRI->pushMatrix();
  ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::DrawShape(const dart::dynamics::Shape*, const Vector4d&)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:322:6: error: ‘mRI’ was not declared in this scope
  if(!mRI)
      ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:328:2: error: ‘mRI’ was not declared in this scope
  mRI->setPenColor(color);
  ^
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp: In member function ‘void MASS::Window::DrawMuscles(const std::vector<MASS::Muscle*>&)’:
/home/efx/Development/PHD/AiriNew/MASS/render/Window.cpp:381:3: error: ‘mRI’ was not declared in this scope
   mRI->setPenColor(color);
   ^
^Crender/CMakeFiles/render.dir/build.make:75: recipe for target 'render/CMakeFiles/render.dir/main.cpp.o' failed
make[2]: *** [render/CMakeFiles/render.dir/main.cpp.o] Interrupt
render/CMakeFiles/render.dir/build.make:62: recipe for target 'render/CMakeFiles/render.dir/Window.cpp.o' failed
make[2]: *** [render/CMakeFiles/render.dir/Window.cpp.o] Interrupt
CMakeFiles/Makefile2:145: recipe for target 'render/CMakeFiles/render.dir/all' failed
make[1]: *** [render/CMakeFiles/render.dir/all] Interrupt
Makefile:83: recipe for target 'all' failed
make: *** [all] Interrupt

Many thanks in advance for the help !

Failed to run trained data

Thank you for your perfect work. When I try to run trained data using the command:'' ./render/render ../data/metadata.txt ../nn/max.pt ../nn/max_muscle.pt'', I met the error:"-bash: ./render/render: No such file or directory".

Do you know how to deal with it？

What's more, do you know how to visualize the results on Unbuntu?

Thank you very much.

how to train a torque-only actuated model?

Please tell me how to train a torque-only actuated model?or which file needs to be modified?

I get an error when I change use muscle to false in Metadata.txt

thanks very much.

Are you planning to release the 346 muscle model?

I was wondering if you would include the 346 muscle model to the repo. I am assuming this model is the one mentioned here Multi-Segment Foot Modeling for Human Animation. I was not able simulate a stable walk with the 284 muscle model.

Unable to view trained data

I am trying to visualize the trained data, but the renderer fails with boost::python::error_already_set().
From debugging I have determined the error occurs on execution of line 32 of window.cpp.
At some point of the codes execution,
boost::python::api::object_base::object_base (this=0x7fffffffe180, rhs=...)
in boost/python/object_core.hpp throws the error_already_set

I built mass in the conda environment of python 3.6.10 and am using boost 1.71. I built dart 6.3 with this boost version.

Missing Deadlift Experiment Data

Hi,

Thank you for your excellent work on this project.

I noticed that the deadlift experiment mentioned in the paper does not have its data in the data folder. Could you please make this data available, or let me know where I can find it?

Unable to execute cmake .. to install/compile MASS

CMake Error at /usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find DART (missing: collision-bullet)
Call Stack (most recent call first):
/usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/dart/cmake/DARTConfig.cmake:174 (find_package_handle_standard_args)
core/CMakeLists.txt:9 (find_package)

-- Configuring incomplete, errors occurred!

I tried reinstall DART with all forms, collision-bullet was install