kexin-wei / soft-tissue-deformation-in-laparoscopic-surgery-planning-with-drl

Soft Tissue Simulation for laparoscopic surgery based CoppeliaSim with basic task managed by DRL agent (spinningup algorithm)

Soft tissue deformation in laparoscopic surgery planning with DRL

Using Deep Reinforcement Learning to plan a laparoscopic surgical tool (5 Dofs) to reach certain point behind the soft tissue.

• DRL algorithm from SpinningUp
• Soft tissue deformation: from paper ( Cotin, Stéphane, Hervé Delingette, and Nicholas Ayache. "A hybrid elastic model allowing real-time cutting, deformations and force-feedback for surgery training and simulation." Visual Computer 16.8 (2000): 437-452.)
• Simulation platform: CoppeliaSim

Required

• Spinningup: Installation Guide
• PyRep: Installation Guide
  • CoppeliaSim: download the lastest version from official website
  • if CoppeliaSim crash after install ubuntu package, show error #98

    ImportError: libcoppeliaSim.so.1: cannot open shared object file: No such file or directory
    

    may consider use python 3.7, not 3.6 as spinningup required.

Reward shaping:

• test t8 on test-i7(ddpg) and drl-ubuntu(ppo)
  • ddpg(test-i7):
    • s0: not learn yet, best dist 0.03-0.04, give up at 300 ep
    • s1: learning from 20 ep, best dist 0.03-0.04
    • s2:
  • ppo(drl-ubuntu)
    • s0:learning from 0 ep, best dist 0.03
    • s1:learning from 0 ep, best diet 0.06->0.02
• test t9 on test-amd
  • Wait until 200 epochs
  • ddpg: 3000/50
    • s0: not learn yet，need to stop, no progress at all, give up at 300 ep
  • ddpg: 3000/80
    • s1:
  • ppo
    • s0: learning from 0 ep
• test new ddpg parameter on test i7
  • Wait test t8 s2 finished at least 200 epochs
  • 3000/80, pi_lr = 3e-4