naleraphael / cartpole-control

An OpenAI-CartPole solver with DQN and control theory

CartPole-control

An OpenAI-CartPole solver implemented with DQN and control theory.

Environment

• Python: 3.5.2
• tensorflow: 1.10.0
• keras: 2.2.2
• openai-gym: 0.10.5

Usage

$ python main.py [--solver SOLVER_TYPE]

• SOLVER_TYPE: basic(default), dqn, simple_control, pid_control

Introduction

• env: Environment of problems to be solved.
• model: Model of agent to behave (and learn).
• agent: A player interact with environment.
• solver: A problem solver controlling the interaction between agent and environment. (3 stages: pretraining, training, solving)

Implemented agents

(more agents are going to be added)

• Basic agent: Actions in all stage are generated by env.action_space.sample().
• DQN agent: Actions are determined by deep Q network.
• Simple control agent: It cares about the angle of pole only. When pole is falling to right (angle > 0), it push the cart to right. Otherwise, it push the cart to left.
• PID control agent: It continuously generate actions to minimized the error between measured angle and desired setpoint. The output of PID controller is based on propotional, integral, derivative terms of the error of angle.
  • P: propotional controller, its output relates to angle (unit: degree).
  • I: integral controller, its output relates to cumulated angle (unit: degree*sec).
  • D: derivative controller, its output relates to variation of angle (unit: degree/sec).

Reference

• Udacity | Q learning cart
• Github | hope-yao/cartpole
• Github | ivPID

cartpole-control's Issues

Errors of patch

I am trying to run your codes on macOS BigSur 11.1 with PyChram 2020.3. While running main.py I'm getting the following errors. Would you please help me to resolve the errors? Which versions of the gym, TensorFlow python, etc. I will need? Thanks.

Failed to patch CartPoleEnv.__init__Traceback (most recent call last):
File "/Users/xxxx/Documents/Python_Deepak/cartpole-control-master/main.py", line 6, in
from cpc.env import CartPoleEnv, AdvancedCartPoleEnv
File "/Users/xxxx/Documents/Python_Deepak/cartpole-control-master/cpc/init.py", line 94, in
CartPoleEnv.init = dirty_patch(CartPoleEnv.init)
File "/Users/xxxx/Documents/Python_Deepak/cartpole-control-master/cpc/init.py", line 52, in dirty_patch
idx_call = payload.index(ins)
ValueError: subsection not found

Learning PID controller

Hey great work! I am trying to extend your work and get a learning agent to auto-tune the PID parameters, and I was just wondering how you would personally go about doing this??

Thanks!