onewarmheart / intelligentcontrol

Intelligent control algorithm and simulation environment.

IntelligentControl

Intelligent control algorithm and simulation environments.(TO DO)

To see the articles, follow us on zhihu.com, here is the special column.

Algorithm:

1.Reinforcement learning
 (1)DDPG
 (2)DDPG+HER
 (3)PPO
 (4)SAC

2.Model predictive control
 (1)Daynamic matrix control
 (2)General Predictive Control

3.Iterative learning control

Environment:

1.Linear system

2.Nonlinear system

intelligentcontrol's Issues

I think there are some small bugs in your code.

In train.py, your code is as follows:

PART II hindsight replay

for i, transition in enumerate(episode_cache):
    new_goals = generate_goals(i, episode_cache, args.HER_sample_num)
    for new_goal in new_goals:
        reward = calcu_reward(new_goal, state, action) 
        state, action, new_state = gene_new_sas(new_goal, transition)
        ram.add(state, action, reward, new_state)

But I think it should be like that:

PART II hindsight replay

for i, transition in enumerate(episode_cache):
    new_goals = generate_goals(i, episode_cache, args.HER_sample_num)
    for new_goal in new_goals:
        state = transition[0]
        action = transition[1]
        reward = calcu_reward(new_goal, state, action) 
        state, action, new_state = gene_new_sas(new_goal, transition) # 一个transition被换成了各种goals
        ram.add(state, action, reward, new_state)

Otherwise, this algorithm is not convergent. I have tried to train it.