skylark0924 / gamma_reward

Hi there 👋

• ✨ My main research interests are 5Rs: Reinforcement learning, Reasoning, and Representation learning used in Robotics, aiming at Real intelligence.

• 🔭 You can also follow my Zhihu blog: Skylark - Zhihu

• 📫 Reach me through [email protected]

gamma_reward's Issues

There is no YELLOW_TIME

In the cityflow env, there is no evidence considering YELLOW_TIME between the transition.

`def step(self, action):
"""
Calculate the state in time
"""
step_start_time = time.time()
for i in range(self.min_action_time):
self._inner_step(action)

    state = self.get_state()
    reward = self.get_raw_reward()

    # 判断是否已经出现拥堵
    self.congestion = self.compute_congestion()
    self.done = {id_: False for id_ in self.intersection_id}
    self.done['__all__'] = False
    # if self.count >= self.num_step:
    #     self.done = {id_: True for id_ in self.intersection_id}
    #     self.done['__all__'] = True
    # if self.count == 3600:
    #     self.reset()
    return state, reward, self.done, {}

def _inner_step(self, action):
    self.update_previous_measurements()

    for id_, a in action.items():  # intersection_id, corresponding action
        if self.current_phase[id_] == self.phase_list[id_][a]:
            self.current_phase_time[id_] += 1
        else:
            self.current_phase[id_] = self.phase_list[id_][a]
            self.current_phase_time[id_] = 1
        self.eng.set_tl_phase(id_, self.current_phase[id_])  # set phase of traffic light

    self.eng.next_step()
    self.count += 1

    # print(self.count)

    self.system_states = {"get_lane_vehicles": self.eng.get_lane_vehicles(),
                          "get_lane_waiting_vehicle_count": self.eng.get_lane_waiting_vehicle_count(),
                          "get_vehicle_speed": None,
                          "get_vehicle_distance": None
                          }
    for id_ in self.intersection_id:
        self.update_current_measurements_map(id_, self.system_states)`