Using the cares rl NetworkFactory will do this. This can be bundled in together with #121

There is unnecessary files that have since become outdated. Remove these

Desired Behaviour

Implements the OpenAI environment methods step and reset.
Responsible for termination conditions, reward functions, and state listening

Desired Behavior

Write a training loop for the CarGoal and CarWall Environments

This is dependent on #1 #2 #3 #5 completing

Since the cares learning package is not inside the repo then when it is training on the servers it cannot find it and so crashs. When it is git submodule added it also needs to be added to the dockerfile so that it can be used and included

Ideally we want every class to have a single parent class.

Potential implementation plan is to have the parent responsible for handling all the set up for subscribing/publishing to the relevant topics.

The class would only house functions that are relevent to all environments – the topic sub/pub – the fetching of data. But environment specific information – like the number of observations, get observations eg. should be in the child class

Current Behaviour

The goal is hardcoded to spawn within a 10x10 grid.

Desired Behaviour

The user should be able to define the grid when instantiating the CarGoalEnvironment class.

Desired Behavior

Create an environment, for the CarWall Reinforcement Learning Environment
Client-side environment is reference to the class that the training loop interacts with. Implements methods like reset and step

Environment Details

Here, we define the details of the reinforcement learning environment.

Map

A 10x10 area surrounded with a wall.

Task

Agent drives f1tenth robot to goal position

Observation

F1tenth's position (x, y), orientation (w, x, y, z), goal's position (x, y), & lidar points (approx. 640)

Action

F1tenth's linear and angular velocities

Reward

Not set in stone
F1tenth's progress toward or away from the goal
If it reaches the goal, +100
If it crashes into a wall -100

Termination Condition

Reaching the goal, and crashing into the wall

Truncation Condition

If the number of steps exceeds the max step parameter (this should be dynamic)

Current Behavior

Calling the set_pose service (and presumably any service), inside the reset_service callback crashes the simulation

Expected Behvaior

Not crash

Current Work Around

Creating the set_pose client directly inside the reset node – which may be a better alternative design wise

The older environments, CarGoal, CarWall etc. don't have dynamic car names that can be changed via launch file. Include this functionality

The number of cars is something that is environment specific. So it should be the environment's launch file that is responsible for spawning the necessary amount of cars

Desired Behaviour

Currently, the training loop spawns the goal. The reset node, only handles the moving of entities that are already spawned. It is necessary to have the environments package handle the spawning of the necessary models in the environment at launch time (and then have the reset node only worry about the repositioning of the different elements)

Current Behaviour

The goal can spawn anywhere within the 10x10 grid. This means it can spawn really close to the car, and trigger a massive reward even though the agent did not take any meaningful actions.

Desired Behaviour

Spawn the goal randomly but within bounds that mean that the car must take meaningful actions in order to reach the goal. This problem exists for both the CarGoal and CarWall environments.

Desired Behavior

Implement a reset service for the car goal environment.
On reset:

• Set the position of the car back to 0, 0
• Set the position of the goal to be somewhere random

Desired Behavior

Reset service for the CarWall environment. When called, the car should relocate to 0, 0 and the goal a random new position.

Desired Behaviour

Create an environment, for the CarBlock Reinforcement Learning Environment
Both the Client-side and the Backend

Client-side environment is reference to the class that the training loop interacts with. Implements methods like reset and step

Backend is reference to the back end processes that need to be spun up in order to simulate the environment: launch files and CarBlockReset service

Environment Details

Here, we define the details of the reinforcement learning environment.

Map

A 10x10 area with obstacles – can be static or dynamic (additional feature)

Task

Agent drives f1tenth robot to goal position

Observation

Its position (x, y), orientation (w, x, y, z), lidar points (approx. 640), & goal's position (x, y)

Action

Its linear and angular velocities

Reward

Not set in stone
Its progress toward or away from the goal
If it reaches the goal, +100
If it crashes into an obstacle -50

Termination Condition

Reaching the goal, and crashing into the wall

Truncation Condition

If the number of steps exceeds the max step parameter (this should be dynamic)

The only difference between the different CarTrack environments (client side) is the all_goals and car_reset_positions. This could easily be encompassed into one class, and have an argument to the class decide which one it is.

There is a delay between the publishing of the command to the cmd_vel topic and the execution of the command on the actual car itself

Desired Behavior

Currently, in the launch files, the f1tenth robot is hardcoded with the name f1tenth. Remove the hardcoded-ness. When a user includes the f1tenth_simulation.launch.py they should pass as args the name they want to name the car.