Traffic Light Detection and Classification with TensorFlow Object Detection API

A brief introduction to the project is available here

AWS AMI with all the software dependencies like TensorFlow and Anaconda (in the community AMIs) - udacity-carnd-advanced-deep-learning

Get the dataset

Drive location

Get the models

Do git clone https://github.com/tensorflow/models.git inside the tensorflow directory

Follow the instructions at this page for installing some simple dependencies.

All the files have to be kept inside the tensorflow/models/research/ directory - data/, config/, data_conversion python files, .record files and utilitites/ ,etc.

Location of pre-trained models:

pre-trained models zoo

Download the required model tar.gz files and untar them into /tensorflow/models/research/ directory with tar -xvzf name_of_tar_file.

Creating TFRecord files:

python data_conversion_udacity_sim.py --output_path sim_data.record

python data_conversion_udacity_real.py --output_path real_data.record

Commands for training the models and saving the weights for inference.

Using Faster-RCNN model

For Simulator Data

Training

python object_detection/train.py --pipeline_config_path=config/faster_rcnn-traffic-udacity_sim.config --train_dir=data/sim_training_data/sim_data_capture

Saving for Inference

python object_detection/export_inference_graph.py --pipeline_config_path=config/faster_rcnn-traffic-udacity_sim.config --trained_checkpoint_prefix=data/sim_training_data/sim_data_capture/model.ckpt-5000 --output_directory=frozen_sim/

For Real Data

Training

python object_detection/train.py --pipeline_config_path=config/faster_rcnn-traffic_udacity_real.config --train_dir=data/real_training_data

Saving for Inference

python object_detection/export_inference_graph.py --pipeline_config_path=config/faster_rcnn-traffic_udacity_real.config --trained_checkpoint_prefix=data/real_training_data/model.ckpt-10000 --output_directory=frozen_real/

Using Inception SSD v2

For Simulator Data

Training

python object_detection/train.py --pipeline_config_path=config/ssd_inception-traffic-udacity_sim.config --train_dir=data/sim_training_data/sim_data_capture

Saving for Inference

python object_detection/export_inference_graph.py --pipeline_config_path=config/ssd_inception-traffic-udacity_sim.config --trained_checkpoint_prefix=data/sim_training_data/sim_data_capture/model.ckpt-5000 --output_directory=frozen_models/frozen_sim_inception/

For Real Data

Training

python object_detection/train.py --pipeline_config_path=config/ssd_inception-traffic_udacity_real.config --train_dir=data/real_training_data

Saving for Inference

python object_detection/export_inference_graph.py --pipeline_config_path=config/ssd_inception-traffic_udacity_real.config --trained_checkpoint_prefix=data/real_training_data/model.ckpt-10000 --output_directory=frozen_models/frozen_real_inception/

Using MobileNet SSD v1

(Due to some unknown reasons the model gets trained but does not save for inference. Ignoring this for now.)

For Simulator Data

Training

python object_detection/train.py --pipeline_config_path=config/ssd_mobilenet-traffic-udacity_sim.config --train_dir=data/sim_training_data/sim_data_capture

Saving for Inference

python object_detection/export_inference_graph.py --pipeline_config_path=config/ssd_mobilenet-traffic-udacity_sim.config --trained_checkpoint_prefix=data/sim_training_data/sim_data_capture/model.ckpt-5000 --output_directory=frozen_models/frozen_sim_mobile/

For Real Data

Training

python object_detection/train.py --pipeline_config_path=config/ssd_mobilenet-traffic_udacity_real.config --train_dir=data/real_training_data

Saving for Inference

python object_detection/export_inference_graph.py --pipeline_config_path=config/ssd_mobilenet-traffic_udacity_real.config --trained_checkpoint_prefix=data/real_training_data/model.ckpt-10000 --output_directory=frozen_models/frozen_real_mobile/

Inference results can be viewed using the TrafficLightDetection-Inference.ipynb or .html files.