bellspringsteen / ourcamera

https://www.nytimes.com/2018/03/15/nyregion/bike-lane-blocked-new-york.html

OurCamera

Read About Project Here

This project uses Google's TensorFlow Machine learning package to identify and categorize types of vehicles on NYC streets.

The data images are downloaded from NYC DOT cameras.

The training data folder has a series of images and training annotations.

Setup

• Install Tensorflow from https://www.tensorflow.org/install/
• Download the github repo for tensorflow models and place it in the top level folder
• Follow the Object Detection Installation instructions here

Run

Create a test.record file and a train.record file:

./generate_tfrecord --folder=path/to/data_dir --train_ratio=.70

Download COCO-pretrained Faster R-CNN with Resnet-101 model Unzip model in the data/models/ folder

Your data structure should look like

data/test.record data/train.record models/model/faster_rcnn_resnet101_coco_11_06_2017 models/model/train models/model/eval

In three seperate command line windows run:

export PYTHONPATH=$PYTHONPATH:pwd:pwd/slim

python models-master/research/object_detection/train.py     --logtostderr    
    --pipeline_config_path=./data/faster_rcnn_resnet101_cars.config  
    --train_dir=./data/models/model/train

python models-master/research/object_detection/eval.py     --logtostderr    
    --pipeline_config_path=./data/faster_rcnn_resnet101_cars.config    
    --checkpoint_dir=./data/models/model/train     
    --eval_dir=./data/models/model/eval

tensorboard --logdir=./data/models/model/

After a thousand or so steps you should be getting results. Look at your tensorflow eval images to gauge when to stop. FYI it took my 4770k Intel i7 about 24 hours to train.

Create a frozen version of your graph by selecting a checkpoint:

python object_detection/export_inference_graph.py \
    --input_type image_tensor \
    --pipeline_config_path /path/data/models/faster_rcnn_resnet101_cars.config \
    --trained_checkpoint_prefix /path/data/models/model/train/model.ckpt-18557 \
    --output_directory output_inference_graph.pb

Run saveimages.py to create a folder of images. This will save an image every second so leave it open as long as you want.

Analyze the images

analyzeimages \
        -path_images ./data/rawimages/ 
        -path_labels_map data/car_label_map.pbtxt 
        -save_directory data/processedimages/

Gotchas:

• When you start up an Amazon EC2 Instance using the AWS Deep Learning AMI you have to enable tensorflow

ourcamera's Issues

"raise ValueError('First step cannot be zero.')" in the training stage

In the training stage with command:
python models-master/research/object_detection/train.py --logtostderr
--pipeline_config_path=./data/faster_rcnn_resnet101_cars.config
--train_dir=./data/models/model/train
shows the error message: "raise ValueError('First step cannot be zero.')".

eval.py and tensorboard --logdir=./data/models/model/ works well.

I confirmed all the paths in generate_tfrecord.py and faster_rcnn_resnet101_cars.config are correct, and test.record and train.record was generated successfully(I don't know the content are correct, if there is a method to check them please let me know). Would you please let me know the possible situation may cause the problem? Thank you.

Colons in /data/labels XML files prevents them from being checked out on Windows and Mac

The Windows and Mac file systems don't allow colons in filenames, which means that checking out the project generates multiple errors like the following:

error: unable to create file data/labels/2017-12-27 12:14:19.731720.xml: Invalid argument

No bounding boxes

I trained for 1400 steps and yet I am unable to see the bonding boxes on eval in tensor board, can you help me with this ?

Some questions

Hi,

It's an awesome project thanks for this, I tried to reproduce it and do the same in my city, but the manual is not that detailed and sometimes it is confusing. In the last few days I tried to make it work but I couldn't based on the read.me so I would have a few questions.

I am able to do the first three steps, and the test is fine:

python object_detection/builders/model_builder_test.py
.............
----------------------------------------------------------------------
Ran 13 tests in 0.066s

OK

Then we have to run the following:

./generate_tfrecord --folder=path/to/data_dir --train_ratio=.70

I suppose the path/to/data_dir int this example is OurCamera/data/ folder.
It gives an error, I am still trying to figure out why, maybe I am doing something wrong but it looks like it gets str type instead of bytes:

(data) root@mlearning:~/data# python ./generate_tfrecord.py -folder=/root/data/data/ -train_ratio=.70
Running on /root/data/data/ with ratio 0.7
Traceback (most recent call last):
  File "./generate_tfrecord.py", line 132, in <module>
    tf.app.run()
  File "/root/data/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "./generate_tfrecord.py", line 124, in main
    tf_example = create_tf_example_from_jpg_xml(images_path+filename+'.jpg', labels_path+filename+'.xml')
  File "./generate_tfrecord.py", line 82, in create_tf_example_from_jpg_xml
    'image/filename': dataset_util.bytes_feature(jpg_file),
  File "/root/data/models/research/object_detection/utils/dataset_util.py", line 30, in bytes_feature
    return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
TypeError: '/root/data/data/images/2017-12-22 19:55:19.816746.jpg' has type str, but expected one of: bytes

You write unzip Unzip model in the data/models/ folder , in that case the the data structure should look like this:

data/test.record data/train.record data/models/model/faster_rcnn_resnet101_coco_11_06_2017 data/models/model/train data/models/model/eval

Am I right?

I am still trying to make it work, but if you have any comments I would appreciate it..

question about training

What does this mean, from the README?

Look at your tensorflow eval images to gauge when to stop.

Where do i find tensorflow eval images?

Solving traffic with AI

Hey I have been thinking of making a similar project which can solve traffic issue throughout the city.

Undefined name 'csv_file' in analyzeimages.py

flake8 testing of https://github.com/Bellspringsteen/OurCamera on Python 3.6.3

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./analyzeimages.py:320:20: F821 undefined name 'csv_file'
    analyzeresults(csv_file)                   ^

1     F821 undefined name 'csv_file'

Hello, sincerely thank you for sharing, I have the following error when running generate_tfrecord.py, can you help me solve this problem?

Traceback (most recent call last): File "/media/junhao/SSD2/Download/OurCamera-master/OurCamera-master/generate_tfrecord.py", line 132, in <module> tf.app.run() File "/home/junhao/.local/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "/media/junhao/SSD2/Download/OurCamera-master/OurCamera-master/generate_tfrecord.py", line 124, in main tf_example = create_tf_example_from_jpg_xml(images_path+filename+'.jpg', labels_path+filename+'.xml') File "/media/junhao/SSD2/Download/OurCamera-master/OurCamera-master/generate_tfrecord.py", line 82, in create_tf_example_from_jpg_xml 'image/filename': dataset_util.bytes_feature(jpg_file), File "./models-master/research/object_detection/utils/dataset_util.py", line 30, in bytes_feature return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value])) TypeError: './666/images/2017-12-15 12_59_56.052543.jpg' has type str, but expected one of: bytes

Can't use pre-trained data frozen_inference_graph.pbsegmentaa, ab, ac and ad

Hi,

Thank you for your help on the previous issue. I successfully go through all the steps, but the output images don't contain any identified objects. To figure out the problem I'm trying to use your pre-trained data but analyzeimages.py can't read them because it's separated as frozen_inference_graph.pbsegmentaa, frozen_inference_graph.pbsegmentab, frozen_inference_graph.pbsegmentac and frozen_inference_graph.pbsegmentad. Would you please let me know how to use them with analyzeimages.py?

Edit: I just found the detection scores are pretty low with my training data which has more than 20,000 steps. So it looks like my training has some problem. I used the sample images in this git. Any comments about the problem?

Thank you.

Problems when running on MacOS

I have a couple of issues when running on OSX.

1. I cannot run "generate_tfrecord" like this ./generate_tfrecord I must first change it chmod +x and then run it like this ./generate_tfrecord.py (including =.py at the end).
2. When running the command I get a strange error, both activating and not activating my tensorflow environment (source ./bin/activate)

./generate_tfrecord.py --folder=data/ --train_ratio=.70

(tensorflow3) NicoPro:OurCamera nriesco$ ./generate_tfrecord.py --folder=data/ --train_ratio=.70
./generate_tfrecord.py: line 14: Processing Image and Annotation Dataset

This executable is used to convert the image and label dataset to test and train TF Records:

A folder should be specified which contains folders labeled images/ and labels/
Each jpg image file should contain a corresponding label file with the same name but with .xml extension
A ratio of test to training should be specified i.e. .70

Example usage:
    ./generate_tfrecord         --folder=path/to/data_dir         --train_ratio=.70

: No such file or directory
from: can't read /var/mail/__future__
from: can't read /var/mail/PIL
./generate_tfrecord.py: line 20: import: command not found
./generate_tfrecord.py: line 21: import: command not found
from: can't read /var/mail/argparse
./generate_tfrecord.py: line 23: import: command not found
./generate_tfrecord.py: line 24: import: command not found
./generate_tfrecord.py: line 25: import: command not found
./generate_tfrecord.py: line 26: import: command not found
./generate_tfrecord.py: line 27: import: command not found
./generate_tfrecord.py: line 28: syntax error near unexpected token `'./models-master/research/''
./generate_tfrecord.py: line 28: `sys.path.append('./models-master/research/')'

It does not matter if I change the path to an absoulte one, add a trailing slash, etc.