medicjue / wm811_automl

============== (Julius) Based on the dataset to benchmark AutoML

This repo provide a wafer-inseption pototype include all tensorflow profiler tools.(tensorboard, timeline)
PS. Loging hardware message by timline cause huge overhead, pls turn it off if you don't need it.
You can also use NVprofiler(.nvvp) to moniter your model performance without overhead.

You can easiely import an other neural network to your application by modify model_zoo. The coding style is friendly for beginer.

run under: python 2.7 in tensorflow1.4 cudnn6.1 cuda8.0

Get start

Few folder for log info. you need to build.

mkdir -p logs/train/meta
mkdir -p logs/train/tfbord
mkdir -p logs/train/timeline
mkdir -p logs/test/tfbord
mkdir -p logs/test/timeline

Execute pkl2tfrecord.py in dataset to build dataset, or use the sub dataset We provide in this repo
Execute model_train.py to start training. But make sure your dataset is place on the right TRRECORD_PATH
Execute model_restore_wafer.py can get the trained model to do the final test.

option

if you need full dataset:
You can download dataset here: https://www.kaggle.com/qingyi/wm811k-wafer-map
In the link, is a .pkl file. We provide a pkl2tfrecord.py script in dataset After turn dataset into tfrecord, you can start your training.

profiler

tensorflow timeline

use chrome forchrome://tracing, load file logs/test/timeline

tensorboard

use cmd in logs/test/tfbord
tensorboard --logdir=./
use browser goto localhost:6006

nvprof (nvidia virtual profiler)

nvprof [-f] -o ./logs/nvvp/name.nvvp python model_train.py

Current result

#accuracy up to 97%

`

  70000 training data 10w testing data
       resnet 50 epec
               train accuracy:  0.9894
               test accuracy:   0.948283950617
        wnet 82 epec
               train accuracy:  0.9977
               test accuracy:   0.973216049383

      inception_v4
              train accuracy:  0.9937
              test accuracy:   0.975475308642

`