My SqueezeNext implementation is not any better than my Tiny Darknet implementation about squeezenext

[amirgholami]

Hi Fred,

Thanks for reaching out. Without analyzing the particular dataset that you are using it is hard to give useful feedback. However, we have tested SqueezeNext on a range of tasks including style transfer and face detection. Note that for each application you will need to modify the neural network

[Fred-Erik]

Thank you so much for your response!

The data is IR and RGB cutouts of license plates. I sadly cannot give you the dataset because it is not legally allowed to share license plates due to privacy laws, but here is an example image of my own license plate. The dataset is about 500.000 images and augmented in all kinds of ways to capture different angles and lighting conditions, so I would say this should allow me to train such a deep model.

However, we have tested SqueezeNext on a range of tasks including style transfer and face detection. Note that for each application you will need to modify the neural network

And what kind of modifications would be needed? I would suspect that because there is not one object like in ImageNet but multiple objects that have to be detected in both OCR and face detection, I would need more channels in the the beginning than in the ImageNet-optimized 1.0-SqNxt-23v5. Are those your findings as well, and/or would you be able to share the structure for the face detection network?

This results in about the same performance (in terms of accuracy and speed) on my target platform, an ARMv7 mobile chip

The performance in the paper is calculated on a simulated accelerator, have you by any chance also done tests on real-life low power hardware like an ARM processor?

[mrgloom]

I can be because Tiny Darknet just have less layers, it seems not only MACCs or FLOPS are matter but also memory access:

The number of computations — whether you count them as MACCs or FLOPS — is only part of the story. Memory bandwidth is the other part, and most of the time is even more important!

On current computer architectures, a single memory access from main memory is much slower than a single computation — by a factor of about 100 or more!

You just saw that these neural networks do a lot of computations, but how many memory accesses do they perform?

For each layer, the device needs to:

read the input vector or feature map from main memory
compute the dot products — which involves reading the layer’s weights from main memory too
write the result back to main memory as a new vector or feature map.
This involves a lot of memory accesses. Since memory is pretty slow, the amount of memory read/writes performed by the layer will have a big impact on its speed too — bigger perhaps than the number of computations.

https://machinethink.net/blog/how-fast-is-my-model/

btw CNN+RNN+Attention model to do OCR any reference to this kind of models?