airicbear / cis-4496-project Goto Github PK

It takes a long time to download the models on the website and the ONNX models take an additional amount of time (rough estimate ~10s) for inferencing. Not only do the ONNX models take more time to inference, but they freeze the entire website when running the inference.

To reproduce this issue, simply upload any image to either model on the website.

Initial ideas:

• Add progress indicator for model downloading and inferencing
• Cache models on client side

The ONNX and TensorFlow.js models are being reloaded each time the ImageInput components are rendered. For example, when you toggle the dark/light theme, the ImageInput component is refreshed which forces the models to be loaded again. We may want to consider decoupling the models from the components.

Following these references:

• tfjs iris example
• Image recognition with TensorFlow.js

I attempted to load model via tf.loadLayersModel but ran into the following error:

Error: Unknown layer: Addons>InstanceNormalization. This may be due to one of the following reasons:
1. The layer is defined in Python, in which case it needs to be ported to TensorFlow.js or your JavaScript code.
2. The custom layer is defined in JavaScript, but is not registered properly with tf.serialization.registerClass().

Some references suggested using graph model instead:

• tensorflow/tfjs#3728
• tensorflow/tfjs#3410

References for saving and loading model:

• https://www.tensorflow.org/js/tutorials/conversion/import_saved_model
• https://www.tensorflow.org/tutorials/keras/save_and_load

The background image for the ImageInput component is not properly sized for small screens. It shows a crop of the image, but I want it to show the entire image.