dcsil / athology-ml Goto Github PK

Setup endpoint to accept x,y,z accelerometer data

Given an athletes ID returns, the corresponding MongoDB table.

get_athelete_by_id (GET)
get_all_atheletes (GET)
create_new_athelete (POST)
add_athelete_sessions (POST)

TensorFlows data loading is a little complicated. There are a lot of little, rather arcane tips and tricks for optimizing it. I don't expect this to be a big deal for our use case because we train on a single accelerator (GPU/TPU), but it's good practice and will allow our code to scale to multiple accelerators in the future, where data loading can become a bottleneck.

Thankfully TensorFlow has collected these tips/tricks in a single document, here, so it is just a matter of going through that and making the changes in our code.

The heroku app is crashing because TensorFlow is consuming more than 200% (or 1024mb) of the memory available in the free heroku dyno:

2021-03-20T23:23:21.739571+00:00 heroku[web.1]: Process running mem=1025M(200.3%)

2021-03-20T23:23:21.745592+00:00 heroku[web.1]: Error R15 (Memory quota vastly exceeded)

2021-03-20T23:23:21.750354+00:00 heroku[web.1]: Stopping process with SIGKILL

2021-03-20T23:23:22.120420+00:00 heroku[web.1]: Process exited with status 137

2021-03-20T23:23:22.184859+00:00 heroku[web.1]: State changed from up to crashed

I think the solution is two parts:

1: Train a smaller ML model
2. Limit the Docker to 512mb of RAM.

For 2., I know I can do this locally by passing -m "512mb" to docker run, and I even confirmed it works, but I have no idea how to get Heroku to call this.

Another option is to try and significantly shrink the model with techniques like pruning and quantization. See here: https://colab.research.google.com/github/tensorflow/model-optimization/blob/master/tensorflow_model_optimization/g3doc/guide/pruning/pruning_with_keras.ipynb

Congratulations on finishing the course!

To make sure we have enough GitHub Actions minutes for the next term, we have automatically disabled your repository's actions.

Feel free to make this repo public or fork to your own org, but we do ask that you leave a copy in the DCSIL org :)

Current docker does not install the repos dependencies, and defaults to exposing port 80. Need to fix both of these things.

https://izziswift.com/integrating-python-poetry-with-docker/
https://fastapi.tiangolo.com/deployment/docker/

I noticed that there is a few-second delay for the first request from each new client. This delay likely corresponds to the loading of the model. The confusing thing is that I would expect the model to be loaded once and then cached, and then not loaded again for the life cycle of the web services, regardless of the client. Figure out why this is not the case.

The CI should run the pytest code in parallel. This will start to have a big impact as the number of tests grows.

Currently, sentry monitoring will be activated even when a local instance is started, as long as the sentry-sdk is installed. It would be nice to be able to provide some kind of develop setting (say, and environment variable) that would not initiate sentry monitoring when running the instance locally.

One of the requirements for the final deliverable is that CodeClimate is setup. Each repo needs a minimum o a "B" average in "maintainability" and coverage of at least 65% (see here for more details).

Once CodeClimate is integrated with our CI, it should be easy to hit these metrics.