yejingxin / ml-testing-accelerators

Testing framework for Deep Learning models (Tensorflow and PyTorch) on Google Cloud hardware accelerators (TPU and GPU)

ML Testing Accelerators

A set of tools and examples to run machine learning tests on ML hardware accelerators (TPUs or GPUs) using Google Cloud Platform.

This is not an officially supported Google product.

Getting Started

In this mode, your tests and/or models run on an automated schedule in GKE. Results are collected by the "Metrics Handler" and written to BigQuery.

1. Install all of our development prerequisites.
2. Follow instructions in the deployments directory to set up a Kubernetes Cluster.
3. Follow instructions in the images directory to set up the Docker image that your tests will run.
4. Deploy the metrics handler to Google Cloud Functions.
5. Deploy the event publisher to you GKE cluster.
6. See templates directory for a JSonnet template library to generate test config files.
7. (Optional) Set up a dashboard to view test results. See dashboard directory for instructions.

Are you interested in using ML Testing Accelerators? E-mail [email protected] and tell us about your use-case. We're happy to help you get started.