Coder Social home page Coder Social logo

spotify / zoltar Goto Github PK

View Code? Open in Web Editor NEW
138.0 25.0 33.0 1.92 MB

Common library for serving TensorFlow, XGBoost and scikit-learn models in production.

Home Page: https://spotify.github.io/zoltar/

License: Apache License 2.0

Shell 0.67% Makefile 0.39% Python 2.18% Java 92.58% Scala 4.14% Dockerfile 0.03%
model-serving machine-learning xgboost tensorflow java

zoltar's Introduction

zoltar's People

Contributors

andrewsmartin avatar brianmartin avatar dependabot-preview[bot] avatar dependabot[bot] avatar fallonchen avatar guidj avatar jwiklund avatar kellen avatar kination avatar krshn368 avatar mattnworb avatar mziccard avatar nevillelyh avatar perploug avatar ravwojdyla avatar rculbertson avatar regadas avatar yonromai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zoltar's Issues

Upgrade to TF 1.6.0

1.5.0 did not support ubuntu 14.04, which is our default, was suppose to be solved in 1.6.0 - see if we can upgrade.

Add startup check(s)

Add a support for startup checks, the flow would be:

  • load model
  • run user defined check on the model
  • allow prediction

This would be useful in canary environments to fail on "startup" - in case there is some skew between expected features and model(s).

Research how to expose model and feature metadata from a service

What and how do we want to expose metadata about a model and feature spec/settings:

  • how do squads want to consume it?
  • what is this information gonna be used for?

Requires talking with the squads/cream and figure out what can we do here. Might be that we will leave this open for some time, until we have better set of requirements.

TEST

testing github integration

Add ML Engine model

Add ML engine model to abstract away serving from ML Engine. Users would push models to ML Engine and still use Zoltar to get predictions, zoltar would make call to ML Engine and retrieve prediction.

Models hot-swap

Research how to and if we can/want-to do hot-swap of models to upgrade/downgrade models without service downtime. There is obviously a lot of corner cases and things to consider with this functionality - should we just keep it simple and delegate this problem to standard backend rollouts, how important is it to have hot-swaps?

Skipton has some of that functionality.

Related: #15

Remove the obsolete IrisExtras

IrisExtras is needed because during release a javadoc.jar is needed for sub-module zoltar-tests (sonatype requires this jar). The only way i got it to generate was by adding this dummy class.

Support date partition parsing?

romain:

Disclaimer: I'm aware the feature bellow sounds like scope creeping, but I think it's important to think about whether it should part of the lib MVP or not.

The scope of the lib is intentionally quite limited, for very good reasons, so I think it's good to exclude things like versioning.

Yet, today most teams do a very simple versioning based on date partitions. For example, the model(& features) is trained daily and the date in the bucket path is used to resolve the latest model when the service starts, without having to do a config modification.

It seems like it could be fairly simple to offer some utils code to resolve the latest model form a path (gs://my_bucket/path_to_my_model/%Y%m%d/my_model.bin). I believe this feature would be really useful since most teams would want to do this.

Thoughts?

Clean up the code

We should:

  • poms use the same style
  • the same use of final
  • review public APIs - which methods/classes are open
  • document all the public APIs
  • no unused imports
  • document that both TF and XGBoost are thread-safe

Add bootstrap/setup stage on model loading

Add a stage/API to allow model setup/bootstrap before we run prediction.

This might be useful for example in case there is some data embedded in the graph, that we might want to retrieve.

Research if we can make TF prediction more user friendly by packaging model and its metadata

Right now TF prediction is rather cumbersome and error-prone, it includes:

  • specify input and output operations
  • shaping of the input Tensors
  • reshaping of output Tensors

Research if we could provide an easy path from https://github.com/spotify/spotify-tensorflow to package a model/graph together with all the necessary metadata so that users don't need to worry about the low level TensorFlow constructs like operations, shapes and tensors in zoltar. This approach could cover 80% of use cases, we should still allow for a completely custom prediction.

Add helper for TF predict results

Right now TF predict library returns a List of Tensors in the order of fetch operations, this is error prone, we should provide a helper that given a list of fetch ops, should return a map (?) of op name -> Tensor.

Better documentation

We're going to implement better documentation by mid-April. Let's use this issue to keep track of topics that should be covered.

Consider a single element ExtractFn

Consider a single element ExtractFn interface, high level API should still allow for a list/vargs (any maybe a single element). Consider - cause we need to see if how will this work with Featran.

Improve apollo example

right now the example seems a bit messy. I think we should improve:

  • config parsing
  • predictor creation
  • endpoint handling

Add example/tests of using multiple Predictors for the same input

Users should be able to for given input, specify multiple Predictors and then map input over the Predictors to get predictions.

Something like:

val input = List(...)
val predictors = List(Predictor.create(XGBoostModel, ...), Predictor.create(TFModle, ...))
predictors.map(_.predict(input))

Simplify predictor building logic

Right now construction the prediction objects is convoluted, we should provide helper methods to hide this complexity for the most common use cases.

Provide BOM

Right now there is multitude of conflicts between dependencies in common cases of mixing scio, featran, zoltar and apollo. For most of our sbt projects where we don't enforce enforcer - it's less of a "problem" (or should i see we are just postponing the problem), in most maven we do enforce it tho. It might be worth to provide a BOM for zoltar so that ppl can include it in their dependency management and get zoltar with fixed dependencies for combined zoltar+scio+featran+apollo.

Add operational metrics

We can start with a small set of metrics:

  • prediction duration
  • prediction rate
  • feature extraction duration
  • feature extraction rate

Review/improve prediction API

rav:

Get some feedback on the API for prediction, I have tried to use it myself on a completely new example, and quite quickly got confused, it's either that we need an easier API or some higher level API on top of the current building blocks.

Helper method to feed sparse arrays into TF

We should add some convenience methods, maybe in TensorFlowExtras, to feed Featran FloatSparseArray and DoubleSparseArray into TensorFlow.

https://github.com/spotify/featran/blob/master/java/src/main/scala/com/spotify/featran/java/JavaOps.scala#L126

The feeding code might look like this:

runner
            .feed("input/raw_indices", Tensors.create(new long[]{0, 5, 9}))
            .feed("input/raw_shape", Tensors.create(new long[]{10}))
            .feed("input/raw_data", Tensors.create(new double[]{1.0, 2.0, 3.0}));

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.