caraml-dev / mlp

A platform for developing and operating the machine learning systems at the various stages of machine learning life cycle.

Machine Learning Platform

Machine Learning Platform (MLP) is a unified set of products for developing and operating the machine learning systems at the various stages of machine learning life cycle. The typical ML life cycle can be viewed through the following nine stages:

MLP Products are systems and services that are specifically built to solve one or multiple stages of the machine learning life cycle's problems. Currently, we have published the following MLP products:

• Feast - For managing and serving machine learning features.
• Merlin - For deploying, serving, and monitoring machine learning models.
• Turing - For designing, deploying, and evaluating machine learning experiments.

Architecture overview

The MLP Server provides REST API used across MLP Products. It exposes a shared concepts such as ML Project. This repository also hosts Go and React (@caraml-dev/ui-lib) libraries used to build a common MLP functionailty.

Getting started

Prerequisites

1. Google Oauth credential

   MLP uses Google Sign-in to authenticate the user to access the API and UI. After you get the client ID, specify it into REACT_APP_OAUTH_CLIENT_ID in .env.development file.

From Docker Compose

If you already have Docker installed, you can spin up MLP and its dependencies by running:

docker-compose up

MLP will now be reachable at http://localhost:8080.

From source

To build and run MLP from the source code, you need to have Go, Node.js, and Yarn installed. You will also need a running Postgresql database, Keto, and Vault servers. MLP uses Docker to make the task of setting up the dependencies a little easier. You can run make local-env to starting up all those dependencies.

make local-env
make run

OR

# `make` will execute `make local-env` and `make run`
make

Documentation

Go to the docs folder for the full documentation and guides.

React UI development

For more information on building, running, and developing the UI app and library, see the UI's README.md.

mlp's Issues

codahale/hdrhistogram repo url has been transferred under the github HdrHstogram umbrella

Problem

The codahale/hdrhistogram repo has been transferred under the github HdrHstogram umbrella with the help from the original author in Sept 2020 (new repo url https://github.com/HdrHistogram/hdrhistogram-go). The main reasons are to group all implementations under the same roof and to provide more active contribution from the community as the original repository was archived several years ago.

The dependency URL should be modified to point to the new repository URL. The tag "v0.9.0" was applied at the point of transfer and will reflect the exact code that was frozen in the original repository.

If you are using Go modules, you can update to the exact point of transfer using the @v0.9.0 tag in your go get command.

go mod edit -replace github.com/codahale/hdrhistogram=github.com/HdrHistogram/[email protected]

Performance Improvements

From the point of transfer, up until now (mon 16 aug 2021), we've released 3 versions that aim support the standard HdrHistogram serialization/exposition formats, and deeply improve READ performance.
We recommend to update to the latest version.