Topic: model-serving Goto Github

Some thing interesting about model-serving

👇 Here are 119 public repositories matching this topic...

ahkarami / deep-learning-in-production

model-serving,In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

User: ahkarami

deep-learning deep-neural-networks python pytorch tesnorflow keras mxnet caffe2 production serving

alibaba / rtp-llm

model-serving,RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

Organization: alibaba

gpt inference llama llm llm-serving llmops model-serving

allegroai / clearml-serving

model-serving,ClearML - Model-Serving Orchestration and Repository Solution

Organization: allegroai

Home Page: https://clear.ml

machine-learning mlops devops deep-learning kubernetes ai clearml model-serving serving serving-pytorch-models

alvarobartt / serving-pytorch-models

model-serving,Serving PyTorch models with TorchServe :fire:

User: alvarobartt

Home Page: https://pytorch.org/serve/

torchserve pytorch model-serving pytorch-cnn model-deployment machine-learning mlops image-classification serve-pytorch

alvarobartt / serving-tensorflow-models

model-serving,Serving TensorFlow models with TensorFlow Serving :orange_book:

User: alvarobartt

Home Page: https://www.tensorflow.org/tfx/guide/serving

tensorflow tensorflow-serving mlops machine-learning model-serving model-deployment image-classification serve-tensorflow-models

aniketmaurya / chitra

model-serving,A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

User: aniketmaurya

Home Page: https://chitra.readthedocs.io/en/latest/

tensorflow image-processing object-detection deep-learning image-dataset visualization gradcam model-interpretation model-visualization image-classification

aporia-ai / inferencedb

model-serving,🚀 Stream inferences of real-time ML models in production to any data lake (Experimental)

Organization: aporia-ai

machine-learning model-serving kafka s3 model-monitoring mlops

asprenger / ray_vllm_inference

model-serving,A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.

User: asprenger

inference llm llm-serving llmops mlops model-serving pytorch ray transformer vllm

balavenkatesh3322 / model_deployment

model-serving,A collection of model deployment library and technique.

User: balavenkatesh3322

aws azure caffe data-science deep-learning keras machine-learning model model-deployment model-server model-serving mxnet neural-network pytorch serving serving-pytorch-models serving-recommendation serving-tensors tensorflow

basetenlabs / truss

model-serving,The simplest way to serve AI/ML models in production

Organization: basetenlabs

Home Page: https://truss.baseten.co

machine-learning artificial-intelligence easy-to-use inference-api inference-server model-serving open-source packaging falcon stable-diffusion

bentoml / bentoml

model-serving,The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!

Organization: bentoml

Home Page: https://bentoml.com

model-serving model-deployment model-management ml-platform ai machine-learning mlops bentoml kubernetes deep-learning

bentoml / clip-api-service

model-serving,CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search

Organization: bentoml

Home Page: https://bentoml.com

ai-applications clip cloud-native mlops model-inference model-inference-service model-serving openai-clip

bentoml / gallery

model-serving,BentoML Example Projects 🎨

Organization: bentoml

Home Page: https://bentoml.com

machine-learning machine-learning-library model-serving model-management model-deployment aws-sagemaker serverless aws-lambda gcp-cloud-functions azure-machine-learning

bentoml / ocr-as-a-service

model-serving,Turn any OCR models into online inference API endpoint 🚀 🌖

Organization: bentoml

Home Page: https://bentoml.com

ocr ocr-python ai-applications model-deployment model-serving

bentoml / onediffusion

model-serving,OneDiffusion: Run any Stable Diffusion models and fine-tuned weights with ease

Organization: bentoml

Home Page: https://bentoml.com

ai diffusion-models fine-tuning kubernetes lora model-serving stable-diffusion

bentoml / transformers-nlp-service

model-serving,Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more

Organization: bentoml

Home Page: https://bentoml.com

llm mlops model-deployment model-inference-service model-serving nlp nlp-machine-learning online-inference transformer llmops

bentoml / yatai

model-serving,Model Deployment at Scale on Kubernetes 🦄️

Organization: bentoml

Home Page: https://bentoml.com

bentoml kubernetes mlops model-deployment model-serving k8s machine-learning

eightbec / fastapi-ml-skeleton

model-serving,FastAPI Skeleton App to serve machine learning models production-ready.

User: eightbec

python python3 fastapi machine-learning model-serving

federatedai / fate-serving

model-serving,A scalable, high-performance serving system for federated learning models

Organization: federatedai

federated-learning model-serving model-versioning monitor inference

fedml-ai / fedml

model-serving,FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://fedml.ai) is your generative AI platform at scale.

Organization: fedml-ai

Home Page: https://fedml.ai

federated-learning deep-learning distributed-training edge-ai machine-learning on-device-training inference-engine mlops model-deployment model-serving

flinkml / flink-jpmml

model-serving,flink-jpmml is a fresh-made library for dynamic real time machine learning predictions built on top of PMML standard models and Apache Flink streaming engine

Organization: flinkml

apache-flink flink-jpmml model-serving pmml prediction

galileo-galilei / kedro-mlflow-tutorial

model-serving,A tutorial on how to use kedro-mlflow plugin (https://github.com/Galileo-Galilei/kedro-mlflow) to synchronize training and inference and serve kedro pipeline

User: galileo-galilei

kedro mlflow kedro-mlflow kedro-tutorial mlops model-serving

google / jetstream

model-serving,JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

Organization: google

gemma gpt gpu inference jax large-language-models llama llama2 llm model-serving

hamedonline / ml-workflow

model-serving,A hands-on case study for demonstrating the stages involved in a machine learning project, from EDA to production.

User: hamedonline

machine-learning ml mlflow data-science ai deployment regression model-serving tabular-data jupyter-notebooks

huggingbench / huggingbench

model-serving,Find the optimal model serving solution for 🤗 Hugging Face models 🚀

Organization: huggingbench

huggingface inference machine-learning model-serving

instill-ai / console

model-serving,⛅ Versatile Data Pipeline (VDP) console website

Organization: instill-ai

Home Page: https://www.instill.tech

no-code console computer-vision deep-learning unstructured-data frontend data-connector ui vdp object-detection

kserve / kserve

model-serving,Standardized Serverless ML Inference Platform on Kubernetes

Organization: kserve

Home Page: https://kserve.github.io/website/

knative machine-learning model-interpretability model-serving istio kubeflow kubeflow-pipelines artificial-intelligence tensorflow pytorch

lightbend / kafka-with-akka-streams-kafka-streams-tutorial

model-serving,Code samples for the Lightbend tutorial on writing microservices with Akka Streams, Kafka Streams, and Kafka

Organization: lightbend

akka kafka-streams model-serving

lightbend / kubeflow-recommender

model-serving,Kubeflow example of machine learning/model serving

Organization: lightbend

kubeflow model-serving machine-learning

lightbend / model-serving-tutorial

model-serving,Code and presentation for Strata Model Serving tutorial

Organization: lightbend

model-serving kafka akka-streams flink spark pmml tensorflow tensorflow-serving

lightning-universe / stable-diffusion-deploy

model-serving,Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.

Organization: lightning-universe

Home Page: https://lightning.ai/muse

model-serving stable-diffusion

logicalclocks / hopsworks

model-serving,Hopsworks - Data-Intensive AI platform with a Feature Store

Organization: logicalclocks

Home Page: https://hopsworks.ai

feature-store aws azure data-science feature-engineering feature-management gcp governance kserve machine-learning mlops model-serving pyspark python serverless ml hopsworks

messense / fasttext-serving

model-serving,fastText model serving service

User: messense

fasttext model-serving model-server nlp

microsoft / aici

model-serving,AICI: Prompts as (Wasm) Programs

Organization: microsoft

ai rust wasm wasmtime inference language-model llm llm-framework llm-inference llm-serving

mlrun / mlrun

model-serving,MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

Organization: mlrun

Home Page: https://mlrun.org

mlops python data-science machine-learning data-engineering experiment-tracking model-serving mlops-workflow workflow kubernetes

modeltc / lightllm

model-serving,LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Organization: modeltc

deep-learning gpt llama llm model-serving nlp openai-triton

modzy / hugging-face-raspberry-pi

model-serving,Deploy, serve, and run a Hugging Face model on a Raspberry Pi with just a few lines of code

Organization: modzy

docker flask-application huggingface huggingface-transformers machine-learning model-serving raspberry-pi raspberry-pi-3

mosecorg / mosec

model-serving,A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Organization: mosecorg

Home Page: https://mosec.readthedocs.io/

model-serving deep-learning machine-learning nerual-network mlops machine-learning-platform hacktoberfest gpu python pytorch

nimbleboxai / nbox

model-serving,The official python package for NimbleBox. Exposes all APIs as CLIs and contains modules to make ML 🌸

Organization: nimbleboxai

Home Page: https://nimblebox.ai/docs/nbox/start

data-science machine-learning ml-infrastructure ml-platform ml-service mlops mlops-pipeline mlops-workflow model-deployment model-management

notai-tech / fastdeploy

model-serving,Deploy DL/ ML inference pipelines with minimal extra code.

Organization: notai-tech

deep-learning tensorflow-serving tf-serving pytorch serving falcon gevent docker model-deployment model-serving

openvinotoolkit / model_server

model-serving,A scalable inference server for models optimized with OpenVINO™

Organization: openvinotoolkit

Home Page: https://docs.openvino.ai/2024/ovms_what_is_openvino_model_server.html

openvino inference ai edge cloud deep-learning serving dag kubernetes machine-learning

predibase / lorax

model-serving,Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Organization: predibase

Home Page: https://loraexchange.ai

fine-tuning gpt llama llm llm-inference llm-serving llmops lora model-serving pytorch

project-monai / monai-deploy-app-sdk

model-serving,MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.

Organization: project-monai

monai healthcare medical-imaging image-processing dicom deploy workflow pipeline ai ml