Topic: inference Goto Github

👇 Here are 1124 public repositories matching this topic...

argmaxinc / whisperkit
inference,Swift native on-device speech recognition with Whisper for Apple Silicon
Organization: argmaxinc
Home Page: https://takeargmax.com/blog/whisperkit
inference ios pretrained-models speech-recognition swift whisper transformers macos visionos watchos

autogptq / autogptq
inference,An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
Organization: autogptq
transformers deep-learning inference large-language-models llms nlp pytorch quantization transformer

aws / amazon-sagemaker-examples
inference,Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
Organization: aws
Home Page: https://sagemaker-examples.readthedocs.io
sagemaker aws reinforcement-learning machine-learning deep-learning examples jupyter-notebook mlops data-science training

awslabs / multi-model-server
inference,Multi Model Server is a tool for serving neural net models for inference
Organization: awslabs
mxnet deep-learning inference ai neural-network onnx server

bytedance / lightseq
inference,LightSeq: A High Performance Library for Sequence Processing and Generation
Organization: bytedance
inference transformer beam-search bert cuda sampling diverse-decoding multilingual-nmt training bart

delta-ml / delta
inference,DELTA is a deep learning based natural language and speech processing platform.
Organization: delta-ml
Home Page: https://delta-didi.readthedocs.io/
nlp deep-learning tensorflow speech sequence-to-sequence seq2seq speech-recognition text-classification speaker-verification nlu

dusty-nv / jetson-inference
inference,Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
User: dusty-nv
Home Page: https://developer.nvidia.com/embedded/twodaystoademo
deep-learning inference computer-vision embedded image-recognition object-detection segmentation jetson jetson-tx1 jetson-tx2

ebhy / budgetml
inference,Deploy a ML inference service on a budget in less than 10 lines of code.
Organization: ebhy
api data-science deployment fastapi inference machine-learning mlops

els-rd / transformer-deploy
inference,Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
Organization: els-rd
Home Page: https://els-rd.github.io/transformer-deploy/
inference deep-learning natural-language-processing deployment machine-learning server

fentechsolutions / causaldiscoverytoolbox
inference,Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.
Organization: fentechsolutions
Home Page: https://fentechsolutions.github.io/CausalDiscoveryToolbox/html/index.html
causal-inference graph causality causal-models algorithm machine-learning graph-structure-recovery python causal-discovery toolbox

gcanti / io-ts
inference,Runtime type system for IO decoding/encoding
User: gcanti
Home Page: https://gcanti.github.io/io-ts/
typescript validation inference types runtime

ggerganov / whisper.cpp
inference,Port of OpenAI's Whisper model in C/C++
User: ggerganov
inference openai speech-recognition speech-to-text transformer whisper

google / mediapipe
inference,Cross-platform, customizable ML solutions for live and streaming media.
Organization: google
Home Page: https://mediapipe.dev
mediapipe c-plus-plus computer-vision deep-learning android video-processing audio-processing mobile-development machine-learning inference

google / xnnpack
inference,High-efficiency floating-point neural network inference operators for mobile, server, and Web
Organization: google
neural-networks inference inference-optimization simd cpu multithreading matrix-multiplication convolutional-neural-networks convolutional-neural-network neural-network

gvergnaud / ts-pattern
inference,🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.
User: gvergnaud
pattern-matching typescript ts pattern matching inference type-inference exhaustive conditions branching

hpcaitech / colossalai
inference,Making large AI models cheaper, faster and more accessible
Organization: hpcaitech
Home Page: https://www.colossalai.org
ai big-model data-parallelism deep-learning distributed-computing foundation-models heterogeneous-training hpc inference large-scale model-parallelism pipeline-parallelism

huaizhengzhang / awesome-system-for-machine-learning
inference,A curated list of research in machine learning systems (MLSys). Paper notes are also provided.
User: huaizhengzhang
Home Page: https://ai-engineering.club/
machine-learning system deep-learning optimization paper resouce-management deep-reinforcement-learning edge-computing computer-system distributed-systems infrastructure deep-neural-networks automl model-database inference

huggingface / huggingface.js
inference,Utilities to use the Hugging Face Hub API
Organization: huggingface
Home Page: https://hf.co/docs/huggingface.js
api-client hub inference machine-learning huggingface

huggingface / optimum
inference,🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools
Organization: huggingface
Home Page: https://huggingface.co/docs/optimum/main/
onnx pytorch inference training intel graphcore onnxruntime transformers quantization habana

huggingface / text-generation-inference
inference,Large Language Model Text Generation Inference
Organization: huggingface
Home Page: http://hf.co/docs/text-generation-inference
bloom nlp pytorch inference gpt deep-learning transformer falcon starcoder

kamalkraj / bert-ner
inference,Pytorch-Named-Entity-Recognition-with-BERT
User: kamalkraj
bert named-entity-recognition pytorch conll-2003 cpp11 bert-ner inference curl postman pretrained-models

linzaer / ultra-light-fast-generic-face-detector-1mb
inference, 💎1MB lightweight face detection model (1MB轻量级人脸检测模型)
User: linzaer
face-detection arm inference mnn ncnn

maratyszcza / nnpack
inference,Acceleration package for neural networks on multi-core CPUs
User: maratyszcza
neural-network neural-networks convolutional-layers inference high-performance high-performance-computing simd cpu multithreading fast-fourier-transform

microsoft / aici
inference,AICI: Prompts as (Wasm) Programs
Organization: microsoft
ai rust wasm wasmtime inference language-model llm llm-framework llm-inference llm-serving

microsoft / deepspeed
inference,DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Organization: microsoft
Home Page: https://www.deepspeed.ai/
deep-learning pytorch gpu machine-learning billion-parameters data-parallelism model-parallelism inference pipeline-parallelism compression

microsoft / deepspeed-mii
inference,MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Organization: microsoft
deep-learning inference pytorch

neuralmagic / deepsparse
inference,Sparsity-aware deep learning inference runtime for CPUs
Organization: neuralmagic
Home Page: https://neuralmagic.com/deepsparse/
machinelearning onnx inference computer-vision object-detection pruning quantization pretrained-models nlp cpus

nvidia-ai-iot / torch2trt
inference,An easy to use PyTorch to TensorRT converter
Organization: nvidia-ai-iot
jetson-nano jetson-tx2 jetson-xavier pytorch tensorrt inference classification

nvidia / tensorrt
inference,NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Organization: nvidia
Home Page: https://developer.nvidia.com/tensorrt
deep-learning gpu-acceleration inference nvidia tensorrt

opennmt / ctranslate2
inference,Fast inference engine for Transformer models
Organization: opennmt
Home Page: https://opennmt.net/CTranslate2
neural-machine-translation cpp mkl quantization cuda thrust opennmt deep-neural-networks openmp onednn

openvinotoolkit / open_model_zoo
inference,Pre-trained Deep Learning models and demos (high quality and extremely fast)
Organization: openvinotoolkit
Home Page: https://docs.openvino.ai/latest/model_zoo.html
models caffemodel demo tensorflow-models model-zoo model deep-learning-models cnn-model openvino inference

openvinotoolkit / openvino
inference,OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
Organization: openvinotoolkit
Home Page: https://docs.openvino.ai
inference deep-learning openvino ai computer-vision diffusion-models generative-ai llm-inference natural-language-processing nlp

openvinotoolkit / openvino_notebooks
inference,📚 Jupyter notebook tutorials for OpenVINO™
Organization: openvinotoolkit
deep-learning machine-learning computer-vision openvino inference

roboflow / inference
inference,A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.
Organization: roboflow
Home Page: https://inference.roboflow.com
computer-vision inference-api inference-server vit yolact yolov5 yolov7 yolov8 jetson tensorrt

superduperdb / superduperdb
inference,🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
Organization: superduperdb
Home Page: https://superduperdb.com
ai mlops torch transformers mongodb python pytorch ml database data

systran / faster-whisper
inference,Faster Whisper transcription with CTranslate2
Organization: systran
deep-learning inference quantization speech-recognition speech-to-text transformer whisper openai

tairov / llama2.mojo
inference,Inference Llama 2 in one file of pure 🔥
User: tairov
Home Page: https://www.modular.com/blog/community-spotlight-how-i-built-llama2-by-aydyn-tairov
inference llama llama2 modular mojo performance simd vectorization parallelize tensor

tencent / ncnn
inference,ncnn is a high-performance neural network inference framework optimized for the mobile platform
Organization: tencent
inference high-preformance simd arm-neon deep-learning artificial-intelligence android ios ncnn vulkan

tencent / tnn
inference,TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
Organization: tencent
deep-learning mnn ncnn inference pytorch tensorflow coreml tensorrt tengine openvino

tencent / turbotransformers
inference,a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
Organization: tencent
nlp transformer bert decoder gpu machine-translation inference huggingface-transformers pytorch albert

tencentmusic / cube-studio
inference,cube studio开源云原生一站式机器学习/深度学习AI平台，支持sso登录，多租户/多项目组，大数据平台对接，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU，边缘计算，serverless，标注平台，自动化标注，数据集管理，大模型微调，vllm大模型推理，llmops，私有知识库，AI模型应用商店，支持模型一键开发/推理/微调，支持国产cpu/gpu/npu芯片，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/spark/ray/volcano分布式
Organization: tencentmusic
kubernetes inference mlops workflow ai pytorch spark argo kubeflow automl

tobegit3hub / tensorflow_template_application
inference,TensorFlow template application for deep learning
User: tobegit3hub
tensorflow tfrecords libsvm csv deep-learning machine-learning mlp cnn lstm inference

triton-inference-server / server
inference,The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Organization: triton-inference-server
Home Page: https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
cloud datacenter deep-learning edge gpu inference machine-learning

trusted-ai / adversarial-robustness-toolbox
inference,Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
Organization: trusted-ai
Home Page: https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
python attack adversarial-machine-learning poisoning trusted-ai artificial-intelligence extraction adversarial-attacks adversarial-examples evasion

tucan9389 / awesome-ml-demos-with-ios
inference,The challenge projects for Inferencing machine learning models on iOS
User: tucan9389
ios machine-learning coreml mlkit tensorflow tensorflow-lite demo awesome inference

uber / neuropod
inference,A uniform interface to run deep learning models from multiple frameworks
Organization: uber
Home Page: https://neuropod.ai
tensorflow pytorch keras deep-learning deeplearning machine-learning machinelearning inference incubation

vaticle / typedb
inference,TypeDB: the polymorphic database powered by types
Organization: vaticle
Home Page: https://typedb.com
database knowledge-base knowledge-representation reasoning logic inference typedb typeql type-system strongly-typed

vllm-project / vllm
inference,A high-throughput and memory-efficient inference and serving engine for LLMs
Organization: vllm-project
Home Page: https://docs.vllm.ai
gpt llm pytorch llmops mlops model-serving transformer llm-serving inference llama

xorbitsai / inference
inference,Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
Organization: xorbitsai
Home Page: https://inference.readthedocs.io
artificial-intelligence chatglm chatglm2 deployment flan-t5 gemma ggml inference llama llama2 llamacpp llm machine-learning mistral openai-api pytorch qwen vllm whisper wizardlm

zjhellofss / kuiperinfer
inference,带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step
User: zjhellofss
inference inference-engine deep-learning deep-neural-networks convolution relu sigmoid graph-algorithms maxpooling caffe