Neural Magic's Projects
An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
Hackathon 2022
CLIP-like model evaluation
A safetensors extension to efficiently store sparse quantized tensors on disk
π€ The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools
Sparsity-aware deep learning inference runtime for CPUs
Repo for building and packaging a 1-click app for DigitalOcean
Top-level directory for documentation and general content
Notebooks using the Neural Magic libraries π
woop wooop
Helm charts for deploying NM VLLM
Reference implementations of MLPerfβ’ inference benchmarks
β‘ Building applications with LLMs through composability β‘
A framework for few-shot evaluation of autoregressive language models.
Neural Magic GHA
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
Neural Magic Docker
A high-throughput and memory-efficient inference and serving engine for LLMs
Various utilities for use with nm-vllm
PyTorch image models, scripts, pretrained weights -- ResNet, ResNeXT, EfficientNet, EfficientNetV2, NFNet, Vision Transformer, MixNet, MobileNet-V3/V2, RegNet, DPN, CSPNet, and more
Framework agnostic sliced/tiled inference + interactive ui + error analysis plots
Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models
Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes
ML model optimization product to accelerate inference.
LLM training code for MosaicML foundation models