turboderp Goto Github PK

alpaca_lora_4bit

exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

exui

Web UI for ExLlamaV2

formatron

Formatron empowers everyone to control the format of language models' output with minimal overhead.

lm-format-enforcer

Enforce the output format (JSON Schema, Regex etc) of a language model

tabbyapi

An OAI compatible exllamav2 API that's both lightweight and fast

text-generation-webui

A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (GGUF), Llama models.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.