janhq Goto Github PK

app-backend

architecture

awesome-local-ai

An awesome repository of local AI tools

charts

This repository contains helm chart for our team

docs

Jan.ai Website & Documentation

extension-template

infinity

The AI-native database built for LLM applications, providing incredibly fast vector and full-text search

jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

jan-web

langchainjs

llama.cpp-avx-vnni

Port of Facebook's LLaMA model in C/C++

model-converter

models

nitro

An inference server on top of llama.cpp. OpenAI-compatible API, queue, & scaling. Embed a prod-ready, local inference engine in your apps. Powers Jan

nitro-tensorrt-llm

Nitro is an C++ inference server on top of TensorRT-LLM. OpenAI-compatible API. Run blazing fast inference on Nvidia GPUs. Used in Jan

node-nitro

open-foundry

Containing all of Jan's researches

openai_trtllm

OpenAI compatible API for TensorRT LLM triton backend

plugin-catalog

py-nitro

real-esrgan

Real-ESRGAN aims at developing Practical Algorithms for General Image/Video Restoration.

tensorrt

NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications.

tensorrtllm_backend

The Triton TensorRT-LLM Backend

triton_tensorrt_llm

trt-llm-as-openai-windows

This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows instead of cloud.