zochory / ai-chat-protocol

A library for easily building AI Chat interfaces.

Microsoft AI Chat Protocol

The Microsoft AI Chat Protocol SDK is a library for easily building AI Chat interfaces from services that follow the AI Chat Protocol API Specification, both of which are located in this repository.

By agreeing on a standard API contract, AI backend consumption and evaluation can be performed easily and consistently across different services regardless of the models, orchestration tooling, or design patterns used.

Note: we are currently in public preview. Your feedback is greatly appreciated as we get ready to be generally available!

With the AI Chat Protocol, you will be able to:

• Develop AI chat interfaces, components, and applications in JavaScript/TypeScript (more languages to follow!)
• Consistently consume and evaluate AI inference backends and middle tiers with ease, either synchronously or by streaming
• Easily incorporate HTTP middleware for logging, authentication, and more.

Please star the repo to show your support for this project!

Getting Started

Our comprehensive getting started guide is coming soon! Be sure to check out the samples and API specification for more details.

• Samples
• API Specification
• Samples on Azure

To take a look locally, install the library via npm:

npm install @microsoft/ai-chat-protocol

Create the client object:

const client = new AIChatProtocolClient("/api/chat");

Stream completions to your UI:

let sessionState = undefined;

// add any logic to handle state here
function setSessionState(value) {
    sessionState = value;
}

const message: AIChatMessage = {
    role: "user",
    content: "Hello World!",
};

const result = await client.getStreamedCompletion([message], {
    sessionState: sessionState,
});

for await (const response of result) {
    if (response.sessionState) {
        //do something with the session state returned
    }
    if (response.delta.role) {
        // do something with the information about the role
    }
    if (response.delta.content) { 
        // do something with the content of the message
    }
}

Samples on Azure

If you're curious on samples hosted on Azure, the following samples utilize the AI Chat Protocol SDK on the frontend:

• Serverless AI Chat with RAG using LangChain.js
• Chat Application using Azure OpenAI (Python)
• OpenAI Chat Application with Microsoft Entra Authentication (Python) - Local
• OpenAI Chat Application with Microsoft Entra Authentication (Python) - Builtin
• OpenAI Chat App Frontend (Vanilla JS)
• Chat Application using Azure OpenAI (Python)

Additionally, many Azure AI sample projects utilize the AI Chat Protocol API spec without the SDK, either because they don't have a frontend, or because they were made before the library's release:

• ChatGPT + Enterprise data with Azure OpenAI and AI Search in Python
• ChatGPT + Enterprise data with Azure OpenAI and Azure AI Search in JavaScript
• Chat with GPT Modes - FastAPI backend
• Evaluating a RAG Chat App

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

License

Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT license.