euskoog / openai-assistants-evals

Visualize LLM Evaluations for OpenAI Assistants

OpenAI Assistants Evals

Remix dashboard for analyzing LLM evaluations

---

You can access the live demo here.

❓ What This Is

OpenAI Assistants Evals is a repo for analyzing LLM evaluations. It acts as a guide for understanding how to collect custom evaluations on OpenAI Assistants. The repository features two main pages:

• Home (blogpost on method and reasoning)
• Evaluations (interactive dashboard)

All data is set up and collected through an adjacent API project, OpenAI Assistants Link. If you want to set up your own evaluations exactly like I have, I would recommend checking out the Link repo to create your own API and database connections.

All data that is displayed in the dashboard is my own local data.

🚀 Getting Started

If you want to run this project locally with your own data, you will first need to follow the instructions from the OpenAI Assistants Link repo. Once you have completed all of those steps, you should have a railway deployment link for your live server. (This will still work even if you use a different hosting service) This link is important because this FE project uses the npm package openapi-typescript-codegen, which generates a local lib for your API types and endpoints. This makes setting up network requests a breeze.

Assuming you have a valid API link, you will need to create a .env file and store your variable as:

API_URL=<YOUR API LINK>

Once this is complete, you will just need to install the required npm packages and run the app.

Install dependencies:

pnpm i

Run codegen, build, and local server:

pnpm dev

This should set you up to run the dashboard with your local data.

Notes:

• This project was created using Remix, Typescript, and Tailwind CSS. To learn more about how Remix works, check out their docs.