adamelliotfields / chat

LLMs in your browser | chat.aef.me 💬

chat's Issues

Settings menu

TBD

Adapters

Currently the app only works for people with Android phones or gaming laptops, which is pretty sad.

Implement adapters so different backends can be used. The first adapter will be for the Hugging Face Inference API.

To support streaming text, you’ll have to use the Hugging Face JS library or write something similar. Edit: TGI Messages API.

Also want to support OpenAI, Perplexity, Anthropic, and Goose.ai. (Mistral and Replicate don’t offer credit-based billing yet).

Assistants

Assistants are just presets with a system prompt.

Need to redesign the layout first and think about how to implement.

Improve prompt input

Increase the textarea height by a fixed amount (e.g., 1.5rem) for each additional row.

Listen for Enter keypress while respecting Shift+Enter.

See react-textarea-autosize.

See example from labs.pplx.ai below.

pplx-labs-input-example.mp4

Import/export conversations

Allow importing an entire conversation object, complete with settings, system prompt, and messages. Likewise, allow exporting the current conversation with settings.

Remove Web LLM

Focus on web APIs until Web GPU is more widespread.

Can add an "offline" toggle in the future. Also need to run inference on a worker thread and possibly use million.