shroominic / fastui-chat

💬 minimalistic ChatBot Interface in pure python

fastui-chat

A minimalistic ChatBot Interface in pure python.
Build on top of FastUI and Funcchain.

Usage

pip install fastui-chat

from fastui_chat import ChatUI

# chatui inherits from FastAPI so you can use it as a FastAPI app
app = ChatUI()

# Run with:
# uvicorn examples.minimal:app

# for hot reloading:
# uvicorn examples.minimal:app --reload

# or use the built-in method
if __name__ == "__main__":
    app.start_with_uvicorn()

Extend FastAPI

You can also only use the router and extend your existing FastAPI app.

from fastapi import FastAPI
from fastui_chat import create_chat_handler, create_history_factory
from fastui_chat.chat import ChatAPIRouter
from fastui_chat.history import InMemoryChatMessageHistory
from fastui_chat.runtime import router as fastui_runtime

# callable that returns a ChatMessageHistory given a session_id
history_factory = create_history_factory(
    # swap out with any from langchain_community.chat_message_histories
    InMemoryChatMessageHistory,
)

# a chat handler generates an AIMessage based on a given HumanMessage and ChatHistory
chat_handler = create_chat_handler(
    llm="openai/gpt-4-turbo-preview",
    history_factory=history_factory,
)

# setup your fastapi app
app = FastAPI()

# add the chatui router to your app
app.include_router(
    ChatAPIRouter(history_factory, chat_handler),
    prefix="/api",
)

# make sure to add the runtime router as latest since it has a catch-all route
app.include_router(fastui_runtime)

# start the server with `uvicorn examples.fastapi_router:app`

Features

• Python Only
• Easy to use
• Minimalistic & Lightweight
• LangChain Compatible
• FastAPI Compatible
• Parallel Chat Sessions
• Switchable ChatHistory Backends
• Insert your custom chat handler

Development Setup

git clone https://github.com/shroominic/fastui-chat.git && cd fastui-chat

./dev-install.sh

Roadmap

If you want to contribute or see whats coming soon checkout the roadmap.todo file for open todos.

fastui-chat's Issues

ModuleNotFoundError: No module named 'fastui_chat' when using the provided dev script

I uninstalled fastui-chat and tried the ./dev-install.sh (rye) way

It starts OK:

$ ./dev-install.sh
SYNC: setup .venv
Reusing already existing virtualenv
Generating production lockfile: /private/tmp/fastui-chat/requirements.lock
Generating dev lockfile: /private/tmp/fastui-chat/requirements-dev.lock
Installing dependencies
Found existing installation: fastui-chat 0.0.2
Uninstalling fastui-chat-0.0.2:
  Successfully uninstalled fastui-chat-0.0.2
Looking in indexes: https://pypi.org/simple/
Obtaining file:///. (from -r /var/folders/11/phrmvr5n0js581n6rgz37gww0000gp/T/tmpx9menndl (line 1))
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: fastui-chat
  Building editable for fastui-chat (pyproject.toml) ... done
  Created wheel for fastui-chat: filename=fastui_chat-0.0.2-py3-none-any.whl size=1648 sha256=63ba8c4a2200cb9a9ee5bacc79ec793b4fe837f665c435ce4356b48a7cc5e157
  Stored in directory: /private/var/folders/11/phrmvr5n0js581n6rgz37gww0000gp/T/pip-ephem-wheel-cache-jc0g6f02/wheels/78/12/b9/db4c72ae1aebdfd866f484b59eb7b39f206b7d7d1080ae28b1
Successfully built fastui-chat
Installing collected packages: fastui-chat
Successfully installed fastui-chat-0.0.2
Done!
ACTIVATE: activate .venv
Spawning virtualenv shell from /private/tmp/fastui-chat/.venv
Leave shell with 'exit'

But when trying to run the simple_chatbot.py example I got this error:

$ python examples/simple_chatbot.py
Traceback (most recent call last):
  File "/private/tmp/fastui-chat/examples/simple_chatbot.py", line 1, in <module>
    from fastui_chat import ChatUI, basic_chat_handler
ModuleNotFoundError: No module named 'fastui_chat'

Am I doing something wrong? (first time using rye here)

scrollable chat history

Any ideas about chat histories that over time become too large that we cannot scroll up and down and no longer fit in the page itself ? Is this a limittion of fastUI rather than your app ?

        @self.get(
            "/chat/{session_id}",
            response_model=FastUI,
            response_model_exclude_none=True,
        )
        async def chat_ui(
            session_id: Annotated[str, Path(title="The ChatHistory SessionID.")],
        ) -> list[AnyComponent]:
            """
            Main endpoint for showing the Chat UI and handling user input.
            """
            return [
                c.Page(
                    components=[
                        c.ServerLoad(
                            path=f"/chat/{session_id}/history",
                            load_trigger=PageEvent(name="chat-load"),
                            components=[],
                        ),
                        ChatInputForm(
                            submit_url=f"/api/chat/{session_id}/generate",
                            fire_page_event="chat-load",
                        ),
                    ],
                    class_name="fixed-bottom row justify-content-center row-cols-lg-2",
                )
            ]

Most likely due to a circular import

Great project idea!

poetry add fastui-chat
Using version ^0.3.4 for fastui-chat

Updating dependencies
Resolving dependencies... (4.3s)

Package operations: 9 installs, 0 updates, 0 removals

• Installing dnspython (2.6.1)
• Installing email-validator (2.1.1)
• Installing docstring-parser (0.16)
• Installing jinja2 (3.1.4)
• Installing pydantic-settings (2.2.1)
• Installing python-multipart (0.0.9)
• Installing fastui (0.6.0)
• Installing funcchain (0.3.3)
• Installing fastui-chat (0.3.4)

LOG_LEVEL="DEBUG" python playground/fastui_chat.py Traceback (most recent call last): File "/Users/derevo/MEV.nosync/SANTA/GIT/ECU/HOMA/playground/fastui_chat.py", line 1, in <module> from fastui_chat import ChatUI File "/Users/derevo/MEV.nosync/SANTA/GIT/ECU/HOMA/playground/fastui_chat.py", line 1, in <module> from fastui_chat import ChatUI ImportError: cannot import name 'ChatUI' from partially initialized module 'fastui_chat' (most likely due to a circular import)

no "simple_chatbot.py" in /examples folder

I cloned the repo and installed it (pip version).
But when I ran python ./examples/simple_chabot.py there is no file found.

What does it look like?

Hi, it says "minimalistic ChatBot Interface" but you did not even add screenshot of the interface in your README.md file.

Response not valid JSON for simple_chatbot

Hi
I created a Github codespace, installed using ./dev-install.sh and pip, then:

export OPENAI_API_KEY=sk...
cd examples
python simple_chatbot.py

No errors in logs but on the page:

Add document retrieving capabilities

Rather than an issue , I would be interested on where I could develop some document retrieving (RAG) capabilities into the chatUI. Which files to extend ?

The simple_chatbot example doesn't work

Glad to see new and interesting projects from you, @shroominic

I installed the package, set and exported the OPENAI API key and copy pasted the simple_chatbot example
https://raw.githubusercontent.com/shroominic/fastui-chat/main/examples/simple_chatbot.py
Then, I tried to run it, to no avail.
The server is started and I can type a question in the UI, but it keeps giving me this error in a loop (until I hit the ctrl-c key)

$  pip install fastui-chat
...
$ export OPENAI_API_KEY=xxxxxxx

$ python simple_chatbot.py
INFO:     Started server process [37535]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:59603 - "GET /api/chat/generate/response?user_msg=what%27s%20the%20capital%20of%20France? HTTP/1.1" 200 OK
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/routing.py", line 69, in app
    await response(scope, receive, send)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/responses.py", line 270, in __call__
    async with anyio.create_task_group() as task_group:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 597, in __aexit__
    raise exceptions[0]
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/responses.py", line 273, in wrap
    await func()
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/responses.py", line 262, in stream_response
    async for chunk in self.body_iterator:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/fastui_chat/chat.py", line 84, in ai_response_generator
    async for chunk in session.astream(user_msg):
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/fastui_chat/session.py", line 20, in astream
    async for message in self.chat_handler.astream(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2109, in astream
    async for chunk in self.atransform(input_aiter(), config, **kwargs):
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2092, in atransform
    async for chunk in self._atransform_stream_with_config(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1266, in _atransform_stream_with_config
    async for chunk in iterator:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2062, in _atransform
    async for output in final_pipeline:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3515, in atransform
    async for item in self.bound.atransform(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3515, in atransform
    async for item in self.bound.atransform(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2092, in atransform
    async for chunk in self._atransform_stream_with_config(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1266, in _atransform_stream_with_config
    async for chunk in iterator:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2062, in _atransform
    async for output in final_pipeline:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3515, in atransform
    async for item in self.bound.atransform(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2092, in atransform
    async for chunk in self._atransform_stream_with_config(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1266, in _atransform_stream_with_config
    async for chunk in iterator:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2062, in _atransform
    async for output in final_pipeline:
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3092, in atransform
    async for output in self._atransform_stream_with_config(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1245, in _atransform_stream_with_config
    final_input: Optional[Input] = await py_anext(input_for_tracing, None)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/utils/aiter.py", line 62, in anext_impl
    return await __anext__(iterator)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/utils/aiter.py", line 97, in tee_peer
 97     def __init__(self, variable_name: str, *, optional: bool = False, **kwargs: Any):
    item = await iterator.__anext__()
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 791, in atransform
    async for output in self.astream(final, config, **kwargs):
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 572, in astream
    yield await self.ainvoke(input, config, **kwargs)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 476, in ainvoke
    return await asyncio.get_running_loop().run_in_executor(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/prompts/base.py", line 93, in invoke
    return self._call_with_config(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 955, in _call_with_config
    output = call_func_with_variable_args(
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 309, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/prompts/base.py", line 88, in _format_prompt_with_error_handling
    return self.format_prompt(**inner_input)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/prompts/chat.py", line 365, in format_prompt
    messages = self.format_messages(**kwargs)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/prompts/chat.py", line 614, in format_messages
    message = message_template.format_messages(**kwargs)
  File "/Users/juanan/.pyenv/versions/3.10.8/lib/python3.10/site-packages/langchain_core/prompts/chat.py", line 123, in format_messages
    raise ValueError(
ValueError: variable history should be a list of base messages, got [AIMessage(content='How can I help you today?')]

This was the question that I tried:

output of [INST] <<SYS>><</SYS>> at the end of every answer

I am using your example and ChatOllama instead of openai chatmodel and I get some UI strange characters at the end of every answer.Following is an image :