huggingface / chat-ui Goto Github PK

https://huggingface.co/spaces/huggingchat/chat-ui/discussions/51

Will allow calling /api directly without prefixing by the origin

Given that the cookie is "Same-Site": "None", we should double check everything is ok security-wise, harden if not.

https://github.com/metonym/svelte-highlight/

Currently it's done by a call from the frontend, but ideally it should be directly from the backend.

Maybe related to #117 (websockets)

• When opening a new conversation: create conversation, then call to send first message, then set conversation title
• When sending a message on a shared conversation: duplicate conversation, then call to send first message
• When retrying a message on a shared conversation: duplicate conversation, then retry

Each of those is two or three calls, they should only be one.

In the backend of message sending, we can maybe send a first token with the conversation id, and sometimes a last token with the conversation title?

It will also help with #19

@gary149 lmk what do you think of something like this?

refs: https://huggingface.slack.com/archives/C052EKHD1U0/p1682440056354789

https://github.com/huggingface/moon-landing/pull/6000/files
https://github.com/huggingface/moon-landing/blob/1cc6f9bb53f1b8087d5f0c21be1f3ec593bd167c/server/views/layout/layout.html#L64

Can use "zod", or "arktype" to validate inputs (or even Joi)

For all front-end requests

Prepare the repository to be made public:

• scan the repo for secrets? (I used gitleaks and seems ok)
• add "contribute" section to README
• add banner/logo to README
• add a license
• add a description in the Github "About" section

Either before or after the call to marked.

We probably need to load something like jsdom on the backend (but not the frontend) for dompurify to work.

Inspiration from hub code:

let purify: createDOMPurify.DOMPurifyI;

async function loadDOMPurify() {
	if (purify) {
		return;
	}
	if (typeof window === "undefined") {
		/// To avoid needing the dependency on the frontend, we load it dynamically on the backend
		// eslint-disable-next-line @typescript-eslint/no-shadow
		const { JSDOM } = await import("jsdom");
		purify = createDOMPurify(new JSDOM().window as unknown as Window);
	} else {
		purify = createDOMPurify(window);
	}

To take advantage of the two vCPUs 🤯

• keyboard navigation on modals (ESC key to close) + click on background
• keyboard navigation on conversation list, currently you can't access edit/delete buttons with keyboard
• keyboard navigation on chat, currently you can't access retry (and soon thumb up/down) buttons with keyboard
• minor but maybe make buttons consistent (some don't have hover/active state)

Once #156 is merged:

• Add a property "currentModel" in Settings (to remember a user's preferred model), and enable changing this setting (in the settings modal or the chat interface)
• Pass it along instead of models[0] to the backend calls

The rest should be UI/UX/Frontend

It may be useful, as in "this was not a good response", like thumbs down

When on HF.

Currently the input is disabled, we should show something when the model is generating.

Because sometimes it's stuck for a while, and we wonder what's happening

Sometimes the model returns weird things like on the following screenshots:

Code not parsed properly:

The scrollToBottom seems to be kicking in even on the homepage. It should not.

Convos are created but their messages aren't saved.

We should save the messages in DB

First draft we can save directly from frontend

• Add a confirmation before deleting chats
• Eventually add a confirmation on retry button if not the last one (as a retry deletes all following messages)

Context:

• Having a conversation
• Scroll up
• Enter a message
• Scroll remains at the same place

I would expect the scroll to be at the message level

Some ideas:

1. Same as on https://play.vercel.ai/r/QI768eA we could have a instead of the current "Share this conversation" once the conv has been shared which copies to clipboard directly on click
2. Maybe have a custom modal instead of window.alert for better look?
3. You can't copy the link on Safari desktop with the share API

cc @gary149

If there is an HTML tag that we don't explicitly support in the markdown, it will appear in plain text. For example some users see <p> tags.

ref: https://huggingface.co/spaces/huggingchat/chat-ui/discussions/69

I don't think we need to go w/ dompurify, maybe we can just unescape the characters before highlighting them

There's a short flash from white to dark (when on dark mode) when refreshing the page. I think it was not the case before 👀

Since we sanitize the markdown before parsing it. We are disallowing every HTML tag within markdown apart from code blocks. We might want to selectively authorise some tags, or use a sanitizer...

I know it's something we built on purpose but thinking about it more and playing with it I don't think we should stop the generation when user is switching between conversations:
Imagine you want to check another conversation while waiting for the assistant to finish answering on a long request, you loose the answer.

          but this should be made configurable through an env var no?

Originally posted by @julien-c in #119 (comment)

https://huggingface.co/spaces/huggingchat/chat-ui/discussions/24

Those last commands:

RUN --mount=type=secret,id=DOTENV_LOCAL,mode=0444,required=true cat /run/secrets/DOTENV_LOCAL > .env.local
   
RUN npm run build

ENV PORT 7860

CMD pm2 start build/index.js -i $CPU_CORES --no-daemon

Should be run in a .sh script called by the last CMD of the Dockerfile. Because the cache layers aren't invalidated when secrets change.

It would also allow people to download the docker image (would probably need to delete & recreate the space though, as for now we allow to download either all or none of the docker images)

cc @XciD @christophe-rannou

• Create MongoDB database/collections
• Create endpoint /api/conversations
  • POST: create a new conversation
  • GET retrieve a list of conversations
• Create endpoint /api/conversation
  • POST(conversation_id): send a new message in a conversation
• Create endpoint /api/conversation/:id
  • GET retrieve an existing conversation

UI glitches briefly:

Currently we don't have any target in our tsconfig. The problem is that by default TS will load some recent ecmascript version which are not compatible with fairly recent browsers such as Safari 14.

For example we are using methods such as Array.prototype.at() which is ECMAScript 2022 or replaceAll().
So we need to either set something like "lib": ["ES2018", "DOM"] so TS doesn't compile if we use those, or just polyfill them ( not sure doing "target": "ES5" would work?).

ref: https://huggingface.slack.com/archives/C055NNJF98Q/p1682531350797939

Currently user loose their input as well.
https://huggingface.co/spaces/huggingchat/chat-ui/discussions/20#6447f6133e7b3c11be63f8aa

To release

• Mobile height (with Safari navigation bar it's overflowing)
• Add social meta
• Make "share" more visible
• Code blocks screen overflow
• Scrollbar adjustments for code blocks and navigation
• Add disclaimer
• Persistent conversations
• Toggle for dark mode
• Models selector
• use .prose class for assistant messages #16
• Share session with others
• Errors handling
• Auto-scroll #10
• Retry with the same message?
• Handle max-tokens
• Stop generating #86
• Privacy modal
• Icons
• #12

Others

• Data export?
• Auto-name conversations
• Expose and being able to configure model params
• Search conversations
• Like/dislikes responses
• Arrow up/down to browse user input history

Find a way to reorder the convo list when sending a message, or update the convo's title

Currently it's:

		headers: {
			...request.headers,
			'Content-Type': 'application/json',
			Authorization: `Basic ${HF_TOKEN}`
		},

I think ...request.headers is not valid unfortunately.

By fixing we can

• remove 'Content-Type': 'application/json'
• Maybe even remove the guard against the route in hooks.server.ts (maybe...)

After #21

In the proxy, listen to the response to save the message in DB afterwards.

May need huggingface/huggingface.js#155 (maybe not!!)

I think maybe the last output from the response contains the whole message, we can use that instead of parsing tokens one by one if possible

Or is this intended?

cc @OlivierDehaene

Below are a series of dialogues between various people and an AI assistant. The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable. The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed. It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer. That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.
-----
<current prompt>
-----

Is this something we want to do ASAP @julien-c @gary149 ?

Sometimes it shows when it shouldn't (nothing to scroll).

Currently we call a specific endpoint to abort a generation.

But ideally, the frontend would abort their request, and it'd be detected by the backend.

Maybe this would help: https://kit.svelte.dev/docs/adapter-node#custom-server

It's possible to support No-JS elegantly code-wise (may need users to Ctrl+Enter rather than Enter)

Eg https://kit.svelte.dev/docs/form-actions

Low priority though