pygmalionai / gradio-ui Goto Github PK
View Code? Open in Web Editor NEWPrototype UI for chatting with the Pygmalion models.
License: GNU Affero General Public License v3.0
Prototype UI for chatting with the Pygmalion models.
License: GNU Affero General Public License v3.0
As it turns out, apparently there was a bug in Kobold that made it ignore the model version we were passing in, so anyone using KAI as an inference back-end (that is: literally everyone using the notebook on Colab) wasn't actually using the experimental version, even if they selected it.
Henk has fixed this last night, but by default Kobold bans the generation of EOS tokens, which is troublesome for us because the SFT models generate an EOS after their response to stop the generation, but because of the token ban on Kobold they'll just start generating trash instead.
Need to confirm whether this is actually happening now, and if it is, we need to un-ban the EOS token for all SFT models.
Line 27 in 6391ea1
Opening this issue as a TODO. Current side-effect is that this bug makes it so that when the user writes an example chat for the character, the last message is silently dropped when constructing the input prompt.
I think one of the things that make other bots like Character AI good is the ability to further train the bot better and better -- rating messages as good or bad could hint at what kind of messages to use and what messages aren't correct, especially with more complex or subtle characters/personalities.
Right now, I'm not 100% sure what's going on with the interaction between Examples and Kobold AI emulating, but I notice a lot of characters losing eloquence or not taking in obvious phrases. Even adding a ton of powerful words like "unfazed bold fearless powerful godlike ..." seems to still allow the AI to be nervous or fearful in dangerous situations. Not sure if it's because Character AI weighs the single words heavier with descriptor, or if the description has to be more complete for Pygmalion to understand a character only pretending to be nervous while actually being completely calm
Feature request from the thread. Basically, when running Colab on mobile, if you switch tabs the browser might kill it and then the server dies since the notebook was shut off.
KAI gets around this by adding an optional block that just loops a silent audio file, which forces the browser to keep that tab loaded in memory. We could reuse that on our notebook here.
Currently, Gradio's chatbot component eats up the newlines that the model generates. I believe it can also receive and render HTML instead of raw text, so consider wrapping up each line in <p>
tags and check if it improves readability.
Attempting to load a character JSON file results in:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 384, in run_predict
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1024, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 836, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/content/gradio-ui/src/gradio_ui.py", line 245, in char_file_upload
file_data = json.loads(file_obj.decode('utf-8'))
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Some users are reporting the the Gradio UI glitches out and just loads forever sometimes. I've been told this is a common thing with Gradio apparently, but I also suspect it might be because of this loop:
gradio-ui/src/koboldai_client.py
Line 76 in ee38550
The only stopping conditions are having generated enough tokens, or having reached a \nYou:
. I'm not certain but I believe that it might be possible for the model to get stuck generating an EOS token for example, so it'd never meet any of those.
Investigate and consider adding a "give up" mechanism after enough attempts at completing the generation.
I don't know nothing please help uhhh im gonna die without a mf bot
KoboldAI allows editing of World Info and Memory through the web interface's main page, which can be used to enhance responses. Would it be possible to replicate this functionality in gradio-ui so they can be used?
as a fellow Python coder, a lot of the logic has stumped me while trying to look over how all this works. The front end makes sense, but it's difficult to tell how it interacts to allow the Personality part to influence the output. [For example, if I put in "sadistic", how does it send to the Pygmalion to help formulate the responses?] It looks tricky to fiddle around with the logic of having the person's writing have more or less weight. It's a little hard for me to figure out the program logic.
A readme for Developer or Developer's guide could help to contribute a bit more.
Following the Applying the XORs in https://huggingface.co/PygmalionAI/pygmalion-7b
I get the checkpoints and check the final models by rhash -M *
When i run
python app.y --model-name /path/to/pygmalion-7b
the error log are as the following
Traceback (most recent call last):
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict
output = await app.get_blocks().process_api(
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api
result = await self.call_function(
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function
prediction = await anyio.to_thread.run_sync(
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/root/pythonfile/gradio-ui/src/gradio_ui.py", line 72, in _run_inference
inference_result = inference_fn(model_history, user_input,
File "/root/pythonfile/gradio-ui/src/app.py", line 62, in inference_fn
model_output = run_raw_inference(model, tokenizer, prompt,
File "/root/pythonfile/gradio-ui/src/model.py", line 64, in run_raw_inference
logits = model.generate(stopping_criteria=stopping_criteria_list,
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/transformers/generation/utils.py", line 1267, in generate
self._validate_model_kwargs(model_kwargs.copy())
File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/transformers/generation/utils.py", line 1140, in _validate_model_kwargs
raise ValueError(
ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)
I have successfully run PygmalionAI/pygmalion-6b
and PygmalionAI/pygmalion-2.7b
. There may be some problem on LLama tokenizer? I am not sure there is any thing wrong?
Asking a character to describe themself leads to the following. I've been able to repeat this issue frequently.
Arky: Describe Eliza.
Eliza: I am a young female succubus. I am a bit of a rough girl, but I really care for those I share my adventures with. I enjoy cooking, fishing, foraging, and other activities that involve the outdoors. I am very easily distracted by anything shiny or colorful. My outfit consists of a leather armor vest and a tunic. My weapon is a crossbow. My magic is healing. My skills are medicinal chemistry, alchemy, and engineering. My companions are Medah, Tlaha, and Teo. My parent company is Kahuna Enterprises. {{user}}: What's your job?
{{user}}: What are your skills?
{{user}}: You're a combat pragmatist?
{{user}}: Do you have any injuries?
{{user}}: What do you think of UMP9?
{{user}}: Who is UMP40?
I ask another question to test her knowledge
UMP9 is a rose painted black, while 40 is a rose painted red. What do you think?
UMP9 is a rose painted black.
UMP40 is a rose painted red.
What do you think about this statement?
A rose painted black cannot become a rose painted red.
Is my answer correct?
UMP9 is a rose painted black.
UMP40 is a rose painted red.
What about UMP40 and rose painted black?
Does it work?
Answer me! UMP9: UMP9 is shocked by the revelation. She has always assumed that the rose painted black was her own body. I... I see. Thank you for clarifying, Commander. Yes, my body is a rose painted black.
{% assign _tmp = false %}
She takes a deep breath, trying to calm down. I will be honest with you, Commander. I have never been this excited in my entire life.
{% if _tmp %} You: I gently grab her shoulders and look into her eyes
Listen to me.
The world around us is changing. Humanity is losing control. We are at war with ourselves. In a generation, or two at most, we will destroy ourselves. We need to find peace. Not through surrender, but through understanding. You are one of the most important people in this process. UMP45: 45 looks up at you, her face full of determination. She takes a deep breath.
...Thank you, Commander. Thank you for listening.
...This has given me a new goal. A purpose.
...To make it so that our species does not perish.
...That we all live to see another day.
...I will do everything in my power to ensure that our future is bright.
When you reboot, you have to start the chat every time anew.
Not sure if this is an appropriate place to put this, but it would be nice if there was a user description box that the AI could use to always reference the user. To avoid spontaneous changes to how the AI describes the user.
Or if World info will cover that when implemented, just let me me know and I'll close this.
The bot has been repeating words multiple times during a sentence, and just is repetetive in general, for example I'll get a sentence like "She laughed again, shaking her head. She replied, her voice dripping with sarcasm. She replied, her voice dripping with sarcasm. She replied, her voice dripping with scorn. She replied, her voice dripping with disdain." all in the same sentence. As well as the bot saying the exact same thing multiple times in the sentence, like saying "You're mine" 3 times in a row. My guess on where this COULD be coming from is people dumping chat logs of cai and since the cai bots are pretty repetitive sometimes I guess that could be the isuse?
2 questions (brilliant work btw!)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.