Coder Social home page Coder Social logo

org-assistant's Introduction

org-assistant.el - License MELPA Version

Org babel extension for Chat Assistant APIs

org-assistant provides support for accessing chat APIs such as ChatGPT in the context of an org notebook.

example

Installation

Org-assistant is available on MELPA

M-x package-install [RET] org-assistant [RET]

Usage

It provides a function named org-assistant that serves as entrypoint for displaying an org assistant buffer. Also, it can be used in any org file by using a src block like #+BEGIN_SRC assistant or #+BEGIN_SRC ?.

The API Key is looked up via org-assistant-auth-function, which has meen tested using the MacOS Keychain. Alternatively, org-assistant-auth-function can be a string and directly set to your API key.

(setq org-assistant-auth-function "<YOUR_API_KEY>")

Calling org-assistant interactively will generate an org-assistant buffer for you. It can be set to a keybinding for quick use like below:

(global-set-key (kbd "C-x C-o") #'org-assistant)

Conversation Evaluation Rules

  • The org tree is traversed up in order to generate the message list when sending information to the chat endpoint.
  • It will only use messages from the branch of the tree that the block that initiated the request is in.
  • It does not include example blocks or source blocks that appear later in the org buffer than the initiating block.
  • noweb support is enabled for all blocks in the conversation based on the initiating block having the :noweb flag set.
  • Example blocks are treated as being responses from the assistant by default if they occur after user messages.
  • If the example block is before any user source block, they are treated as system messages to the assistant instead.

See org-babel-execute:assistant for more details.

Example

* Chat User Question
#+BEGIN_SRC ?
Hi
#+END_SRC

AI Response
#+BEGIN_SRC assistant :sender assistant
Hello! How can I assist you today?
#+END_SRC

When the output is set to png file, the image generation APIs are called instead.

* Image Generation User Question
#+BEGIN_SRC ? :file sphere.png
Generate a sphere
#+END_SRC

AI Response
#+RESULTS:
file:sphere.png

You can introspect the sent conversation using the :echo flag.

* Branching Echo
#+BEGIN_SRC ?
This is the user.  Repeat verbatim only: "This is the system"
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant
"This is the system"
#+END_SRC

** Branch A
#+BEGIN_SRC ? :echo
Response A
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant
(user . "This is the user.  Repeat verbatim only: \"This is the system\"")
(assistant . "\"This is the system\"")
(user . "Response A")
#+END_SRC

** Branch B
#+BEGIN_SRC ? :echo
Response B
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant
(user . "This is the user.  Repeat verbatim only: \"This is the system\"")
(assistant . "\"This is the system\"")
(user . "Response B")
#+END_SRC

Comparison With Other AI Packages

org-assistant.el and org-ai.el

  • org-ai.el is focused more on runtime interaction with AI
  • org-assistant.el is focused more on reproducible sessions via org babel
  • org-assistant.el supports branching conversations
  • org-assistant.el is not meant to be used downstream as a library for AI endpoint interactions.
  • In org-assistant.el, all interaction is async using org-babel, which allows for notebook style prompt development
  • In org-ai.el, interaction is synchronous and inline, which is better for in-editor use cases
  • org-ai.el supports a lot of other AI use cases like text to speech

org-assistant.el and gptel

Feel free to add a pull request detailing the differences if there is a package I missed

Commands

org-assistant-setup

Set up optional libraries like ‘markdown-mode’ to work with ‘org-assistant’.

If ‘markdown-mode’ is available, it will be used.

org-assistant

Prompt the user for an initial prompt for the assistant.

Then display a window with the buffer containing the response.

org-assistant--execute-curl-shell-command-request

Execute a curl shell command for ‘org-assistant’. URL is the endpoint called from ‘org-assistant’. METHOD is the Http method used in the request. REQUEST-ID is the request id, used for debugging purposes. BUFFER is the buffer that the process should output to HEADERS are the headers to send in the request BODY is a JSON object encoded as a string.

(fn &key URL METHOD REQUEST-ID BUFFER HEADERS BODY)

org-babel-execute:assistant

Execute an ‘org-assistant’ in an org-babel context.

PARAMS is used to enable noweb mode. If :echo is set, return the conversation that would be sent to the endpoint instead of evaluating.

If :list-models is set, the ‘org-assistant-models-endpoint’ will be called instead.

:params can be set to a list like ’((max_tokens . 1) (stop . "stop")).

See https://platform.openai.com/docs/api-reference/chat/create for parameters.

TEXT must be empty if :list-models is set.

This is intended to be called via org babel in a src block with Ctrl-C Ctrl-C like:

#+BEGIN_SRC assistant
Hi
#+END_SRC

The response from the assistant will be in the example block following:

#+BEGIN_SRC assistant :sender assistant
Response
#+END_SRC

All of the messages that are in the same branch of the org tree are included in the request to the assistant.

* Question
#+BEGIN_SRC assistant
Hi
#+END_SRC

#+BEGIN_SRC assistant :sender assistant
Response
#+END_SRC

#+BEGIN_SRC assistant
What’s up?
#+END_SRC

Running babel on the second assistant block will send the conversation:

User: Hi
Assistant: Response
User: What’s up?

Running babel on the first assistant block will only include the messages before it:

User: Hi

Only messages in the same branch will be included:

* Question
#+BEGIN_SRC assistant
Hi
#+END_SRC

#+BEGIN_SRC assistant :sender assistant
Response
#+END_SRC
** Branch A
#+BEGIN_SRC assistant
Branch A
#+END_SRC

#+BEGIN_SRC assistant :sender assistant
Branch A Response
#+END_SRC

** Branch B
#+BEGIN_SRC assistant
Branch B
#+END_SRC

#+BEGIN_SRC assistant :sender assistant
Branch B Response
#+END_SRC

If you ran Ctrl-C Ctrl-C on Branch B’s src block the conversation sent to the endpoint would be:

User: Hi
Assistant: Response
User: Branch B
Assistant: Branch B Response

‘org-assistant’ also supports image generation. If the :file attribute is set, the image API will be used.

The following is an example of using the image endpoint:

#+BEGIN_SRC assistant :file output.png
An image of the GNU mascot
#+END_SRC

org-babel-execute:?

See ‘org-babel-execute:assistant’.

ARGS is routed as is.

org-assistant-explain-function

Ask the assistant to explain the function at point.

org-assistant-write-docstring

Ask the assistant to generate a docstring for the function at point.

Customization

Known keys are: temperature top_p n stream stop max-tokens presence-penalty frequency-penalty logit-bias user Should be a alist like '((max_tokens . 10) (user . "emacs")).

This can be overriden on a per-src block basis by specifying the :params argument.

See https://platform.openai.com/docs/api-reference/chat/create for reference.

#+BEGIN_SRC assistant :params '((max_tokens . 10) (user . "emacs"))
Hi
#+END_SRC

Known allowed keys are: size user Should be a alist like '((size . "1024x1024") (user . "emacs")).

This can be overriden on a per-src block basis by specifying the :params argument.

See https://platform.openai.com/docs/api-reference/chat/create for reference.

#+BEGIN_SRC assistant :params '((max_tokens . 10) (user . "emacs"))
Hi
#+END_SRC

Contributing

Contributions welcome, but forking preferred. I plan to actively maintain this, but I will be prioritizing features that impact me first.

I'll look at most pull requests eventually, but there is no SLA on those being accepted.

Also, I will only respond to pull requests on a case by case basis. I have no obligation to comment on, justify not accepting, or accept any given pull request. Feel free to start a fork that has more support in that area.

If there's a great pull request that I'm slow on accepting, feel free to fork and rename the project.

org-assistant's People

Contributors

spenczar avatar tyler-dodge avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

emacsmirror swannedlakee spenczar

org-assistant's Issues

Doesn't support indented org blocks

Hello!

Currently, org-assistant fails silently if the src block is indented. Examples:

Fails:

   #+BEGIN_SRC assistant
   foo
   #+END_SRC

Works:

#+BEGIN_SRC assistant
bar
#+END_SRC

I think having indented SRC blocks is valid org syntax. The view src bind still works on them, exporting them works, and they even get the text indented to match the indent of the block whenever you exit their src edit mode.

The reason why I ran into this issue is because I'm using org-modern, and it doesn't display any leading spaces in src blocks. Org-assistant failed silently on those blocks because they had that leading space.

Thoughts?

Thanks!

Evaluating while queries are in queue

Hello!

It appears that when executing an org-block multiple times while another is being streamed in, the one being executed multiple times will generate multiple replies, instead of cancelling earlier invocations.

Example:

#+BEGIN_SRC assistant
Write a 50 word poem.
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant :stream-id 241f9d18-6034-4220-bb8f-a19808985674
Sipping coffee,
Watching life go by.
Savoring moments,
Life's little highs.
From the delicate chirping,
Of the birds up high,
To the bustling street sounds,
Of
#+END_SRC

#+BEGIN_SRC assistant
Write a 50 word poem.
#+END_SRC

#+RESULTS:
b5eab1b2-d1a3-4d35-81d5-b3d23a77ae35
466b8920-3c3d-4403-b88e-dc15ae31cf2e
a1379ef7-d25a-49ad-8658-77127dc16260
2bee4a15-aeac-40c5-9d7e-bea9d1965bdf
c424d095-dc1c-460a-a68b-946ba4f6ba2b
5b66f48b-4cd5-44c1-b69b-9225fdfc32dc

I executed the last block 6 times while the first block was being written to.

First Impressions

  1. Using the tree structure is great.
  2. Using ‘?’ feels invasive. This should be behind a flag that is by default disabled.
  3. The model used for the request must be specifiable using a header argument of the source block and also a heading property that sets the default model fir all subtrees.
  4. The answer must also be a source block, as you can complete an assistant message, too!
  5. The message type must be a header argument of the source block.
  6. Image generation must use a different lang name, not assistant. It should also support specifying the model used.
  7. What’s the backend? Ideally must be use a modular backend that supports different models including local.
  8. ♥️

Cancel all tasks

Is there a way to cancel all running tasks?

Sometimes I have started many and delete the buffer they are running in, for instance, meaning that all tasks will be run, but trigger errors as they are completed one by one.

Relating to that, it would make sense to add the cancel all tasks function to the kill-buffer hook of the buffer you run it in.

Enable markdown mode by default?

org-assisnant's source code blocks are not associated with any particular major mode, currently. That can be changed, though. I have this in my init.el:

(add-to-list 'org-src-lang-modes '("?" . markdown))
(add-to-list 'org-src-lang-modes '("assistant" . markdown))

This makes it so that I'm using markdown-mode when within the block. This has a bunch of nice effects, but a very nice one is fontification. When combined with (setq markdown-fontify-code-blocks-natively t), this gets very nice indeed:
image

I can see arguments against this:

  • markdown-mode is, incredibly, not bundled into Emacs. Maybe there is some mailing list thread about this, I don't know. But this would add a dependency, anyway.
  • Isn't org better than markdown? Well, yes, but ChatGPT is pretty bad at responding in properly-formatted org text, and also nested org-within-org is horrible to write, so prompting in markdown is a lot easier.
  • The configuration is pretty easy to do as a user, although it took some hunting to figure out exactly the right way to specify it.

But it's so pleasant to work with, that maybe that outweighs the costs. Or maybe just document it? I don't know - up to you.

Consider splitting package

I believe this package is really two packages in one.

One is the org component that searches org heading trees and such.

The other is a excellent reliable AI query library, with queues among other things.

Perhaps splitting this package in two would be worthwhile?

Thoughts?

Undo and streaming

Hello!

Sometimes I press undo while something is being streamed in.

That irreversibly deletes part of what's been generated, as the text will keep getting streamed in while I try to recover it through redoing.

Is there a solution to this? Perhaps the streaming should be wrapped in with-undo-amalgamate, so that it all gets undone in one go? That might be slightly better, as it's then clear what you just undid.

Another, more pressing problem is that streaming creates a ton of undo points, meaning you have to press undo many times to undo a long stream of text.

Thoughts on this?

Encoding issue when chaining replies

Hello again!

Thank you so much for the fixes! I found another one.

There appears to be an encoding issue when chaining replies.

#+BEGIN_SRC assistant
Give me 5 emojis.
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
🤔😂🙌🌞😍
#+END_EXAMPLE

#+BEGIN_SRC assistant
Give me 5 more
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
\360\237\240\272\360\237\224\260\360\237\230\157\360\237\230\146\360\237\215\211
#+END_EXAMPLE

Preview what will be sent to API

When I have long branching trees of AI conversations, it can be difficult to grasp what exactly will be sent to the API. Especially if you add in noweb into the mix.

It would be useful if there was a way to preview exactly what will be sent, along with what users will send each message.

I have made a small function below that shows me what has been sent to the API, after its already been sent.

(defvar my/org-assistant-debug-buf nil)
(defvar my/org-assistant-debug-buf-name "*org-assistant-debug-buf*")

(defun my/org-assistant-debug-buffer-get ()
  (or (and (buffer-live-p my/org-assistant-debug-buf) my/org-assistant-debug-buf)
      (with-current-buffer (generate-new-buffer my/org-assistant-debug-buf-name)
        (setq my/org-assistant-debug-buf (current-buffer))
        (visual-line-mode)
        (current-buffer))))

(defun my/org-assistant-format (blocks)
  (->> blocks
       (--map (list
               (cons "content" (cdr it))
               (cons "role" (symbol-name (car it)))))
       (vconcat)))

(defun my/org-assistant--debug (fun &rest args)
  (let ((res (funcall fun args)))
    (with-current-buffer (my/org-assistant-debug-buffer-get)
      (erase-buffer)
      (insert
       (format "%S"
               (my/org-assistant-format res)
               )))
    res))

(advice-add #'org-assistant--org-blocks :around #'my/org-assistant--debug)

The UI isn't great, it would be better if the different messages were more clearly distinguished.

Evaluating thread while streaming

Currently, when evaluating blocks that require a parent assistant block that's incomplete and being currently streamed in, the block isn't delayed until all parent blocks in the conversation are finished. Instead, it takes what has been streamed in at the moment of evaluation and sends that to the AI API.

It would make more sense if child blocks waited until all parent blocks were streamed in fully.

Perhaps having this check would be easier if the streaming status was present in the org block src header line, in case the conversation is regarding UUIDs, which might be mistaken for the UUID being used to stream in the content. Ref: #21.

Here's an example where I evaluate the child block before the parent block finishes, leading to an inaccurate reply of '272 words'.

#+BEGIN_SRC assistant
Write me a 500 word poem about emacs
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant
In the world of text editors, few can compare
To the power and flexibility of Emacs software
It's a tool that's been crafted with care
And with each release, it becomes more rare

It's a program that's been written in Lisp
And it's often used to write a script
For those who prefer to work in a terminal
Emacs is the editor that's truly indispensable

It's not just for coding
a956aadf-9939-4f58-81d6-220c35925e5b
#+END_SRC


#+BEGIN_SRC assistant
How many words is the poem you just wrote?
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant
272 words.
#+END_SRC

Confusing error on curl version <7.82.0: "exited abnormally with code 2"

org-assistant generates a curl command that looks like this:

curl https://api.openai.com/v1/chat/completions \
  -H 'Authorization:Bearer <REDACTED>' \
  -H 'Content-Type:application/json; charset=utf-8' \
  -X POST \
  --json @/var/folders/kg/jqdn2j5s4n3f4y9_1swlftrm0000gn/T/jsonsZglKi

The --json argument was added in v7.82.0 from March 5, 2022 (https://curl.se/mail/lib-2022-03/0011.html).

If the local version of curl predates that, you get a very confusing output:

* Question
#+BEGIN_SRC ?
hi
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
(error "exited abnormally with code 2
")
#+END_EXAMPLE

#+BEGIN_SRC ?

#+END_SRC

(this is with curl 7.79.1 from 2021-09-22)

It turns out that replacing --json with --data (no other changes!) works on my version of curl, 7.79.1, since the curl command is already setting the content-type header. So changing the command to call --data might be an easy fix to be compatible with older curl versions. Of course, upgrading cURL is what I'll do in the short term :)

Clarification: How do you send System messages to the Assistant?

Hi, Really cool plugin!

One thing that I dont get is how you send system messages to the assistant.
e.g.

{
  "messages": [
    {
      "role": "system",
      "content": "Act as an elisp developer"
    } 
 ]
}

You write something about example blocks before any user input being interpreted as system messages, is this what you are referring to?

Does that include ALL example blocks not following a src block?

Warning on latest commit

Hi!

I get this warning whenever I call an org-assistant block on the latest commit.

WARNING: Callback that takes no argument may be specified. Passing callback with no argument is deprecated.
Callback must take one argument. Or, this error is coming from somewhere inside of the callback: %S

I don't get a backtrace. I will see if I can extract some more information later.

Progress/status indicator

When the API returns long replies, there can be long periods of waiting where I don't know whether its taking so long because my internet went down, or because the reply is really long.

It would be useful if there were some progress or status indicator to each job displayed in the mode line.

Bug related to freezing prompt & cancelling blocks

Hello!

Sometimes when I use this package on an unreliable network, the prompt will simply freeze without explanation.

What's more, when I execute the block again, the overlay jumps up to the character at the start of the org block and stays there while a new prompt is created below. Usually when this happens, the new overlay will freeze as well, and the only way to fix it is to kill the buffer.

I'm not sure how to debug this issue. Perhaps there should be an interactive function that displays all the relevant state and pretty prints them for bug reports and diagnostics.

Here's an image of one of my src blocks experiencing the issue:
org-assistant

The _ before the src block is the original overlay. The reason why it doesn't say BEGIN_SRC is because I'm hiding that part through org-modern

Org blocks aren't properly escaped

Hello again!

It appears that the output returned by the assistant isn't being escaped so that it won't break the org block it's written in.

For instance:

#+BEGIN_SRC assistant
Write back a sample org file with a single heading.
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
* My Heading

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce eget ante eu elit facilisis dignissim. Curabitur orci sapien, interdum ac odio vel, porttitor pulvinar lectus. Nam ullamcorper metus vitae libero blandit, vel dapibus nisi consectetur. Proin pretium dolor et velit bibendum, ut facilisis neque tincidunt. Mauris gravida quam eu ex euismod consequat. Duis bibendum cursus libero, ut ullamcorper justo placerat ac.

Ut elit elit, vestibulum id tellus id, tempus hendrerit orci. Nullam bibendum euismod lorem eu varius. Aliquam dictum semper mauris, ac aliquam justo bibendum ut. Suspendisse potenti. In hac habitasse platea dictumst. Integer euismod euismod faucibus. In hac habitasse platea dictumst. Aliquam erat volutpat. Donec quis augue nec ante bibendum maximus.

Duis vel erat mi. Vivamus vel purus id lectus suscipit sagittis. Nulla et quam sagittis, feugiat tortor vel, consequat tellus. Duis rutrum pulvinar nibh id gravida. Suspendisse potenti. Sed nec quam quis metus lobortis sodales in nec ipsum. Integer lobortis arcu a magna vestibulum pharetra. Nam dictum turpis non erat auctor ullamcorper. Suspendisse non magna ligula. Sed quis felis arcu.
#+END_EXAMPLE

Anything after * My Heading won't be inside an org-block, as the heading ends the block statement.

There should be a function that sanitizes a string for insertion into an org block.
In the case of the heading, a comma is used: .* My Heading

Errors when adding to queue

I keep getting strange errors when evaluating multiple blocks at the same time.

It might be because I have org-assistant-parallelism set to 3.

Here's the backtrace I got:

Debugger entered--Lisp error: (error "Not currently in a code block")
  error("Not currently in a code block")
  org-babel-goto-src-block-head()
  org-assistant--org-src-arguments()
  org-assistant-cancel-block-at-point()
  org-babel-execute:assistant("Back of book blurb: Listen close..." ((:colname-names) (:rowname-names) (:result-params "replace" "raw") (:result-type . value) (:results . "replace raw") (:exports . "code") (:session . "none") (:cache . "no") (:noweb . "yes") (:hlines . "no") (:tangle . "no")))
  org-babel-execute-src-block(nil ("assistant" "\nBack of book blurb: Listen close..." ((:colname-names) (:rowname-names) (:result-params "raw" "replace") (:result-type . value) (:results . "raw replace") (:exports . "code") (:tangle . "no") (:hlines . "no") (:noweb . "yes") (:cache . "no") (:session . "none")) "" nil 7921441 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)

I will try to reproduce it reliably.

Noweb doesn't work for system prompts

Hello!

When running system prompts, it appears that noweb references aren't expanded.

Works:

#+NAME: FOO
#+BEGIN_SRC text
bar
#+END_SRC

#+BEGIN_SRC assistant :echo t
<<FOO>>
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
(user . "bar")
#+END_EXAMPLE

Doesn't work:

#+NAME: FOO
#+BEGIN_SRC text
bar
#+END_SRC

#+BEGIN_SRC assistant :sender system :echo t
<<FOO>>
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
(system . "<<FOO>>")
#+END_EXAMPLE

Thanks for any help!

Set model used per-block

Hello!

I can't seem to set the model per block. Neither of these appear to work:

#+BEGIN_SRC assistant :params '((model . "gpt-4"))

#+BEGIN_SRC assistant :model "gpt-4"

Instead, it seems my only option is to set it globally or buffer locally like this: (setq org-assistant-model "gpt-4")

It would be useful to have the ability to set it by block, as not all of my AI queries need GPT-4.

Thanks!

Running block multiple times quickly

Hello again!

When running a block multiple times in quick succession, multiple results under the result tag are created:

#+BEGIN_SRC assistant
1+1
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
2
#+END_EXAMPLE
#+BEGIN_EXAMPLE
As an AI language model, I can give you the answer that 1+1 equals 2.
#+END_EXAMPLE

If I now evaluate the block again, the lower example block stays:

#+BEGIN_SRC assistant
1+1
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
As an AI language model, I can give you the answer that 1+1 equals 2.
#+END_EXAMPLE
#+BEGIN_EXAMPLE
As an AI language model, I can give you the answer that 1+1 equals 2.
#+END_EXAMPLE

While, if I had waited a bit between each invocation, it would have overridden the previous block and looked like this.

#+BEGIN_SRC assistant
1+1
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
As an AI language model, I can give you the answer that 1+1 equals 2.
#+END_EXAMPLE

I think the most logical behavior for when you run a block that's already running is to cancel the current job and start up a new job.

That would also be very useful if you have a long-running prompt that you noticed you made a mistake in while it's running.

Streaming id location

Hello!

I have been trying out the streaming branch, and it's excellent so far.

One minor issue I have with it is that it puts the ID of the block that's being streamed in at the end of the source block, inside the contents.

It might be better to put it inside the SRC header line, so it's not present in the contents.

That would make it easier to copy the text while it's being streamed in, especially when using functions that copy the entire block contents, excluding the header arguments and such.

For instance, it could look something like this:

#+BEGIN_SRC assistant :sender assistant :stream-id ...

The block id would then be removed when the stream is completed, just as it is at the moment.

Perhaps there are some downsides to this. Thoughts?

Accessing text field while it's being streamed in

Hello!

The overlays for streaming are working brilliantly so far, however, due to the way overlays work, the cursor can't enter them, and so text can't be copied while content is being streamed in.

I wonder if there are any overlay properties that can be set to allow this, but I doubt it, as I believe overlays occupy a single character in the buffer.

Thoughts?

Noweb bug

Hello!

There is some strange noweb bug occurring when I use multiple lines in the noweb snippet and put the noweb snippet marker on a non-blank line.

It might only be present in the latest streaming branch. That's the only commit I have tried this on.

Examples:

No issue when the snippet marker is on its own line:

#+NAME: Test
#+BEGIN_SRC text
1
+ 1
#+END_SRC

#+BEGIN_SRC assistant :echo
What's
<<Test>>
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
(user . "What's
1
+ 1")
#+END_EXAMPLE

Bug occurs when it's moved to a non-empty line:

#+NAME: Test
#+BEGIN_SRC text
1
+ 1
#+END_SRC

#+BEGIN_SRC assistant :echo
What's <<Test>>
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
(user . "What's 1
What's + 1")
#+END_EXAMPLE

Too much is escaped

Hello!

It appears that all star characters that are printed gets escaped with a comma.

However, only stars at BOL should be escaped, as those are the only ones that could be org syntax.

Example:

#+BEGIN_SRC assistant
Give me examples of the markdown star character in use
#+END_SRC

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant
1. Emphasis: `_This text will be italicized_.`
2. Strong emphasis: `,**This text will be bold,**`.
3. Strikethrough: `~~This text will be crossed out~~`.
4. Unordered list:

See that * gets escaped as ,*, thereby breaking Markdown syntax even if you were to enter the block.

system prompt and tuning parameters

Thanks for this package; the approach to iteration is really interesting and helpful for exploration. Something I miss from org-ai is the ability to set the system-prompt and to tune the model manually. Do you have plans to add defcustoms for these parameters? Just wondering. Thanks again.

Evaluating blocks fail silently

Hello!

Sometimes when evaluating too many blocks at the same time (perhaps around 5 concurrently), some blocks will fail and return an empty SRC block.

The result will usually look like this:

#+BEGIN_SRC assistant
foo
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE

#+END_EXAMPLE

I suspect this is due to OpenAI being rejecting requests when there are too many of them at the same time.

It would be better if an error of some sort was triggered instead, and if you could limit and queue the amount of jobs, perhaps.

I imagine this being a problem, especially if you are evaluating them all when tangling an org file.

OpenAI bullet pointed lists aren't formatted properly

It appears as if OpenAI has some special formatting when sending lists.

The items all appear on the same line:

#+BEGIN_SRC assistant
Make a bullet point list of the 10 most common names in the US
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
Here is a list of the 10 most common names in the US, based on data from the
Social Security Administration:

1. Emma 2. Olivia 3. Noah 4. Liam 5. Ava 6. Sophia 7. Isabella 8. Jackson 9.
Lucas 10. Aiden

Note: This list is based on data from 2020 and may change slightly year to year.
#+END_EXAMPLE

I'm using the latest commit: aac1ce6

SRC block escaped by org syntax

Hello!

EDIT: This issue has re-appeared when using streaming: #4

Running:

#+BEGIN_SRC assistant
Give me org-mode headings
#+END_SRC

results in:

#+RESULTS:
#+BEGIN_SRC assistant :sender assistant#+BEGIN_SRC assistant :sender assistant#+BEGIN_SRC assistant :sender assistant#+BEGIN_SRC assistant :sender assistant#+BEGIN_SRC assistant :sender assistant#+BEGIN_SRC assistant :sender assistant
 Level Heading
#+END_SRC
 Level Heading
***** Fifth
#+END_SRC
 Level Heading
**** Fourth
#+END_SRC
 Level Heading
*** Third
#+END_SRC
 Level Heading
** Second
#+END_SRC
* Top
#+END_SRC

Doesn't work with emojis

Thanks so much for this package! The branching feature is amazing!

There appears to be some encoding issues when printing emojis.

Example:

#+BEGIN_SRC assistant
Print a few emojis
#+END_SRC

#+RESULTS:
#+BEGIN_EXAMPLE
�����������
#+END_EXAMPLE

#+BEGIN_SRC ?

#+END_SRC

noweb support

It appears that noweb isn't supported when running the blocks.

What's more, the text block named 'example' here gets sent to OpenAI when you put it above the prompt, which can be a problem. Maybe there should be a way to designate blocks to not be included in the request?

#+NAME: bar
#+BEGIN_SRC text
foo
#+END_SRC

#+BEGIN_SRC assistant :noweb yes
Read back to me the following text:
<<bar>>
#+END_SRC

Streaming output

gpt-4 is very slow to generate large output, which is painful when using org-assistant. It can take a few minutes to receive a full response - but you get no feedback, just waiting.

ChatGPT supports streaming output to help with this. org-assistant could stream results and write into the buffer one token at a time.

Turning on the behavior for the API is easy; the hard part is parsing the results and handling them token-by-token. To enable the API, the client would need to set "stream": true (at the top level of the JSON payload, next to "model": "gpt-3.5-turbo" for example). Curl would need to be invoked with --no-buffer to avoid any internal buffering.

Then, the SSE stream would need to be parsed and each datagram would need to be handled. The SSE spec is relatively simple: data messages are sent as lines prefixed withdata:. A blank line separates data messages. In the OpenAI implementation, each data message is either a JSON object which describes a single token to insert, or the exact string "[DONE]" to indicate termination.

I think handling the SSE stream would require a change in org-assistant's implementation away from using set-process-sentinel, which fires just once when the curl process exits, to using accept-process-output to do incremental reads from stdout. It could get hairy, I'm not sure.

Here is an example of what the curl output looks like, for reference, when streaming:

$ curl https://api.openai.com/v1/chat/completions -H 'Authorization:Bearer <REDACTED>' -H 'Content-Type:application/json; charset=utf-8' -X POST --data @./gpt4.json --no-buffer
data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"I"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":" am"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":" powered"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":" by"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":" Open"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"AI"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"'s"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":" G"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"PT"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"-"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"3"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{"content":"."},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-7CZ5sgsMjO6HZuAgFDj2AZEb1ZfIG","object":"chat.completion.chunk","created":1683229192,"model":"gpt-4-0314","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]}

data: [DONE]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.