Provide an option to use either selected text in the browser or text copied to the clipboard (necessary for using Google docs etc) about lumos

[andrewnguonly]

Thanks for the suggestion @nausher!

I've been meaning to find a workaround for the issue you're describing (i.e. parsing content from Google docs, etc). Even the Highlighted Content feature does not work as expected. Let me see what I can come up with.

[sublimator]

Some info, though perhaps stale:
https://stackoverflow.com/questions/75019646/how-to-get-selected-text-in-google-docs-with-javascript

[andrewnguonly]

Ability to load clipboard content as an attachment will released in v1.0.15.

[andrewnguonly]

A couple of things I noted - (1) Clicking the 'paper clip icon' immediately opens the file dialog box, perhaps a different icon for clipboard history can be used?

I'll keep iterating on the UI/UX for this feature. Thanks for the feedback! One of the challenges with this feature is that browsers cannot access clipboard content without explicit action from the user. I haven't found a way to optionally render a UI component based on if clipboard content is available. Ideally, I'd like to do that so the UI isn't cluttered with more buttons. I may need to remove some of the existing buttons.

(2) When I copy the content from the Lumos chat pop-up, the text seems to be rich and I get the background yellow copied over too. If this could be a setting or a configuration, there could be an option to copy only the plain text without any formatting from the Lumos window.

Created an issue for this: #181. In the meantime, the cmd + c shortcut copies the last AI message and it doesn't include the yellow background.

[sublimator]

I've noticed similar issues with Google Docs, though never dove in to see exactly what it is. Is it using a lot of nested iframes?

No, actually, I have a vague memory that the window.getSelection() api was not working. Perhaps they implement their own editor control and selection geometry system?

[andrewnguonly]

I looked at it briefly a while back and I think they implemented some kind of custom editor. Other tools like Grammarly are able to access the content (seemingly without access to Google APIs), so there must be some way to access it. I'll need to investigate.

[nausher]

Would a simpler fix (hack) be to provide a checkbox option to use what's in the clipboard similar to "Disable content parsing".
The user could then just copy the content of the Google doc to their clipboard, the content can be then chunked or truncated depending on the context length and passed to Ollama.

[sublimator]

https://workspaceupdates.googleblog.com/2021/05/Google-Docs-Canvas-Based-Rendering-Update.html

[andrewnguonly]

Would a simpler fix (hack) be to provide a checkbox option to use what's in the clipboard similar to "Disable content parsing". The user could then just copy the content of the Google doc to their clipboard, the content can be then chunked or truncated depending on the context length and passed to Ollama.

I'll consider this. However, I prefer to avoid having too many input components cluttering the UI, which is already pretty cluttered IMO.

[andrewnguonly]

I think a reasonable product experience is to treat clipboard content like an attachment. If the attachment/clipboard content is present, then those contents will take precedence over the web page's content for parsing. A user must explicitly remove the attachment/clipboard content to resume parsing the web page's content.

This feels simple to understand given the attachment functionality already exists. The Disable content parsing flag will remain as is. If it's checked, Lumos will skip parsing the clipboard content.

[nausher]

Hi, I updated to the 1.0.15 version but I'm not seeing a attach file option.  Please see screenshot attached. 

[andrewnguonly]

Hi, I updated to the 1.0.15 version but I'm not seeing a attach file option.  Please see screenshot attached.

The sideways paperclip icon is the button to attach a file. When you attach a file, another icon will appear that lets you remove the attachment.

However, you don't need to click the button to load clipboard content. Just copy text to your clipboard from any app (i.e. cmd + c), open Lumos, and press the cmd + b shortcut key. Now the clipboard content is loaded. You should see the same icon that lets you remove the attachment, but instead it will be for removing the clipboard content.

[nausher]

Thanks, I finally was able to get it working after a few attempts. 
A couple of things I noted - (1) Clicking the 'paper clip icon' immediately opens the file dialog box, perhaps a different icon for clipboard history can be used?
(2) When I copy the content from the Lumos chat pop-up, the text seems to be rich and I get the background yellow copied over too. If this could be a setting or a configuration, there could be an option to copy only the plain text without any formatting from the Lumos window.

Thanks for quickly fixing this issue. I'm now able to use Lumos in Google Apps.