SBCL warnings seem to break the REPL connection about vlime

[belaja-akacija]

Getting this as well.
However, I don't even need an error to be thrown at me in order for this to issue to crop up.
It just decides to randomly not show output other than "--" after a certain point.
I am running Neovim on Arch, using Packer as my package manager, all latest version.

[jam1015]

getting this on arch too

[phmarek]

Hmmm, I can't reproduce. Which vlime (version, git commit?) and which swank (in the slime QL) are you using?

[chrisjdavies]

vlime version (installed and managed with vim-plug):

chris@chris-air vlime % pwd
/Users/chris/.vim/plugged/vlime
chris@chris-air vlime % git log
commit 3205f02306314ab8cfc9034cf72097891c923e9d (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Philipp Marek <[email protected]>
Date:   Tue Nov 9 05:56:38 2021 +0000

    Merge pull request #68 from ab-10/patch-1
    
    Update SBCL version

SWANK version shows as 2.28 in the vlime REPL buffer.

[phmarek]

Well, these are quite old versions.
Would you please update vlime and swank and try again?

The matching swank is https://github.com/vlime/slime.

Thanks!

[chrisjdavies]

:PlugUpdate tells me this is the latest version of vlime - and GitHub seems to agree? Double-checked and I'm definitely on master, which seems to be correct.

For SWANK, I think vlime is using the quicklisp one, right? I seem to be using the latest of that:

* (ql:update-dist "quicklisp")
You already have the latest version of "quicklisp": 2023-02-15.
NIL
* (ql:quickload 'swank)
To load "swank":
  Load 1 ASDF system:
    swank
; Loading "swank"

(SWANK)

[phmarek]

Would you please try a new version? The direct-json branch from https://github.com/vlime/vlime, and master from https://github.com/vlime/slime?

Thanks.

[chrisjdavies]

Checked out that branch, but now I'm getting:

#<THREAD "main thread" RUNNING {70087A06B3}>:
  READ error during LOAD:

    Package SWANK does not exist.

      Line: 44, Column: 36, File-Position: 1236

      Stream: #<SB-INT:FORM-TRACKING-STREAM for "file /Users/chris/.vim/plugged/vlime/lisp/src/vlime.lisp" {7006F155A3}>

It's definitely installed with quicklisp though, so I'm not sure why it's not resolving.

For the slime repo, where am I checking that out to for SBCL/vlime to find it?

Worth adding, I tried doing a manual load through SBCL by first (ql:quickload 'swank) and then (load "~/.vim/plugged/vlime/lisp/start-vlime.lisp") -- which gets further, but then hits this instead:

* (ql:quickload 'swank)
To load "swank":
  Load 1 ASDF system:
    swank
; Loading "swank"
.
(SWANK)
* (load "~/.vim/plugged/vlime/lisp/start-vlime.lisp")
While evaluating the form starting at line 33, column 0
  of #P"~/.vim/plugged/vlime/lisp/start-vlime.lisp":

debugger invoked on a SB-INT:SIMPLE-PROGRAM-ERROR @7005A74B10 in thread
#<THREAD "main thread" RUNNING {70087906B3}>:
  invalid number of arguments: 1

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [REPLACE-FUNCTION     ] Call a different function with the same arguments
  1: [CALL-FORM            ] Call a different form
  2: [CHOOSE-DIFFERENT-PORT] Choose a different port
  3: [RETRY                ] Retry EVAL of current toplevel form.
  4: [CONTINUE             ] Ignore error and continue loading file "/Users/chris/.vim/plugged/vlime/lisp/start-vlime.lisp".
  5: [ABORT                ] Abort loading file "/Users/chris/.vim/plugged/vlime/lisp/start-vlime.lisp".
  6:                         Exit debugger, returning to top level.

((LAMBDA (VLIME::ADDR VLIME::PORT) :IN VLIME:MAIN) 7002) [external]
   source: (LAMBDA (ADDR PORT) (FORMAT T "Server created: (~a ~a)~%" ADDR PORT))

[equwal]

Also a problem in neovim

[equwal]

I was able to load up vlime manually in SBCL, but it makes no difference for the output buffer bug.

[equwal]

I managed to downgrade my swank and get it to work so this is a regression

[equwal]

This is a sufficient test case:

(defun foo ())
; FOO
(defun foo ())
; FOO (or nothing -- failed)

[equwal]

first bad commit is 78ad57b7455be3f34a38da456183ddf8d604bdf8

commit 78ad57b7455be3f34a38da456183ddf8d604bdf8
Author: Stas Boukarev <[email protected]>
Date:   Mon Nov 28 00:07:53 2022 +0300

    Handle a lot of output more responsively.

    Wait for Emacs to finish processing output, otherwise on continuous
    output its input buffer will fill up and nothing else will be
    processed, most importantly an interrupt-thread request.

 contrib/slime-repl.el   | 4 +++-
 contrib/swank-repl.lisp | 6 +++++-
 swank.lisp              | 3 ++-
 swank/gray.lisp         | 2 +-

[equwal]

Git apply my patch y'all

From 9f4ed641c3117efe71f9015afa33a26331bc457a Mon Sep 17 00:00:00 2001
From: Spenser Truex <[email protected]>
Date: Fri, 26 May 2023 18:52:35 -0300
Subject: [PATCH] Revert commit that broke vlime's output buffer

While this might be more responsive, it broke vlime. I left the buffer
size increase in though.

Test case was:
start vlime
load (defun foo ()) ; FOO
load (defun foo ()) again
did vlime output?: ; FOO
---
 contrib/slime-repl.el   | 4 +---
 contrib/swank-repl.lisp | 6 +-----
 swank.lisp              | 3 +--
 swank/gray.lisp         | 4 +++-
 4 files changed, 6 insertions(+), 11 deletions(-)

diff --git a/contrib/slime-repl.el b/contrib/slime-repl.el
index d0d48831..03c726eb 100644
--- a/contrib/slime-repl.el
+++ b/contrib/slime-repl.el
@@ -1728,10 +1728,8 @@ expansion will be added to the REPL's history.)"
 
 (defun slime-repl-event-hook-function (event)
   (slime-dcase event
-    ((:write-string output &optional target thread)
+    ((:write-string output &optional target)
      (slime-write-string output target)
-     (when thread
-       (slime-send `(:write-done ,thread)))
      t)
     ((:read-string thread tag)
      (cl-assert thread)
diff --git a/contrib/swank-repl.lisp b/contrib/swank-repl.lisp
index f6db28b9..259c9ea3 100644
--- a/contrib/swank-repl.lisp
+++ b/contrib/swank-repl.lisp
@@ -122,11 +122,7 @@ DEDICATED-OUTPUT INPUT OUTPUT IO REPL-RESULTS"
   "Create function to send user output to Emacs."
   (lambda (string)
     (with-connection (connection)
-      (send-to-emacs `(:write-string ,string nil ,(current-thread-id)))
-      ;; Wait for Emacs to finish writing, otherwise on continuous
-      ;; output its input buffer will fill up and nothing else will be
-      ;; processed, most importantly an interrupt-thread request.
-      (wait-for-event `(:write-done)))))
+      (send-to-emacs `(:write-string ,string)))))
 
 (defun open-dedicated-output-stream (connection coding-system)
   "Open a dedicated output connection to the Emacs on SOCKET-IO.
diff --git a/swank.lisp b/swank.lisp
index 07af0954..162ffb7e 100644
--- a/swank.lisp
+++ b/swank.lisp
@@ -1010,8 +1010,7 @@ The processing is done in the extent of the toplevel restart."
           &rest _)
          (declare (ignore _))
          (encode-message event (current-socket-io)))
-        (((:emacs-pong :emacs-return :emacs-return-string :ed-rpc-forbidden
-           :write-done)
+        (((:emacs-pong :emacs-return :emacs-return-string :ed-rpc-forbidden)
           thread-id &rest args)
          (send-event (find-thread thread-id) (cons (car event) args)))
         ((:emacs-channel-send channel-id msg)
diff --git a/swank/gray.lisp b/swank/gray.lisp
index 162e344a..6a3794c4 100644
--- a/swank/gray.lisp
+++ b/swank/gray.lisp
@@ -39,9 +39,11 @@
 
 (in-package swank/gray)
 
+(defconstant +string-maximum+ 64000)
+
 (defclass slime-output-stream (fundamental-character-output-stream)
   ((output-fn :initarg :output-fn)
-   (buffer :initform (make-string 64000))
+   (buffer :initform (make-string +string-maximum+))
    (fill-pointer :initform 0)
    (column :initform 0)
    (lock :initform (make-lock :name "buffer write lock"))
-- 
2.35.1

[gajon]

@equwal I've applied your patch and the vlime repl buffer is working fine now. Thank you!

[phmarek]

ac21427 might help for the non-JSON version as well - that sends the :WRITE-DONE reply.

@chrisjdavies I've got a symlink to the vlime checkout in my quicklisp/local-projects/ directory, and the vlime/swank checkout is in there too. Shall we find out what goes wrong?

[equwal]

ac21427 might help for the non-JSON version as well - that sends the :WRITE-DONE reply.

@chrisjdavies I've got a symlink to the vlime checkout in my quicklisp/local-projects/ directory, and the vlime/swank checkout is in there too. Shall we find out what goes wrong?

That's probably the Right Way To Do It with the current status of the latest developments. I made a very cheap hack above:

• I went into quicklisp and deleted swank 2.28 that I had installed with (ql:quickload :swank)
• I remade the directory with a git clone
• I bisected it so I could make that patch
• I checked out 2.28 again
• I applied my patch
• it werks!

I just wanted to hack some CL code, I'm not any good at vimscript. I hope this explanation cleans things up.