Potential Issue w/ `analyze-last-stacktrace` op about cider-nrepl

[r0man]

Hi @rschmukler,

thanks for opening an issue. I haven't seen this myself so far while working with and on the stacktrace middleware.

I tried to see what is happening when an exception occurs in the handler. I added a (/ 1 0) form at the beginning of the handle-analyze-stacktrace-op and called it with Cider. When this is happening I see the exception in the *nrepl-server*, but my REPL still stays intact.

If this is an issue with the exception analysis, we need to track down which exception is causing this. Do you see anything in the *nrepl-server* buffer when this is happening? Something like ERROR: Unhandled REPL handler exception processing message ...?

If not, could you please try this branch [1] and again look at the *nrepl-server* buffer. If you find anything there, please post it on this issue.

[1] https://github.com/r0man/cider-nrepl/commits/try-catch

[rschmukler]

@r0man thank you so much for taking a look at this. I realize it's a somewhat abstract problem and it may have nothing to do with you; so again, thank you for giving it your attention.

I have a repo / sequence of evaluations in which I can now get it to reliably trigger the error. This enabled me to start adding printlns to everything to see where the error actually arises.

So, as it turns out, there is no exception thrown - instead the function actually freezes.

After a lot of debugging, I have narrowed it down to an interaction between delay, pmap and changing the send-off executor (which is used by pmap) to the (Executors/newVirtualThreadPerTaskExecutor) (a JDK 19 preview feature w/ Loom).

This interaction can be seen with the use of pmap inside analyze-stacktrace-data and the delay used by ns-common-prefix in haystack.

I suspect it might be some weird interaction with haystack being an inlined dependency.

Anyway, I believe I created a repo that can be used for a minimal reproduction. Note that you will need OpenJDK 19 to test it. This likely isn't an issue with haystack or nrepl itself - perhaps something w/ Clojure or even the JDK itself. Let me know if you investigate further or have any ideas on how best to proceed.

Thanks again for your time.

[rschmukler]

A real time update, just in case anyone happens to be looking:

It looks like the issue is the number of nested pmap calls (both within haystack and orchard) once the executor has been changed. Still digging further down the call stack, but I can trigger a freeze w/ minimal reproduction for now, w/ JDK 19 with --enable-preview

(let [n 50]
    (->> (range n)
         (pmap
           (fn [_]
             (->> (range n)
                  (pmap (fn [_]
                          (->> (range n)
                               (pmap (fn [_]
                                       (pmap identity (range n))))))))
             ))
         (into #{})))

[rschmukler]

Probably the final comment w/ regards to cider-nrepl. This looks to be an issue with Project Loom more than anything. As I said above, orchard and haystack make heavy use of pmap but it seems like recursive thread thread spawning can cause loom to lock up pretty easily. I'll see about how to file an issue with them. A minimal reproduction:

(defn ppmap
  [f coll]
  (->> coll
       (mapv
         (fn [x]
           (let [cf (java.util.concurrent.CompletableFuture.)]
             (Thread/startVirtualThread
               (bound-fn []
                 (try
                   (.complete cf (f x))
                   (catch Exception e
                     (.completeExceptionally cf e)))))
             cf)))
       (map deref)))
       
 (let [n 4]
    (->> (range n)
         (ppmap
           (fn [_]
             (->> (range n)
                  (ppmap (fn [_]
                           (->> (range n)
                                (ppmap (fn [_]
                                         (ppmap identity (range n))))
                                (apply concat)
                                (into #{}))))
                  (apply concat)
                  (into #{}))))
         (apply concat)
         (into #{})))

Thanks again for taking a look.