Sparkmonitor failure for kernel restart - java.net.SocketException: Broken pipe (Write failed) about sparkmonitor

[utkarshgupta137]

@rdhara I used github actions for building JAR files by modifying it to upload jar files as a release: https://github.com/utkarshgupta137/sparkmonitor/blob/master/.github/workflows/publish.yml#L41

[rdhara]

Yup I figured it out (same step you have) and got it to work, thank you @creepysta!

[krishnan-r]

This probably is the same as #7, see my comment there. I will draft a PR soon with a fix.

[rdhara]

@creepysta Can you please explain the steps you took after adding that one line to the listener file? I'm new to sbt and Scala and am having some trouble generating the JARs referenced in the build.sbt:

assembly / assemblyOutputPath := {
  scalaBinaryVersion.value match {
    case "2.11" => (baseDirectory { base => base / ("../sparkmonitor/listener_2.11.jar") }).value
    case "2.12" => (baseDirectory { base => base / ("../sparkmonitor/listener_2.12.jar") }).value
  }
}

These jars appear in site-packages when I just run pip install sparkmonitor. I'm not even sure where the JAR building is happening, but based on a search for sbt I found this on line 75 of the packages.json: "build:scalalistener": "cd scalalistener && sbt +assembly", but the command failed with an error message.

My current plan is to make the change in a separate repo and try to pip install that repo after cloning it in. I tried this but I'm not able to get the JARs to appear after the setup.py runs; any guidance you can provide on building this locally would be very helpful - thanks in advance!

[rdhara]

@rdhara I used github actions for building JAR files by modifying it to upload jar files as a release: https://github.com/utkarshgupta137/sparkmonitor/blob/master/.github/workflows/publish.yml#L41

Thank you I'll try this!

[creepysta]

Hi @rdhara , apologies for the late reply. Can you try assembly followed by package in sbt shell? Just open sbt shell in the same folder as sbt.build file.
The output jar should be present in ../sparkmonitor considering your cwd is sparkmonitor/scalalistener