scalacenter / bloop

Bloop is a build server and CLI tool to compile, test and run Scala fast from any editor or build tool.

Compile, test and run Scala fast

Bloop is a build server for the Scala programming language. Bloop aspires to provide the best out-of-the-box experience to Scala developers and a solid platform for build tool authors to consume the Scala toolchain; to compile, test or run Scala code.

Bloop integrates with editors, build tools and other Scala tooling project to support many different workflows and allow bespoke integrations.

Pointers 👇
💻 Install bloop in your computer or CI server by following our Installation page
📚 Learn more about bloop and how to use it from your favorite build tool and editor in our website
⚙️ Tool author? Integrate your tool with bloop by reading the Integration Guide
❓Questions? Unsure if bloop is useful for your use case? Ask right away in our Discord channel!

bloop's Issues

`install` should run source and resource generators

Otherwise, the project won't compile or the tests won't pass if they are needed.

Bloop will compile multiple times the same bridge concurrently

For instance when compiling sbt/sbt, there are multiple leave projects that will be built concurrently. If Bloop needs a new compiler bridge, it'll compile multiple times the same bridge concurrently.

Remove macros and reduce dependencies in bloop

• utest -> junit
• caseapp -> roll our own? find something macro-less?

Let users configure the size of the thread pool

#72 introduced a thread pool to execute the task. By default, it's size is the Runtime.getRuntime.availableProcessors(). It should be configurable.

Test projects should depend on their "normal" projects

Currently, for every project in the sbt build, we define 2 projects: the "normal" project, and the test project. The test project should have a dependency on the normal project.

The issue arises when we try to build everything concurrently. Because the test projects do not depend on their normal project, we build them without having built the normal project before, and compilation will fail.

Use idiomatic configuration folders instead of `$HOME/.bloop`

We can rely on https://github.com/soc/directories, LICENSE is good.

Find a better name

Blossom is the best that @Duhemm and I came up with for now, but we didn't give it too much thought. We're looking for a better name, that is simpler to pronounce and ideally related to nature.

We're open for suggestions.

Log4j error when running tests

jvican in /data/rw/code/scala/loop                                                                                                                                                                   [13:08:42] 
> $ time coursier launch ch.epfl.scala:bloop_2.12:4a9b08e5 -- test -c /data/rw/code/scala/loop/.bloop-config/ -p frontend-test                                                       [±topic/reorder-classpath]
Loading 54 projects from '/data/rw/code/scala/loop/.bloop-config'...
Elapsed: 742.512836 ms
-------------------------------- Running Tests --------------------------------
+ bloop.tasks.TaskTest.Simple dependency chain 17ms  
+ bloop.tasks.TaskTest.Results are combined 22ms  
+ bloop.tasks.TaskTest.Task failure reports partial results 141ms  
+ bloop.tasks.TaskTest.Un-necessary tasks are not run 19ms  
+ bloop.tasks.TaskTest.Two roots, one dependent 19ms  
X bloop.tasks.IntegrationTestSuite 0ms 
  java.util.ServiceConfigurationError: org.apache.logging.log4j.spi.Provider: Provider org.apache.lo
  gging.log4j.core.impl.Log4jProvider not a subtype
    java.util.ServiceLoader.fail(ServiceLoader.java:239)
    java.util.ServiceLoader.access$300(ServiceLoader.java:185)
    java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
    java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    org.apache.logging.log4j.util.ProviderUtil.loadProviders(ProviderUtil.java:101)
    org.apache.logging.log4j.util.ProviderUtil.<init>(ProviderUtil.java:67)
    org.apache.logging.log4j.util.ProviderUtil.lazyInit(ProviderUtil.java:142)
    org.apache.logging.log4j.util.ProviderUtil.hasProviders(ProviderUtil.java:126)
    org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:89)
    bloop.logging.Logger.<init>(Logger.scala:17)
    bloop.logging.Logger$.get(Logger.scala:12)
    bloop.tasks.IntegrationTestSuite$.<init>(IntegrationTestSuite.scala:12)
    bloop.tasks.IntegrationTestSuite$.<clinit>(IntegrationTestSuite.scala:-1)
    sun.misc.Unsafe.ensureClassInitialized(Unsafe.java:-2)
    sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
    java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    java.lang.reflect.Field.get(Field.java:393)
    utest.runner.BaseRunner.$anonfun$runSuite$1(BaseRunner.scala:97)
X bloop.tasks.CompilationTaskTest 0ms 
  java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.LogManager
    bloop.logging.Logger.<init>(Logger.scala:17)
    bloop.logging.Logger$.get(Logger.scala:12)
    bloop.tasks.CompilationTaskTest$.<init>(CompilationTaskTest.scala:11)
    bloop.tasks.CompilationTaskTest$.<clinit>(CompilationTaskTest.scala:-1)
    sun.misc.Unsafe.ensureClassInitialized(Unsafe.java:-2)
    sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
    java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    java.lang.reflect.Field.get(Field.java:393)
    utest.runner.BaseRunner.$anonfun$runSuite$1(BaseRunner.scala:97)
X bloop.tasks.TestTaskTest 0ms 
  java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.LogManager
    bloop.logging.Logger.<init>(Logger.scala:17)
    bloop.logging.Logger$.get(Logger.scala:12)
    bloop.tasks.TestTaskTest$.<init>(TestTaskTest.scala:11)
    bloop.tasks.TestTaskTest$.<clinit>(TestTaskTest.scala:-1)
    sun.misc.Unsafe.ensureClassInitialized(Unsafe.java:-2)
    sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
    java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    java.lang.reflect.Field.get(Field.java:393)
    utest.runner.BaseRunner.$anonfun$runSuite$1(BaseRunner.scala:97)
Tests: 5, Passed: 5, Failed: 0
coursier launch ch.epfl.scala:bloop_2.12:4a9b08e5 -- test -c  -p frontend-tes  9.94s user 0.25s system 286% cpu 3.557 total

Reuse classloaders of the same Scala version

We're potentially classloading the Scala universe twice, one in ScalaInstance and another one in the ClassLoaderCache created internally by ZincUtil.scalaCompiler. We should unify these two so that we construct the classloader cache ourselves, we share it across the two different use sites, and we pass it in explicitly to create AnalyzingCompiler.

Project loading is slower than it should be

It should be super fast. I have a strong feeling it comes from the creation of all the ScalaInstances (which all go through Coursier). We should probably cache / share ScalaInstances.

It needs to be profiled and fixed.

Recycle compilation across different live loop instances

Unable to to pass option to bloop script

MacOS 10.13.1
Zsh
Bloop version: 66018e0a

➜  some-project git:(feature/some-branch) ✗ bloop projects
Projects loaded from '/Users/lgr/projects/some-project/.bloop-config':
 * core
 * core-test
 * root
 * root-test
 * web
 * web-test

➜  some-project git:(feature/some-branch) ✗ bloop compile -p root
Usage: bloop-ng.py [options] cmd arg1 arg2 ...

bloop-ng.py: error: no such option: -p

➜  some-project git:(feature/some-branch) ✗ bloop compile root
Required option --project / -p not specified

➜  some-project git:(feature/some-branch) ✗ bloop compile --project root
Usage: bloop-ng.py [options] cmd arg1 arg2 ...

bloop-ng.py: error: no such option: --project

Support `--verbose`

Most of the commands expose --verbose to enable verbose logging, but this option is not passed anywhere at the moment.

Enabling verbose logging in Log4J:

Configurator.setRootLevel(Level.DEBUG)

The issue is that we must disable it after the command has run (otherwise, all the commands will run with verbose logging enabled, and there's no obvious place to do it elegantly at the moment.

Add lichess to our community build

Current projects:

• sbt/sbt
• lihaoyi/utest
• scalacenter/versions

We need more. I propose to add heavy projects, possibly from the community build. Extra ideas:

• ornicar/lila
• quasar-analytics/quasar
• And similar projects... but we need a big testing suite to catch performance regressions.

Nailgun prints out cryptic error messages

While trying out bloop I get some errors from the nailgun server

$. ./bloop-server
Unable to load nailgun-version.properties.
NGServer [UNKNOWN] started on address localhost/127.0.0.1 port 2113.

Compared to when I start scalafmt with nailgun

$ ./scalafmt_ng
NGServer 0.9.1 started on all interfaces, port 2113.

Every bloop command ends with a nailgun exception

$ ng bloop.Cli help
bloop f24267cc
Usage: bloop [options] [command] [command-options]


Available commands: about, clean, compile, help, projects, test
Type `bloop 'command' --help` for help on an individual command

Nov 30, 2017 10:59:51 AM com.martiansoftware.nailgun.NGInputStream$1 run
WARNING: Nailgun client read future was interrupted
java.lang.InterruptedException
	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
	at java.util.concurrent.FutureTask.get(FutureTask.java:204)
	at com.martiansoftware.nailgun.NGInputStream$1.run(NGInputStream.java:91)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Nov 30, 2017 10:59:51 AM com.martiansoftware.nailgun.NGSession run
INFO: NGSession shutting down

I have not seen these errors when running scalafmt with nailgun. However, I have no idea what's causing those errors. I tried

• using the same nailgun version as scalafmt
• redirect out/in/err in case the logging infrastructure was messing it up

    Console.withErr(ngContext.err) {
      Console.withIn(ngContext.in) {
        Console.withOut(ngContext.out) {
          val exitStatus = run(cmd, logger)
          ngContext.exit(exitStatus.code)
        }
      }
    }

Add parallel execution

Remove redundant loging

We have redundant labels for the logs. Example (both times [W] and [Warn]):

[W] [Warn] /data/rw/code/scala/loop/frontend/target/scala-2.12/test-classes/projects/sbt/main/src/main/scala/sbt/internal/CommandExchange.scala:28: Unused import

Add nailgun integration and instructions

Add integration tests for the CLI

Now we have a cute CLI. We need to have integration tests for it, to make sure the most important commands work! So far we're testing this manually.

Bring Zinc as a source dependency

We should bring Zinc as a source dependency so that we can have make changes really fast to the project and publish it on our own. This is something I'll do before doing the Nailgun integration.

Add support for file watching `~`

We should probably add a really fast source file watching for ~compile and ~test. We may want to add this directly to Zinc rather than experiment with it directly in blossom. I prefer this approach since it's more generic.

Compilation error shouldn't freeze bloop

If a compilation task fails, then all the dependent tasks will be blocked, waiting for the failed task to finish.

Running tests does not read from resources directory

Bloop version: 66018e0a

I have Akka-http project and application.conf in src/main/resources as well some overrided properties in src/test/resources directories. When I run tests in sbt all is fine, but with bloop I get errors about missing configuration:

com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka'
  at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:156)
  at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:149)
  at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:176)
  at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:188)
  at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:193)
  at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:250)
  at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:314)
  at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:650)
  at akka.actor.ActorSystem$.apply(ActorSystem.scala:244)
  at akka.actor.ActorSystem$.apply(ActorSystem.scala:287)

Running a non-existing command throws a scary exception

docs+1… bloop ❯ ./bloop foobar                                                  
Nov 30, 2017 9:14:12 AM com.martiansoftware.nailgun.NGSession run
INFO: Server unexpectedly exited with unhandled exception
java.lang.ClassNotFoundException: foobar
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at com.martiansoftware.nailgun.NGSession.run(NGSession.java:271)

java.lang.ClassNotFoundException: foobar
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at com.martiansoftware.nailgun.NGSession.run(NGSession.java:271)

Use the resident compiler when caching works

The resident compiler avoids classfiles parsing, which has a significant cost (!): scala/compiler-benchmark#39. The argument goes that is quite expensive for compilation of small projects (or for incremental compiles that only compile a small subset), and so bloop is the most likely candidate to benefit from this work.

The cache of projects should take into account config file metadata

In the same way we do for classpath hashing. Otherwise, users will need to kill the server to have their projects detected.

Add support for running tests

Support for running tests is crucial. Adding support for it will require to also comply with sbt's test interface so that we can already reuse some of the implementations written by test frameworks. Or, at least, we need to figure out some strategy.

Use custom reporter *from within Scalac*

This is a crazy idea.

I didn't realize this was possible as of 2.12: scala/scala#4544.

I think it would be cool to experiment with better error messages only for 2.12.x. That approach would allow us to create a reporter that would understand concrete error messages (by using regexes, since we don't have context information of where those messages were created) and then prettify them.

Having this in bloop may be nice, but what I think it would be nicer is that we consider making this change upstream in scalac. If we make it upstream, we have the benefits of:

1. Much better implementation since we can touch the call sites creating the error messages.
2. Bigger impact and less ad-hoc solution.

Figure out the proper error handling abstraction

The state of errors in this codebase is a little bit... messy, let's say. We need to have a good story with regard to error handling here, and make it impossible that the nailgun server crashes because of an unwanted exception.

Change logger for zinc

I want colors, highlighted stack traces, nicer way of reporting log levels, and a simpler logger architecture than the one that sbt has.

Add support for console

Find out why the jars of scala instance are required for sbt to compile

Our sbt integration test fails to compile when we don't add the scala instance jars at the end of the classpath for some misterious reason. I need to figure out why, because these jars should not be there in the first place (they should come from the classpath we get from the .bloop-config). See a227c4e.

Add some tests

Our wrapper needs some tests 😄. @Duhemm volunteered to make some tomorrow, I'm assingning this task to you.

No tests run for project which aggregates others

Bloop version: 66018e0a

I have multi module sbt project.

• core
• web (which depends on core)
• root (which aggregates core and web).

Running tests on root does nothing

bloop -- test -p root
bloop -- test -p root-test

No output at all.

I have to run tests on project which contains them.

Cross publishing sbt-bloop does a weird thing with sbt 0.13.16

Not sure what kind of sick joke this is (publishes twice for sbt 1 / scala 2.12):

Workaround:

> set sbtVersion in sbtBloop := "0.13.16"
> sbtBloop/publishLocal

Problem:

sbt:bloop> ^ sbtBloop/publishLocal
[info] Setting `sbtVersion in pluginCrossBuild` to 1.0.3
[info]       _____            __         ______           __
[info]      / ___/_________ _/ /___ _   / ____/__  ____  / /____  _____
[info]      \__ \/ ___/ __ `/ / __ `/  / /   / _ \/ __ \/ __/ _ \/ ___/
[info]     ___/ / /__/ /_/ / / /_/ /  / /___/ /__/ / / / /_/ /__/ /
[info]    /____/\___/\__,_/_/\__,_/   \____/\___/_/ /_/\__/\___/_/
[info]
[info]    ***********************************************************
[info]    ***       Welcome to the build of `loooooooooop`        ***
[info]    *** An effort funded by the Scala Center Advisory Board ***
[info]    ***********************************************************
[info]
[info] Setting up the integration builds.
[info] Writing Ivy file /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/resolution-cache/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/resolved.xml.xml
[info] Packaging /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/sbt-bloop-dcd81ce5-sources.jar ...
[info] Wrote /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/sbt-bloop-dcd81ce5.pom
[info] Done packaging.
[info] Packaging /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/sbt-bloop-dcd81ce5.jar ...
[info] Done packaging.
[info] :: delivering :: ch.epfl.scala#sbt-bloop;dcd81ce5 :: dcd81ce5 :: integration :: Tue Nov 21 19:28:11 CET 2017
[info]  delivering ivy file to /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/ivy-dcd81ce5.xml
[info]  published sbt-bloop to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/poms/sbt-bloop.pom
[info]  published sbt-bloop to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/jars/sbt-bloop.jar
[info]  published sbt-bloop to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/srcs/sbt-bloop-sources.jar
[info]  published ivy to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/ivys/ivy.xml
[success] Total time: 2 s, completed Nov 21, 2017 7:28:11 PM
[info] Setting `sbtVersion in pluginCrossBuild` to 0.13.16
[info]       _____            __         ______           __
[info]      / ___/_________ _/ /___ _   / ____/__  ____  / /____  _____
[info]      \__ \/ ___/ __ `/ / __ `/  / /   / _ \/ __ \/ __/ _ \/ ___/
[info]     ___/ / /__/ /_/ / / /_/ /  / /___/ /__/ / / / /_/ /__/ /
[info]    /____/\___/\__,_/_/\__,_/   \____/\___/_/ /_/\__/\___/_/
[info]
[info]    ***********************************************************
[info]    ***       Welcome to the build of `loooooooooop`        ***
[info]    *** An effort funded by the Scala Center Advisory Board ***
[info]    ***********************************************************
[info]
[info] Setting up the integration builds.
[info] Writing Ivy file /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/resolution-cache/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/resolved.xml.xml
[info] Wrote /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/sbt-bloop-dcd81ce5.pom
[info] Packaging /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/sbt-bloop-dcd81ce5.jar ...
[info] Done packaging.
[info] :: delivering :: ch.epfl.scala#sbt-bloop;dcd81ce5 :: dcd81ce5 :: integration :: Tue Nov 21 19:28:16 CET 2017
[info]  delivering ivy file to /Users/martin/Documents/Projects.nosync/Duhemm/bloop/sbt-bloop/target/scala-2.12/sbt-1.0/ivy-dcd81ce5.xml
[info]  published sbt-bloop to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/poms/sbt-bloop.pom
[info]  published sbt-bloop to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/jars/sbt-bloop.jar
[info]  published sbt-bloop to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/srcs/sbt-bloop-sources.jar
[info]  published ivy to /Users/martin/.ivy2/local/ch.epfl.scala/sbt-bloop/scala_2.12/sbt_1.0/dcd81ce5/ivys/ivy.xml

Report compilation messages in a readable way

Running `bloop --help` shows Nailgun help

bloop --help shows nailgun help:

docs+1… bloop ❯ ./bloop --help                                                                    
Usage: bloop [options] cmd arg1 arg2 ...

Options:
  -h, --help            show this help message and exit
  --nailgun-server=NAILGUN_SERVER
  --nailgun-port=NAILGUN_PORT
  --nailgun-filearg=NAILGUN_FILEARG
  --nailgun-showversion
  --nailgun-help

Use a better hashing algorithm for classpath hashing

We're currently using SHA-1. I believe a cryptographic hash is not necessary for hashing and detecting changes in jars, so we should use something faster and more lightweight like xxHash, see my previous attempt to merge this into Zinc here: sbt/zinc#371. We could in theory implement it in bloop since we have the classpath hooks in latest Zinc 1.x for this.

However, this is only useful if we happen to find out that either of the following hypothesis are true:

1. Hashing more than one classpath entry for multi-module builds happens often; and,
2. The hash takes up a non-negligible time of the compilation.

I believe that these assumptions are most likely false, so this ticket is only for documenting the possibility of improving the hash. In practice, the cost of hashing the classpath happens only once (the first time you run the compiler) and after that the classpath should be the same, but this claim requires investigation.

I have no idea how bearable this process is for projects with gigantic classpaths that are likely to change (in essence, huge multi-module builds). So maybe it's worth it for them after all.

Use a standard configuration format that other tools can reuse

This is a ticket to keep track of collaboration with other tools in the Scala commuinty. Bloop should settle on a configuration format that can be read by any tool (from LSP servers, scalafix to other build tools).

As this format doesn't exist, we should probably pioneer one. Another possibility is to reuse .eclipse configuration files -- that are widely supported by Eclipse, Intellij and vscode (not natively, but some plugin extensions do) -- or directly generate .idea configuration files. However, we may want to encode more information in the file and it's not clear yet whether eclipse configuration files allow it.

Supporting all this configuration files is easy, so we may also consider understanding all of these so that people that heavily use Intellij and rely on sbt-idea to import their projects do not need to run install in the build. This would mean there's a standard, preferred configuration file but bloop can also understand others. This is a nitpick and it's only UX work, so this won't happen anytime soon. But it's a thing to keep an eye on.

My ideal format is json because it can be edited and it's readable (unlike protobuf). This json format should have also a library that provides bincompat readers and writers so that other tools that want to read/write from it only need to depend on the library and use it.

When bloop is public, I'll start summoning people up in this ticket so that we converge to one solution and explore how useful this file would be, and how it could help make better tooling.

Make the sbt plugin reconstruct the dependency classpath manually

Because of the way sbt works, when you get the dependencyClasspath of a root project you need to compile dependent projects. However, this is in theory not required because we can reconstruct the classpath based on the target directories of the project dependencies and the jars of the library dependencies (and probably also resource target directories -- this needs research). We don't care if the target directories are populated with class files or not.

Our sbt plugin should avoid compilation of projects to generate install. Why?

• People may want to run install in broken code (they are in the middle of a rebase), and they don't want to stage all their changes just to generate the bloop config.
• Compilation of projects can be very timeconsuming, especifically for large scala projects.

I propose our plugin reconstructs the dependency classpath on its own and doesn't rely on our dear sbt to do the work.

Clean puts bloop in bad state

Using bloop to compile bloop

bloop compile -p frontend # OK
bloop clean -p backend # OK
bloop compile -p frontend
Nov 30, 2017 1:08:08 PM com.martiansoftware.nailgun.NGSession run
INFO: Server unexpectedly exited with unhandled exception
java.util.NoSuchElementException: key not found: frontend
	at scala.collection.MapLike.default(MapLike.scala:232)
	at scala.collection.MapLike.default$(MapLike.scala:231)
	at scala.collection.AbstractMap.default(Map.scala:59)
	at scala.collection.MapLike.apply(MapLike.scala:141)
	at scala.collection.MapLike.apply$(MapLike.scala:140)
	at scala.collection.AbstractMap.apply(Map.scala:59)
	at bloop.engine.Interpreter$.$anonfun$compile$1(Interpreter.scala:105)

It seems the exception gets caught by nailgun

Add support for test quick execution

I never use it so I completely forgot about it.

Do people actually use it often? If so, we should probably support it since the idea is nice.

Port nice term display for the progress bar

From coursier and ammonite: https://github.com/coursier/coursier/blob/master/cache/src/main/scala/coursier/TermDisplay.scala

Dotty support

Would be cool to support Dotty someday.

See #30 (comment)

Show the compilation speed in LOC/s

See scala/bug#10601

Add fast check mode

This is just an idea, and it's long-term.

We should add a fast check mode to bloop that allows the compiler to run only until refchecks while still being incremental and being able to do things like semanticdb generation (to be tooling friendly). This would require changes to the compiler because we need to have the mapping between classess and targeted class files right after rechecks.

The main blocker for this feature is compilation of downstream projects (projects that depend on a project we're editing in fast mode), since they are not able to reuse the symbol and type information that comes from the dependee projects.

I have some ideas about how to tackle this problem:

1. Let the compiler persist independent ScalaSig information in an independent IR and create readers for this IR the compiler.
2. Add readers/writers for Tasty inside the compiler.

Running commands against an non-existing project crashes bloop

Should bloop remove old projects when doing install?

Steps:

• I run installBloop
• I remove a project from my build.sbt
• I re-run installBloop

Problem:

• The configuration file for the old project is still there

Question:

• Should bloop remove all the existing configuration files when doing bloopInstall?

The nailgun script is only compatible with Python 2

got following error when tried to use bloop cli

% bloop projects
  File "/Users/evgeniy/.bloop/bloop-ng.py", line 245
    self.sockpath = ur'\\.\pipe\{0}'.format(sockpath)
                                   ^
SyntaxError: invalid syntax

seems reason for that 3x python installed on system as default python version

% python --version
Python 3.6.3 :: Anaconda, Inc.

It works well after downgrade to 2.7 by conda install python=2.7

Prefer incremental compiles with jars over classes directory

There's this hypothesis that I haven't had yet the time to test that the JVM prefers jars rather than classes and performs better with the former. This has already been documented upstream: sbt/zinc#305.

This ticket wants to investigate this further by:

1. Confirming the claim.
2. If true, implementing it in bloop.