scalameta / metals Goto Github PK
View Code? Open in Web Editor NEWScala language server with rich IDE features ๐
Home Page: https://scalameta.org/metals/
License: Apache License 2.0
Scala language server with rich IDE features ๐
Home Page: https://scalameta.org/metals/
License: Apache License 2.0
The indexDependencyClasspath should be moved out of Compiler.scala since it doesn't do anything related to the compiler. See https://github.com/scalameta/language-server/pull/45/files#r152680485
Or it should default to the current working directory. This would make it easier to use coursier bootstrap -o metaserver
since the workspace can be passed dynamically on start. Related #111
Sometimes it's necessary to clear the index cache (see for instance #105 (comment)). This command must be executed on the server, since the client doesn't know the cache location (which is machine-dependent). It's also a good opportunity to learn about workspace/executeCommand
.
In #51 and #55 we moved signatureHelp and hover to a new testing infrastructure. This new infrastructure makes it very productive to polish the output from the presentation compiler using ~metaserver/testOnly -- tests.compiler
. This workflow is way more productive than doing publishLocal
, open vscode, manually type code and see what it produces.
The completion provider can also benefit from some polishing in how the completions are ordered, for example that fields of a class take prioritity over defaults like toString
/finalize
package example
import a._
class UserTest extends org.scalatest.FunSuite {
val user = a.User("Bob", 42)
val list = List(1, 2, 3).map(x => x.toString)
}
03:05:13.798 DEBUG l.core.MessageReader - Received headers:
Content-Length: 242
03:05:13.798 DEBUG l.core.Connection - Received {"jsonrpc":"2.0","id":228,"method":"textDocument/hover","params":{"textDocument":{"uri":"file:///Users/laughedelic/dev/laughedelic/language-server/test-workspace/a/src/test/scala/example/UserTest.scala"},"position":{"line":8,"character":15}}}
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - hover at a/src/test/scala/example/UserTest.scala:8:15
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - Database for a/src/test/scala/example/UserTest.scala
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.example. at a/src/test/scala/example/UserTest.scala:0:8
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.a. at a/src/test/scala/example/UserTest.scala:2:7
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.example.UserTest# at a/src/test/scala/example/UserTest.scala:4:6
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.example.UserTest#`<init>`()V. at a/src/test/scala/example/UserTest.scala:4:15
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.org. at a/src/test/scala/example/UserTest.scala:4:23
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.org.scalatest. at a/src/test/scala/example/UserTest.scala:4:27
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.org.scalatest.FunSuite# at a/src/test/scala/example/UserTest.scala:4:37
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.example.UserTest#user. at a/src/test/scala/example/UserTest.scala:6:6
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.a. at a/src/test/scala/example/UserTest.scala:6:13
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.a.User. at a/src/test/scala/example/UserTest.scala:6:15
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.example.UserTest#list. at a/src/test/scala/example/UserTest.scala:8:6
03:05:13.799 INFO s.m.l.ScalametaLanguageServer - _root_.scala.collection.immutable.List. at a/src/test/scala/example/UserTest.scala:8:13
03:05:13.800 DEBUG l.core.MessageWriter - Content-Length: 158
{"jsonrpc":"2.0","result":{"contents":[{"language":"scala","value":""}],"range":{"start":{"line":8,"character":13},"end":{"line":8,"character":17}}},"id":228}
03:05:13.800 DEBUG l.core.MessageWriter - payload: {"jsonrpc":"2.0","result":{"contents":[{"language":"scala","value":""}],"range":{"start":{"line":8,"character":13},"end":{"line":8,"character":17}}},"id":228}
Completion items are missing the kind
property, so they all have the same icon in editors.
We should populate the kind
property based on the symbol type, similarly to what @laughedelic has already done for textDocument/symbols
Currently symbol type is used in place of its hierarchical parent reference:
Citing LSP spec (textDocument/documentSymbol
response):
/**
* The name of the symbol containing this symbol. This information is for
* user interface purposes (e.g. to render a qaulifier in the user interface
* if necessary). It can't be used to re-infer a hierarchy for the document
* symbols.
*/
containerName?: string;
So symbols get grouped by type instead of the declaration hierarchy. I don't know if there is such thing in the VS Code UI, but in Atom there is an Outline view panel which directly corresponds to the textDocument/documentSymbol
feature:
And here is an example Flow JS source:
I'd like to fix it, but I don't know how to get this hierarchical information from the SemanticDB.
I'm just starting to use language-server
with vscode so this may not be a bug.
I normally use other triggered build commands like ~fastOptJS
or ~reStart
. It seems language-server
only updates when ~compile
or ~test
is called. Is that correct? Can it be improved?
Currently we only check for the JAVA_HOME
env variable, but that's not guaranteed to be set (
for instance it's not set on my machine)
See comment from @gabro https://github.com/scalameta/language-server/pull/23/files#r150784897
Just as a comparison, the Java LSP jumps to a .class file containing the source.
I'm not sure I understand the implementation, but here's the relevant bit https://github.com/eclipse/eclipse.jdt.ls/blob/2127f12fb3700dd783b75d973e4f2ecc134b9ddf/org.eclipse.jdt.ls.core/src/org/eclipse/jdt/ls/core/internal/handlers/NavigateToDefinitionHandler.java#L52
Now that you've setup Travis CI, what do you think about using scalacenter/sbt-release-early? I guess you know this project well and I don't need to explain why it's useful, but I also understand that you may not want to use right now while things are moving too fast.
After #37, JDK indexing is disabled even if indexJDK = true
in ServerConfig. I tried to enable it but then one unit test <<CharRef>>
fails.
The ctags indices currently store the full file contents of all sources in your classpath. There is no need for that since we can store just the paths to the entries in the zip/jar files instead.
/**
* Defines how the host (editor) should sync document changes to the language server.
*/
export namespace TextDocumentSyncKind {
/**
* Documents should not be synced at all.
*/
export const None = 0;
/**
* Documents are synced by always sending the full content
* of the document.
*/
export const Full = 1;
/**
* Documents are synced by sending the full content on open.
* After that only incremental updates to the document are
* send.
*/
export const Incremental = 2;
}
Currently we do the full-sync, which sends the whole file contents on every keystroke
See textDocument/signatureHelp
. @olafurpg at a first glance, do you think we can pull this off using denotations only?
In #51 signatureHelp only highlights the first argument, we should be able to highlight the second argument once it's active by implementing a push/pop stack of open/close parens and count the number of commas in the current active level.
LSP supports sending log messages to the client, using connection.logMessage
.
Those messages are then handled by the client in different ways, for instance VSCode displays them in the Scalameta
output channel.
I think it would be more convenient than outputting to a file like we do now, or at least we could have both, so that one can access those logs more conveniently without leaving the editor.
Currently, completions only suggest symbols that are available in your active scope. It would be nice if the completions also included symbols that are accessible with an import. LSP CompletionItem has an optional textEdit
field to support inserting the import on completion
/**
* An edit which is applied to a document when selecting this completion. When an edit is provided the value of
* `insertText` is ignored.
*
* *Note:* The range of the edit must be a single line range and it must contain the position at which completion
* has been requested.
*/
textEdit?: TextEdit;
The Java LSP seems to be taking advantage of this.
Essentially this:
This can leverage the LSP notifications of type "status". The LS will send a notification indicating the status (Starting, Started, Error) and a message.
Each specific editor can decide how to show this message. The example in the GIF above is a mocked implementation in VSCode.
The Java LSP does it exactly like this:
User is val
(icon is kinda ugly in vscode with https://marketplace.visualstudio.com/items?itemName=patrys.vscode-code-outline)
I feel that constant is not appropriate for vals since vals can be very stateful objects, constants are IMO literals like "string"
and 2
Currently completion items are sorted alphabetically.
Ideally we would want them sorted by relevance, where relevance can be a combination of multiple factors, such as (but not limited too):
The references request is sent from the client to the server to resolve project-wide references for the symbol denoted by the given text document position.
The data already exists in SymbolIndex
, but the server needs to register this capability and listen to those requests. This commit here fe14433 can give an idea how to add support for new capabilities.
A key feature that is currently missing is red squigglies for compile errors. I've been hesitant to switch on the error messages from the presentation compiler in fear of reporting false negatives by stale/incorrect classpaths (I'm scarred by false negatives in IntelliJ). However, I think red squigglies from the presentation compiler can help surface integration issues when the classpath/setup is stale/broken.
Open question: should we run the compiler on file save on every keystroke?
We can run on every keystroke here
https://github.com/scalameta/language-server/blob/d600fc0881cd19600132a7ee3b6a234c8557e1f9/metaserver/src/main/scala/scala/meta/languageserver/ScalametaLanguageServer.scala#L172-L180
We could create an Observable[FileChanged] where something like case class FileChanged(uri: String, contents: String)
that gets then mapped through the presentation compiler with something like
askLoadedTyped(unit).get
unit.problems.map(toLSPDiagnostic).foreach(notifications.publishDiagnostics)
Effects.CompilerNotifications
Similar to how scalafix linter messages get reported.
In #37, we merged the new persistent schema for #35. It's still not clear however much much memory the new symbol indexer needs and how fast it can index a very large corpus of code. We should add some benchmarks to validate performance improvements or catch regressions. Here is a semanticdb corpus of ~3.000.000 LOC from the community builds that we can use to load test the indexer: https://drive.google.com/open?id=1Xu7Rl7EBhrUVu698HzIhc0QQNuhclVk1
https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#workspace_symbol
The workspace symbol request is sent from the client to the server to list project-wide symbols matching the query string.
Request:
method: 'workspace/symbol'
params: WorkspaceSymbolParams defined as follows:
/**
* The parameters of a Workspace Symbol Request.
*/
interface WorkspaceSymbolParams {
/**
* A non-empty query string
*/
query: string;
}
I'd like to try and use mtags alone for this, since that would enable workspace/symbols to work without any semanticdb/build integration.
Currently, we wait for .semanticdb files to appear to support go to definition. This means we don't support go to definition for WIP buffers or even for working programs if you haven't run sbt compile
.
We should be able to support go to definition as you type by fetching symbols from the presentation compiler and converting them to scala.meta.Symbol.
Currently we only return completions suggested by the PC, which don't include language keywords.
This does not work
trait Foo
new <<Foo>> {}
The logs show
08:52:32.592 INFO s.m.l.ScalametaLanguageServer - Found symbol _root_.example.a.f.$anon#
Reported by @laughedelic
Repro: start VSCode extension as documented in https://github.com/scalameta/language-server/blob/master/CONTRIBUTING.md.
Try jumping to Either
, e.g. in
case class User(name: String, age: Int, x: Either[String, Int])
Result: jumping to Either
doesn't work, while jumping to Int
does.
Scenario:
Expected result:
Actual result:
A simple solution is to try search for a semanticdb file on textDocument/open
or even to get them all when the server starts and feed them to the semanticdbs monix observable.
Hi! I'm trying to use autozimu/LanguageClient-neovim
with metaserver
, but can't find a way to start my client-metaserver... Is there some command for language server?
For example:
let g:LanguageClient_serverCommands = { \ 'rust': ['rustup', 'run', 'nightly', 'rls'], \ 'javascript': ['javascript-typescript-stdio'], \ }
Running scalametaEnableCompletions triggers the correct events but it doesn't show "new compiler config" and completions/hover don't work.
00:15:16.974 DEBUG l.core.MessageReader - <-- {"jsonrpc":"2.0","method":"workspace/didChangeWatchedFiles","params":{"changes":[{"uri":"file:///Users/ollie/dev/slick/slick/target/compile.compilerconfig","type":2}]}}
00:15:16.981 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/slick/target/compile.compilerconfig changed, extension=compilerconfig
00:15:17.418 DEBUG l.core.MessageReader - <-- {"jsonrpc":"2.0","method":"workspace/didChangeWatchedFiles","params":{"changes":[{"uri":"file:///Users/ollie/dev/slick/slick/target/test.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/osgi-tests/target/compile.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/osgi-tests/target/test.compilerconfig","type":2}]}}
00:15:17.419 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/slick/target/test.compilerconfig changed, extension=compilerconfig
00:15:17.419 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/osgi-tests/target/compile.compilerconfig changed, extension=compilerconfig
00:15:17.419 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/osgi-tests/target/test.compilerconfig changed, extension=compilerconfig
00:15:18.202 DEBUG l.core.MessageReader - <-- {"jsonrpc":"2.0","method":"workspace/didChangeWatchedFiles","params":{"changes":[{"uri":"file:///Users/ollie/dev/slick/reactive-streams-tests/target/test.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/reactive-streams-tests/target/compile.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/slick-hikaricp/target/compile.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/slick-hikaricp/target/test.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/target/test.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/target/compile.compilerconfig","type":2}]}}
00:15:18.203 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/reactive-streams-tests/target/test.compilerconfig changed, extension=compilerconfig
00:15:18.203 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/reactive-streams-tests/target/compile.compilerconfig changed, extension=compilerconfig
00:15:18.203 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/slick-hikaricp/target/compile.compilerconfig changed, extension=compilerconfig
00:15:18.203 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/slick-hikaricp/target/test.compilerconfig changed, extension=compilerconfig
00:15:18.203 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/target/test.compilerconfig changed, extension=compilerconfig
00:15:18.203 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/target/compile.compilerconfig changed, extension=compilerconfig
00:15:18.496 DEBUG l.core.MessageReader - <-- {"jsonrpc":"2.0","method":"workspace/didChangeWatchedFiles","params":{"changes":[{"uri":"file:///Users/ollie/dev/slick/slick-testkit/target/compile.compilerconfig","type":2},{"uri":"file:///Users/ollie/dev/slick/slick-testkit/target/test.compilerconfig","type":2}]}}
00:15:18.497 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/slick-testkit/target/compile.compilerconfig changed, extension=compilerconfig
00:15:18.497 INFO s.m.l.ScalametaLanguageServer - File /Users/ollie/dev/slick/slick-testkit/target/test.compilerconfig changed, extension=compilerconfig
It would be helpful if the logs included more details when no presentation compiler is found for a path. It used to log before but not anymore.
See #23 (comment). Storing symbol definition position instead of or in addition to the name position would make index reusable not only for definition/references, but also for the symbols outline (which currently has a broken temporary implementation).
This issue is to discuss the details and decide how to proceed.
By default, scalafix complains about the lack of .scalafix.conf because it doesn't know which rules to enable. Should we enable some set of rules by default?
I noticed that server responses with an error to the shutdown
request:
01:04:47.884 DEBUG l.core.MessageReader - Received headers: Content-Length: 59
01:04:47.884 DEBUG l.core.Connection - Received {"jsonrpc":"2.0","id":14,"method":"shutdown","params":null}
01:04:47.900 DEBUG l.core.MessageWriter - Content-Length: 199
{"jsonrpc":"2.0","error":{"code":-32602,"message":"Invalid params","data":{"meaning":"Invalid method parameter(s).","error":{"obj":[{"msg":["command parameters must be given"],"args":[]}]}}},"id":14}
01:04:47.900 DEBUG l.core.MessageWriter - payload: {"jsonrpc":"2.0","error":{"code":-32602,"message":"Invalid params","data":{"meaning":"Invalid method parameter(s).","error":{"obj":[{"msg":["command parameters must be given"],"args":[]}]}}},"id":14}
shutdown
request is not supposed to have any parameters, so I think that the underlying LSP library doesn't handle it well.
It causes some extra lags in Atom (atom/atom-languageclient#119) and I suspect after this response server never receives an exit
request. By the way, I didn't find how exit
is handled in the server.
Example
Use enumero
to define an enum as:
@enum trait Planet {
object Earth
object Venus
object Mercury
}
The generated code is something like
sealed abstract trait Planet
object Planet {
case object Earth extends Planet
case object Venus extends Planet
case object Mercury extends Planet
}
When using it, the definition provider finds nothing:
val x = Planet.Earth // no definition found for `Earth`
This is not super-obvious to fix, since the positions depend on the macro implementation.
@olafurpg suggested we may run the syntactic indexer (ScalaMtags
) to fill-in positions that are not present in the semanticdb.
We need configuration in few places (e.g. the path of .scalafmt.conf
and similar files).
The LSP way of doing this is to react to workspace/didChangeConfiguration
.
signatureHelp currently uses completions, which can often fail with a string index out of bounds error. This can be fixed by inserting CURSOR in the right place before invoking completions, as we did in #64 .
While testing the server in slick/slick
23:45:52.280 ERROR s.m.l.mtags.Mtags$ - Error indexing jar:file:///Users/ollie/.ivy2/cache/net.sourceforge.fmpp/fmpp/srcs/fmpp-0.9.15-sources.jar!/fmpp/tools/CommandLine.java
java.lang.IllegalArgumentException: 1101 is not a valid line number, allowed [0..1083]
at org.langmeta.internal.inputs.InternalInput.lineToOffset(InternalInput.scala:31)
at org.langmeta.internal.inputs.InternalInput.lineToOffset$(InternalInput.scala:27)
at org.langmeta.inputs.Input$VirtualFile.lineToOffset(Input.scala:74)
at org.langmeta.languageserver.InputEnrichments$XtensionInputOffset$.toOffset$extension(InputEnrichments.scala:32)
at scala.meta.languageserver.mtags.JavaMtags$$anon$1.toRangePosition(JavaMtags.scala:58)
at scala.meta.languageserver.mtags.JavaMtags$$anon$1.$anonfun$visitClass$1(JavaMtags.scala:84)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.meta.languageserver.mtags.MtagsIndexer.withOwner(MtagsIndexer.scala:32)
When using in VSCode, the output channel of the extension is full of
[Error - 4:50:59 PM] Request completionItem/resolve failed.
Message: Invalid params
Code: -32602
[object Object]
There's probably something wrong with the languageserver
bindings
The server currently re-builds a semanticdb index on every startup. This is fine for small projects, but is not scalable for larger projects. Also with #23, indexing will always take at least ~10-15s since we now index the sources of your full classpath, including the JDK (2.5M loc). If we persist the semanticdb index, the language server only needs to run this expensive computation once and startup will be instant from there on.
There are many alternatives to store these indices
For now, I think we should go with the protobuf approach since I believe that our schema can and should remain dead simple and the size of the indices stay small enough to fit in memory. On server startup, we load a .metaserver/index.zip
file (filename up for debate). On server shutdown, we optionally dump the index to disk for use next time we open the editor.
I propose we make minor adaptations to the metadoc schema
syntax = "proto3";
package scala.meta.languageserver.index;
// All of the metadata associated with a single symbol definition
message SymbolIndex {
string symbol = 1;
// The Position where this symbol is defined.
Position definition = 2;
map<string, Ranges> references = 3;
int64 flags = 4;
string name = 6;
string signature = 7;
// Planned for Scalameta v2.2, see https://github.com/scalameta/scalameta/milestone/9
// string docstring = 6; // javadoc/scaladoc
// string overrides = 7; // parent symbol that this symbol overrides
}
// A standalone position used by "Go to definition" when jumping to a symbol
// definition.
message Position {
// can be a local file, entry inside local zip file, http url, or anything
// else.
string uri = 1;
int32 start = 2;
int32 end = 3;
}
// A hack to workaround the fact that protobuf can't
// encode `map<string, repeated string>`
message Ranges {
repeated Range ranges = 1;
}
// A slim range position, the filename is embedded in the
// key of `references: map<string, ranges>`
message Range {
int32 start = 2;
int32 end = 3;
}
Observe that the index does not include the uri file contents or the original semanticdb Document
, since that would be prohibitive to store in memory.
The language server must be able to retrieve the original semanticdb Document
from a URI, which includes the full file contents and all of the resolved names, symbols and synthetics.
They could be a lot.
LSP supports many interesting use cases. I'm going to list a few that are within reach:
document formatting
linting
symbols (listing, renaming)
workspace/symbol
requests to list all the symbols in the workspace matching a query stringtextDocument/documentSymbol
to list all the symbols in a documenttextDocument/rename
for renaming a symbol in the workspace (this could be cool, and potentially achievable with Scalafix)go to definition
textDocument/definition
is for jumping to the definition of a symbol (this has been done already in metadoc)autocompletion
A reference to the full spec of LSP is here: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md
We currently only manually test interactions with the presentation compiler. We should add unit tests for it to make it easier to implement features like #20
In #45 we currently lock the global cache during the entire period of indexing a classpath. We should explore ways to minimize locking as much as possible, and also improve error handling (e.g., by retry) when the lock is taken.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.