Implementing_a_server_for_the_Language_Server

We all have our favorite tools, something that we are used to or something that makes us most productive. Yet, it’s often that a particular language is not really supported by that favorite IDE or editor of ours. And even if it is supported it is clearly not at the level that we would expect from modern tooling. It’s obvious why that is, implementing a whole new integration takes time, mostly from OSS contributors, and is certainly something that we all lack.

This is where Language Server Protocol (LSP for short) comes into play. Its aim is to decouple the presentation of the code from the actual language implementation and was created by the folks in Microsoft. It has a clear specification on what a potential client can require from a server and the server tries to implement all possible methods through an interface. This makes it quite easy to implement any language server since we know exactly what might be required. It’s also language independent, therefore it can be implemented in anything that is best suited for the language. An LSP client can be a separate application implemented at another time. This reduces the usual problem of n*m integrations between editors and language backends to n+m number of integrations, which greatly reduces the time requirements for the developers.

Metals

Metals is an implementation of the Language Server that uses Scalameta SemanticDB plugin and Scala presentation compiler to implement a lot of the needed methods. SemanticDB compiler plugin gives us additional information about the source code elements such as symbols and types. And presentation compiler offers us, among others, information about possible completions in a source code document.

Metals already allows us to use most of the major editors and IDEs with pretty simple steps. However, it must be said that the integration with Visual Studio Code is probably the cleanest, but it’s not a surprise since that also comes from Microsoft. There is a dedicated extension in the Visual Studio Code Marketplace that easily bootstraps the metals integration and works out of the box. It adds all the needed configuration for Bloop, which is a compiler server used for quicker compilation in Metals.

Example

Let’s go through implementing a feature in Metals, so that we see exactly how does LSP work.

One of the easiest features to implement for an LSP server would be documentHighlight. This feature highlights anything that is related to the symbol currently where the cursor is.

Implementation

Our goal is to check all the symbols in a given document and compare them to the symbol we are at. That should be simple to do with some additional information from SemanticDB. The code below is a bit simplified with some Metals specific implementation details left out.

Let’s start by implementing the request itself. Metals implementation is based on `org.eclipse.lsp4j` LSP implementation. It provides all the needed interfaces and classes for a full language server.

1@JsonRequest("textDocument/documentHighlight") 
2def documentHighlights(params: TextDocumentPositionParams)
3  : CompletableFuture[util.List[DocumentHighlight]] = ???

Above, we define a method for the LSP server that works for the document highlighting. `@JsonRequest()` indicates the name of the LSP endpoint to be implemented, which is in this case textDocument/documentHighlight. TextDocumentPositionParams are the parameters required by that endpoint and util.List[DocumentHighlight] is the required return type. All of that is based directly on the LSP specification.

Ok, now let’s try to get all the needed information:

1// Let's get the path to the file we are working on
2val source: Path = params.getTextDocument.getUri.toAbsolutePath
3// SemanticDB is located in META-INF with the same prefix as the source
4// this should work assuming sourceRoot and targetDir are specified before
5val fileName: String = source.getFileName.toString
6val relativePath: Path = sourceRoot.relativize(source)
7val semanticDBPath: Path = targetDir
8  .resolve("target/classes/META-INF/semanticdb")
9  .resolve(relativePath)
10  .resolveSibling(fileName + ".semanticdb")
11// and then get the semanticDB information
12val in: InputStream = Files.newInputStream(semanticDBPath)
13val doc: TextDocument = TextDocuments.parseFrom(in).documents.head

This will get us the symbol that we are pointing to based on the cursor position and the SemanticDB symbol information, which tells us the exact range where each symbol is located. The only thing that’s left is match all the symbols:

1// Find matching symbol occurrence in SemanticDB
2val position: Position = params.position
3val symbolOccurrence: Option[SymbolOccurrence] = 
4  doc.occurrences.find{ occurence =>
5    val range: Range = occurence.range.get
6    range.startLine <= position.getLine &&
7    range.endLine >= position.getLine &&
8    range.startCharacter <= position.getCharacter && 
9    range.endCharacter >= position.getCharacter

In a production code we would need to use our own concurrency model that would map to the java futures, but for the sake of simplicity we left that out.

Altogether the implementation looks like this:

1val occurencesList = for{
2  occurence <- symbolOccurrence
3  otherOccurence <- doc.occurrences
4  if occurence.symbol == otherOccurence.symbol
5  range <- otherOccurence.range
6} yield new DocumentHighlight(range)

Further issues

There is a number of further issues that we didn’t cover here. One of those is what happens when we edit the file? How should we update the positions? Another issue might come up with Scala symbols, which aren’t always the same underneath, but we would want to highlight them together. In Scala`val a = 1` and `a + 3` would actually be different symbols since first is really a setter and the second a getter. Solving those problems is however a bit more complex, so it will not be part of this blog post. Full implementation is in my PR, which I recommend to look up if you plan to work with LSP servers or Scalameta.

1val completableFuture = new CompletableFuture[util.List[DocumentHighlight]]()
2completableFuture.complete(occurencesList)

Links

LSP - https://microsoft.github.io/language-server-protocol/specification
Scalameta - https://scalameta.org/
Metals - https://scalameta.org/metals/
Visual Studio Code - https://code.visualstudio.com/
Bloop - https://scalacenter.github.io/bloop/
Document highlight - https://microsoft.github.io/language-server-protocol/specification#textDocument_documentHighlight
Metals PR - https://github.com/scalameta/metals/pull/621

1@JsonRequest("textDocument/documentHighlight")
2def documentHighlights(params: TextDocumentPositionParams) : 
3  CompletableFuture[util.List[DocumentHighlight]] = {
4    val source: Path = params.getTextDocument.getUri.toAbsolutePath
5    val fileName: String = source.getFileName.toString
6    val relativePath: Path = sourceRoot.relativize(source)
7    val semanticDBPath: Path = targetDir
8      .resolve("target/classes/META-INF/semanticdb")
9      .resolve(relativePath)
10      .resolveSibling(fileName + ".semanticdb")
11    val in: InputStream = Files.newInputStream(semanticDBPath)
12    val doc: TextDocument = TextDocuments.parseFrom(in).documents.head
13    val position: Position = params.position
14    val symbolOccurrence: Option[SymbolOccurrence] = 
15      doc.occurrences.find{ occurence =>
16        val range: Range = occurence.range.get
17        range.startLine <= position.getLine &&
18        range.endLine >= position.getLine &&
19        range.startCharacter <= position.getCharacter && 
20        range.endCharacter >= position.getCharacter
21      }
22    val occurencesList = for{
23      occurence <- symbolOccurrence
24      otherOccurence <- doc.occurrences
25      if occurence.symbol == otherOccurence.symbol
26      range <- otherOccurence.range
27    } yield new DocumentHighlight(range)
28    val completableFuture = new CompletableFuture[util.List[DocumentHighlight]]()
29    completableFuture.complete(occurencesList)
30}

Obviously we took a few shortcuts here just to show quickly how to work with LSP. We have left out some implementation details like methods for Java/Scala compatibility and you might notice couple of “not-so-safe” programming choices. However, this short example shows how one would go about creating and working with LSP. After starting the server, multiple clients can use the same backend with the properly defined methods.

Further issues

Summing up

Hopefully, this gives you a look on how to work with LSP servers or how to start working on your own. If you want to contribute we are always looking for a helping hand in metals and while creating new features might be a bit too time consuming or too complex, there is always the option to implement some small fixes or improvements. I also recommend trying out Metals on your own and reporting anything that might be a problem.