sourcegraph / scip-kotlin

SCIP indexer for Kotlin. Implemented as a SemanticDB compiler plugin.

Kotlin SCIP support

This codebase implements a Kotlin compiler plugin that can be used together with scip-java to emit SCIP indexes for Kotlin projects.

Getting started

This project must be used together with scip-java. Visit scip-java for instructions on how to index Kotlin projects with scip-java. Note that scip-java indexes Kotlin sources even if you have no Java code.

Kotlin version compatibility

Any given release of scip-kotlin only supports one major version of Kotlin. Use the table below to find the version of scip-kotlin that matches the Kotlin version you are using in your project.

Kotlin version	scip-kotlin version
1.8.x	0.3.2
1.9.x	0.4.0

SemanticDB support

This project is implemented as a SemanticDB compiler plugin. To generate SCIP, you first compile the Kotlin sources with the SemanticDB compiler plugin and then convert SemanticDB files into SCIP using scip-java index-semanticdb. See Low-level usage for more details on how to generate SemanticDB files and convert SemanticDB into SCIP.

Low-level usage

First, fetch the jar file of the SemanticDB compiler plugin: com.sourcegraph:semanticdb-kotlinc:VERSION. For example, you can use Coursier to download the jar file.

curl -fLo coursier https://github.com/coursier/launchers/raw/master/coursier && chmod +x ./coursier
export SEMANTICDB_KOTLIN_VERSION="latest.release" # or replace with a particular version
export SEMANTICDB_KOTLIN_JAR=$(./coursier fetch com.sourcegraph:semanticdb-kotlinc:$SEMANTICDB_KOTLIN_VERSION)

Once you have the jar file, you need to determine two compiler options:

• sourceroot: the absolute path to the root directory of your codebase. All source files that you want to index should be under this directory. For Gradle codebases, this is typically the toplevel build.gradle file. For Maven codebases, this is typically the toplevel pom.xml file.
• targetroot: the absolute path to the directory where you want the compiler plugin to write SemanticDB files. This can be any directory on your computer.

Now you have all the necessary parameters to invoke the Kotlin compiler with the SemanticDB compiler plugin.

kotlinc -Xplugin=${SEMANTICDB_KOTLIN_JAR} \
  -P plugin:semanticdb-kotlinc:sourceroot=SOURCEROOT_DIRECTORY \
  -P plugin:semanticdb-kotlinc:targetroot=TARGETROOT_DIRECTORY

Once the compilation is complete, the targetroot should contain *.semanticdb files in the META-INF/semanticdb sub-directory.

To convert the SemanticDB files into SCIP, run scip-java index TARGETROOT_DIRECTORY. If you have Coursier installed, you can run scip-java directly like this

cd $SOURCEROOT_DIRECTORY
./coursier launch --contrib scip-java -- index-semanticdb TARGETROOT_DIRECTORY

scip-kotlin's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

• WARN: Fallback to renovate.json file as a preset is deprecated, please use a default.json file instead.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update plugin org.jetbrains.kotlin.jvm to v2
• fix(deps): update kotlin monorepo to v2 (major) (script-runtime, org.jetbrains.kotlin:kotlin-stdlib-jdk8)

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• chore(deps): update plugin com.github.johnrengelman.shadow to v7.1.2
• fix(deps): update dependency io.kotest:kotest-assertions-core to v4.6.4
• chore(deps): update plugin com.palantir.git-version to v0.15.0

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update plugin org.jetbrains.kotlin.jvm to v1.9.24
• fix(deps): update dependency dev.zacsweers.kctfork:core to v0.4.1
• chore(deps): update dependency gradle to v7.6.4
• chore(deps): update plugin com.google.protobuf to v0.9.4
• chore(deps): update plugin io.github.gradle-nexus.publish-plugin to v1.3.0
• fix(deps): update dependency com.google.protobuf:protobuf-java to v3.25.3
• fix(deps): update dependency com.google.protobuf:protoc to v3.25.3
• fix(deps): update dependency com.sourcegraph:scip-java_2.13 to v0.9.10
• fix(deps): update dependency com.sourcegraph:semanticdb-javac to v0.9.10
• fix(deps): update dependency org.junit.jupiter:junit-jupiter-params to v5.10.2
• fix(deps): update kotlin monorepo to v1.9.24 (script-runtime, org.jetbrains.kotlin:kotlin-stdlib-jdk8)
• chore(deps): update actions/checkout action to v4
• chore(deps): update actions/setup-go action to v5
• chore(deps): update actions/setup-java action to v4
• chore(deps): update dependency gradle to v8
• chore(deps): update plugin com.diffplug.spotless to v6
• chore(deps): update plugin com.github.johnrengelman.shadow to v8
• chore(deps): update plugin com.palantir.git-version to v3
• chore(deps): update plugin io.github.gradle-nexus.publish-plugin to v2
• fix(deps): update dependency com.google.protobuf:protobuf-java to v4
• fix(deps): update dependency com.google.protobuf:protoc to v4
• fix(deps): update dependency io.kotest:kotest-assertions-core to v5
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

github-actions

.github/workflows/ci.yml

• actions/checkout v3
• actions/setup-java v3

.github/workflows/labeler.yml

• github/issue-labeler v3.4

.github/workflows/pr-auditor.yml

• actions/checkout v4
• actions/setup-go v4

.github/workflows/release.yml

• actions/checkout v3
• actions/setup-java v3

gradle

gradle.properties

settings.gradle.kts

build.gradle.kts

• org.jetbrains.kotlin.jvm 1.9.22
• com.github.johnrengelman.shadow 7.1.0
• com.palantir.git-version 0.12.3
• io.github.gradle-nexus.publish-plugin 1.1.0
• com.diffplug.spotless 5.17.1

debug-project/build.gradle.kts

semanticdb-kotlin/build.gradle.kts

• com.google.protobuf 0.8.17
• com.google.protobuf:protobuf-java 3.17.3
• com.google.protobuf:protobuf-java 3.17.3
• com.sourcegraph:semanticdb-javac 0.8.23
• com.google.protobuf:protoc 3.17.3

semanticdb-kotlinc/build.gradle.kts

• com.google.protobuf:protobuf-java 3.15.7
• io.kotest:kotest-assertions-core 4.6.3
• dev.zacsweers.kctfork:core 0.4.0
• org.junit.jupiter:junit-jupiter-params 5.8.1
• org.jetbrains.kotlin:kotlin-stdlib-jdk8 1.5.0
• script-runtime 1.5.0
• com.sourcegraph:scip-java_2.13 0.8.24
• com.sourcegraph:semanticdb-javac 0.6.12

gradle-wrapper

gradle/wrapper/gradle-wrapper.properties

• gradle 7.2

---

• Check this box to trigger a request for Renovate to run again on this repository

`java.lang.IndexOutOfBoundsException` indexing `com.fasterxml.jackson.module__jackson-module-kotlin__2.11.4`

[info] Caused by: java.lang.IndexOutOfBoundsException: start -1, end -1, length 0
[info] 	at java.base/java.lang.AbstractStringBuilder.checkRange(AbstractStringBuilder.java:1802)
[info] 	at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:680)
[info] 	at java.base/java.lang.StringBuilder.append(StringBuilder.java:218)
[info] 	at com.sourcegraph.semanticdb_kotlinc.SemanticdbTextDocumentBuilder.stripKDocAsterisks(SemanticdbTextDocumentBuilder.kt:160)

NoClassDefFoundError: org/jetbrains/kotlin/resolve/source/PsiSourceElementKt indexing square/okhttp repo

Running lsif-java index in the square/okhttp repo

e: java.lang.NoClassDefFoundError: org/jetbrains/kotlin/resolve/source/PsiSourceElementKt
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.getAllMethods(SymbolsCache.kt:253)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.methodDisambiguator(SymbolsCache.kt:197)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.semanticdbDescriptor(SymbolsCache.kt:178)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.uncachedSemanticdbSymbol-BfSPOvA(SymbolsCache.kt:109)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.getSymbol-BfSPOvA(SymbolsCache.kt:71)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.emitSymbols(SymbolsCache.kt:51)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache.access$emitSymbols(SymbolsCache.kt:30)
	at com.sourcegraph.semanticdb_kotlinc.GlobalSymbolsCache$get$1.invokeSuspend(SymbolsCache.kt:40)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlin.sequences.SequenceBuilderIterator.hasNext(SequenceBuilder.kt:140)
	at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
	at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
	at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)

@Strum355 This looks like a Kotlin compiler version incompatibility. Do you think we'll need to cross-build/cross-publish the semanticdb-kotlinc compiler plugin?

method disambiguator fails to find method for type alias

Question in the Kotlin Slack unfortunately received no responses. We successfully extract the methods of the aliased type which look to include the method we're looking for, but indexOf returns -1. Following image shows the exception:

While this image shows the view in the debugger. The variables to observe are originalDesc (the "unwrapped" version of desc, the constructor), and the list of methods.

Commented-out test that triggers the bug: https://github.com/sourcegraph/lsif-kotlin/blob/main/semanticdb-kotlinc/src/test/kotlin/com/sourcegraph/semanticdb_kotlinc/test/SemanticdbSymbolsTest.kt#L92-L101

`java.lang.NoSuchMethodError: 'java.lang.String org.jetbrains.kotlin.ir.backend.js.lower.serialization.ir.JsManglerDesc.fqnString(org.jetbrains.kotlin.descriptors.DeclarationDescriptor, boolean)'`

CI error from the JetBrains plugin https://github.com/sourcegraph/jetbrains/actions/runs/7752759066/job/21142806054?pr=367#step:8:12011

Root problem seems to be caused by the Kotlin version mismatch: 1.9.22 in the JB plugin and 1.8.21 in scip-kotlin.

The regression started happening after this PR here sourcegraph/jetbrains#343

 Task :jetbrains-shared:scipCompileAll UP-TO-DATE
e: java.lang.NoSuchMethodError: 'java.lang.String org.jetbrains.kotlin.ir.backend.js.lower.serialization.ir.JsManglerDesc.fqnString(org.jetbrains.kotlin.descriptors.DeclarationDescriptor, boolean)'
	at com.sourcegraph.semanticdb_kotlinc.SemanticdbTextDocumentBuilder$symbolInformation$supers$1.invoke(SemanticdbTextDocumentBuilder.kt:89)

> Task :compileKotlin FAILED
	at com.sourcegraph.semanticdb_kotlinc.SemanticdbTextDocumentBuilder$symbolInformation$supers$1.invoke(SemanticdbTextDocumentBuilder.kt:88)
	at kotlin.sequences.FilteringSequence$iterator$1.calcNext(Sequences.kt:171)
	at kotlin.sequences.FilteringSequence$iterator$1.hasNext(Sequences.kt:194)
	at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:311)
	at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303)
	at kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)
	at com.google.protobuf.AbstractMessageLite$Builder.addAllCheckingNulls(AbstractMessageLite.java:372)
	at com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:434)
	at com.sourcegraph.semanticdb_kotlinc.Semanticdb$SymbolInformation$Builder.addAllOverriddenSymbols(Semanticdb.java:10992)
	at com.sourcegraph.semanticdb_kotlinc.SemanticdbTextDocumentBuilder.symbolInformation-vWNn3L8(SemanticdbTextDocumentBuilder.kt:102)
	at com.sourcegraph.semanticdb_kotlinc.SemanticdbTextDocumentBuilder.emitSemanticdbData-WnVtuAA(SemanticdbTextDocumentBuilder.kt:52)
	at com.sourcegraph.semanticdb_kotlinc.SemanticdbVisitor$emitAll$1.invoke(SemanticdbVisitor.kt:38)

Populate `kind` and `properties` fields in `SymbolInformation`

While the semanticdb SymbolInformation has kind and properties fields, they are currently not set by semanticdb-kotlinc. I propose to add those towards sourcegraph/scip-java/issues/666.

Rename this repo into semanticdb-kotlinc

wrong semanticdb symbol for function types' parameters

For the given construct private val onErrorStub: (Throwable) -> Unit = {}, we get a semanticdb symbol occurrence as follows:

  symbols {
    symbol: "io/reactivex/rxjava3/kotlin/subscribersKt#onErrorStub.`<no name provided>`."
    display_name: "Throwable"
    language: KOTLIN
  }

Notice the display_name, indicating its a symbol occurrence for the function type's parameter of type Throwable, which is incorrect

Kotlin 2.0.0 support

Hello, first of all, thank you very much for your work on this project! This plugin has been immensely useful for our project and we appreciate all the time and effort you've put into maintaining it.

I was wondering, with Kotlin 2.0.0 being released today, is there any ETA on this plugin being updated to support it so projects can begin picking up the newer release? Thanks in advance!