kotlin / kotlin-spec

Kotlin Language Specification:

Kotlin Language Specification

This repository contains the specification of the Kotlin programming language, which describes how parts of the language should function in more detail, as compared to a more traditional user documentation on the Kotlin Website.

It would be most useful to those who are interested in how Kotlin works on a finer level and how its features interoperate, e.g., language enthusiasts, compiler writers and Kotlin power-users. However, if you are simply wondering, why some code you wrote works the way it does, this specification might help you get an answer to that.

Currently, the specification covers only what we call Kotlin/Core: fundamental parts of Kotlin which should function the same way irregardless of the underlying platform. In the future, we plan to extend it with additional platform-specific sections covering Kotlin/JVM, Kotlin/JS and Kotlin/Native.

Compatibility

Kotlin Language Specification is still in progress and has experimental stability level, meaning no compatibility should be expected between even incremental releases, any functionality can be added, removed or changed without warning.

Artifacts

HTML version of the specification is available on the Kotlin Website.

PDF version can be downloaded using the links in the HTML version or using the direct link.

Kotlin grammar

The grammar is a part of the specification and is also contained in this repository.

The reference grammar files in ANTLR4 format are located in the grammar folder. The human-friendly version is available on the Kotlin Website, in the HTML or PDF version of the specification.

Building the specification

The specification is built using Gradle. Therefore, most dependencies are downloaded by the build system.

However, there are several external dependencies which should be installed separately. For instructions on how to do this, please refer to your operating system documentation.

• Pandoc (tested with version 2.9.1)
• npm (tested with version 6.14.6)
• gpp (tested with version 2.25)
• bash (tested with version 5.0.17)

After installing these dependencies, building the specification is as easy as running:

./gradlew buildWeb buildPdf

which creates both HTML and PDF versions of the specification in ./build/spec.

When doing the build for the very first time, you will see Gradle downloading ideaIC-LATEST-EAP-SNAPSHOT.zip, which may take quite a long time. This IDEA snapshot is used for specification and grammar tests, and will be cached (until a new version is available), so this lengthy download should happen only once (in a while).

The PDF version is available in two flavours: single-file PDF, containing the complete specification, and a set of section-based PDFs, containing their respective sections.

The HTML version consists of a set of section-based pages linked together via an interactive table of contents.

Contributing

If you want to contribute to the specification, that's great! You can help us make the Kotlin specification better by one of the following ways.

1. Create an issue and describe what you think can be improved
2. Make a pull request and extends the specification
3. Discuss the specification on the Kotlin forums
4. Drop an email to Marat Akhin or Mikhail Belyaev with your suggestions

Note: if doing a pull request, it should be based on the develop branch.

We welcome any and all feedback to the specification, but may tweak, change or iterate with you on the contribution before including it in the specification.

Further details on how to contribute to the specification are available in CONTRIBUTING.md.

Reference

If one needs to reference this specification, they may use the following:

Marat Akhin, Mikhail Belyaev et al. "Kotlin language specification: Kotlin/Core", JetBrains / JetBrains Research, 2020

kotlin-spec's Issues

Add preliminary information about lattices to the Control- and data-flow analysis section

For additional information on which parts need clarification, see https://youtrack.jetbrains.com/issue/KT-48580

Html version of the specification is not available on

Got html error 404 on requesting link to html artifact http://jetbrains.github.io/kotlin-spec/

Purpose of modes stack size checking in lexer

Could you explain the purpose of modes stack size checking before popMode()?

kotlin-spec/grammar/src/main/antlr/KotlinLexer.g4

Line 44 in 091d421

RCURL: '}' { if (!_modeStack.isEmpty()) { popMode(); } };

Is this a workaround for more reliable error handling if the first open LCURL is missed?

Is this related to antlr/antlr4#2006

ANTLR parser fails when a newline exists inside an expression interpolated in a string

Hi team,

I noticed a problem when parsing files using the official Kotlin ANTLR grammar from https://kotlinlang.org/docs/reference/grammar.html.

This parses OK:

val numbers = mapOf("1" to 1, "2" to 2, "3" to 3)
println("output: ${numbers.entries.joinToString { "${it.key}: ${it.value}" }}")

This fails (but compiles without issue in e.g. Android Studio):

val numbers = mapOf("1" to 1, "2" to 2, "3" to 3)
println("output: ${
    numbers.entries.joinToString { "${it.key}: ${it.value}" }}")

This issue is possibly related to https://youtrack.jetbrains.com/issue/KT-26077. When adding parentheses around the expression, ANTRL does parse the statement just fine. Assuming this is the same issue, and since KT-26077 hasn't been touched in over a year, is there anything I might be able to do in the short term to work around this?

Many thanks!

Cover Kotlin/Core parts of coroutines in more detail

Using https://github.com/JetBrains/kotlin/blob/document-coroutines-codegen/compiler/backend/src/org/jetbrains/kotlin/codegen/coroutines/coroutines-codegen.md as a reference

incomplete document

too many "TODO" items to be useful at this point.

Discriminate overload resolution candidates which require (SAM) conversions

Exhibit 1:

package foo

fun interface Callback {
    fun doit(): Unit
}

fun foo(cb: () -> Unit) {} // (1)
fun foo(cb: Callback) {} // (2)

fun test() {
    foo {} // resolved to (1)
}

Exhibit 2:

package foo

fun ambiguous(sfn: suspend () -> Unit) = sfn // (1)
fun ambiguous(fn: () -> Unit) = fn // (2)

fun test(fn: () -> Unit) = ambiguous(fn) // resolved to (2)
// Note: this will need fixing after the compatibility mode disabling

Some additional info available at Quip (Disabling new inference compatibility mode).

Consider adding commentaries from docs to grammar in spec

The doc team expressed an idea to remove grammar from language documentation completely and just provide a link to the spec. This involves translating (and maybe rewriting to support the spec style) grammar comments from the docs to the spec grammar.

We need to devise some solution to do this both in the build (rewrite grammar-converter) and in the text.

Grammar: annotations in valueArgument

Current valueArgument non-terminal does not allow for multiple annotations:

valueArgument:
    [annotation]
    {NL}
    [simpleIdentifier {NL} '=' {NL}]
    ['*']
    {NL}
    expression 

Grammar rule 'classDeclaration' error

Description

I litterally just copy pasted the 3 kotlin .g4 files and ran grun to check the generated AST using the MutableStack<E> code from the example here : https://play.kotlinlang.org/byExample/01_introduction/06_Generics
And it seems like it can't match the LCURLY bracket of the classBody rule ? Could anyone confirm please

Output + AST

Talk about "bare types" (parameterized types without parameters)

Bare types can be encountered, for example, in type checking or cast expressions, and are somewhat related to runtime-available types.

Also, they currently have some problems (see KT-15533 / KT-15706).

Inconsistent formatting for rule alternatives

The following alternatives are declared without line breaks:

additiveOperator
    : ADD | SUB
    ;

But the following - with line breaks:

multiplicativeOperator
    : MULT
    | DIV
    | MOD
    ;

I suggest always using line breaks for rule alternatives.

Two test cases fail after a refresh checkout

Hi,

After checking out the 3ee8926 latest commit, I ran the grammar test and found the following errors:

org.jetbrains.kotlin.spec.grammar.TestRunner > doTest[grammar$functions$lt-gt-ambiguity.kt] FAILED
    com.intellij.rt.execution.junit.FileComparisonFailure at TestRunner.kt:83

org.jetbrains.kotlin.spec.grammar.TestRunner > doTest[psi$stringTemplates$StringTemplateWithTryWithoutBlockInShortEntry.kt] FAILED
    java.lang.AssertionError at TestRunner.kt:107

5342 tests completed, 2 failed, 69 skipped

Add details of how `val`/`var` properties work w.r.t. inheritance and overriding

interface I1 {
    var ordinal: Int
}

interface I2 {
    val ordinal: Int
}

class Case1(override var ordinal: Int) : I2, I1

// ERROR!
class Case2(override val ordinal: Int) : I2, I1 // VAR_OVERRIDDEN_BY_VAL

Set of restricted chars for JVM

I've noticed that despite following statement in syntax.md#identifiers, Kotlin JVM compiler is fine with escaped identifiers with '(' in it. I've also found this checker which seems to exactly follow JVM specification

Note: for example, the following characters are not allowed in identifiers used as declaration names on the JVM platform even when escaped due to JVM restrictions: (, ), {, }, [, ], .

Licensing question

Hi,

I'd like to implement support for Kotlin in Universal Ctags project. This project is in turn used in Geany (my IDE of choice), and also can be used in vim, Emacs and other editors.

I have used the ANTLR grammar files present in this repository to create a peg file describing Kotlin language, which probably makes it a derivative work, right? My intention was to use this derived grammar in to generate a Kotlin parser in C to be used in Ctags. Now the problem is, that both Universal Ctags and Geany are GPLv2, which is as far as I know not compatible with Apache license used for this repository.

I know I've been a fool to not check the license beforehand... But I believe that many Kotlin users would benefit from this, so I'd like to ask if there is some way to resolve the license incompatibility? I assume it would be too big of a change to re-license or dual-license Kotlin grammar just to allow it's use in other projects. Could it perhaps be possible to grant me/the projects a permission to use it for this specific purpose? I am not a lawyer, so I actually have a very vague idea of what is possible...

Grammar: incorrect parsing of the last method call in expression

val bar = 1 + foo() is parsed as:

----declaration|propertyDeclaration : null
------VAL : val
------variableDeclaration|simpleIdentifier|Identifier : bar
------ASSIGNMENT : =
------expression|...|genericCallLikeComparison : null
--------infixOperation|...|additiveExpression : null
----------multiplicativeExpression|...|IntegerLiteral : 1
----------additiveOperator|ADD : +
----------multiplicativeExpression|...|Identifier : foo
--------callSuffix|valueArguments : null
----------LPAREN : (
----------RPAREN : )
----semis|EOF : <EOF>

with ANTLR 4.9.1 and the last version of grammar, but version from the 3ee8926 gives different result, that looks like the correct one:

----declaration|propertyDeclaration : null
------VAL : val
------variableDeclaration|simpleIdentifier|Identifier : bar
------ASSIGNMENT : =
------expression|...|additiveExpression : null
--------multiplicativeExpression|...|IntegerLiteral : 1
--------additiveOperator|ADD : +
--------multiplicativeExpression|...|postfixUnaryExpression : null
----------primaryExpression|simpleIdentifier|Identifier : foo
----------postfixUnarySuffix|callSuffix|valueArguments : null
------------LPAREN : (
------------RPAREN : )
----semis|EOF : <EOF>

Grammar: inconsistency with in vs &&

In actual kotlin, the following code has different priority:

when (x) {
       in y && z -> /* parsed as in (y && z) */
}

when {
      x in y && z -> /*  parsed as (x in y) && z */
}

We need to recheck it against the grammar

Wrong link to increment/decrement expressions specification

https://kotlin.github.io/kotlin-spec/sections/statements.html#operator-assignments

Note: although for most real-world use cases operators ++ and -- are similar to operator assignments, in Kotlin they are expressions and are described in the corresponding section of this specification.

Here "corresponding section" link points to https://kotlin.github.io/kotlin-spec/sections/control--and-data-flow-analysis.html#expressions-1, but it doesn't describe increment/decrement operators and their desugaring at all.
In fact, increment and decrement operators are described in corresponding subsections of https://kotlin.github.io/kotlin-spec/sections/expressions.html#prefix-expressions and https://kotlin.github.io/kotlin-spec/sections/expressions.html#postfix-operator-expressions.

Not all expressions are supported condition form of the "when expression"

In list of the supported condition forms of the "when" expressions says in the last item, that any other expression is supported. But "continue" expression and "break" expression (without labels) can't be used. Also can't be used "spread" operator as "when" condition, which is in the "Operator expressions" section.

Maybe make a correction here?

See this spec test.
(I will add a negative test for spread operators a little latter)

where is getCompilerTests.sh

The grammar document reads about grammar test:

Download compiler tests using the run script: ./getCompilerTests.sh (you must have svn installed for this);

Where can I find this script? I'd like to run the grammar test and add some testing code too.

Missing KSP specification

I am still missing KSP specification for Servlets written in Kotlin. Please, can you make one? I found it a blockage point for making Kotlin more widespread on Kotlin(Java) EE environment.

gradle build error

I got the following build error message when I tried to build and run grammar tests. What should I do?

grammar$ gradle tasks --all

Configure project :
e: ~/kotlin-spec/build.gradle.kts:20:6: Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
public fun Project.sourceSets(configure: Action): Unit defined in org.gradle.kotlin.dsl
public fun KotlinJvmProjectExtension.sourceSets(configure: Action<NamedDomainObjectContainer>): Unit defined in org.gradle.kotlin.dsl
e: ~/kotlin-spec/build.gradle.kts:21:5: Expression '"main"' of type 'String' cannot be invoked as a function. The function 'invoke()' is not found
e: ~/kotlin-spec/build.gradle.kts:22:14: Unresolved reference: srcDir
e: ~/kotlin-spec/build.gradle.kts:115:22: Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
public val Project.sourceSets: SourceSetContainer defined in org.gradle.kotlin.dsl
public val KotlinJvmProjectExtension.sourceSets: NamedDomainObjectContainer defined in org.gradle.kotlin.dsl
e: ~/kotlin-spec/build.gradle.kts:121:22: Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
public val Project.sourceSets: SourceSetContainer defined in org.gradle.kotlin.dsl
public val KotlinJvmProjectExtension.sourceSets: NamedDomainObjectContainer defined in org.gradle.kotlin.dsl

FAILURE: Build failed with an exception.

• Where:
  Build file '~/kotlin-spec/build.gradle.kts' line: 20

• What went wrong:
  Script compilation errors:

  Line 020: java.sourceSets {
  ^ Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
  public fun Project.sourceSets(configure: Action): Unit defined in org.gradle.kotlin.dsl
  public fun KotlinJvmProjectExtension.sourceSets(configure: Action<NamedDomainObjectContainer>): Unit defined in org.gradle.kotlin.dsl

  Line 021: "main" {
  ^ Expression '"main"' of type 'String' cannot be invoked as a function. The function 'invoke()' is not found

  Line 022: java.srcDir("build-utils/src")
  ^ Unresolved reference: srcDir

  Line 115: classpath = java.sourceSets["main"].runtimeClasspath
  ^ Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
  public val Project.sourceSets: SourceSetContainer defined in org.gradle.kotlin.dsl
  public val KotlinJvmProjectExtension.sourceSets: NamedDomainObjectContainer defined in org.gradle.kotlin.dsl

  Line 121: classpath = java.sourceSets["main"].runtimeClasspath
  ^ Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
  public val Project.sourceSets: SourceSetContainer defined in org.gradle.kotlin.dsl
  public val KotlinJvmProjectExtension.sourceSets: NamedDomainObjectContainer defined in org.gradle.kotlin.dsl

5 errors

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

Left recursion for expression rules?

ANTLR 4 supports left recursion. Grammar with them looks more clearly and performance of generated parser is faster.

Compare:

expression
    : disjunction
    ;

disjunction
    : conjunction (NL* DISJ NL* conjunction)*
    ;

conjunction
    : equality (NL* CONJ NL* equality)*
    ;

equality
    : comparison (equalityOperator NL* comparison)*
    ;

... // Other rules

And

expression
    : expression equalityOperator NL* expression
    | expression NL* CONJ NL* expression
    | expression NL* DISJ NL* expression
    | ... // Other alternatives including tokens
    ;

IEEE 754 comparison rules are not documented in the spec

See details here https://youtrack.jetbrains.com/issue/KT-48489

How to run a specific test in the test suite?

Hi,

I'm new to the Java test framework. How can I run a single test?

The :grammar:test command run all tests, which I don't want.

Thanks.

describe default value evaluation

Right now the spec doesn't specify when default values are actually evaluated. Would be nice to highlight that this is the case every time when a default value is required, but not in other cases.

So explain that someFunction is invoked every time no value for the parameter is provided and in other cases it is not invoked (so no overhead).

fun test(a:Int = someFunction()) {
 }

This might look like an obvious clarification, but different languages solve this in different ways. So might take away some doubt depending where you coming from.

Garbage Collection

Hi,

I know that the spec is still in development. But, I would like to know more how the garbage collection works with Kotlin and implications for Strong References.

I would like to understand better how Kotlin interacts with the garbage collector.
When variables are ready to be destroyed, how we can manage memory leaks...

I tried to search, but I haven't found much information.

Thanks in advance

Define which scope function parameters are in

https://youtrack.jetbrains.com/issue/KT-28932#focus=Comments-27-5169515.0-0

Function parameters are a little bit weird: they're obviously in the scope of a function, but their defaults a little bit different to the scope of the function, and you can use local variables which shadow them (which is explicitly written as impossible for decls in the same scope in spec).

Also: primary constructor parameters, they're even weirder.

Using of string literals instead of token names for clarity

Since ANTLR 4.7.2 supports linking of token names and string literal for all grammars (see antlr/antlr4#2347), maybe it makes sense to use string literals for some tokens like LPAREN, COMMA and similar for clarity?

Compare the following rule:

ifExpression
    : IF NL* LPAREN NL* expression NL* RPAREN NL* (controlStructureBody | SEMICOLON)
    | IF NL* LPAREN NL* expression NL* RPAREN NL* controlStructureBody? NL* SEMICOLON? NL* ELSE NL* (controlStructureBody | SEMICOLON)
    ;

And the following with string literals:

ifExpression
    : IF NL* '(' NL* expression NL* ')' NL* (controlStructureBody | ';')
    | IF NL* '(' NL* expression NL* ')' NL* controlStructureBody? NL* ';'? NL* ELSE NL* (controlStructureBody | ';')
    ;

ANTLR generates identical code for both grammars.

NOT_IS token never generated

I believe that the NOT_IS token is never generated due to EXCL_WS and EXCL_NO_WS being declared above it.

In my test, a simple test written in Kotlin:

val lexer = KotlinLexer(CharStreams.fromString("!is"))
val tokenStream = CommonTokenStream(lexer).apply { fill() }
val tokens = tokenStream.tokens
assertThat(tokens).hasSize(2) // This fails
assertThat(tokens[0].type).isEqualTo(KotlinLexer.NOT_IS)
assertThat(tokens[1].type).isEqualTo(KotlinLexer.EOF)

What I actually get is 3 tokens, !, is, and <EOF>.

downloading ideaIC-LATEST-EAP-SNAPSHOT.zip?

Hi,

When I run the grammar test, I have noticed that Gradle downloads the large file ideaIC-LATEST-EAP-SNAPSHOT.zip automatically from time to time when there is an update. Is the IDE package mandatory to run the tests? Is there a way to disable the downloading?

Thank you.

Type system / Glossary / Type parameter

Actual: "Formal type argument of a type constructor"
Proposal: "Formal type parameter of a type constructor" or better "Formal parameter of a type constructor"

Typo in "Reoad tests" button label

Grammar: dot and save navigation operators allow for newlines, while reference operator does not

This is correct:

x
.y
.z
.f

This is not:

x
::y
::z
::f

(well, it is correct, but is not a navigation chain as the first example)

In the "Exhaustive when expressions" no says about objects, which belong to the sealed classes

In the list of the cases, when "when" expression will be exhaustive says about sealed class subtypes covering using type test conditions (type test condition is type checking operator followed by type).
But we can "when" make exhaustive using just an enumeration of objects.

See this spec test (case 4).

Maybe add info about exhaustive "when" written using objects, which belong to the sealed classes?

Add examples to "type capturing" topic

Type capturing is a quite complex topic so it'd be nice to add at least some examples there.
https://kotlinlang.org/spec/type-system.html#type-capturing

Refine how renaming import works

TL;DR it changes the name of the imported entity for the purposes of resolution, even in the same file; i.e., if one were to reimport test as foo in the same file, it is no longer available as test and must be referenced as foo.

Systemically replace "undefined behavior" with "unspecified behavior"

The term "undefined behavior" is generally associated with C/C++, where it means "program my produce arbitrary results", while in Kotlin, especially on JVM, such arbitrary behaviors are not possible. Some constructs may simply behave in unspecified ways.

P.S. The fact that K/N currently allows truly undefined behaviors in C/C++ sense is not relevant. I still think a less scary wording should be used.

Refine `type-checking expression`

The workings of is/!is should be defined via subtyping rules, as type [...] is the same as [...] type is just plain wrong

Annotation classes can have type parameters

The current spec says otherwise.

Add SAM(lambda) conversion syntax

The following syntax for SAM conversion is supported

val lambda = { "OK" }

fun box() = Sam(lambda).get()

fun interface Sam {
    fun get(): String
}

Refine explanation about integer literals and their type w.r.t. context

1. Better describe integer literal type dependency on its context and/or bit size
2. We also must not forget about type selection priority

Is keyword "this" in memberAccessOperator an identifier or thisExpression?

In the psi/examples/collections/ArrayList.kt grammar test file, an expression uses "this" in the following way:

class ArrayList<T> : IMutableList<T> {
    ...
    override fun remove() {
      checkVersion()
      val result = ArrayList.this.remove(index - 1)
      ...
    }
  }

The expected antlrtree is:

      propertyDeclaration
        VAL("val")
        variableDeclaration
          simpleIdentifier
            Identifier("result")
        ASSIGNMENT("=")
        expression
          disjunction
            conjunction
              equality
                comparison
                  infixOperation
                    elvisExpression
                      infixFunctionCall
                        rangeExpression
                          additiveExpression
                            multiplicativeExpression
                              asExpression
                                prefixUnaryExpression
                                  postfixUnaryExpression
                                    primaryExpression
                                      simpleIdentifier
                                        Identifier("ArrayList")
        DOT(".")
        THIS("this")
        DOT(".")
        Identifier("remove")
        LPAREN("(")
        Identifier("index")
        SUB("-")
        IntegerLiteral("1")
        RPAREN(")")

The ".this" is not a part of the expression. What's the semantics of "this" here? I would assume that it has the same semantic meaning as in Java, i.e., it refers to an instance of ArrayList. Is this a bug in the grammar?

HTML sections are currently broken

Links in HTML-sections build in develop use "pdf" as file extension. Probably an aftermath of the whole compoundFilter.kt business

Describe type approximation in more detail

Grammar: empty body after delegate parsed as lambda expression

object : Base, Base2 by Delegate() {     }

the {} in the end is parsed as a lambda expression, but

• trailing lambdas are forbidden for delegation expressions
• it is, in fact, a class body

Get rid of duplication in rule alternatives

Consider the following rule:

getter
    : modifiers? GET
    | modifiers? GET NL* LPAREN NL* RPAREN (NL* COLON NL* type)? NL* functionBody
    ;

Why don't mark the tail just by the optional operator (?)?

getter
    : modifiers? GET (NL* LPAREN NL* RPAREN (NL* COLON NL* type)? NL* functionBody)?
    ;

Also:

parenthesizedUserType
    : LPAREN NL* userType NL* RPAREN
    | LPAREN NL* parenthesizedUserType NL* RPAREN
    ;

to

parenthesizedUserType
    : LPAREN NL* (userType | parenthesizedUserType) NL* RPAREN
    ;

And others.

Introduction to Grammar should specify how encoding for source files is specified

Before the lexical grammar it might be worth specifying how a source file is decoded from bytes to code-units.

For example, the Go spec mandates UTF-8

Source code is Unicode text encoded in UTF-8.

Maybe at the top of ¶ Syntax

Overload resolution intro is misleading

Currently it states that it only applies to functions and function-like callables, which is incorrect