Repository navigation
scip-kotlinc Kotlin update, fixes and improvements - #973
Conversation
|
Hi @nicolas-guichard, thank you for this outstanding contribution! It's true that we experimented a bit with switching to |
5aebd5e to
d778a65
Compare
|
That should fix the CI failures except the test runs against https://github.com/square/okio and https://github.com/hoc081098/FlowExt because they target 2.2.0. square/okio master has updated to 2.2.21 and hoc081098/FlowExt master still targets 2.2.0. An option would be to patch them to 2.4.0 in our GitHub Actions workflow between the “Clone repository” and “Run scip-java index” steps. Hopefully that should just work™ unless they also use private API. Another option would be to replace them with other projects that target 2.4.0. What do you prefer, or do you have a better option in mind? |
This is definitely a preferred option. We do not have repo-specific tests (we do not check contents of the generated indexes). Please just pick anything that works and is relatively popular/maintained. :) |
d778a65 to
365d6c3
Compare
This goes one bit further than a3eb2f2 and doesn't even try to set project repositories when settings repositories are set. This prevents the build from failing on projects that set repositories in Settings but don't set FAIL_ON_PROJECT_REPOS, such as the KotlinConf app.
For scip-kotlinc: * CheckerContext.containingFile was renamed to containingFileSymbol. * FirCallableSymbol<*>.directOverriddenSymbolsSafe now takes context by context parameter. * FirCallableSymbol.callableId was made nullable and is replaced by FirCallableSymbol.callableIdForRendering for rendering purpose. * Anonymous objects are now considered as locals. Ported from mozsearch/semanticdb-kotlinc@3f47564
This will allow us to distinguish between method and constructors, and get better kind information for locals. Ported from mozsearch/semanticdb-kotlinc@6affaf7
This changes the symbols of getters and setters to be `x.get().` and `x.set().` instead of `getX().` and `setX().`. Ported from mozsearch/semanticdb-kotlinc@7b4bd70
This will allow us to get the parent symbol for locals as well. For non-locals, it should be equal to the current symbol minus the last segment. Ported from mozsearch/semanticdb-kotlinc@63c9b65
Reduces the debug log clutter. Ported from https://github.com//mozsearch/semanticdb-kotlinc/commit/4b7afc21337cba75c2c7b724043bcc9204eb9eb0
This should reduce memory usage a bit and avoids lots of “given file is not under the sourceroot” clutter when running the tests. Ported from https://github.com//mozsearch/semanticdb-kotlinc/commit/65b1898de26c02929f1fbd445a3c9c92e8fd3bab
scip-kotlinc changes: getContainingSymbol was moved from o.j.k.fir.analysis.checkers to org.jetbrains.kotlin.fir.resolve. CompilerPluginRegistrar now has a virtual pluginId which must match the CommandLineProcessor. Ported from mozsearch/semanticdb-kotlinc@cde86db
- Remove unnecessary cast to FirClass in SymbolsCache (FirClassSymbol.fir already returns FirClass) - Use parameterless toClassLikeSymbol() overload in AnalyzerCheckers - Remove unnecessary inline modifier from test snippet Ported from mozsearch/semanticdb-kotlinc@eb68377
…age collector Previously handleException constructed a fresh CompilerConfiguration() to retrieve the message collector key, which always fell back to PrintingMessageCollector(System.err) — ignoring whatever collector the actual compilation was configured with. Pass the real CompilerConfiguration into PostAnalysisExtension so that exception messages are routed through the same collector as all other diagnostics. Also change the severity from EXCEPTION to WARNING, since EXCEPTION is treated as isError=true by the Kotlin compiler and would cause the build to fail — contrary to the plugin's intentional "log-and-continue" behaviour. The exception test now captures compiler output via result.messages and asserts that the warning is actually emitted with the expected content, rather than only checking the exit code. Ported from mozsearch/semanticdb-kotlinc@e6b72f5
This is a test for 88f6272784c5e56eb135019a4b0dda5dd0808017. Ported from mozsearch/semanticdb-kotlinc@2c1f170
…turn types - local functions: verifies that named functions declared inside a function body receive local symbols (local0, local1), that explicit return type references are emitted (kotlin/Int#), and that call-site references resolve to the same local symbol. Also adds a SemanticdbSymbolsTest entry confirming the locals counter increments correctly. - user-defined class as return type: verifies that SemanticSimple- FunctionChecker emits a REFERENCE occurrence for a user-defined class appearing in the return-type position (sample/MyClass#), exercising the returnTypeRef path that built-in types do not reach. Ported from mozsearch/semanticdb-kotlinc@ff23a7e
displayName() in SemanticdbTextDocumentBuilder fell through to `firBasedSymbol.toString()` for two symbol types, producing strings like "FirTypeAliasSymbol sample/MyAlias" and "FirTypeParameterSymbol T" instead of the short name. - Broaden the FirClassSymbol branch to FirClassLikeSymbol so that FirTypeAliasSymbol (a subtype of FirClassLikeSymbol but not of FirClassSymbol) also uses classId.shortClassName. - Add an explicit FirTypeParameterSymbol branch that returns symbol.name.asString(). Add tests that exercise the corrected paths: - typealias: first test for SemanticTypeAliasChecker; verifies the DEFINITION occurrence and SymbolInformation (including the now-correct displayName "MyAlias"). Notes that type-alias references in value declarations are not currently tracked (the property checker resolves aliases to their expansion). - type parameters: first test for SemanticTypeParameterChecker; verifies the DEFINITION occurrence and SymbolInformation (displayName "T"). Notes that T in type-annotation positions does not produce REFERENCE occurrences because toClassLikeSymbol() returns null for type parameters. Ported from mozsearch/semanticdb-kotlinc@20b1d20
…rrences - Fix SemanticSimpleFunctionChecker and SemanticPropertyChecker to emit REFERENCE occurrences for extension receiver types - Add SemanticEnumEntryChecker to emit DEFINITION occurrences for enum entries (previously missing from AnalyzerDeclarationCheckers) - Fix semanticdbKind() to return Kind.ENUM_MEMBER for FirEnumEntry (FirEnumEntry extends FirVariable, which would otherwise map to Kind.LOCAL) - Fix SemanticClassReferenceExpressionChecker (is/as operators) to use the already-extracted typeRef and source locals consistently rather than re-accessing expression.conversionTypeRef on each line - Extract emitTypeRef() helper to reduce duplication across checkers; use it for supertype references in SemanticClassLikeChecker so they get the same fake-source guard as other type reference emissions - Add tests for extension receivers, enum entries, named/unnamed companion objects, string template references, and is/as type references Ported from mozsearch/semanticdb-kotlinc@d823d35
…uators - multiple supertype references: verifies that SemanticClassLikeChecker emits REFERENCE occurrences for every entry in superTypeRefs, and that SymbolInformation.overriddenSymbols is populated for all supertypes - three-way overload disambiguator: extends the existing two-overload coverage to three, verifying the (+1)/(+2) suffix counting logic in methodDisambiguator() Ported from mozsearch/semanticdb-kotlinc@24b6433
The "non-0-based" phrasing predates this branch but is misleading: line helpers return 1-based values (callers subtract 1 for protobuf), while column helpers already return 0-based values. Make the docstrings say what they actually return. Ported from mozsearch/semanticdb-kotlinc@3059517
Add two test cases:
- enum entry with body (Op.PLUS { override ... }): the body is modeled
as a synthetic anonymous subclass, so the entry gets a multi-line
enclosing_range and the overridden member surfaces as a local symbol
rather than a sample/Op#PLUS.apply() global. Asserts both.
- multi-line generic class declaration: general coverage we lacked for
generic classes. Asserts enclosing_range for the class, its type
parameter, and a member.
Ported from mozsearch/semanticdb-kotlinc@979f1f3
Changes in scip-kotlinc: - Rename FirSimpleFunction -> FirNamedFunction in SemanticSimpleFunctionChecker - Replace removed isLocalMember with isLocalDeclaredInBlock (moved within org.jetbrains.kotlin.fir.analysis.checkers.declaration) Ported from mozsearch/semanticdb-kotlinc@832bf44
Changes to scip-kotlinc: - -Xcontext-parameters argument isn't required anymore - FirAllModifierRenderer now takes an argument This also replaces the Kotlin test repos with repos using Kotlin 2.4.0. Ported from mozsearch/semanticdb-kotlinc@89427a5
365d6c3 to
a269bdb
Compare
|
The KotlinConf App was failing because it doesn't specify I've included a fix for this here, but I'm happy to split that into another PR if you prefer. |
jupblb
left a comment
There was a problem hiding this comment.
Thank you! I believe this PR is ok to merge as-is. Awesome work! :)
Port the remaining SymbolInformation fields #973 added to scip-kotlinc. - kind follows the declaration: Class (including objects, enums and annotation classes), Interface, TypeAlias, Constructor, Method (including accessors and local functions), Property (including local variables, loop and catch variables, and destructuring entries), Parameter, TypeParameter and EnumMember. - enclosing_symbol is the nearest containing declaration, or the package for top-level declarations. Synthetic accessors are enclosed by their property, and `val`/`var` constructor parameter properties by the class. Declarations inside lambdas have no enclosing symbol. On the Kotlin snapshot corpus, every global symbol and every local emitted by both indexers has the same kind and enclosing_symbol as scip-kotlinc on main. The ported AnalyzerTest cases get back their kind and enclosing_symbol expectations, and the snapshot goldens gain `kind` lines. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's accessor symbols: getters and setters are now `Class#banana.get().` / `Class#banana.set().` instead of `Class#getBanana().` / `Class#setBanana().`, for source properties, `val`/`var` constructor parameters, explicit accessors, the enum `entries` getter and references to library properties (`kotlin/String#length.get().`). Setter parameters follow (`Class#banana.set().(value)`). Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's extension symbols: a top-level extension function or property is owned by its receiver type within the extension's own package, so `fun String.foo()` in `sample` is `sample/String#foo().` and `kotlin.collections.forEachIndexed` on `Iterable` is `kotlin/collections/Iterable#forEachIndexed().`. This applies to source declarations and to references to library extensions. Extension disambiguators count the same-named members of the receiver class (when it is in the extension's package) followed by the same-named extensions on that receiver in the package, so a member `MyClass#foo().` and an extension `MyClass#foo(+1).` stay distinct. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's local symbols: anonymous objects and their members are now local symbols, numbered in declaration order, instead of `<anonymous object at N>` globals. An enum entry body declares a local anonymous subclass of the enum, anchored at the entry identifier, which owns the body's members, so `PLUS { override fun apply() }` defines `local 0` (`object : Op`), its constructor and `apply` as locals rather than `Op#PLUS.apply().`. The implicit constructor of a local class is now a local symbol too; it was emitted as the invalid symbol "local 0`<init>`().". Unlike scip-kotlinc, enum entries with constructor arguments but no body do not declare an anonymous class: that is an artifact of FIR's desugaring, and those entries are plain instances of the enum class. Port the `local classes` and `anonymous object` cases from scip-kotlinc's AnalyzerTest to cover this. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's resolved-qualifier references: a class name used as its companion object's receiver (`Bar.instance()`, `CompanionOwner.create()`) now references the class (`Bar#`) instead of the companion (`Bar#Companion#`). Naming the companion explicitly (`Bar.Companion`, `Foo.Factory`) still references it. Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce Co-authored-by: Amp <amp@ampcode.com>
Port #973's range for unnamed companion objects: the definition, and that of its implicit constructor, now covers the `companion` keyword instead of the declaration's first line (`companion object {`). scip-kotlinc's implicit-constructor range ran past the end of the line; here both definitions share the keyword range. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Match the FirRenderer output that #973's Kotlin 2.4 update introduced: - Enum entries render as `public final static enum entry RED: Color` instead of `public final val RED: Color`, and the `entries` getter as `public static get()`. - Functions and classes declared in a code block render as `local` (`local final fun inner(): Unit`); members of local classes and anonymous objects keep their visibility. - Extension functions and properties render their receiver (`public final fun String.foo(): Int`). The other differences in scip-kotlinc's 2.4 output are renderer internals and are not reproduced: evaluated annotation arguments (`@Tagged(tag = String(run) [evaluated = ...])`) and expanded type alias markers (`{snapshots/StringContainer=} Container<String>`). The ported AnalyzerTest cases now use scip-kotlinc's expectations for these signatures. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
#973 added 17 test cases to scip-kotlinc's AnalyzerTest and extended `type operators`; replacing scip-kotlinc deleted them. Run the same sources through KotlinAnalysisIndexer instead, using a copy of the ScipBuilders DSL. The #973 output changes are not part of scip-kotlin-analysis yet, so the expectations follow its current output: - No `kind` or `enclosing_symbol` assertions. - Extension functions and properties keep top-level symbols (`sample/foo().` instead of `sample/String#foo().`). - Enum entries render as `val`, and members of an enum entry body are `sample/Op#PLUS.apply().` instead of locals of an anonymous class. - `Foo.Factory` emits separate `Foo#` and `Foo#Factory#` references, an implicit companion receiver references `Bar#Companion#`, and the unnamed companion definition range ends at the end of its first line. Signature assertions for local functions and extension receivers are dropped: scip-kotlin-analysis currently renders `public final fun inner()` for local functions and omits the extension receiver. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port the remaining SymbolInformation fields #973 added to scip-kotlinc. - kind follows the declaration: Class (including objects, enums and annotation classes), Interface, TypeAlias, Constructor, Method (including accessors and local functions), Property (including local variables, loop and catch variables, and destructuring entries), Parameter, TypeParameter and EnumMember. - enclosing_symbol is the nearest containing declaration, or the package for top-level declarations. Synthetic accessors are enclosed by their property, and `val`/`var` constructor parameter properties by the class. Declarations inside lambdas have no enclosing symbol. On the Kotlin snapshot corpus, every global symbol and every local emitted by both indexers has the same kind and enclosing_symbol as scip-kotlinc on main. The ported AnalyzerTest cases get back their kind and enclosing_symbol expectations, and the snapshot goldens gain `kind` lines. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's accessor symbols: getters and setters are now `Class#banana.get().` / `Class#banana.set().` instead of `Class#getBanana().` / `Class#setBanana().`, for source properties, `val`/`var` constructor parameters, explicit accessors, the enum `entries` getter and references to library properties (`kotlin/String#length.get().`). Setter parameters follow (`Class#banana.set().(value)`). Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's extension symbols: a top-level extension function or property is owned by its receiver type within the extension's own package, so `fun String.foo()` in `sample` is `sample/String#foo().` and `kotlin.collections.forEachIndexed` on `Iterable` is `kotlin/collections/Iterable#forEachIndexed().`. This applies to source declarations and to references to library extensions. Extension disambiguators count the same-named members of the receiver class (when it is in the extension's package) followed by the same-named extensions on that receiver in the package, so a member `MyClass#foo().` and an extension `MyClass#foo(+1).` stay distinct. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's local symbols: anonymous objects and their members are now local symbols, numbered in declaration order, instead of `<anonymous object at N>` globals. An enum entry body declares a local anonymous subclass of the enum, anchored at the entry identifier, which owns the body's members, so `PLUS { override fun apply() }` defines `local 0` (`object : Op`), its constructor and `apply` as locals rather than `Op#PLUS.apply().`. The implicit constructor of a local class is now a local symbol too; it was emitted as the invalid symbol "local 0`<init>`().". Unlike scip-kotlinc, enum entries with constructor arguments but no body do not declare an anonymous class: that is an artifact of FIR's desugaring, and those entries are plain instances of the enum class. Port the `local classes` and `anonymous object` cases from scip-kotlinc's AnalyzerTest to cover this. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Port #973's resolved-qualifier references: a class name used as its companion object's receiver (`Bar.instance()`, `CompanionOwner.create()`) now references the class (`Bar#`) instead of the companion (`Bar#Companion#`). Naming the companion explicitly (`Bar.Companion`, `Foo.Factory`) still references it. Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce Co-authored-by: Amp <amp@ampcode.com>
Port #973's range for unnamed companion objects: the definition, and that of its implicit constructor, now covers the `companion` keyword instead of the declaration's first line (`companion object {`). scip-kotlinc's implicit-constructor range ran past the end of the line; here both definitions share the keyword range. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
Match the FirRenderer output that #973's Kotlin 2.4 update introduced: - Enum entries render as `public final static enum entry RED: Color` instead of `public final val RED: Color`, and the `entries` getter as `public static get()`. - Functions and classes declared in a code block render as `local` (`local final fun inner(): Unit`); members of local classes and anonymous objects keep their visibility. - Extension functions and properties render their receiver (`public final fun String.foo(): Int`). The other differences in scip-kotlinc's 2.4 output are renderer internals and are not reproduced: evaluated annotation arguments (`@Tagged(tag = String(run) [evaluated = ...])`) and expanded type alias markers (`{snapshots/StringContainer=} Container<String>`). The ported AnalyzerTest cases now use scip-kotlinc's expectations for these signatures. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a153c0-ec1f-774a-993c-bca3256017ce
We've been maintaining a semanticdb-kotlinc fork at https://github.com/mozsearch/semanticdb-kotlinc for some time now, which was kept up-to-date with Kotlin updates with a couple of extra fixes and improvements along the way.
Since this repo seems to be actively maintained again, I've rebased our changes on top of the new scip-kotlinc to submit them here.
Now I only just saw #963 which plans to drop the plugin in favor of the Analysis API (which makes sense), so… This may still be useful until that PR is fully fleshed-out.