Validate KLIB ABI compatibility on Multiplatform

[fzhinkin]

There's an API to verify KLIB's binary compatibility that'll be released soon:
https://youtrack.jetbrains.com/Issue/KT-54402
https://github.com/JetBrains/kotlin/blob/master/compiler/util-klib-abi/ReadMe.md

BCV should use it to support KLIBs validation in addition to JVM bytecode validation.

[qwwdfsad]

Things that are worth keeping in mind (or maybe even split into a few subtasks):

Dependency on the Kotlin compiler

• We can hardcode the dependency for the first version, but then we'll have to update it each Kotlin release
• We can later provide a plugin extension that takes the dependency of the same version as KGP applied to the project

Default behaviour and migration path

• When we should enable Klib dump by default
• What is the naming scheme/migration path for non-JVM dumps, and how are they different (or the same?) across platform-specific, JS IR and WASM source-sets

[ddolovov]

I would recommend to use v2 of binary signatures (with mangled names included into the rendered signature text) vs v1 (with only hashes).

[fzhinkin]

What is the naming scheme/migration path for non-JVM dumps, and how are they different (or the same?) across platform-specific, JS IR and WASM source-sets

I see a few options here:

1. use unified naming for both jvm and non-jvm target: api/<target>/<projectName>.api

• pros: we already use such naming if a project contains multiple jvm-targets;
• cons: for an existing multiplatform project it means that an old api-file for a jvm-target need to be moved to jvm subdirectory.

2. for jvm targets don't do anything, for native targets use naming scheme mentioned above

• pros: much pleasant migration path as there's no need to update jvm-specific dumps
• cons: there will be a dependency between the jvm'a apiDump task and native tasks, need to look carefully how to deal with it

3. dump klibs to their own abi-directory: abi/<target>/<projectName>.abi

• pros: easier to implement (compared to (2))
• cons: adds some complexity by introducing an additional directory for dumps: its unclear why KLIB ABI dumps deserved a different top-level directory

It's also unclear if there's anything we should do about identical KLIB ABI dumps across different native targets.
For example, for kotlinx-io-core, dumps for linuxX64, linuxArm64, mingwX64 and all androidNative*-targets are identical, as well as all dumps for apple-targets are identical.
I don't really like the idea of storing identical files within the project and I don't like the idea of updating all these dumps once an API changed even more. But at the same time it's unclear if having identical ABI dumps for almost all targets is a common case (I would guess that it is) and how to group these dumps (assuming that they can diverge in the future).
It's tempting to group dumps by the source sets instead of targets, but a target may use multiple source sets.
For me, it's an open question and I can't name a good solution at the moment.

[qwwdfsad]

My vote is for option #1, hidden under the flag initially (because this functionality is going to be wildly unstable).
Cons are not that serious -- this will be a one-time change.

It's also unclear if there's anything we should do about identical KLIB ABI dumps across different native targets.

Probably we'll need to dump ABI for configured source-sets, not targets, implying that different targets in the same source-sets are identical

[fzhinkin]

Probably we'll need to dump ABI for configured source-sets, not targets, implying that different targets in the same source-sets are identical

A downside of such an approach is that a logic of searching for a source set dominating multiple targets tends to be fragile (it could work fine for default hierarchies, but I'm afraid that for non-default hierarchies we may end up with a bunch of unique source sets tied to targets).
And it also doesn't match how we generate dumps for JVM.

[fzhinkin]

So far, here's how it going to work:

• KLib validation will be disabled by default, to enable it one has to set klibValidationEnabled flag in apiValidation settings:

apiValidation {
    klibValidationEnabled = true
}

• when enabled, KLib validation will use v2-signatures in dumps, it could be changed using klibSignatureVersion option:

apiValidation {
    klibValidationEnabled = true
    klibSignatureVersion = 1 // if validated klib doesn't contain appropriate signatures, an exception will be thrown during the validation
}

• KLib validation supports ignoredPackages, ignoredClasses and nonPublicMarkers options the same way classfiles validation does, but all public* options are unsupported and will be ignored.

Right now, dumps will be generated (and verified) for each configured target that is supported by the host.

[ddolovov]

In the future we might implement v3 of signatures. Do I get it right that no changes will be required in the compatibility validator, it will support new version of signatures just out of box and all that will be actually needed is that the user should specify klibSignatureVersion = 1 in Gradle build file?

[fzhinkin]

@ddolovov You got it right: once v3 of signatures is available, users need only specify klibSignatureVersion = 3 to switch from a version they used previously.