Publish Gradle Module Metadata and variant info

[jeffdgr8]

This allows Gradle to identify the com.couchbase.lite:couchbase-lite-java and com.couchbase.lite:couchbase-lite-android artifacts as platform-specific variants of the same library. If both dependencies show up in a Gradle configuration, Gradle will choose the one appropriate for the platform being compiled and exclude the other variant.

These changes allow the publication to include a Gradle Module Metadata .module file, which contains similar info to the .pom, but has additional features, such as being variant aware.

These changes define the other platform variant in the .module metadata. Since the java and android variants are each published from their own module, the other platform variant is published as a variant dependency, similar to this workaround, but utilizing Gradle APIs.

The current versions of the Android Gradle Plugin (7.1-8.2) are using the Kotlin Gradle Plugin to create their publication variant components, which wraps the AdhocComponentWithVariants API that is required to add additional variants to a component. So reflection is needed to access this API in the Android module, but this can be replaced with a direct cast of the component, the same as the Java module, if my fix is accepted in the Kotlin Gradle Plugin.

[bmeike]

Jeff...
Sorry I've ignored this for so long.

I can't do without the USE_MAVEN_LOCAL flag: it is the foundation of dev testing.

Can you do this without removing it?

[jeffdgr8]

@bmeike yes, sorry, I can revert that change. It's not necessary. I removed it to get it working locally, changing to how you have it here, rather than setting the USE_MAVEN_LOCAL flag. (Done)

[bmeike]

Ok... I'm basically good with this.

I need to verify that the changes to the POM generation work exactly as we need. Our build team runs BlackDuck over this stuff and they get all excited if the dependencies don't look right.

I also need to understand the whonking big hunks of code in those androidApiConfig defs. I need to be sure I can maintain them.

Out of curiosity: is this something that could go in the community edition only? Does it actually affect anything I will build or is it useful only when the client is doing the build?

[jeffdgr8]

I need to verify that the changes to the POM generation work exactly as we need. Our build team runs BlackDuck over this stuff and they get all excited if the dependencies don't look right.

Definitely, please verify. I did file comparisons myself between the two local maven publication outputs (with and without the changes) to ensure the .pom is identical to before, as well as the .module having similar metadata as the .pom, but with the additional variant metadata.

I also need to understand the whonking big hunks of code in those androidApiConfig defs. I need to be sure I can maintain them.

The androidApiConfig and androidRuntimeConfig defs are made to mirror the .module metadata generated by the Android library variant within the Java .module. Similarly, the javaApiConfig and javaRuntimeConfig defs do the same thing in reverse. If you compare the .module files generated by each library, you should see these attributes and properties being the same as what Gradle generates for the other library variant. If you build the libraries with just these changes, you'll see the .module metadata that is generated without adding the additional variant metadata.

Category.LIBRARY identifies the project as a library. TargetJvmEnvironment and KotlinPlatformType each identify the platform as either JVM or Android. These attributes are what allow Gradle to select the correct library variant. (In practice I've found Gradle can use either of these attributes. The one is from Gradle for Java and the other comes from Kotlin.). The Usage attribute value differentiates the compile-time (API) and runtime variants. My understanding is Gradle uses the API variant for compileOnly dependencies and the runtime variant for implementation and api dependencies.

Out of curiosity: is this something that could go in the community edition only?

I'm applying the same workaround code, which injects this Gradle module metadata, for both the CE and EE libraries in my project. I similarly have separate artifacts to support both editions of CBL. So it's helpful in both publications.

Does it actually affect anything I will build or is it useful only when the client is doing the build?

This Gradle module metadata is useful in any situation where a Gradle configuration sees both the -android and -java dependencies in its dependency graph, such that it can identify both artifacts as variants of the same library and exclude the one that's not for the platform being built. Since both artifacts implement most of the same APIs, this avoids classpath conflicts, but also ensures the correct native library is used at runtime. This could occur with transitive dependencies coming from other dependencies, for example. It also has the benefit of fixing an incorrect dependency declaration. If someone declares the -java dependency in their Android app, Gradle will automatically pull in the correct -android platform variant instead.

In my case it's a dependency inherited from an intermediate source set in Kotlin Multiplatform merged with the dependency used in the root platform source set. I have a shared JVM+Android source set, which implements most of the Java and Android CBL APIs that are common. It depends on the -java artifact to see these CBL APIs. But then it's merged with both the JVM and Android root target source sets, which implement the parts of the CBL API that differ. In the Android case, the -android CBL dependency is introduced and the inherited -java dependency should be excluded.

There might be a situation where you found this useful yourself, possibly with something like the -ktx library which builds on top of the base library. This likely wouldn't be a factor with how you're currently building the CBL library with separate Gradle modules and shared source sets. There might be a way to rework how code is shared to build the library variants from within the same Gradle module, in which case Gradle would be able to publish the variant metadata itself automatically, without the need to manually define it as I have here. This is what Kotlin Multiplatform projects do. See Okio's .module file, for example, with each of the platform variants defined in the root artifact. OkHttp also publishes Gradle module metadata, but since it's not multiplatform, the only variants are the API, runtime, sources, and javadocs (v5 actually introduces multiplatform support for JVM and JS).

[bmeike]

@jeffdgr8 : sorry to be so long with this. I've been pulled into a different project for a couple (2w) sprints. This is a big enough change so that I need to be sure I can live with it, before I merge it.

Back soon....

[jeffdgr8]

@bmeike no worries. Since I have a workaround which injects the gradle module metadata rules, this isn't a blocker. Just will be nice to remove the workaround eventually with native support in the publications at some point. Thanks.

[bmeike]

Sorry it's taken me so long to look at this.
Can you say a bit more about the point of doing this and provide refs to docs for how it works?
NVM about the point. I see the point now. Docs for the mechanisms you are using and how the libs actually are provided at runtime, though, I'd like to see.

[bmeike]

Jeff: How does this work? Who provides the library at runtime?

[jeffdgr8]

This is just to filter the coroutines dependency from being published as an explicit runtime dependency in both the .pom and .module, which is what you were doing already when manually filtering the published pom dependencies.

By specifying the dependency as compileOnly, the library consumer needs to provide the dependency if they make use of any of the CBL KTX APIs that depend on coroutines-core APIs (same with work-runtime-ktx). But it makes the dependency optional if they don't use those APIs.

The tests add the dependencies as androidTestImplementation, so they're added to the test binaries.

Again, this is how the behavior was already, this just makes it explicit and allows it to be the same in both the .pom and the .module metadata.

[bmeike]

Is this going to work for both the CE and EE versions?

[jeffdgr8]

All of this should work the same for both CE and EE, just with the variant artifacts being different for EE. So

javaApi("com.couchbase.lite:couchbase-lite-java-ee:$version")

here and

androidApi("com.couchbase.lite:couchbase-lite-android-ee:$version")

in the Java lib's build.gradle.

It might make sense to use a variable, like CBL_ANDROID_LIB, as is used in other places currently.

[jeffdgr8]

This is the code (and where it's applied) that injects this variant metadata for my CBL KMP library. Once there are library artifacts that have this .module variant info published, this workaround will no longer be necessary.

Without the workaround or this published metadata, the Android source set sees both the CBL Android and inherited Java dependency, which results in errors from APIs that conflict (for example, Android-specific code that sees errors with workaround removed). The Gradle Module Metadata tells Gradle that these CBL artifacts are both platform variants of the same library, so they shouldn't both exist on the classpath at once.

The libraries are configured as dependencies the same as they currently are. The only difference this makes is that if both libraries appear as dependencies, Gradle will choose the variant that's appropriate for the platform being compiled and exclude the other. Or even if only one variant is declared, but it's the wrong variant for the platform (e.g. -java for an Android app), then it will replace it with the correct platform (e.g. -android) variant.

There are several links to docs describing Gradle Module Metadata and configuring variants in publications in the PR description comment, including:

• Gradle Module Metadata spec (variants value)
• Modifying and adding variants to existing components for publishing

Let me know if there's any additional info that would be helpful.

[jeffdgr8]

KT-58830 is resolved! Kotlin 2.1.20 adds official support for accessing the AdhocComponentWithVariants API to add custom Gradle publication variants. So reflection is no longer required in the Android library configuration, once the KOTLIN_VERSION is 2.1.20 (currently 2.1.10).