dataconnect: integrate codegen into the gradle build

[dconeybe]

This PR adds a custom gradle plugin to the "dataconnect" project to hook into the gradle build and generate the Data Connect source code automatically as part of the build. There is no longer a requirement to manually download and launch the Data Connect emulator just to build the project. This behaves very similarly to the protobuf gradle plugin (https://github.com/google/protobuf-gradle-plugin) and how it compiles the .proto files to java/kt files on-demand when the project is built. This also unblocks compiling the dataconnect module in GitHub CI.

The plugin first creates a node project directory (by creating an empty directory that contains an empty package.json) and installs the configured version of firebase-tools into it by running npm install. Then, it uses that firebase-tools to run the dataconnect:sdk:generate subcommand. In order to control where the generated code gets placed, a copy of the dataconnect.yaml et al files is made, and the connector.yaml files are modified to only produce kotlin connectors and to put them into the directory of the gradle plugin's choosing. All of this is done in the "build" directory, and, therefore, is wiped out by a "gradle clean".

As a nice side effect, the GitHub Actions CI can now compile dataconnect just fine. Therefore, this PR also removes the hacks that opted dataconnect out of the CI.

The following new tasks are now present in the :dataconnect:app module:

1. setupFirebaseToolsForDataConnect
   • Creates the directory dataconnect/app/build/dataconnect/firebase-tools with a single file in it, an empty package.json.
   • Runs npm install [email protected] in that directory (the version is configured in build.gradle.kts).
2. generateDebugDataConnectSources and generateReleaseDataConnectSources
   • Depends on the setupFirebaseToolsForDataConnect task to set up the firebase command
   • Copies the contents of dataconnect/dataconnect into dataconnect/app/build/dataconnect/variants/{debug,release}/config
   • Edits the connector.yaml file(s) to remove all SDKs under the generate section, except for kotlinSdk. This is because since we're building the kotlin SDK, we don't want to generate code for other SDKs.
   • Modifies the outputDir of the kotlinSdk to use the directory specified by the gradle plugin for placing the generated code. This is because gradle dictates where the generated code is to go and this directory can't be known in advance and hardcoded into the file. Moreover, the directories are different for the "debug" and "release" builds.
   • Runs firebase dataconnect:sdk:generate to generate the code into the directory dictated by gradle.

This plugin is configured by a new "extension" added to dataconnect/app/build.gradle.kts. It looks like this:

dataconnect {
    // The version of https://www.npmjs.com/package/firebase-tools to use to perform the
    // Data Connect code generation.
    firebaseToolsVersion = "13.25.0"

    // The directory that contains dataconnect.yaml to use as input when performing
    // the Data Connect code generation.
    dataConnectConfigDir = file("../dataconnect")
}

This gradle plugin is a proof-of-concept and isn't quite production ready. If it proves to be valuable, customers can at least copy/paste it into their own Android projects. Or, we could make it an "official" plugin. There is even a way to remove the need for node altogether, by just calling the data connect toolkit cli binary directly (without needing to install firebase-tools). This plugin was largely based on my experience writing a similar gradle plugin in the dataconnect android sdk itself: https://github.com/firebase/firebase-android-sdk/blob/866e6bd246fde0e54fdeb5f5d0c043632edf4161/firebase-dataconnect/gradleplugin/plugin/src/main/kotlin/com/google/firebase/dataconnect/gradle/plugin/DataConnectGradlePlugin.kt.

[thatfiredev]

(No action required)

We use ktlint in this repo, but I've considered moving to ktfmt so that it aligns with the SDK formatting.

I'm happy to move forward with ktfmt only applied to dataconnect in this PR and we can apply to other modules and remove ktlint in a follow-up PR.

[dconeybe]

Ahh ok. Yeah I think this isn't the PR to change the code formatting tool. I'll switch to ktlint.

[thatfiredev]

Where does it generate? I checked out your repo and tried running the ./gradlew generateDebugDataConnectSources command but couldn't see the generated files.

[dconeybe]

I'll update the comment to be more clear about where the code gets generated by gradle.

In the meantime, if you run ./gradlew --info generateDebugDataConnectSources (i.e. add the --info argument) the logs state the directory into which the code is generated.

[thatfiredev]

Oh, cool, I see it in quickstart-android/dataconnect/app/build/generated/java/generateDebugDataConnectSources/com/google/firebase/dataconnect/movies

[dconeybe]

Comment updated to this:

... app/build.gradle.kts will look after generating the code on-demand into the directory prescribed by Gradle (e.g. app/build/generated/java/generateDebugDataConnectSources).

[dconeybe]

(oops, forgot to publish my comments 2 weeks ago [facepalm])

[dconeybe]

Comment updated to this:

... app/build.gradle.kts will look after generating the code on-demand into the directory prescribed by Gradle (e.g. app/build/generated/java/generateDebugDataConnectSources).

[dconeybe]

Ahh ok. Yeah I think this isn't the PR to change the code formatting tool. I'll switch to ktlint.

[dconeybe]

This PR is on hold for now. It was just taking me wayyyy to long and I needed to timebox it. Anything gradle related usually takes forever!