-
Notifications
You must be signed in to change notification settings - Fork 564
Description
This is the D8 and R8 integration specification for Xamarin.Android.
What is D8? What is R8?
At a high level, here are the steps that occur during an Android application's Java compilation:
javaccompiles Java codedesugarremove's the "sugar" (from Java 8 features) that are not fully supported on Androidproguardshrinks compiled Java codedx"dexes" compiled Java code into Android dex format. This is an alternate Java bytecode format supported by the Android platform.
This process has a few issues, such as:
- proguard is made by a third party, and aimed for Java in general (not Android specific)
dxis slower than it could be
So in 2017, Google announced a "next-generation" dex compiler named D8.
- D8 is a direct replacement for
dx - R8 is a replacement for
proguard, that also "dexes" at the same time. If using R8, a D8 call is not needed.
Both tools have support for various other Android-specifics:
- Both
desugarby default unless the--no-desugaringswitch is specified - Both support multidex
- R8 additionally has support to generate a default
multidex.keepfile, thatproguardcan generate
- R8 additionally has support to generate a default
Additionally, R8 is geared to be backwards compatible to proguard. It uses the same file format for configuration and command-line parameters as proguard. However, at the time of writing this, there are still several flags/features not implemented in R8 yet.
For more information on how R8 compares to proguard, see a great article written by the proguard team here.
You can find the source for D8 and R8 here.
For reference, d8 --help:
Usage: d8 [options] <input-files>
where <input-files> are any combination of dex, class, zip, jar, or apk files
and options are:
--debug # Compile with debugging information (default).
--release # Compile without debugging information.
--output <file> # Output result in <outfile>.
# <file> must be an existing directory or a zip file.
--lib <file> # Add <file> as a library resource.
--classpath <file> # Add <file> as a classpath resource.
--min-api # Minimum Android API level compatibility
--intermediate # Compile an intermediate result intended for later
# merging.
--file-per-class # Produce a separate dex file per input class
--no-desugaring # Force disable desugaring.
--main-dex-list <file> # List of classes to place in the primary dex file.
--version # Print the version of d8.
--help # Print this message.
For reference, r8 --help:
Usage: r8 [options] <input-files>
where <input-files> are any combination of dex, class, zip, jar, or apk files
and options are:
--release # Compile without debugging information (default).
--debug # Compile with debugging information.
--output <file> # Output result in <file>.
# <file> must be an existing directory or a zip file.
--lib <file> # Add <file> as a library resource.
--min-api # Minimum Android API level compatibility.
--pg-conf <file> # Proguard configuration <file>.
--pg-map-output <file> # Output the resulting name and line mapping to <file>.
--no-tree-shaking # Force disable tree shaking of unreachable classes.
--no-minification # Force disable minification of names.
--no-desugaring # Force disable desugaring.
--main-dex-rules <file> # Proguard keep rules for classes to place in the
# primary dex file.
--main-dex-list <file> # List of classes to place in the primary dex file.
--main-dex-list-output <file> # Output the full main-dex list in <file>.
--version # Print the version of r8.
--help # Print this message.
What does Xamarin.Android do now?
In other words, what is currently happening before we introduce D8/R8 support?
- The Javac MSBuild task compiles
*.javafiles to aclasses.zipfile. - The Desugar MSBuild task "desugars" using
desugar_deploy.jarif$(AndroidEnableDesugar)isTrue. - The Proguard MSBuild task shrinks the compiled Java code if
$(AndroidEnableProguard)isTrue. Developers may also supply custom proguard configuration files viaProguardConfigurationbuild items. - The CreateMultiDexMainDexClassList MSBuild task runs
proguardto generate a final, combinedmultidex.keepfile if$(AndroidEnableMultiDex)isTrue. Developers can also supply custommultidex.keepfiles viaMultiDexMainDexListbuild items. - The CompileToDalvik MSBuild task runs
dx.jarto generate a finalclasses.dexfile in$(IntermediateOutputPath)android\bin. Ifmultidexis enabled, aclasses2.dex(and potentially more) are also generated in this location.
What would this process look like with D8 / R8?
Two new MSBuild tasks named R8 and D8 will be created.
- The Javac MSBuild task will remain unchanged.
R8will be invoked to create amultidex.keepfile if$(AndroidEnableMultiDex)isTrue.D8will run if$(AndroidEnableProguard)isFalseand "desugar" by default.- Otherwise,
R8will run if$(AndroidEnableProguard)isTrueand will also "desugar" by default.
So in addition to be being faster (if Google's claims are true), we will be calling less tooling to accomplish the same results.
So how do developers use it? What are sensible MSBuild property defaults?
Currently, a csproj file might have the following properties:
<Project>
<PropertyGroup>
<AndroidEnableProguard>True</AndroidEnableProguard>
<AndroidEnableMultiDex>True</AndroidEnableMultiDex>
<AndroidEnableDesugar>True</AndroidEnableDesugar>
</PropertyGroup>
</Project>To enable the new behavior, we should introduce two new enum-style properties:
$(AndroidDexGenerator)- supportsdxord8$(AndroidLinkTool)- supportsproguardorr8
But for an existing project, a developer could opt-in to the new behavior with two properties:
<Project>
<PropertyGroup>
<AndroidEnableProguard>True</AndroidEnableProguard>
<AndroidEnableMultiDex>True</AndroidEnableMultiDex>
<AndroidEnableDesugar>True</AndroidEnableDesugar>
<!--New properties-->
<AndroidDexGenerator>d8</AndroidDexGenerator>
<AndroidLinkTool>r8</AndroidLinkTool>
</PropertyGroup>
</Project>There should be two new MSBuild properties to configure here, because:
- You could use
D8in combination withproguard, asR8is not "feature complete" in comparison toproguard. - You may not want to use code shrinking at all, but still use
D8instead ofdx. - You shouldn't be able to use
dxin combination withR8, it doesn't make sense. - Developers should be able to use the existing properties for enabling code shrinking,
multidex, anddesugar.
Our reasonable defaults would be:
- If
AndroidDexGeneratoris omitted,dxandCompileToDalvikshould be used. Until D8/R8 integration is deemed stable and enabled by default. - If
AndroidDexGeneratorisd8andAndroidEnableDesugaris omitted,AndroidEnableDesugarshould be enabled. - If
AndroidLinkToolis omitted andAndroidEnableProguardistrue, we should defaultAndroidLinkTooltoproguard.
MSBuild properties default to something like:
<AndroidDexGenerator Condition=" '$(AndroidDexGenerator)' == '' ">dx</AndroidDexGenerator>
<!--NOTE: $(AndroidLinkTool) would be blank if code shrinking is not used at all-->
<AndroidLinkTool Condition=" '$(AndroidLinkTool)' == '' And '$(AndroidEnableProguard)' == 'True' ">proguard</AndroidLinkTool>
<AndroidEnableDesugar Condition=" '$(AndroidEnableDesugar)' == '' And ('$(AndroidDexGenerator)' == 'd8' Or '$(AndroidLinkTool)' == 'r8') ">True</AndroidEnableDesugar>If a user specifies combinations of properties:
AndroidDexGenerator=d8andAndroidEnableProguard=TrueAndroidLinkToolwill get set toproguard
AndroidDexGenerator=dxandAndroidLinkTool=r8- This combination doesn't really make sense, but we don't need to do anything: only
R8will be called because it dexes and shrinks at the same time.
- This combination doesn't really make sense, but we don't need to do anything: only
AndroidEnableDesugaris enabled when omitted, if eitherd8orr8are used
For new projects that want to use D8/R8, code shrinking, and multidex, it would make sense to specify:
<Project>
<PropertyGroup>
<AndroidEnableMultiDex>True</AndroidEnableMultiDex>
<AndroidDexGenerator>d8</AndroidDexGenerator>
<AndroidLinkTool>r8</AndroidLinkTool>
</PropertyGroup>
</Project>Additional D8 / R8 settings?
--debug or --release needs to be explicitly specified for both D8 and R8. We should use the AndroidIncludeDebugSymbols property for this.
$(D8ExtraArguments) and $(R8ExtraArguments) can be used to explicitly pass additional flags to D8 and R8.
How are we compiling / shipping D8 and R8?
We will add a submodule to xamarin-android for r8. It should be pinned to a commit with a reasonable release tag, such as 1.2.35 for now.
To build r8, we have to:
- Download and unzip a tool named depot_tools from the Chromium project
- Put the path to
depot_toolsin$PATH - Run
gclientso it will download/bootstrap gradle, python, and other tools - Run
python tools\gradle.py d8 r8to compiled8.jarandr8.jar - We will need to ship
d8.jarandr8.jarin our installers, similar to how we are shippingdesugar_deploy.jar
Current Implementation
PR #2019 is the current implementation.
However, it became clear that a lot of the ideas around D8/R8 are not formalized or known.
The goal here will be to actually explain what everything should do, before I finish the implementation.