-
Notifications
You must be signed in to change notification settings - Fork 565
[xa-prep-tasks] Use the Mono bundle #162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
7528870 to
58ba31a
Compare
Context: dotnet#162 PR dotnet#162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR dotnet#162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono! by design!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR dotnet#162 fix to build, as all required generated files will be present within the bundle. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
Context: dotnet#162 PR dotnet#162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR dotnet#162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR dotnet#162 fix to build, as all required generated files will be present within the bundle. Additionally, add a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
Context: dotnet#162 PR dotnet#162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR dotnet#162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR dotnet#162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
Context: #162 PR #162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR #162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
58972f1 to
6e07fb8
Compare
Context: dotnet#162 PR dotnet#162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR dotnet#162 currently fails][0] when building `src/Mono.Data.Sqlite`: CSC: error CS2001: Source file `/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/external/mono/mcs/build/common/Consts.cs' could not be found Fix this error by copying `external/mono/mcs/build/common/Consts.cs` into `bin/$(Configuration)/include`, and updating the `Consts.cs` references within `Mono.Data.Sqlite.csproj` and `Mono.Posix.csproj` so that they'll use the `include` copy. This should allow us to use the bundle to build these projects. Additionally, `llc.exe` and `opt.exe` are installed into `$prefix/bin`, which isn't correct; `$prefix/bin` as a construct doesn't exist on Windows. Install these into `$prefix/lib/mandroid`. Update the bundle version to note these bundle changes. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/67/console
#165) Context: #162 PR #162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/Mono.Data.Sqlite`: CSC: error CS2001: Source file `/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/external/mono/mcs/build/common/Consts.cs' could not be found Fix this error by copying `external/mono/mcs/build/common/Consts.cs` into `bin/$(Configuration)/include`, and updating the `Consts.cs` references within `Mono.Data.Sqlite.csproj` and `Mono.Posix.csproj` so that they'll use the `include` copy. This should allow us to use the bundle to build these projects. Additionally, `llc.exe` and `opt.exe` are installed into `$prefix/bin`, which isn't correct; `$prefix/bin` as a construct doesn't exist on Windows. Install these into `$prefix/lib/mandroid`. Update the bundle version to note these bundle changes. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/67/console
5f4138b to
d16225b
Compare
|
| prepare:: prepare-external prepare-props | ||
|
|
||
| # $(call GetPath,path) | ||
| GetPath = $(shell $(MSBUILD) $(MSBUILD_FLAGS) /p:DoNotLoadOSProperties=True /nologo /v:minimal /t:Get$(1)FullPath build-tools/scripts/Paths.targets | tr -d '[[:space:]]' ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not a fan of this kind of {x,ms}build invocation, to be honest. It's convenient if it works, but if it breaks... The error output of msbuild is mangled and next to impossible to read, also it gets interpreted by shell as a set of commands. Maybe (not as part of this PR) we should have a wrapper script for such one-liners that will properly check the exit status of the command as well as log and process its output.
|
LGTM |
| <Target Name="GetBundleFileName" | ||
| DependsOnTargets="_GetHashes"> | ||
| <PropertyGroup> | ||
| <XABundleFileName>bundle-v2-$(Configuration)-$(HostOS)-libzip=$(_LibZipHash),llvm=$(_LlvmHash),mono=$(_MonoHash).zip</XABundleFileName> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the v2 be a property? It is being used in more than one place and sounds like something that might be updated in future too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where else is -v2 being used? It should only be used in this property.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
d16225b to
515c1d5
Compare
|
Other than what I commented on, LGTM! |
eccabe3 to
766d869
Compare
Context: dotnet@fbfd676 Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
766d869 to
67c533a
Compare
) Context: dotnet#162 PR dotnet#162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR dotnet#162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR dotnet#162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
dotnet#165) Context: dotnet#162 PR dotnet#162 tries to use the mono bundle (fbfd676) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR dotnet#162 currently fails][0] when building `src/Mono.Data.Sqlite`: CSC: error CS2001: Source file `/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/external/mono/mcs/build/common/Consts.cs' could not be found Fix this error by copying `external/mono/mcs/build/common/Consts.cs` into `bin/$(Configuration)/include`, and updating the `Consts.cs` references within `Mono.Data.Sqlite.csproj` and `Mono.Posix.csproj` so that they'll use the `include` copy. This should allow us to use the bundle to build these projects. Additionally, `llc.exe` and `opt.exe` are installed into `$prefix/bin`, which isn't correct; `$prefix/bin` as a construct doesn't exist on Windows. Install these into `$prefix/lib/mandroid`. Update the bundle version to note these bundle changes. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/67/console
Context: dotnet@fbfd676 Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
Changes: dotnet/android-tools@fc3c2ac...20f6112 * dotnet/android-tools@20f6112: Bump LibZipSharp to 2.0.4 (#166) * dotnet/android-tools@e4f0d59: Add support for writing android:roundIcon to Android manifest (#162) Changes: dotnet/android-libzipsharp@2.0.3...2.0.4 * dotnet/android-libzipsharp@8ac30f3: Bump Version to 2.0.4 for Release (#109) * dotnet/android-libzipsharp@8e22fe1: Use RuntimeInformation to determine the Processor architecture. (#108) * dotnet/android-libzipsharp@33f5b01: Bump vcpkg to latest version (#106) * dotnet/android-libzipsharp@6fd0a14: [ci] Use new EO compliant build pools (#107) * dotnet/android-libzipsharp@d339358: Add a new more complex unit test (#105) Other changes: * 32-bit `libZipSharpNative.dll` is now in `x86\libZipSharpNative.dll`. * 64-bit `lib64\libZipSharpNative.dll` is now in `x64\libZipSharpNative.dll`.
Changes: dotnet/android-tools@fc3c2ac...20f6112 * dotnet/android-tools@20f6112: Bump LibZipSharp to 2.0.4 (dotnet#166) * dotnet/android-tools@e4f0d59: Add support for writing android:roundIcon to Android manifest (dotnet#162) Changes: dotnet/android-libzipsharp@2.0.3...2.0.4 * dotnet/android-libzipsharp@8ac30f3: Bump Version to 2.0.4 for Release (dotnet#109) * dotnet/android-libzipsharp@8e22fe1: Use RuntimeInformation to determine the Processor architecture. (dotnet#108) * dotnet/android-libzipsharp@33f5b01: Bump vcpkg to latest version (dotnet#106) * dotnet/android-libzipsharp@6fd0a14: [ci] Use new EO compliant build pools (dotnet#107) * dotnet/android-libzipsharp@d339358: Add a new more complex unit test (dotnet#105) Other changes: * 32-bit `libZipSharpNative.dll` is now in `x86\libZipSharpNative.dll`. * 64-bit `lib64\libZipSharpNative.dll` is now in `x64\libZipSharpNative.dll`.
Changes: dotnet/android-tools@fc3c2ac...20f6112 * dotnet/android-tools@20f6112: Bump LibZipSharp to 2.0.4 (dotnet#166) * dotnet/android-tools@e4f0d59: Add support for writing android:roundIcon to Android manifest (dotnet#162) Changes: dotnet/android-libzipsharp@2.0.3...2.0.4 * dotnet/android-libzipsharp@8ac30f3: Bump Version to 2.0.4 for Release (#109) * dotnet/android-libzipsharp@8e22fe1: Use RuntimeInformation to determine the Processor architecture. (#108) * dotnet/android-libzipsharp@33f5b01: Bump vcpkg to latest version (#106) * dotnet/android-libzipsharp@6fd0a14: [ci] Use new EO compliant build pools (#107) * dotnet/android-libzipsharp@d339358: Add a new more complex unit test (#105) Other changes: * 32-bit `libZipSharpNative.dll` is now in `x86\libZipSharpNative.dll`. * 64-bit `lib64\libZipSharpNative.dll` is now in `x64\libZipSharpNative.dll`.
DO NOT MERGE
Before merging the bundle filename needs to be changed (see the TODO in the commit). I don't want to do this yet so that I can see if the current bundle is used by the PR bot. :-)
Context: fbfd676
Stage 3 of the cunning plan is to (attempt to) use the mono bundle
introduced in commit fbfd676.
This "simple" desire (ha!) re-raises the architectural project
dependency issue "solved" in fbfd676, but first, a simple question:
What should download the mono bundle?
There are two plausible answers:
make preparecan (somehow) handle it.Both are plausible. The problem with using
maketargets (1) is thereis increased potential for "duplication" -- duplication of the bundle
filename, downloading it, and extracting it. Plus,
makeisn't"Windows friendly", in that GNU make isn't (normally) present with
Visual Studio. (
NMAKEis, but the Makefiles in this project are notcompatible with
NMAKE.)Which brings us to MSBuild (2): can it handle the task?
To tackle that, we need to be able to have an MSBuild task project
which has no dependencies, so that it can download and extract the
mono bundle before anything else runs, as it may be downloading
contents which mean that other projects don't need to run.
The need for a "pre-bootstrap" task assembly -- called
xa-prep-tasks-- thus "undoes" some of the logic regarding
libzip-windows.mdprojand the
<Zip/>task from fbfd676: it isn't possible to rely onlibzipfrom a "pre-build" state, aslibzipis one of the things inthe mono bundle, so now we need two "bootstrap" task assemblies:
one without a
libzipdependency --xa-prep-tasks.dll-- and onewith a
libzipdependency --Xamarin.Android.Tools.BootstrapTasks.dllMove tasks which don't currently require
libzip-- or won't in thefuture, or laziness -- from
Xamarin.Android.Tools.BootstrapTasks.dlland move them into
xa-prep-tasks.dll.With that architectural compromise in place, add
xa-prep-tasksas a@(ProjectReference)to various projects to help ensure it's builtfirst, and rearchitect
bundle.mdprojso thatxa-prep-tasks.targetsandbundle.targetscan use the same targetsto compute the bundle filename, now in
build-tools/bundle/bundle-path.targets.Add a post-build step to
xa-prep-tasks.csprojwhich downloads andextracts the expected mono bundle.
One "problem" (feature?) is that the new
<SystemUnzip/>task doesn'treport errors as errors when unzip'ing the file. This turns out to be
fine here because when downloading the mono bundle from Azure we don't
get a 404 anyway -- Azure instead returns an XML document containing
an error message (wat?!). We can thus ignore most error handling
entirely...though we're also ignoring any checking for invalid
downloads, which is something we should address in the future.
Update the varioous project files so that they won't attempt to
rebuild binaries that were present in the mono bundle.
Finally, the mono bundle isn't quite right, as it installs
llc.exeand
opt.exeintobin/, notlib/mandroid. This was overlooked incommit fbfd676.
Add a version number to the bundle filename so that a unique filename
is used, thus ensuring that we attempt to download an updated mono
bundle in the future, and correct the
CreateBundletarget so that itpackages required files.