Skip to content

Conversation

@akoeplinger
Copy link
Member

This ensures that we can revert mxe to an earlier version.

Background:

The current "do we need to rebuild " is mostly "forward-only". It works properly if you bump forward in time, but if you go backwards, it won't rebuild because it's timestamp based.

For example: if we bump MXE, it'll rebuild the (cached!) MXE on the build server but if we find that MXE is broken and we need to revert... we can't because what we're reverting to is older than the bump!

@jonpryor jonpryor added the full-mono-integration-build For PRs; run a full build (~6-10h for mono bumps), not the faster PR subset (~2h for mono bumps) label Oct 6, 2017
@jonpryor
Copy link
Contributor

jonpryor commented Oct 6, 2017

build

@jonpryor
Copy link
Contributor

The most recent commit does not appear to have improved matters:

dyld: Symbol not found: _clock_gettime
  Referenced from: /Users/builder/android-toolchain/mxe-a926b16/x86_64-apple-darwin15.6.0/bin/patch (which was built for Mac OS X 10.12)
  Expected in: /usr/lib/libSystem.B.dylib
bash: line 1: 55233 Trace/BPT trap: 5       patch -p1 -u

One gets the feeling that patch(1), somehow, has a bad library reference.

Further confounding/confusing is why

  1. ~/android-toolchain/mxe-a926b16/x86_64-apple-darwin15.6.0/bin/patch exists
  2. (1) is being used in the first place

Why isn't /usr/bin/patch being used?

Something is very weird here. :-(

This ensures that we can revert mxe to an earlier version.

Background:

The current "do we need to rebuild <dependency>" is mostly "forward-only".
It works properly if you bump forward in time, but if you go _backwards_,
it won't rebuild because it's timestamp based.

For example: if we bump MXE, it'll rebuild the (cached!) MXE on the build server
but if we find that MXE is broken and we need to revert... we can't because
what we're reverting to is older than the bump!

--

The new mxe now includes a few more "native" plugins in `plugins/native/<OS_SHORT_NAME>`.
This caused issues on OSX where the `patch` that mxe built ran into
a problem because it relied on newer APIs not available in older OSX.

To fix this we just disable building the native plugins and rely
on the system ones to be working. This is what mxe is doing on Travis:
https://github.com/mxe/mxe/blob/16acc77a3d45425d5a7e11d8b2511fc5a28dd56f/.travis.yml#L6
@akoeplinger akoeplinger force-pushed the mxe-bump-and-cache-counter branch from 9b32dae to 8b83f7e Compare October 18, 2017 13:00
@akoeplinger
Copy link
Member Author

This should be good to go now. We removed building patch from mxe by setting OS_SHORT_NAME (which is what mxe is doing in their Travis CI config too).

@jonpryor jonpryor merged commit b28856f into dotnet:master Oct 18, 2017
@akoeplinger akoeplinger deleted the mxe-bump-and-cache-counter branch October 18, 2017 19:39
Redth pushed a commit to Redth/xamarin-android that referenced this pull request Oct 30, 2017
Add commit SHA1 to `$(AndroidMxeInstallPrefix)`, so that we can
revert mxe to an earlier version.

Most of the current "do not rebuild DEPENDENCY" logic is
"forward-only": it only works properly if you go *forward* in time,
not backwards in time.

For example, consider the `ForceBuild` target in
`build-tools/mono-runtimes/mono-runtimes.targets`: if the HEAD commit
within `external/mono` moves "backward" so that the resulting
timestamp of `autogen.sh` is *older* than e.g.
`bin\$(Configuration)\bcl-tests\bcl-tests.zip`, re-running the
`ForceBuild` target won't do anything.

`external/mono` and `mono-runtimes` are forward-only.

Usually this is considered acceptable. If it's truly a problem, nuke
the `bin` directory and rebuild.

"Forward-only" behavior is *not* always acceptable.

Case in point: `external/mxe`. Building MXE takes a significant
amount of time -- roughly 10+ minutes -- so we *cache* the MXE
toolchain into `$(AndroidMxeInstallPrefix)` -- which defaulted to
`$HOME/android-toolchain/mxe` -- and only rebuild the cached MXE
when `external/mxe` changes.

All well and good...except when MXE *cannot* be built *at all*.

Case in point: [MXE couldn't be built with Xcode 8.3][mxe1744].
We didn't notice this for a long period of time, because our Jenkins
machines are fairly stable; they built and cached MXE under
Xcode 8.1, and never had to rebuild MXE again.

[mxe1744]: mxe/mxe#1744

The only reason we noticed is because some developers tried to build
MXE on new machines, with newer Xcode versions, and MXE didn't build.

This raises an interesting conundrum: for *cached* resources such
as MXE, it is *extremely* important to be able to *revert* to a
"known good" version. "Forward only" behavior is at odds with this;
you *can't* revert to a "known good" version, because it may have
been replaced!

Bump to mxe/xamarin/a926b16d, as this version *can* be built under
Xcode 9. (If we're lucky, it will *also* be able to build an updated
commit of `external/libzip`…)

Additionally, alter the semantics of `$(AndroidMxeInstallPrefix)` to
*require* that it *ends with* the abbreviated commit hash of the
`external/mxe` commit; in this commit, `$(AndroidMxeInstallPrefix)`
is `$HOME/android-toolchain/mxe-a926b16d`.

In particular, this allows us to *revert*: bumping `external/mxe`
will build MXE into a new, *unique*, directory, which means that any
previously built directory will not be modified. This in turn allows
us to "revert" the patch to use the previous MXE, if necessary.

The updated MXE now includes a few more "native" plugins in
`plugins/native/$(OS_SHORT_NAME)`. This caused issues on macOS where
the `patch` that MXE built ran into a problem because it relied on
newer APIs not available in older versions of macOS.

Fix this by disabling building the native plugins and instead rely on
the system ones by overriding `$(OS_SHORT_NAME)`.
[This mirrors what MXE is doing on Travis][travis].

[travis]: https://github.com/mxe/mxe/blob/16acc77a3d45425d5a7e11d8b2511fc5a28dd56f/.travis.yml#L6
jonpryor added a commit to jonpryor/xamarin-android that referenced this pull request Nov 30, 2021
Fixes: dotnet/java-interop#815
Fixes: dotnet/java-interop#920

Changes: dotnet/java-interop@0293360...7f1a5ab

  * dotnet/java-interop@7f1a5ab1: [JavaTypeSystem] Enable resolving generic types on declaring types (dotnet#916)
  * dotnet/java-interop@8daa0263: [Xamarin.Android.Tools.Bytecode] Fix Nullability warnings (dotnet#922)
  * dotnet/java-interop@77c9c5fa: [class-parse] Import parameter names for unresolvable types (dotnet#921)
  * dotnet/java-interop@a8b444d5: [Java.Interop.Tools.JavaTypeSystem] ctors w/ generic type parameters (dotnet#919)
jonpryor added a commit that referenced this pull request Nov 30, 2021
Fixes: dotnet/java-interop#815
Fixes: dotnet/java-interop#920

Changes: dotnet/java-interop@0293360...7f1a5ab

  * dotnet/java-interop@7f1a5ab1: [JavaTypeSystem] Enable resolving generic types on declaring types (#916)
  * dotnet/java-interop@8daa0263: [Xamarin.Android.Tools.Bytecode] Fix Nullability warnings (#922)
  * dotnet/java-interop@77c9c5fa: [class-parse] Import parameter names for unresolvable types (#921)
  * dotnet/java-interop@a8b444d5: [Java.Interop.Tools.JavaTypeSystem] ctors w/ generic type parameters (#919)
jonathanpeppers pushed a commit that referenced this pull request Nov 30, 2021
Fixes: dotnet/java-interop#815
Fixes: dotnet/java-interop#920

Changes: dotnet/java-interop@0293360...7f1a5ab

  * dotnet/java-interop@7f1a5ab1: [JavaTypeSystem] Enable resolving generic types on declaring types (#916)
  * dotnet/java-interop@8daa0263: [Xamarin.Android.Tools.Bytecode] Fix Nullability warnings (#922)
  * dotnet/java-interop@77c9c5fa: [class-parse] Import parameter names for unresolvable types (#921)
  * dotnet/java-interop@a8b444d5: [Java.Interop.Tools.JavaTypeSystem] ctors w/ generic type parameters (#919)
@github-actions github-actions bot locked and limited conversation to collaborators Feb 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

full-mono-integration-build For PRs; run a full build (~6-10h for mono bumps), not the faster PR subset (~2h for mono bumps)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants