Fix fstat call in darwin-aarch64. #4107

oraluben · 2021-12-10T09:22:23Z

Looks like fstat and some other system calls don't have $INODE64 suffix on aarch64 anymore, even if _DARWIN_USE_64_BIT_INODE is given, since they're only for a backward-compatible issue which only happens on Intel platform.

This PR is to fix the failure on darwin-aarch64.

Failed test

# yyc @ Yichens-M1 in ~/IdeaProjects/graal/graal/substratevm on git:master o [17:08:52] 
$ mx native-unittest --regex ImageInfoTest 
========================================================================================================================
GraalVM Native Image: Generating 'com.oracle.svm.junit.svmjunitrunner'...
========================================================================================================================
[1/7] Initializing...                                                                                   (18.4s @ 0.21GB)
Version info: 'GraalVM dev Java 17 CE'
4 user-provided feature(s)
- com.oracle.svm.configure.config.SignatureUtilFeature
- com.oracle.svm.junit.JUnitFeature
- com.oracle.svm.test.NativeImageResourceFileSystemProviderTest$RegisterResourceFeature
- com.oracle.svm.test.SerializationRegistrationTestFeature
[2/7] Performing analysis...  [******]                                                                  (58.2s @ 0.69GB)
 3,823 (78.60%) of  4,864 classes reachable
 4,844 (61.95%) of  7,819 fields reachable
17,933 (48.66%) of 36,851 methods reachable
    49 classes,    55 fields, and   622 methods registered for reflection
    58 classes,    60 fields, and    51 methods registered for JNI access
[3/7] Building universe...                                                                               (3.8s @ 1.12GB)
[4/7] Parsing methods...      [**]                                                                       (2.9s @ 2.62GB)
[5/7] Inlining methods...     [****]                                                                     (4.5s @ 1.37GB)
[6/7] Compiling methods...    [*****]                                                                   (31.3s @ 2.35GB)
[7/7] Creating image...Fatal error: java.lang.RuntimeException: There was an error linking the native image: Linker command exited with 1

Linker command executed:
/usr/bin/cc -Wl,-U,___darwin_check_fd_set_overflow -Wl,-no_compact_unwind -Wl,-exported_symbols_list -Wl,/var/folders/yc/q5xjr0614bl08972dyxfjmmm0000gn/T/SVM-6025001424348589003/exported_symbols.list -Wl,-x -arch arm64 -o /Users/yyc/IdeaProjects/graal/graal/substratevm/svmbuild/darwin-aarch64/junit/tmp9zKWLO/com.oracle.svm.junit.svmjunitrunner com.oracle.svm.junit.svmjunitrunner.o /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64/liblibchelper.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libnet.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64/libdarwin.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libnio.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libjava.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libfdlibm.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libzip.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64/libjvm.a -v -L/var/folders/yc/q5xjr0614bl08972dyxfjmmm0000gn/T/SVM-6025001424348589003 -L/Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib -L/Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64 -lz -Wl,-framework,Foundation -lpthread -ldl

Linker command output:
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch arm64 -platform_version macos 11.0.0 12.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o /Users/yyc/IdeaProjects/graal/graal/substratevm/svmbuild/darwin-aarch64/junit/tmp9zKWLO/com.oracle.svm.junit.svmjunitrunner -L/var/folders/yc/q5xjr0614bl08972dyxfjmmm0000gn/T/SVM-6025001424348589003 -L/Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib -L/Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64 -L/usr/local/lib -U ___darwin_check_fd_set_overflow -no_compact_unwind -exported_symbols_list /var/folders/yc/q5xjr0614bl08972dyxfjmmm0000gn/T/SVM-6025001424348589003/exported_symbols.list -x com.oracle.svm.junit.svmjunitrunner.o /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64/liblibchelper.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libnet.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64/libdarwin.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libnio.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libjava.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libfdlibm.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/libzip.a /Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/clibraries/darwin-aarch64/libjvm.a -lz -framework Foundation -lpthread -ldl -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/lib/darwin/libclang_rt.osx.a
Undefined symbols for architecture arm64:
"_fstat$INODE64", referenced from:
    ___svm_version_info in com.oracle.svm.junit.svmjunitrunner.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
      at com.oracle.svm.hosted.image.NativeImageViaCC.handleLinkerFailure(NativeImageViaCC.java:505)
      at com.oracle.svm.hosted.image.NativeImageViaCC.write(NativeImageViaCC.java:452)
      at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:674)
      at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:495)
      at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:424)
      at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:580)
      at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:127)
      at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:610)
------------------------------------------------------------------------------------------------------------------------
                    5.7s (4.2% of total time) in 24 GCs | Peak RSS: 3.30GB | CPU load: ~358.64%
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/Users/yyc/IdeaProjects/graal/graal/substratevm/svmbuild/darwin-aarch64/junit/tmp9zKWLO/com.oracle.svm.junit.svmjunitrunner.build_artifacts.txt
========================================================================================================================
Failed generating 'com.oracle.svm.junit.svmjunitrunner' after 2m 14s.
Error: Image build request failed with exit status 1

Ref: #2666

graalvmbot · 2021-12-10T10:08:09Z

Hello Yichen Yan, thanks for contributing a PR to our project!

We use the Oracle Contributor Agreement to make the copyright of contributions clear. We don't have a record of you having signed this yet, based on your email address wenji -(dot)- yyc -(at)- alibaba-inc -(dot)- com. You can sign it at that link.

If you think you've already signed it, please comment below and we'll check.

oraluben · 2021-12-10T10:15:20Z

As a member of Alibaba, we've signed CLA of Graal.

oraluben · 2021-12-10T13:34:55Z

And I got stuck in this gate task:

Failed test

$ mx native-unittest com.oracle.truffle.nfi.test --build-args --language:nfi -H:MaxRuntimeCompileMethods=2000 -H:+TruffleCheckBlackListedMethods --run-args -Dnative.test.lib=/Users/yyc/IdeaProjects/graal/graal/truffle/mxbuild/darwin-aarch64/truffle-test-native/libnativetest.dylib -Dnative.isolation.test.lib=/Users/yyc/IdeaProjects/graal/graal/truffle/mxbuild/darwin-aarch64/truffle-test-native/libisolationtest.dylib --very-verbose --enable-timing
Building junit image for matching: com.oracle.truffle.nfi.test.ArrayNFITest com.oracle.truffle.nfi.test.DeoptTest com.oracle.truffle.nfi.test.ErrnoNFITest com.oracle.truffle.nfi.test.ExceptionNFITest com.oracle.truffle.nfi.test.FloatingNFITest com.oracle.truffle.nfi.test.ForbiddenNFITest com.oracle.truffle.nfi.test.GlobalNFITest com.oracle.truffle.nfi.test.ImplicitConvertNFITest com.oracle.truffle.nfi.test.IntegerLimitsNFITest com.oracle.truffle.nfi.test.IsolatedNamespaceTest com.oracle.truffle.nfi.test.KeyInfoNFITest com.oracle.truffle.nfi.test.LateBindNFITest com.oracle.truffle.nfi.test.LoadNFILibraryTest com.oracle.truffle.nfi.test.NullArrayNFITest com.oracle.truffle.nfi.test.NullNFITest com.oracle.truffle.nfi.test.NullableNFITest com.oracle.truffle.nfi.test.NumericNFITest com.oracle.truffle.nfi.test.ObjectNFITest com.oracle.truffle.nfi.test.PointerNFITest com.oracle.truffle.nfi.test.RegisterPackageNFITest com.oracle.truffle.nfi.test.StringAsInterfaceNFITest com.oracle.truffle.nfi.test.StringNFITest com.oracle.truffle.nfi.test.ThreadNFITest com.oracle.truffle.nfi.test.VarargsNFITest com.oracle.truffle.nfi.test.WrappedPrimitiveNFITest com.oracle.truffle.nfi.test.parser.ClosureParseSignatureTest com.oracle.truffle.nfi.test.parser.ErrorParseSignatureTest com.oracle.truffle.nfi.test.parser.ParseLibraryDescriptorTest com.oracle.truffle.nfi.test.parser.SimpleParseSignatureTest com.oracle.truffle.nfi.test.parser.VarargsParseSignatureTest
Warning: Option 'TruffleCheckBlackListedMethods' is deprecated and might be removed from future versions
Warning: Invalid option --add-reads org.graalvm.nativeimage.junitsupport=ALL-UNNAMED provided by jar:file:///Users/yyc/IdeaProjects/graal/graal/sdk/mxbuild/darwin-aarch64/GRAALVM_CEE4649546_JAVA17/graalvm-cee4649546-java17-22.1.0-dev/lib/svm/macros/junitcp/junit-support.jar!/META-INF/native-image/org.graalvm.nativeimage.junitsupport/com.oracle.svm.junit/native-image.properties. Specified module 'org.graalvm.nativeimage.junitsupport' is unknown.
========================================================================================================================
GraalVM Native Image: Generating 'com.oracle.svm.junit.svmjunitrunner'...
========================================================================================================================
[1/7] Initializing...                                                                                   (35.5s @ 0.25GB)
 Version info: 'GraalVM dev Java 17 CE'
 3 user-provided feature(s)
  - com.oracle.svm.junit.JUnitFeature
  - com.oracle.truffle.api.test.polyglot.RegisterTestClassesForReflectionFeature
  - com.oracle.truffle.tck.tests.TruffleTCKFeature
[2/7] Performing analysis...  [**********]                                                              (98.5s @ 2.14GB)
  11,541 (91.61%) of 12,598 classes reachable
  15,538 (66.23%) of 23,462 fields reachable
  56,954 (63.15%) of 90,182 methods reachable
   4,157 ( 4.61%) of 90,182 methods included for runtime compilation
     460 classes,   184 fields, and 1,529 methods registered for reflection
      58 classes,    59 fields, and    52 methods registered for JNI access
[3/7] Building universe...                                                                               (0.2s @ 2.16GB)
Error: type is not available in this platform: com.oracle.svm.core.posix.headers.Dlfcn$GNUExtensions
com.oracle.svm.core.util.UserError$UserException: type is not available in this platform: com.oracle.svm.core.posix.headers.Dlfcn$GNUExtensions
        at com.oracle.svm.core.util.UserError.abort(UserError.java:87)
        at com.oracle.svm.hosted.FallbackFeature.reportAsFallback(FallbackFeature.java:233)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:596)
        at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:495)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:424)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:580)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:127)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:610)
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: type is not available in this platform: com.oracle.svm.core.posix.headers.Dlfcn$GNUExtensions
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.createType(AnalysisUniverse.java:214)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:205)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:182)
        at com.oracle.graal.pointsto.meta.AnalysisType.getEnclosingType(AnalysisType.java:1025)
        at com.oracle.svm.hosted.meta.UniverseBuilder.build(UniverseBuilder.java:146)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:555)
        ... 5 more
------------------------------------------------------------------------------------------------------------------------
                      5.9s (4.2% of total time) in 27 GCs | Peak RSS: 4.26GB | CPU load: ~334.02%
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /Users/yyc/IdeaProjects/graal/graal/substratevm/svmbuild/darwin-aarch64/junit/tmp8DFG8C/com.oracle.svm.junit.svmjunitrunner.build_artifacts.txt
========================================================================================================================
Failed generating 'com.oracle.svm.junit.svmjunitrunner' after 2m 18s.
Error: Image build request failed with exit status 1

The class GNUExtensions should not be reachable during static analysis.

~~The task failed on both darwin aarch64 and x86.~~
Fixed.

graalvmbot · 2021-12-17T13:08:20Z

Hello Yichen Yan, thanks for contributing a PR to our project!

We use the Oracle Contributor Agreement to make the copyright of contributions clear. We don't have a record of you having signed this yet, based on your email address oraluben -(at)- outlook -(dot)- com. You can sign it at that link.

If you think you've already signed it, please comment below and we'll check.

lewurm · 2021-12-20T09:47:08Z

Thank you @oraluben for your contribution! The fix looks good to me, I'll send it through our internal CI system.

lewurm · 2021-12-20T10:07:59Z

As a member of Alibaba, we've signed CLA of Graal.

@oraluben can you make sure all your commits are authored with your Alibaba email address?

oraluben · 2021-12-20T10:13:01Z

As a member of Alibaba, we've signed CLA of Graal.

@oraluben can you make sure all your commits are authored with your Alibaba email address?

should be fine now.

graalvmbot · 2021-12-21T11:08:21Z

Yichen Yan has signed the Oracle Contributor Agreement (based on email address wenji -(dot)- yyc -(at)- alibaba-inc -(dot)- com) so can contribute to this repository.

lewurm · 2021-12-22T12:46:46Z

@oraluben your changes are in the merge queue now, please refrain from updating this PR, otherwise the commit hashes won't match anymore

oraluben force-pushed the fix-darwin-aarch64-stat branch from 987438c to 2a671b7 Compare December 10, 2021 10:16

teshull self-assigned this Dec 10, 2021

oraluben force-pushed the fix-darwin-aarch64-stat branch 4 times, most recently from c15d32d to ed50471 Compare December 17, 2021 13:06

oraluben mentioned this pull request Dec 17, 2021

Fix Python version in Github Actions. #4136

Closed

oraluben force-pushed the fix-darwin-aarch64-stat branch from 0196cb3 to 61b5e78 Compare December 20, 2021 10:11

graalvmbot added the oca-signed label Dec 21, 2021

oraluben added 3 commits December 22, 2021 17:52

Fix fstat call in darwin-aarch64.

3158199

Export necessary package for darwin-aarch64.

f905143

Fix darwin platform.

3e0c8ce

oraluben force-pushed the fix-darwin-aarch64-stat branch from 61b5e78 to 3e0c8ce Compare December 22, 2021 09:53

graalvmbot merged commit 3e0c8ce into oracle:master Dec 22, 2021

oraluben deleted the fix-darwin-aarch64-stat branch December 23, 2021 00:42

lewurm mentioned this pull request Feb 4, 2022

Support for Apple M1 (darwin-aarch64) #2666

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix fstat call in darwin-aarch64. #4107

Fix fstat call in darwin-aarch64. #4107

Uh oh!

oraluben commented Dec 10, 2021 •

edited

Loading

Uh oh!

graalvmbot commented Dec 10, 2021

Uh oh!

oraluben commented Dec 10, 2021

Uh oh!

oraluben commented Dec 10, 2021 •

edited

Loading

Uh oh!

graalvmbot commented Dec 17, 2021

Uh oh!

lewurm commented Dec 20, 2021

Uh oh!

lewurm commented Dec 20, 2021

Uh oh!

oraluben commented Dec 20, 2021

Uh oh!

graalvmbot commented Dec 21, 2021

Uh oh!

lewurm commented Dec 22, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Fix fstat call in darwin-aarch64. #4107

Fix fstat call in darwin-aarch64. #4107

Uh oh!

Conversation

oraluben commented Dec 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

graalvmbot commented Dec 10, 2021

Uh oh!

oraluben commented Dec 10, 2021

Uh oh!

oraluben commented Dec 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

graalvmbot commented Dec 17, 2021

Uh oh!

lewurm commented Dec 20, 2021

Uh oh!

lewurm commented Dec 20, 2021

Uh oh!

oraluben commented Dec 20, 2021

Uh oh!

graalvmbot commented Dec 21, 2021

Uh oh!

lewurm commented Dec 22, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

oraluben commented Dec 10, 2021 •

edited

Loading

oraluben commented Dec 10, 2021 •

edited

Loading