[Ruby 3.2] having runtime issues on darwin

[flavorjones]

@larskanis I'm hoping you can help me out here, I'm not sure what's going wrong in the toolchain.

I've built a version of Nokogiri that precompiles Ruby 3.2 shared objects using the docker images published here. (The pipeline to build and test Nokogiri is here if you're curious, and the PR is here).

For both x86_64-darwin and arm64-darwin (the latter I run manually on my dev machine), I see this runtime error:

/Users/runner/hostedtoolcache/Ruby/3.2.0-rc1/x64/lib/ruby/gems/3.2.0+3/gems/nokogiri-1.15.test.2022.1219.1745-x86_64-darwin/lib/nokogiri/extension.rb:7:in `require_relative': dlopen(/Users/runner/hostedtoolcache/Ruby/3.2.0-rc1/x64/lib/ruby/gems/3.2.0+3/gems/nokogiri-1.15.test.2022.1219.1745-x86_64-darwin/lib/nokogiri/3.2/nokogiri.bundle, 0x0009): Symbol not found: (_rb_cObject) (LoadError)
  Referenced from: '/Users/runner/hostedtoolcache/Ruby/3.2.0-rc1/x64/lib/ruby/gems/3.2.0+3/gems/nokogiri-1.15.test.2022.1219.1745-x86_64-darwin/lib/nokogiri/3.2/nokogiri.bundle'
  Expected in: '/Users/runner/hostedtoolcache/Ruby/3.2.0-rc1/x64/bin/ruby' - /Users/runner/hostedtoolcache/Ruby/3.2.0-rc1/x64/lib/ruby/gems/3.2.0+3/gems/nokogiri-1.15.test.2022.1219.1745-x86_64-darwin/lib/nokogiri/3.2/nokogiri.bundle

You can see this in the CI logs here. You can download this gem yourself if you like from the "Artifacts" section of this pipeline run:

• arm64 gem
• x86_64 gem

Comparing the 3.2 dylib and the 3.1 dylib (which still works), both have _rb_cObject listed as Undefined, so I'm not actually sure the problem is in how the shared object is built. Do you have any ideas?

[flavorjones]

Interesting: when I install 3.2.0-rc1 using ruby-build then that version of Ruby is able to run this gem fine. So maybe this is a problem with how ruby/setup-ruby is building 3.2.0-rc1?

[larskanis]

The _rb_cObject should be in libruby, but libruby isn't mentioned in the above error output. I guess it has something to do with this commit ruby/ruby@50d81bf . On Macos we use the same trick like on Linux, that unresolved symbols are automatically resolved to already loaded libraries. Therefore the nokogiri.bundle doesn't link to libruby explicit, but is still resolved at runtime.

[flavorjones]

Looks like if I build ruby with --enable-shared then this happens at runtime. When I build without that flag, ruby loads the bundle fine.

[larskanis]

Here was a similar issue with nokogiri on Macos: sparklemotion/nokogiri#2063 (comment)

[flavorjones]

Wow, thanks for refreshing my memory, I have very little recollection of doing that work. 🤷

So at this point, I think the 3.2 bundle is fine ... do you agree?

I want to understand better what's changed about the Ruby build system, but as long as --disable-shared (which is implicit by default) is set, users should be fine. Do you agree?

[flavorjones]

Tagging @kateinoigakukun in case he spots something that's wrong with either the rake-compiler-dock build process or Ruby's makefiles.

[kateinoigakukun]

@flavorjones Unfortunately, an extension library compiled with a --enable-shared'ed ruby is not compatible with a --disable-shared'ed ruby, and vice versa.

I think the compatibility has not been guaranteed but it just worked so far. The bundle_loader change changed the situation, so we can't assume that fat gems are portable among rubies configured with different options.

[flavorjones]

@kateinoigakukun Thanks for confirming.

@larskanis Do you know why the 3.2.0-rc1 ruby binaries installed by ruby/setup-ruby aren't compatible? I looked in the repo but nothing jumped out at me.

[larskanis]

@flavorjones Which ruby binaries aren't compatible with what? Do you have a CI link?

[flavorjones]

@larskanis Darwin 3.2.0-rc1 binaries aren't compatible with the extensions being built by rake-compiler-dock.

CI link: https://github.com/sparklemotion/nokogiri/actions/runs/3735058984/jobs/6350173407

The CI config is:

  cruby-x86_64-darwin-install:
    needs: ["cruby-package"]
    strategy:
      fail-fast: false
      matrix:
        ruby: ["2.7", "3.0", "3.1", "3.2.0-rc1"]
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: true
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: "${{matrix.ruby}}"
      - uses: actions/download-artifact@v3
        with:
          name: cruby-x86_64-darwin-gem
          path: gems
      - run: ./scripts/test-gem-install gems

and it's passing for the other Ruby versions.

[flavorjones]

#90 reproduces this error in rake-compiler-dock's CI pipeline

[flavorjones]

@larskanis I confirmed that the rubies built by ruby-builder (and downloadable from here) are built with --enable-shared.

Not sure what you think we should do here. I just pushed #91 to see what will break if we set --enable-shared on darwin.

[flavorjones]

Unfortunately, an extension library compiled with a --enable-shared'ed ruby is not compatible with a --disable-shared'ed ruby, and vice versa

@kateinoigakukun This change may have unintended consequences. Over the past two years, nokogiri precompiled for darwin has been downloaded 12,062,659 times, so I'm very worried that if we can no longer ship precompiled binaries for darwin that this is going to be a burden on both users and gem maintainers.

Is the recommended workaround to ship two bundles for darwin? One for rubies configured with --enable-shared and one for rubies configured with --disable-shared?