[macos] native_assets_test.dart: flutter run hot reload and hot restart with native assets macos debug is flaky

[knopp]

Integration test sometimes fails with

34:35 +84 ~1 -2: test/integration.shard/isolated/native_assets_test.dart: flutter run hot reload and hot restart with native assets macos debug [E]
  Expected: not contains 'Invalid argument(s): Couldn\'t resolve native function \'sum\''
    Actual: 'warning: Run script build phase \'Run Script\' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target \'Flutter Assemble\' from project \'Runner\')\n'
              'Performing hot reload...                                        \n'
              'The Flutter DevTools debugger and profiler on macOS is available at: http://127.0.0.1:9106?uri=http://127.0.0.1:53308/xYohxuqAUcQ=/\n'
              'Reloaded 0 libraries in 520ms (compile: 8 ms, reload: 0 ms, reassemble: 492 ms).\n'
              '\n'
              'Performing hot restart...                                       \n'
              'Restarted application in 5,343ms.\n'
              '\n'
              'Performing hot reload...                                        \n'
              '\n'
              '══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════\n'
              'The following ArgumentError was thrown building _FocusInheritedScope:\n'
              'Invalid argument(s): Couldn\'t resolve native function \'sum\' in\n'
              '\'package:package_with_native_assets/package_with_native_assets_bindings_generated.dart\' : Failed to\n'
              'load dynamic library \'package_with_native_assets1.framework/package_with_native_assets1\': Failed to\n'
              'load dynamic library \'package_with_native_assets1.framework/package_with_native_assets1\':\n'
              'dlopen(package_with_native_assets1.framework/package_with_native_assets1, 0x0001): tried:\n'
              '\'package_with_native_assets1.framework/package_with_native_assets1\' (no such file),\n'
              '\'/System/Volumes/Preboot/Cryptexes/OSpackage_with_native_assets1.framework/package_with_native_assets1\'\n'
              '(no such file),\n'

I'm unable to reproduce this locally.

[knopp]

cc @dcharkes

[dcharkes]

I have run the test multiple times locally, but I am also unable to reproduce this.

If it's flaky, the Flutter CI should not notice and file a P0 issue and assign it to me (I'm the test owner).

[knopp]

I opened this in case you have any idea what this could be caused by. It looks like a potential issue with duplicate framework name prevention (package_with_native_assets1.framework), but in macos/debug there's only one architecture so I don't quite see the codepath that could be causing this.

[jmagman]

If it's flaky, the Flutter CI should not notice and file a P0 issue and assign it to me (I'm the test owner).

@dcharkes this is happening in tool_integration_tests_4_4, which doesn't get flagged by the flaky bot:

flutter/TESTOWNERS

Line 334 in c77c0cf

# tool_integration_tests @christopherfujino @flutter/tool

It's failed four times in the last three commits:
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8746080258132885809/+/u/run_test.dart_for_tool_integration_tests_shard_and_subshard_4_4/stdout

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8746098371767711457/+/u/run_test.dart_for_tool_integration_tests_shard_and_subshard_4_4/stdout

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8746095741986732209/+/u/run_test.dart_for_tool_integration_tests_shard_and_subshard_4_4/stdout

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8746047964768234769/+/u/run_test.dart_for_tool_integration_tests_shard_and_subshard_4_4/stdout

[dcharkes]

I can't reproduce this locally, but inspecting the temp folder unit test yields:

format-version:
  - 1
  - 0
  - 0
native-assets:
  macos_arm64:
    package:package_with_native_assets/package_with_native_assets_bindings_generated.dart:
      - absolute
      - package_with_native_assets.framework/package_with_native_assets
    package:link_hook/link_hook_bindings_generated.dart:
      - absolute
      - link_hook.framework/link_hook
  macos_x64:
    package:package_with_native_assets/package_with_native_assets_bindings_generated.dart:
      - absolute
      - package_with_native_assets1.framework/package_with_native_assets1
    package:link_hook/link_hook_bindings_generated.dart:
      - absolute
      - link_hook1.framework/link_hook1

I'm thinking that somehow the order of architectures gets switched during hot reload or hot restart on the bot, causing the two different names to swap archs.

Now, these shouldn't be different names to begin with. This was recently fixed in iOS (02d5286), but not in MacOS. I've created #149761 to address this.