ARM (32-bit) SIGILL

[mc-appl]

I was able to get everything cross-compiled to armv7 32-bit (beaglebone cortex8) fine from the meta-flutter recipes after making a few small changes (flutter/buildroot#499). I am testing out the wayland client but its SIGILL'ing in the flutter lib after the flutter-client is booted and threads are running. Any ideas where to go from here? I am not sure how the performance is going to be once its running, but it would be great to see how it performs on the beaglebone with the PowerVR GPU + Weston8 compositor.

FWIW - I have tried libflutter cross compiled and in debug mode to see if i could step through and figure out what the instruction is that its not liking, but I am unsure of the steps to create an .AOT file that is also in debug mode. When i try and run swap the release libflutter for the the debug libflutter its not able to load (I think a previous issue mentioned how to work around that, but for the desktop linux version).

root@migraine:~/pkg# gdb --args flutter-client --bundle=/usr/share/flutter/examples
GNU gdb (GDB) 9.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-poky-linux-gnueabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from flutter-client...
Reading symbols from /usr/bin/.debug/flutter-client...
(gdb) br main.cc:41
Breakpoint 1 at 0x5a2a0: file /usr/src/debug/flutter-wayland-client/1.0-r0/git/examples/flutter-wayland-client/main.cc, line 41.
(gdb) r
Starting program: /usr/bin/flutter-client --bundle=/usr/share/flutter/examples
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
[New LWP 1256]
[New LWP 1257]
[New LWP 1258]
[New LWP 1259]
[New LWP 1260]

Thread 1 "flutter-client" hit Breakpoint 1, main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/flutter-wayland-client/1.0-r0/git/examples/flutter-wayland-client/main.cc:41
41      /usr/src/debug/flutter-wayland-client/1.0-r0/git/examples/flutter-wayland-client/main.cc: No such file or directory.
(gdb) info thread
  Id   Target Id                 Frame 
* 1    LWP 1253 "flutter-client" main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/flutter-wayland-client/1.0-r0/git/examples/flutter-wayland-client/main.cc:41
  2    LWP 1256 "io.flutter.ui"  0xb6dae75e in ?? () from /usr/lib/libflutter_engine.so
  3    LWP 1257 "flutter-client" 0xb6447af8 in ?? () from /lib/libpthread.so.0
  4    LWP 1258 "io.flutter.io"  0xb65545a4 in epoll_wait () from /lib/libc.so.6
  5    LWP 1259 "io.worker.1"    0xb644aecc in pthread_cond_wait () from /lib/libpthread.so.0
  6    LWP 1260 "flutter-client" 0xb65545a4 in epoll_wait () from /lib/libc.so.6
(gdb) c
Continuing.

Thread 2 "io.flutter.ui" received signal SIGILL, Illegal instruction.
[Switching to LWP 1256]
0xb5424b8c in ?? ()
(gdb) 

vbox$ readelf -A ./deploy/images/dorado/p2/usr/bin/id.coreutils
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3
  Tag_Advanced_SIMD_arch: NEONv1
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_rounding: Needed
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6
  Tag_Virtualization_use: TrustZone


vbox$ readelf -A ./sysroots-components/cortexa8hf-neon/flutter-engine/usr/lib/libflutter_engine.so
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3
  Tag_Advanced_SIMD_arch: NEONv1
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_Virtualization_use: TrustZone

[HidenoriMatsubayashi]

Thank you for your comment and challenge.

I am not sure how the performance is going to be once its running, but it would be great to see how it performs on the beaglebone with the PowerVR GPU + Weston8 compositor.

It's interesting for me, too. I've tried to work flutter apps on raspberry pi4 (32bit) and worked it fine, but it was over a year ago.

Also, you can cross-build libflutter_engine without stripping debug symbols by setting true into is_debug option (in gn debug build) to debug. See also:

• https://www.chromium.org/developers/gn-build-configuration
• https://github.com/flutter/buildroot/blob/fc82ca44f1ae85c5fbfa06bd313477539ca50729/build/config/BUILDCONFIG.gn#L149
• https://github.com/flutter/buildroot/blob/master/build/toolchain/clang.gni#L28

[mc-appl]

@HidenoriMatsubayashi Thanks so much for the response, I was able to get symbols loaded. After trying many things I was convinced there were no issues with the libflutter-engine.so and focused on the VM snapshot side of things. After much digging I found this issue mentioned on the arm7 pi with dart:

dart-lang/sdk#24466

I added both the flags to then gen_snapshot that were mentioned and now everything is working. It is actually quite performant after things get loaded. I need to dig in and understand things a little better. As a side note for reference, you can see all VM options with clang_x64/gen_snapshot --verbose

[HidenoriMatsubayashi]

@mc-appl

Great! Also, I welcome to accept your PR (yocto-recipe) to add flutter-engine 32bit build support in https://github.com/sony/meta-flutter, if you want.