Build with frame pointers for improved profiling

[erikgrinaker]

Release binaries are currently built without frame pointers. This frees up a register for the compiler and avoids a couple of instructions per function call, which can improve performance (typically <1%). However, stack unwinding and profiling then has to use DWARF information to generate backtraces, which is far more expensive and can cause difficulty e.g. for perf and eBPF profilers.

We're considering continuous profiling, and jemalloc heap profiling already probabilistically takes stack traces during allocations. These stack traces will be much cheaper with frame pointers enabled. This might save more CPU than we lose with the dedicated frame pointer register, and allow us to profile at higher frequency.

The Rust stdlib recently enabled frame pointers by default for this reason. It's also possible that frame pointers are already enabled by default on aarch64 CPUs (used for Pageservers), since this architecture uses a dedicated frame pointer register.

Related reading:

• https://www.brendangregg.com/blog/2024-03-17/the-return-of-the-frame-pointers.html
• Enable frame pointers for the standard library rust-lang/rust#122646

[erikgrinaker]

It's also possible that frame pointers are already enabled by default on aarch64 CPUs (used for Pageservers)

This isn't the case. I looked at the assembly function prologue, which doesn't maintain frame pointers. This is confirmed by the rustc target specs, which default to FramePointer::MayOmiton Linux and doesn't override this for the aarch64_unknown_linux_gnu profile:

https://github.com/rust-lang/rust/blob/fd19773d2f8a070dc03f0072f9bc41a65fd04fed/compiler/rustc_target/src/spec/base/linux.rs

https://github.com/rust-lang/rust/blob/fd19773d2f8a070dc03f0072f9bc41a65fd04fed/compiler/rustc_target/src/spec/targets/aarch64_unknown_linux_gnu.rs

Apple aarch64 does enable frame pointers, since this is required by Apple debug tooling:

https://github.com/rust-lang/rust/blob/fd19773d2f8a070dc03f0072f9bc41a65fd04fed/compiler/rustc_target/src/spec/base/apple/mod.rs#L122

[erikgrinaker]

tikv-jemallocator builds jemalloc with frame pointers:

[tikv-jemalloc-sys 0.6.0+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7] CFLAGS="-O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -m64 -Wall"

[erikgrinaker]

The frame-pointer feature in pprof-rs is 10x slower than libunwind without frame pointers. See #10226 (comment).

[erikgrinaker]

Reopening in an attempt to fix pprof-rs seg faults.