Absorb the `libm` repository into `compiler-builtins`. #822

tgross35 · 2025-04-18T23:39:00Z

Merge remote-tracking branch 'libm/merge-into-builtins-prep' into absorb-libm

Absorb the libm repository into compiler-builtins.

This was done using git-filter-repo to ensure hashes mentioned in
commit messages were correctly rewritten, I used the same strategy to
merge ctest into libc 1 and it worked quite well. Approximately:

# `git filter-repo` requires a clean clone
git clone https://github.com/rust-lang/libm.git

# Move all code to a `libm` subdirectory for all history
git filter-repo --to-subdirectory-filter libm

# The default merge messages are "merge pull request #nnn from
# user/branch". GH links these incorrectly in the new repo, so
# rewrite messages from `#nnn` to `rust-lang/libm#nnn`.
echo 'regex:(^|\s)(#\d+)==>\1rust-lang/libm\2' > messages.txt
git filter-repo --replace-message messages.txt

# Re-add a remote and push as a new branch
git remote add upstream https://github.com/rust-lang/libm.git
git switch -c merge-into-builtins-prep
git push --set-upstream upstream merge-into-builtins-prep

# Now in a compiler-builtins, add `libm` as a remote
git remote add libm https://github.com/rust-lang/libm.git
git fetch libm

# Do the merge that creates this commit
git merge libm/merge-into-builtins-prep --allow-unrelated-histories

The result should be correct git history and blame for all files, with
messages that use correct rewritten hashes when they are referenced.
There is some reorganization and CI work needed, but that will be a
follow up.

After this merges I will need to push tags from libm, which I have
already rewritten to include a libm- prefix. Old tags in
compiler-builtins should likely also be rewritten to add a prefix (we
already have this for newer tags), but this can be done at any point.

Original remote: https://github.com/rust-lang/libm.git
Default HEAD: c94017af75c3ec4616d5b7f9b6b1b3826b934469 ("Migrate all
crates except libm to edition 2024")
HEAD after rewriting history: 15fb630
("Migrate all crates except libm to edition 2024")

It is currently getting the default of 1 or 2. Since this operation should always be infinite precision, no deviation is allowed.

Currently `logspace` does a lossy cast from `F::Int` to `usize`. This could be problematic in the rare cases that this is called with a step count exceeding what is representable in `usize`. Resolve this by instead adding bounds so the float's integer type itself can be iterated.

We want to be able to adjust our configuration based on whether we are running in CI, propagate this so our tests can use it.

Rather than collecting a list of file names in `libm-test/build.rs`, just use a script to parse rustdoc's JSON output.

Now that we are using rustdoc output to locate public functions, the test is indicating a few that were missed since they don't have their own function. Update everything to now include the following routines: * `erfc` * `erfcf` * `y0` * `y0f` * `y1` * `y1f` * `yn` * `ynf`

Use `rustdoc` JSON for API list, add functions that were missing

Once we start addinf `f16` and `f128` routines, we will need to have this cfg for almost all uses of `for_each_function`. Rather than needing to specify this each time, always emit `#[cfg(f16_enabled)]` or `#[cfg(f128_enabled)]` for each function that uses `f16` or `f128`, respectively.

Using the same name as the routines themselves means this will correctly get picked up by the CI job looking for exhaustive tests.

Currently our implementations for `abs` and `copysign` are defined on the trait, and these are then called from `generic`. It would be better to call core's `.abs()` / `.copysign(y)`, but we can't do this in the generic because calling the standalone function could become recursive (`fabsf` becomes `intrinsics::fabsf32`, that may lower to a call to `fabsf`). Change this so the traits uses the call to `core` if available, falling back to a call to the standalone generic function. In practice the recursion isn't likely to be a problem since LLVM probably always lowers `abs`/`copysign` to assembly, but this pattern should be more correct for functions that we will add in the future (e.g. `fma`). This should eventually be followed by a change to call the trait methods rather than `fabs`/`copysign` directly.

Make it more obvious what the expected ULP for a given routine is. This also narrows ULP to 0 for operations that require exact results.

This is a nonfunctional change.

There isn't any need to cache the integer since it gets provided as an argument anyway. Simplify this in `jn` and `yn`.

Rug provides `trunc_fract_round`, which implements `modf`, use it to add a test.

This implementation comes from `rug::Float::to_f32_exp` [1]. [1]: https://docs.rs/rug/1.26.1/rug/struct.Float.html#method.to_f32_exp

Occasionally it is useful to see some information from running tests without making everything noisy from `--nocapture`. Add a function to log this kind of output to a file, and print the file as part of CI.

Currently, tests use a handful of constants to determine how many iterations to perform: `NTESTS`, `AROUND`, and `MAX_CHECK_POINTS`. This configuration is not very straightforward to adjust and needs to be repeated everywhere it is used. Replace this with new functions in the `run_cfg` module that determine iteration counts in a more reusable and documented way. This only updates `edge_cases` and `domain_logspace`, `random` is refactored in a later commit.

Introduce the `KnownSize` iterator wrapper, which allows providing the size at construction time. This provides an `ExactSizeIterator` implemenation so we can check a generator's value count during testing.

Currently, all inputs are generated and then cached. This works reasonably well but it isn't very configurable or extensible (adding `f16` and `f128` is awkward). Replace this with a trait for generating random sequences of tuples. This also removes possible storage limitations of caching all inputs.

Streamline the way that test iteration count is determined (replace `NTESTS`)

The `support` module that this feature makes public will be useful for implementations in `compiler-builtins`, not only for testing. Give this feature a more accurate name.

New random seeds seem to indicate that this test does have some more failures, this is a recent failure on i586: ---- musl_random_jnf stdout ---- Random Musl jnf arg 1/2: 100 iterations (10000 total) using `LIBM_SEED=nLfzQ3U1OBVvqWaMBcto84UTMsC5FIaC` Random Musl jnf arg 2/2: 100 iterations (10000 total) using `LIBM_SEED=nLfzQ3U1OBVvqWaMBcto84UTMsC5FIaC` thread 'musl_random_jnf' panicked at crates/libm-test/tests/compare_built_musl.rs:43:51: called `Result::unwrap()` on an `Err` value: input: (205, 5497.891) (0x000000cd, 0x45abcf21) expected: 7.3291517e-6 0x36f5ecef actual: 7.331668e-6 0x36f6028c Caused by: ulp 5533 > 4000 It seems unlikely that `jn` would somehow have better precision than `j0`/`j1`, so just use the same precision.

Update the script to produce, in addition to the simple text list, a JSON file listing routine names, the types they work with, and the source files that contain a function with the routine name. This gets consumed by another script and will be used to determine which extensive CI jobs to run.

Add a generator that will test all inputs for input spaces `u32::MAX` or smaller (e.g. single-argument `f32` routines). For anything larger, still run approximately `u32::MAX` tests, but distribute inputs evenly across the function domain. Since we often only want to run one of these tests at a time, this implementation parallelizes within each test using `rayon`. A custom test runner is used so a progress bar is possible. Specific tests must be enabled by setting the `LIBM_EXTENSIVE_TESTS` environment variable, e.g. LIBM_EXTENSIVE_TESTS=all_f16,cos,cosf cargo run ... Testing on a recent machine, most tests take about two minutes or less. The Bessel functions are quite slow and take closer to 10 minutes, and FMA is increased to run for about the same.

Add a CI job with a dynamically calculated matrix that runs extensive jobs on changed files. This makes use of the new `function-definitions.json` file to determine which changed files require full tests for a routine to run.

Add exhaustive/extensive tests

Since `fmod` is generic, there isn't any need to have the small wrappers in separate files. Most operations was done in [1] but `fmod` was omitted until now. [1]: rust-lang/libm#537

Benchmarks for [1] seemed to indicate that repository organization for some reason had an effect on performance, even though the exact same rustc commands were running (though some with a different order). After investigating more, it appears that dependencies may have an affect on inlining thresholds for generic functions. It is surprising that this happens, we more or less expect that public functions will be standalone but everything they call will be inlined. To help ensure this, mark all generic functions `#[inline]` if they should be merged into the public function. Zulip discussion at [2]. [1]: rust-lang/libm#533 [2]: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Dependencies.20affecting.20codegen/with/513079387

In preparation for switching to a virtual manifest, move the `libm` crate into a subdirectory and update paths to match. Updating `Cargo.toml` is done in the next commit so git tracks the moved file correctly.

Move the workspace configuration to a virtual manifest. This reorganization makes a more clear separation between package contents and support files that don't get distributed. It will also make it easier to merge this repository with `compiler-builtins` which is planned (builtins had a similar update done in [1]). LICENSE.txt and README.md are symlinkedinto the new directory to ensure they get included in the package. [1]: #702

Unfortunately this means we lose use of the convenient name `gen`, so this includes a handful of renaming. We can't increase the edition for `libm` yet due to MSRV, but we can enable `unsafe_op_in_unsafe_fn` to help make that change smoother in the future.

tgross35 · 2025-04-18T23:46:10Z

GH is only showing 250 commits, the relevant merge is a0db2ca.

…orb-libm Absorb the libm repository into `compiler-builtins`. This was done using `git-filter-repo` to ensure hashes mentioned in commit messages were correctly rewritten, I used the same strategy to merge `ctest` into `libc` [1] and it worked quite well. Approximately: # `git filter-repo` requires a clean clone git clone https://github.com/rust-lang/libm.git # Move all code to a `libm` subdirectory for all history git filter-repo --to-subdirectory-filter libm # The default merge messages are "merge pull request #nnn from # user/branch". GH links these incorrectly in the new repo, so # rewrite messages from `#nnn` to `rust-lang/libm#nnn`. echo 'regex:(^|\s)(#\d+)==>\1rust-lang/libm\2' > messages.txt git filter-repo --replace-message messages.txt # Re-add a remote and push as a new branch git remote add upstream https://github.com/rust-lang/libm.git git switch -c merge-into-builtins-prep git push --set-upstream upstream merge-into-builtins-prep # Now in a compiler-builtins, add `libm` as a remote git remote add libm https://github.com/rust-lang/libm.git git fetch libm # Do the merge that creates this commit git merge libm/merge-into-builtins-prep --allow-unrelated-histories The result should be correct git history and blame for all files, with messages that use correct rewritten hashes when they are referenced. There is some reorganization and CI work needed, but that will be a follow up. After this merges I will need to push tags from `libm`, which I have already rewritten to include a `libm-` prefix. Old tags in compiler-builtins should likely also be rewritten to add a prefix (we already have this for newer tags), but this can be done at any point. * Original remote: https://github.com/rust-lang/libm.git * Default HEAD: c94017af75c3ec4616d5b7f9b6b1b3826b934469 ("Migrate all crates except `libm` to edition 2024") * HEAD after rewriting history: 15fb630 ("Migrate all crates except `libm` to edition 2024") [1]: rust-lang/libc#4283 (comment)

tgross35 · 2025-04-19T20:22:16Z

946e467 is now the relevant merge commit

tgross35 · 2025-04-19T20:37:35Z

I asked Mark to fastforward master so GH doesn't interfere with history, everything looks good.

Since [1], `libm` is now part of the `compiler-builtins` repositoy, so this repo will be archived. Update the README to reflect that. [1]: rust-lang/compiler-builtins#822

This has been merged into https://github.com/rust-lang/compiler-builtins. Link: rust-lang/compiler-builtins#822

Mention libm in the description, and remove the homepage that points to a long-closed issue. Also update quotes to match most other files. Link: rust-lang/compiler-builtins#822

This has been merged into https://github.com/rust-lang/compiler-builtins. Link: rust-lang/compiler-builtins#822

Mention libm in the description, and remove the homepage that points to a long-closed issue. Also update quotes to match most other files. Link: rust-lang/compiler-builtins#822

This has been merged into https://github.com/rust-lang/compiler-builtins. Link: rust-lang/compiler-builtins#822

Mention libm in the description, and remove the homepage that points to a long-closed issue. Also update quotes to match most other files. Link: rust-lang/compiler-builtins#822

tgross35 added 30 commits December 29, 2024 04:18

Don't run push CI on anything other than master

ca9f238

Set the allowed FMA ULP to 0

1c224e1

It is currently getting the default of 1 or 2. Since this operation should always be infinite precision, no deviation is allowed.

Forward the CI environment variable when running in Docker

cfad96c

We want to be able to adjust our configuration based on whether we are running in CI, propagate this so our tests can use it.

Use rustdoc output to create a list of public API

bfc6ddd

Rather than collecting a list of file names in `libm-test/build.rs`, just use a script to parse rustdoc's JSON output.

Merge pull request rust-lang/libm#384 from tgross35/api-list

a85942b

Use `rustdoc` JSON for API list, add functions that were missing

Rename generic abs to fabs

7491ed5

Using the same name as the routines themselves means this will correctly get picked up by the CI job looking for exhaustive tests.

Change to exhaustive matching for default_ulp

0dd1d7b

Make it more obvious what the expected ULP for a given routine is. This also narrows ULP to 0 for operations that require exact results.

precision: Sort ilogb with other precise operations

224e3d5

This is a nonfunctional change.

Clean up integers stored in MpTy

5d9371c

There isn't any need to cache the integer since it gets provided as an argument anyway. Simplify this in `jn` and `yn`.

Add tests against MPFR for modf and modff

9560120

Rug provides `trunc_fract_round`, which implements `modf`, use it to add a test.

Add tests against MPFR for frexp and frexpf

7ce5096

This implementation comes from `rug::Float::to_f32_exp` [1]. [1]: https://docs.rs/rug/1.26.1/rug/struct.Float.html#method.to_f32_exp

Add tests against MPFR for scalbn{f} and ldexp{f}

f690245

Add a way for tests to log to a file

3cd43fa

Occasionally it is useful to see some information from running tests without making everything noisy from `--nocapture`. Add a function to log this kind of output to a file, and print the file as part of CI.

Add an iterator that ensures known size

19de96c

Introduce the `KnownSize` iterator wrapper, which allows providing the size at construction time. This provides an `ExactSizeIterator` implemenation so we can check a generator's value count during testing.

Update precision based on new test results

04238bc

Merge pull request rust-lang/libm#379 from rust-lang/tgross35/ntests

c0dda2f

Streamline the way that test iteration count is determined (replace `NTESTS`)

Rename unstable-test-support to unstable-public-internals

5f725f3

The `support` module that this feature makes public will be useful for implementations in `compiler-builtins`, not only for testing. Give this feature a more accurate name.

Add tests against MPFR for ilogb and ilogbf

80a0d38

Update precision based on failures from extensive tests

00bdc58

Run extensive tests in CI when relevant files change

c963b54

Add a CI job with a dynamically calculated matrix that runs extensive jobs on changed files. This makes use of the new `function-definitions.json` file to determine which changed files require full tests for a routine to run.

Merge pull request rust-lang/libm#388 from rust-lang/tgross35/extensive

7b69e7b

Add exhaustive/extensive tests

tgross35 added 5 commits April 18, 2025 15:14

Combine the source files for fmod

c3c363f

Since `fmod` is generic, there isn't any need to have the small wrappers in separate files. Most operations was done in [1] but `fmod` was omitted until now. [1]: rust-lang/libm#537

refactor: Move the libm crate to a subdirectory

54bac41

In preparation for switching to a virtual manifest, move the `libm` crate into a subdirectory and update paths to match. Updating `Cargo.toml` is done in the next commit so git tracks the moved file correctly.

tgross35 requested a review from Amanieu April 18, 2025 23:49

tgross35 changed the title ~~Absorb the libm repository into compiler-builtins.~~ Absorb the libm repository into compiler-builtins. Apr 18, 2025

Amanieu approved these changes Apr 19, 2025

View reviewed changes

tgross35 added 5 commits April 19, 2025 20:16

Remove the libm submodule

cf4fd02

Update submodules after the libm merge

b66086f

Update path to libm after the merge

0e4ba2a

Update .git-blame-ignore-revs after the libm merge

52e748b

tgross35 force-pushed the absorb-libm branch from f47886b to 52e748b Compare April 19, 2025 20:18

Mark-Simulacrum merged commit 52e748b into master Apr 19, 2025
28 checks passed

Mark-Simulacrum deleted the absorb-libm branch April 19, 2025 20:34

tgross35 mentioned this pull request Apr 19, 2025

Libm reorganization #831

Merged

tgross35 mentioned this pull request Apr 20, 2025

Note that this crate has been moved rust-lang/libm#546

Merged

tgross35 added a commit to tgross35/rust-team that referenced this pull request Apr 21, 2025

Archive the libm repository

fce839b

This has been merged into https://github.com/rust-lang/compiler-builtins. Link: rust-lang/compiler-builtins#822

tgross35 mentioned this pull request Apr 21, 2025

Archive rust-lang/libm, update compiler-builtins rust-lang/team#1783

Merged

tgross35 added a commit to tgross35/rust-team that referenced this pull request Apr 22, 2025

Archive the libm repository

31d362f

This has been merged into https://github.com/rust-lang/compiler-builtins. Link: rust-lang/compiler-builtins#822

tgross35 added a commit to tgross35/rust-team that referenced this pull request Apr 22, 2025

Archive the libm repository

c373f63

This has been merged into https://github.com/rust-lang/compiler-builtins. Link: rust-lang/compiler-builtins#822

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Absorb the `libm` repository into `compiler-builtins`. #822

Absorb the `libm` repository into `compiler-builtins`. #822

Uh oh!

tgross35 commented Apr 18, 2025

Uh oh!

tgross35 commented Apr 18, 2025

Uh oh!

tgross35 commented Apr 19, 2025

Uh oh!

Uh oh!

tgross35 commented Apr 19, 2025

Uh oh!

Uh oh!

Absorb the libm repository into compiler-builtins. #822

Absorb the libm repository into compiler-builtins. #822

Uh oh!

Conversation

tgross35 commented Apr 18, 2025

Uh oh!

tgross35 commented Apr 18, 2025

Uh oh!

tgross35 commented Apr 19, 2025

Uh oh!

Uh oh!

tgross35 commented Apr 19, 2025

Uh oh!

Uh oh!

Absorb the `libm` repository into `compiler-builtins`. #822

Absorb the `libm` repository into `compiler-builtins`. #822