misc coercion cleanups and handle safety correctly

[BoxyUwU]

r? lcnr

todo: write better PR description and FCP comment for the safety stuff

"remove normalize call"

Fixes #132765 in theory breaking as we might now treat some previously-ambig hr aliases as rigid when we shouldn't. we may want to explicitly normalize the coercion target in fcx.coerce like the new solver does.

"leak check and lub for closure<->closure coerce-lubs of same defids"

Fixes rust-lang/trait-system-refactor-initiative#233

fn peculiar() -> impl Fn(u8) -> u8 {
    return |x| x + 1
}

the |x| x + 1 expr has a type of Closure(?31t) which we wind up inferring the RPIT to. The CoerceMany ret_coercion for the whole peculiar typeck has an expected type of RPIT (unnormalized). When we type check the return |x| x + 1 expr we go from the never type to Closure(?31t) which then participates in the ret_coercion giving us a coerce-lub(RPIT, Closure(?31t)).

Normalizing RPIT gives us some Closure(?50t) where ?31t and ?50t have been unified with ?31t as the root var. resolve_vars_if_possible doesn't resolve infer vars to their roots so these wind up with different structural identities so the fast path doesn't apply and we fall back to coercing to a fn ptr. cc #147193 which also fixes this

New solver probably just gets more inference variables here because canonicalization + generally different approach to normalization of opaques. Idk :3

This technically allows more code to compile (see the test using TAIT in the commit). I don't think this can be observed on stable though.

In theory this could break existing code that relied on coercing a closure to itself resulting in a fnptr. I don't expect this to really happen in practice and this is an "obvious" bug fix.

leak check fndef<->fndef lubs of the same defids

when coerce-lub'ing a fndef with another fndef we already try to lub it first to avoid coercing to a fnptr unnecessarily. we now leak check after this lub operation.

this avoids us incorrectly considering the fndefs to be equal despite unequal binders and then winding up with borrow checker errors due when coercing to a fnptr would have worked fine. see tests/ui/coercion/leak_check_fndef_lub.rs

this is theoretically a breaking change as there could be dead code that relied on these unsatisfiable region constraints never being checked while also asserting that the output of the lub operation was an fndef. I don't think such code is likely to exist, and even if it does I am comfortable breaking it in favour of making code start working that expects us to coerce to a fnptr for unequal fndefs.

"account for safe target features in fndef<->closure and fndef<->fndef coerce-lubs"

We previously did not take safe target features into account when creating the fn sig for fndefs during fndef<->closure or fndef<->fndef coerce-lubs. We now do allowing coerce-lub to produce a safe fn pointer when fndefs with safe target features are involved. This is consistent with existing (non lub) coercion logic for fndef->fnptr which allows coercing to a safe fn pointer.

[rust-log-analyzer]

The job x86_64-gnu-llvm-20 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

test [ui] tests/ui/coercion/issue-88097.rs#next ... ok
test [ui] tests/ui/coercion/lub_coercion_handles_safety.rs ... ignored, only executed when the architecture is x86_64
test [ui] tests/ui/coercion/leak_check_fndef_lub_deadcode_breakage.rs ... FAILED
test [ui] tests/ui/coercion/issue-39823.rs ... ok
test [ui] tests/ui/coercion/lub_closures_before_fnptr_coercion.rs ... ok
test [ui] tests/ui/coercion/method-return-trait-object-14399.rs ... ok
test [ui] tests/ui/coercion/mut-trait-object-coercion-8398.rs ... ok
test [ui] tests/ui/coercion/mut-mut-wont-coerce.rs ... ok
test [ui] tests/ui/coercion/no_local_for_coerced_const-issue-143671.rs ... ok
test [ui] tests/ui/coercion/pin-dyn-dispatch-sound.rs ... ok
---
test [ui] tests/ui/zero-sized/zero-sized-tuple-struct.rs ... ok

failures:

---- [ui] tests/ui/coercion/leak_check_fndef_lub_deadcode_breakage.rs stdout ----
Saved the actual stderr to `/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/coercion/leak_check_fndef_lub_deadcode_breakage/leak_check_fndef_lub_deadcode_breakage.stderr`
diff of stderr:

4 LL |     unsafe { std::mem::transmute::<_, ()>(lubbed) }
5    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
6    |
-    = note: source type: `fn()` (64 bits)
+    = note: source type: `fn()` (32 bits)
8    = note: target type: `()` (0 bits)
9 
10 error: aborting due to 1 previous error


The actual stderr differed from the expected stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args coercion/leak_check_fndef_lub_deadcode_breakage.rs`

error: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/coercion/leak_check_fndef_lub_deadcode_breakage.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2" "--target=i686-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/coercion/leak_check_fndef_lub_deadcode_breakage" "-A" "unused" "-W" "unused_attributes" "-A" "internal_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/i686-unknown-linux-gnu/native/rust-test-helpers" "-Clinker=x86_64-linux-gnu-gcc"
stdout: none
--- stderr -------------------------------
error[E0512]: cannot transmute between types of different sizes, or dependently-sized types
##[error]  --> /checkout/tests/ui/coercion/leak_check_fndef_lub_deadcode_breakage.rs:25:14
   |
LL |     unsafe { std::mem::transmute::<_, ()>(lubbed) }
   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: source type: `fn()` (32 bits)
   = note: target type: `()` (0 bits)

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0512`.
------------------------------------------

---- [ui] tests/ui/coercion/leak_check_fndef_lub_deadcode_breakage.rs stdout end ----

failures:
    [ui] tests/ui/coercion/leak_check_fndef_lub_deadcode_breakage.rs

test result: FAILED. 19619 passed; 1 failed; 424 ignored; 0 measured; 0 filtered out; finished in 529.21s