Use of `deepcopy` doesn't trim well

[RomeoV]

This package uses deepcopy at 19 locations (rg "deepcopy" | wc -l).
However, deepcopy doesn't work well for trimming binaries.

For instance, in https://github.com/RomeoV/TrimSimpleNonlinearSolve.jl we differentiate though a linear solve
with ForwardDiff which triggers https://github.com/RomeoV/LinearSolve.jl/blob/rv%2Fremove-linsolve-forwarddiff-special-path/ext/LinearSolveForwardDiffExt.jl#L57 and results in errors while trimming.
We can resolve it by removing this extension, but probably we need some better way of handling deepcopy, perhaps a deepcopy_noalias?

[RomeoV]

Probably related to #646 and https://discourse.julialang.org/t/differentiationinterface-forwarddiff-unable-to-diff-ldiv-any-other-options-to-solve-linear-system-w-o-allocations-and-differentiate/131076.

[oscardssmith]

do any of these need more than copy?

[ChrisRackauckas]

They need copy(sol). We really just should define a package with a nice recursive definition of copy and apply it to all of our structs. We don't need the alias preservation stuff of deepcopy, we just need a simple correct copy that recurses into arrays of arrays.

[ChrisRackauckas]

recursivecopy from RecursiveArrayTools.jl is close, but should do a little bit more, handling all fields of a type.

[ChrisRackauckas]

The Enzyme overloads would be faster with this too, this would be helpful in some of the serialization stuff where we are trying to serialize simpler objects, etc. It's just generally what we want instead of deepcopy, so if such a package is made we can probably replace most deepcopy's in the whole org with this.

[RomeoV]

I tried to do something much simpler and just rewrite

  # primal_sol = deepcopy(sol)
  primal_sol = (;
      u = recursivecopy(sol.u),
      resid = recursivecopy(sol.resid),
      retcode = recursivecopy(sol.retcode),
      iters = recursivecopy(sol.iters),
      stats = recursivecopy(sol.stats),
  )

since the primal_sol copy is just used for some basic accesses later.
I tested it in https://github.com/RomeoV/TrimSimpleNonlinearSolve.jl/blob/master/main.jl.
The modification runs fine from the REPL, but trimming doesn't work. A binary is produced but crashes with

romeo@Romeo-P1 ~/D/j/TrimSimpleNonlinearSolve.jl (master)> ./main 1.0
fatal: error thrown and no exception handler available.
Core.MethodError(f=LinearSolveForwardDiffExt.var"#linearsolve_dual_solution"(), args=(Array{Core.Float64, 1}(dims=(2,), mem=Memory{Core.Float64}(2, 0x7f80f2965190)[-0.62487975619289615, -0.11217496281116118]), Array{Core.Array{Core.Float64, 1}, 1}(dims=(1,), mem=Memory{Core.Array{Core.Float64, 1}}(1, 0x7f80f296a020)[Array{Core.Float64, 1}(dims=(2,), mem=Memory{Core.Float64}(2, 0x7f80f29651f0)[0.17027192280605205, -0.063152288523283875])]), ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.NonlinearFunction{false, SciMLBase.FullSpecialize, TrimSimpleNonlinearSolve.var"#f", LinearAlgebra.UniformScaling{Core.Bool}, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing, SciMLBase.var"#DEFAULT_OBSERVED_NO_TIME", Core.Nothing, Core.Nothing, Core.Nothing, Core.Nothing}, Core.Float64}, Core.Float64, 1}), world=0x0000000000009761)

[ChrisRackauckas]

Oh that's an even simpler thing than the copy function I would've written... 😅 that is odd. I'd need help from @gbaraldi to decipher what to do then.

[jClugstor]

If I remember correctly I needed to use deepcopy because something that was aliased was changing during the dual linear solves, changing the results of the primal solution. But yeah actually I don't need the whole solution object, just a few of the relevant fields.

Looks like the trimming issue is coming from linearsolve_dual_solution. I wouldn't be suprised if the issue is with the way that the Dual types get constructed, which is ... probably pretty bad in the first place.

[ChrisRackauckas]

#724

[RomeoV]

why is this reopened?