autodiff in default algorithm

[ArnoStrouwen]

Is it possible to do a try/catch on the evaluation of the Jac to turn on autodiff?
https://github.com/SciML/DifferentialEquations.jl/blob/master/src/ode_default_alg.jl#L49

[ChrisRackauckas]

try/catch always has a ton of overhead so that's not good for the default method. It only works in the adjoints because the cost of adjoint differentiation is already so high when it makes sense, so the overhead is always negligible. For many default ODEs, you'd just hit a standard explicit RK method and in those cases the try/catch would be noticeable but never used.

Maybe the right thing to do would be Core.return_type and check if it's a Union{}. That would tell you if it's provable from the types to error? SciML/OrdinaryDiffEq.jl#2103 should merge this week, and with that we may want to reconsider how this could be done @oscardssmith

[oscardssmith]

I'm really not sure why you think try catch has so much overhead. I measure it as ~10ns in the case where no exception is thrown.

julia> function f(n)
           try
               return n[]+1
           catch
           end
       end
f (generic function with 1 method)

julia> @btime f(Any[100])
  34.174 ns (1 allocation: 64 bytes)
101

julia> function g(n)
           return n[]+1
       end
g (generic function with 1 method)

julia> @btime g(Any[100])
  24.751 ns (1 allocation: 64 bytes)
101

[ChrisRackauckas]

Oh okay, then maybe in the new form we can make the AD more dynamic? Though in future versions when using ADTypes with DI this might be an issue. @gdalle something to consider.

[oscardssmith]

Also worth pointing out that catching an error is pretty expensive.

julia> @btime f(Any["hi"])
  20.599 μs (3 allocations: 128 bytes)

[gdalle]

Oh okay, then maybe in the new form we can make the AD more dynamic? Though in future versions when using ADTypes with DI this might be an issue.

Probably related to JuliaDiff/DifferentiationInterface.jl#164