Can DifferentiationInterface.jl replace the AD backend extensions here?

[gdalle]

As you may know, @adrhill and I are putting the finishing touches to an AD interface package:

https://github.com/gdalle/DifferentiationInterface.jl

One of our goals is to make life easier for everyone who ever had to write one package extension per AD backend. It seems that this is the case for you @tpapp, and that LogDensityProblemsAD.jl falls into the spectrum of cases we cover (one array in, one scalar out).
Would you like to try it out as a potential tool for simplification? Happy to help if that's the case.

Related:

• Can DifferentiationInterface be useful for Turing? TuringLang/Turing.jl#2187

[tpapp]

Yes, I think it would be great to add API for that backend too, but I would be reluctant to replace existing ones since there may be user code out there using it.

[gdalle]

I'm not sure my question was clear, sorry.
DifferentiationInterface.jl is not really a backend, it's a door to a dozen backends. In that way, it contains much of the code that is in your package extensions: take a look at

• https://github.com/tpapp/LogDensityProblemsAD.jl/blob/master/ext/LogDensityProblemsADForwardDiffExt.jl (similar to https://github.com/gdalle/DifferentiationInterface.jl/blob/main/ext/DifferentiationInterfaceForwardDiffExt/allocating.jl, with config setup)
• https://github.com/tpapp/LogDensityProblemsAD.jl/blob/master/ext/LogDensityProblemsADReverseDiffExt.jl (similar to https://github.com/gdalle/DifferentiationInterface.jl/blob/main/ext/DifferentiationInterfaceReverseDiffExt/allocating.jl, with tape setup)

The general idea is that many package developers have had to write such glue code, and if we put it in a single place it will be easier to maintain and optimize

[gdalle]

This would also allow a clean switch from symbol-based to ADTypes.jl-based backend specification, although symbols can still be supported if necessary.

Related:

• how to query all supported interfaces #15

[devmotion]

This would also allow a clean switch from symbol-based to ADTypes.jl-based backend specification

LogDensityProblemsAD already supports ADTypes (this functionality is eg used by Turing).

[gdalle]

True, what I meant is that it supports ADTypes by falling back on symbols

https://github.com/tpapp/LogDensityProblemsAD.jl/blob/master/ext/LogDensityProblemsADADTypesExt.jl

If you wanna give a shot to DifferentiationInterface, it would probably become the other way around, symbols being translated into ADTypes

[tpapp]

@gdalle: Again, I am happy to expose unifying interfaces in this package. We can even use it as a fallback under the hood by translating ADTypes.

@devmotion: We probably should replace the examples in the README and the manual with ADTypes at some point.

[gdalle]

If you want I can open a demo PR changing one backend, so that you see what it would look like? E.g. ForwardDiff?

[tpapp]

Yes, PRs are always welcome.

[wsmoses]

For the reason specified in my comment #29 (review) this would present issues for Enzyme users until DI supports multiple arguments and activity states.