Including VarNames in sampler states, etc. #2511
Description
Many things in the Turing ecosystem assume that parameters have a form of Vector{<:Real}. We currently tend to do a lot of vi[:] and vi = unflatten(vi, params) to convert between this vector-of-real and a VarInfo.
However, this representation leads to information loss in many places. See e.g.
Sometimes we attempt to use a NamedTuple, but even that is not fully safe because NamedTuples are indexed by Symbols and not VarNames. This leads to issues like
TuringLang/DynamicPPL.jl#814
TuringLang/MCMCChains.jl#470
TuringLang/DynamicPPL.jl#702
To this end I've been thinking for a while now about changes to AbstractMCMC. It's also come up in discussions with @yebai and @mhauru. This is quite tough because it's one of the lowest-level packages in TuringLang and everything directly or indirectly depends on it.
I'm opening this issue here to collect some thoughts on it and will edit it slowly. Please bear with me, it's hard to formulate sensible thoughts on this topic, and even harder to write them down in a clear way.
Related: