Roadmap for removal of `Selector` shenanigans

[torfjelde]

The problem

Currently, DynamicPPL.jl supports implementation of Gibbs sampling through usages of Selectors and gids, which are effectively ways for each instance of Sampler to define it's unique "Gibbs ID" and then subsequently which sampler that particular realization present in VarInfo came from.

All of this machinery is complicated and means that samplers outside of DynamicPPL are not compatible without an annoying amount of code needed.

In TuringLang/Turing.jl#2099 we're introducing an alternative approach to a Gibbs sampler which makes use of the condition functionality to define a sequence of conditioned Models, whose random variables are then restricted to only those that a particular sampler needs.

This has the following benefits:

1. Any sampler which wants to be compatible with Gibbs on a Model, simply needs to be able to sample from a Model (which, thanks to DynamicPPL.LogDensityFunction is quite easy in most cases).
2. There is no need for all the code relating to Selector and gids since all the variables present in a VarInfo is supposed to be sampled.
3. Moreover, once we remove this "some variables in VarInfo are not supposed to be sampled and some are"-convention, we can also remove some of the more unfortunate tilde-implementations, e.g. DynamicPPL.get_and_set_val, and instead fully adopt the convention that: if we're using a SamplingContext, things might get resampled, and if we're not using a SamplingContext, then things are not to be sampled. This will then also remove the need for del falgs, etc.

Currently the sampler introduced in TuringLang/Turing.jl#2099 does still depend on a small part of this Selector-business, but it does so only to stay comaptible with the current DynamicPPL. Once we remove this from DynamicPPL, we can remove the old Gibbs sampler in Turing.jl, and finally move completely away from this stuff.

Things we need to do

Here's some rough things we need to get done (this will be probably change):

1. Try out Turing.Experimental.Gibbs on a diverse set of problems to make sure it works as expected.
2. Determine if particle samplers will still be functional if we indeed remove this selector business completely.
3. Remove Selector and gids.
4. Stop using Sampler to as an indexing object, etc.

[torfjelde]

@devmotion @yebai Probably of interest

[yebai]

Good idea to get rid of Selector and Sampler as indexing objects. It is also probably fine to temporarily break particle Gibbs during the transition -- we can fix PG when the design of the new trace object is more stable.

[mhauru]

Closed by release of v0.35.