prob and logprob macros for simple model queries

[mohdibntarek]

This PR is based on #965. In this PR, I implemented a similar syntax to the one proposed in #989. Here is an example:

using Turing

@model demo(y, ::Type{T} = Float64) where {T} = begin
	x = Vector{T}(undef, 10)
	x .~ Normal()
	y .~ Normal.(x, 1.0)
end
chain = sample(demo(rand(10)), HMC(0.64, 1), 1000)

# log likelihood
logprob"y = rand(10) | model = demo, x = rand(10)" 

# all log likelihoods, one for each sample in chain
logprob"y = rand(10) | chain = chain"

# log prior
logprob"x = rand(10) | model = demo" 

The posterior and joint probabilities are next.

Note the use of a string macro which is handled just like a normal macro except there is no syntax highlighting. The main difference under the hood is that this one doesn't go through the Julia parser to parse the input expression. The precedence of | in the Julia parser is quite high compared to how we want it here, so I chose not to go against the flow and just used a string macro.

[mohdibntarek]

Just some dispatch bugs left in Inference, then I will add tests.

[mohdibntarek]

Here are the implemented and remaining features using the above model as an example:

• Query the prior using prob"x = rand(3) | model = demo"
• Query the likelihood using prob"y = rand(3) | x = rand(3), model = demo"
• Query the likelihood given every sample in the chain using prob"y = rand(3) | chain = chain, model = demo"
• Query the joint probability using prob"x = rand(3), y = rand(3) | model = demo"

In a future PR, I will implement:

• Create a prior probability distribution using prob"x | model = demo"
• Create the likelihood probability distribution using prob"y | x = rand(3), model = demo"
• Create a joint probability distribution using prob"x, y | model = demo"
• Create the posterior predictive distribution of y using prob"y | chain = chain, model = demo"

[mohdibntarek]

Because this PR is already quite large, I will leave the syntax for creating a distribution from the model for another PR. For now, I will just finish the joint probability querying and update the docs, then this PR will be ready for review.

[yebai]

Because this PR is already quite large, I will leave the syntax for creating a distribution from the model for another PR. For now, I will just finish the joint probability querying and update the docs, then this PR will be ready for review.

Sounds good.

[mohdibntarek]

Just docs left and maybe a couple more tests.

[trappmartin]

This look really cool. I left a few minor remarks.

[mohdibntarek]

Thanks @trappmartin for your review. I will add the missing docstrings and comments while updating the docs, shortly.

[mohdibntarek]

DistributionsAD-Zygote bug

[mohdibntarek]

Docs, docs and more docs now.

[trappmartin]

See comments for tests.

[trappmartin]

This PR feels like a Christmas present. 🎁

[trappmartin]

I’m happy to approve once we have the issues which discuss the special cases.

[cpfiffer]

I've got no issues -- looks like @trappmartin covered pretty much everything. This is a spectacular PR -- I'm very pleased with the stuff we'll be able to do.

[mohdibntarek]

Some spacings are off, and the docs need an update. Then a rebase and we should be ready for a merge. Should be hopefully done by tomorrow. Would be cool to merge this on Christmas day!

[mohdibntarek]

If tests pass and there are no further comments, this is ready for merge.

[mohdibntarek]

The error is due to a Pkg bug. The only workarounds I know are to either limit our MCMCChains compat to pre-1.0 or to get a new release for CmdStan that supports MCMCChains 1.0.

[mohdibntarek]

StanJulia/CmdStan.jl#76

[mohdibntarek]

We can also just drop CmdStan tests for now, or merge anyways and hope for a new CmdStan release soon.

[cpfiffer]

We could also drop the MCMCChains compat to 0.4, not much changed between 0.4 and 1.0. Then we can keep the CmdStan tests in.

[mohdibntarek]

We can do that but that won't fix the issue. There is no version of CmdStan that supports MCMCChains 1.0 yet. My understanding is that Pkg is struggling to downgrade MCMCChains from 1.0 to the perfectly compatible 0.4 because of a bug. So we have to either stop Pkg from thinking MCMCChains 1.0 is valid to begin with (by limiting our MCMCChains compat in Turing to pre-1.0) or provide a version of CmdStan that supports MCMCChains 1.0 and hence Pkg doesn't need to downgrade anything. This is my best understanding of the dependency error we have.

[mohdibntarek]

We could also drop the MCMCChains compat to 0.4

Oh I think I misread your comment. If you mean to only support MCMCChains 0.4 then yes this should fix the error IIUC.

[mohdibntarek]

Pkg error resolved, tests are running.

[trappmartin]

Looks good so far

[mohdibntarek]

As always @goedman comes and saves the day, thanks Rob!

[trappmartin]

Looks like this can be merged.

[yebai]

Thanks, @mohamed82008!