Avoid unnecessary computation with `generated_quantities`

[tiemvanderdeure]

generated_quantities always re-computes the entire model - even when not the entire model is needed to actually generate the returned values. E.g.

@model function complicated_model()
    a ~ Normal(0, 1)
    sleep(0.1) # some expensive computation
    b ~ Normal(0, 1)
    return a
end

chn = sample(complicated_model(), NUTS(0.65), 100)
generated_quantities(complicated_model(), chn)

What is the easiest way around this? Submodels don't work because the variables have different names - so I suppose the only way is to make a new model that doesn't have the unnecessary expensive bits?

[penelopeysm]

There have been a few discussions around this (#589, #510, potentially more) and it seems to me that there should be a nicer way of doing it, but we haven't seemed to be able to settle on something that makes everyone happy.

Personally, I'd suggest that this is the easiest solution right now:

@model function f(inference=true)
    x ~ Normal()
    if inference
        println("inference")
    else
        println("not inference")
    end
    y ~ Normal(x)
end

chn = sample(f(), NUTS(), 1000)
returned(f(false), chn)

The slightly ugly thing about this is that if you have any submodels inside then you have to pass that parameter into the submodel as well.

(ps. generated_quantities was renamed to returned in v0.31.1. The functionality remains the same but the former might be removed at some point in time.)

[penelopeysm]

You could also use an approach where you have two models, one for inference and one for prediction, and one is nested in the other as a submodel (but you have to disable prefixing to get around the thing you mentioned about the variable names being different). See e.g. Tor's comment here: TuringLang/Turing.jl#2485 (comment) and the following comments.

(Nowadays, you'd disable the prefixing with x ~ to_submodel(inner(), false))

As you can see from that discussion, submodels without prefixing makes me feel slightly uneasy, but if it works for you then it works 😄

[yebai]

Related: #810

[tiemvanderdeure]

Thanks, those are helpful suggestions @penelopeysm!

Feel free to close if you think this is too similar to the other issue.

[penelopeysm]

I'll close, but thanks for opening it! It's helpful if people search for the same thing :)