Vec assume

[xukai92]

On working of #255

I'm trying to vectorize the transformation as well (#276). I already done that for univariate and simplex.

TODOs

• vec mat
• vec mat transformation(s)

[xukai92]

Note about using array of array v.s. mat result from vec_assume_mv.jl:

N = 20
beta = [0.5, 0.5]
setchunksize(N*length(beta))
alg = HMC(1000, 0.2, 4)

# Test for vectorize UnivariateDistribution
@model vdemo() = begin
  phi = Vector{Vector{Real}}(N)
  phi ~ [Dirichlet(beta)]
end

t_vec = @elapsed res_vec = sample(vdemo(), alg)

@model vdemo() = begin
  phi = Matrix(2,N)
  phi ~ [Dirichlet(beta)]
end

t_vec_mat = @elapsed res_vec_mat = sample(vdemo(), alg)

@model vdemo() = begin
  phi = Vector{Vector{Real}}(N)
  for i = 1:N
    phi[i] ~ Dirichlet(beta)
  end
end

t_loop = @elapsed res = sample(vdemo(), alg)

println("Time for")
println("  Loop : $t_loop")
println("  Vec  : $t_vec")
println("  Vec2 : $t_vec_mat")

gives

Time for
  Loop : 9.823776041
  Vec  : 21.934951717
  Vec2 : 6.140146129

with N=20.

Mat seems much faster (should be also related to my implementation of converting mat back to nested array)

[xukai92]

Note: Distributions.jl doesn't support vectorisation version of logpdf of matrixvariate variables.

[xukai92]

Also there is a typo(bug) in their fake verctorization implementation. For this reason I will not include the test file in runtest.jl

[yebai]

@xukai92 Do you want me to review this PR? Seems tests are currently broken.

[xukai92]

Seems there is a bug of the resuming MCMC feature which may cause numerical error very easily. I guess it might be the case that when a chain is resumed, I call the find_good_eps again, which should be disabled. Let me fix this first.

[xukai92]

@yebai I fixed few bugs and improve the stability a bit. Could you have a review now plz?

[yebai]

Time for
Loop : 9.823776041
Vec : 21.934951717
Vec2 : 6.140146129
with N=20.
Mat seems much faster (should be also related to my implementation of converting mat back to nested array)

impressive speedup! I've started reviewing this PR and will add my comments soon.

[yebai]

perhaps we should add a dimensionality assertion here:

@assert size(var) == size(rs)

[xukai92]

Yeah the assertions makes sense to me!

[yebai]

dimensionality assertion?

[yebai]

dimensionality assertion?

[yebai]

dimensionality assertion?

[yebai]

the distributions package does not check d.p[x] > 0.0, so I override their definition here. It's probably better to submit a PR to Distributions package to fix this.

[xukai92]

Yeah I created an issue (#285) for this.

[xukai92]

@yebai Tests on Travis look fine now. But tests on Apprvor got the same problem before - freeze before running actual tests. I'm playing around the settings/scripts to see if I can resolve it.

[xukai92]

NOTE: I did a profiling on the vectorized version of LDA model and it shows >70% time is spent by gradient2().

[xukai92]

@yebai Can you give me permissions on appveyor so that I can terminate or restart tests? I have to wait for each one to be terminated by the sever to try to fix the issue, which is a bit slow-responsed.

[yebai]

Done - I've added you to appveyor permissions list.

[xukai92]

@yebai Appveyor actually freezes when executing using Turing. Any idea why this happens?

[yebai]

@yebai Appveyor actually freezes when executing using Turing. Any idea why this happens?

Turing imports other packages when loaded. It's possible some packages being imported have errors/conflicts, most likely due to pre-compilation. To see these warnings or errors, try temporarily disable @supress_err function in Turing.