Initial Epsilon in find_good_step_size function

[RajDandekar]

I am trying to integrate AdvancedHMC.jl in trying to find terms of a differential equation (in the spirit of Universal Ordinary Differential Equations (https://arxiv.org/abs/2001.04385)

I was curious to learn more about the initialization: ϵ′ = ϵ = T(0.1) in the find_good_step_size function. I noticed that using such a large value caused the derivative of the hamiltonian (dH/ dtheta) to blow up leading to a lot of numerical errors before finding the initial step size.

I found that when I initialize it to a smaller value (ϵ′ = ϵ = T(0.001)), I get about a 50 times performance improvement in the find_good_step_size function.

Any comments on this?

I can share my code if that would help.

Thanks!

[Vaibhavdixit02]

I find this peculiar also because trying it out with Neural ODEs (and DEs in the tutorial) the step sizes in the range of 0.25 gave much more well behaved sampling. It might be worthwhile to expose this since it varies between problems?

[xukai92]

Feel free to fire a PR to allow optional initial step size in find_good_step_size.

[Vaibhavdixit02]

I know @KumarRishabh was looking for a first issue, this should be easy enough if he wants to take a stab

[KumarRishabh]

Thanks @Vaibhavdixit02! Looks like a good issue to get started with. Will start working on this.