From 9c744113309e45c4012dc6498020f9743cb4982b Mon Sep 17 00:00:00 2001 From: Markus Hauru Date: Tue, 4 Jun 2024 11:07:17 +0100 Subject: [PATCH 1/2] Add mode estimation page --- _quarto.yml | 2 + .../docs-12-using-turing-guide/Project.toml | 1 - .../docs-12-using-turing-guide/index.qmd | 114 +- .../docs-17-mode-estimation/Manifest.toml | 1559 +++++++++++++++++ .../docs-17-mode-estimation/Project.toml | 8 + tutorials/docs-17-mode-estimation/index.qmd | 120 ++ 6 files changed, 1693 insertions(+), 111 deletions(-) create mode 100755 tutorials/docs-17-mode-estimation/Manifest.toml create mode 100755 tutorials/docs-17-mode-estimation/Project.toml create mode 100755 tutorials/docs-17-mode-estimation/index.qmd diff --git a/_quarto.yml b/_quarto.yml index 9bfc84d73..39f061140 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -53,6 +53,8 @@ website: - text: "Quick Start" href: tutorials/docs-14-using-turing-quick-start/index.qmd - tutorials/docs-12-using-turing-guide/index.qmd + - text: "Mode Estimation" + href: tutorials/docs-17-mode-estimation/index.qmd - tutorials/docs-09-using-turing-advanced/index.qmd - tutorials/docs-10-using-turing-autodiff/index.qmd - tutorials/docs-13-using-turing-performance-tips/index.qmd diff --git a/tutorials/docs-12-using-turing-guide/Project.toml b/tutorials/docs-12-using-turing-guide/Project.toml index 8f0177a9b..ac63fc48c 100755 --- a/tutorials/docs-12-using-turing-guide/Project.toml +++ b/tutorials/docs-12-using-turing-guide/Project.toml @@ -1,6 +1,5 @@ [deps] Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b" -Optim = "429524aa-4258-5aef-a3af-852621145aeb" StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" StatsPlots = "f3b207a7-027a-5e70-b257-86293d7955fd" Turing = "fce5fe82-541a-59a6-adf8-730c64b5f9a0" diff --git a/tutorials/docs-12-using-turing-guide/index.qmd b/tutorials/docs-12-using-turing-guide/index.qmd index 43c363628..b66b4d29d 100755 --- a/tutorials/docs-12-using-turing-guide/index.qmd +++ b/tutorials/docs-12-using-turing-guide/index.qmd @@ -418,120 +418,14 @@ loglikelihood(model, chn) ### Maximum likelihood and maximum a posterior estimates -Turing provides support for two mode estimation techniques, [maximum likelihood estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) and [maximum a posterior](https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation) (MAP) estimation. Optimization is performed by the [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl) package. Mode estimation is currently a optional tool, and will not be available to you unless you have manually installed Optim and loaded the package with a `using` statement. To install Optim, run `import Pkg; Pkg.add("Optim")`. - -Mode estimation only works when all model parameters are continuous -- discrete parameters cannot be estimated with MLE/MAP as of yet. - -To understand how mode estimation works, let us first load Turing and Optim to enable mode estimation, and then declare a model: - -```{julia} -# Note that loading Optim explicitly is required for mode estimation to function, -# as Turing does not load the opimization suite unless Optim is loaded as well. -using Turing -using Optim - -@model function gdemo(x) - s² ~ InverseGamma(2, 3) - m ~ Normal(0, sqrt(s²)) - - for i in eachindex(x) - x[i] ~ Normal(m, sqrt(s²)) - end -end -``` - -Once the model is defined, we can construct a model instance as we normally would: - -```{julia} -# Create some data to pass to the model. -data = [1.5, 2.0] - -# Instantiate the gdemo model with our data. -model = gdemo(data) -``` - -Mode estimation is typically quick and easy at this point. Turing extends the function `Optim.optimize` and accepts the structs `MLE()` or `MAP()`, which inform Turing whether to provide an MLE or MAP estimate, respectively. By default, the [LBFGS optimizer](https://julianlsolvers.github.io/Optim.jl/stable/#algo/lbfgs/) is used, though this can be changed. Basic usage is: +Turing also has functions for estimating the maximum aposteriori and maximum likelihood parameters of a model. This can be done with ```{julia} -# Generate a MLE estimate. -mle_estimate = optimize(model, MLE()) - -# Generate a MAP estimate. -map_estimate = optimize(model, MAP()) +mle_estimate = maximum_likelihood(model) +map_estimate = maximum_a_posteriori(model) ``` -If you wish to change to a different optimizer, such as `NelderMead`, simply place your optimizer in the third argument slot: - -```{julia} -#| eval: false -# Use NelderMead -mle_estimate = optimize(model, MLE(), NelderMead()) - -# Use SimulatedAnnealing -mle_estimate = optimize(model, MLE(), SimulatedAnnealing()) - -# Use ParticleSwarm -mle_estimate = optimize(model, MLE(), ParticleSwarm()) - -# Use Newton -mle_estimate = optimize(model, MLE(), Newton()) - -# Use AcceleratedGradientDescent -mle_estimate = optimize(model, MLE(), AcceleratedGradientDescent()) -``` - -Some methods may have trouble calculating the mode because not enough iterations were allowed, or the target function moved upwards between function calls. Turing will warn you if Optim fails to converge by running `Optim.converge`. A typical solution to this might be to add more iterations, or allow the optimizer to increase between function iterations: - -```{julia} -#| eval: false -# Increase the iterations and allow function eval to increase between calls. -mle_estimate = optimize( - model, MLE(), Newton(), Optim.Options(; iterations=10_000, allow_f_increases=true) -) -``` - -More options for Optim are available [here](https://julianlsolvers.github.io/Optim.jl/stable/#user/config/). - -#### Analyzing your mode estimate - -Turing extends several methods from `StatsBase` that can be used to analyze your mode estimation results. Methods implemented include `vcov`, `informationmatrix`, `coeftable`, `params`, and `coef`, among others. - -For example, let's examine our ML estimate from above using `coeftable`: - -```{julia} -#| eval: false -# Import StatsBase to use it's statistical methods. -using StatsBase - -# Print out the coefficient table. -coeftable(mle_estimate) -``` - - - -```{.cell-bg} -───────────────────────────── - estimate stderror tstat -───────────────────────────── -s 0.0625 0.0625 1.0 -m 1.75 0.176777 9.8995 -───────────────────────────── -``` - -Standard errors are calculated from the Fisher information matrix (inverse Hessian of the log likelihood or log joint). t-statistics will be familiar to frequentist statisticians. Warning -- standard errors calculated in this way may not always be appropriate for MAP estimates, so please be cautious in interpreting them. - -#### Sampling with the MAP/MLE as initial states - -You can begin sampling your chain from an MLE/MAP estimate by extracting the vector of parameter values and providing it to the `sample` function with the keyword `initial_params`. For example, here is how to sample from the full posterior using the MAP estimate as the starting point: - -```{julia} -#| eval: false -# Generate an MAP estimate. -map_estimate = optimize(model, MAP()) - -# Sample with the MAP estimate as the starting point. -chain = sample(model, NUTS(), 1_000; initial_params=map_estimate.values.array) -``` +For more details see the [mode estimation page](../docs-17-mode-estimation/index.qmd). ## Beyond the Basics diff --git a/tutorials/docs-17-mode-estimation/Manifest.toml b/tutorials/docs-17-mode-estimation/Manifest.toml new file mode 100755 index 000000000..4ba574a15 --- /dev/null +++ b/tutorials/docs-17-mode-estimation/Manifest.toml @@ -0,0 +1,1559 @@ +# This file is machine-generated - editing it directly is not advised + +julia_version = "1.10.3" +manifest_format = "2.0" +project_hash = "83d22eae75c51307701765c5fbe616f2da4efa69" + +[[deps.ADTypes]] +git-tree-sha1 = "daf26bbdec60d9ca1c0003b70f389d821ddb4224" +uuid = "47edcb42-4c32-4615-8424-f2b9edc5f35b" +version = "1.2.1" + + [deps.ADTypes.extensions] + ADTypesChainRulesCoreExt = "ChainRulesCore" + ADTypesEnzymeCoreExt = "EnzymeCore" + + [deps.ADTypes.weakdeps] + ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" + EnzymeCore = "f151be2c-9106-41f4-ab19-57ee4f262869" + +[[deps.AbstractFFTs]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "d92ad398961a3ed262d8bf04a1a2b8340f915fef" +uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c" +version = "1.5.0" +weakdeps = ["ChainRulesCore", "Test"] + + [deps.AbstractFFTs.extensions] + AbstractFFTsChainRulesCoreExt = "ChainRulesCore" + AbstractFFTsTestExt = "Test" + +[[deps.AbstractMCMC]] +deps = ["BangBang", "ConsoleProgressMonitor", "Distributed", "FillArrays", "LogDensityProblems", "Logging", "LoggingExtras", "ProgressLogging", "Random", "StatsBase", "TerminalLoggers", "Transducers"] +git-tree-sha1 = "b0489adc45a7c8cf0d8e2ddf764f89c1c3decebd" +uuid = "80f14c24-f653-4e6a-9b94-39d6b0f70001" +version = "5.2.0" + +[[deps.AbstractPPL]] +deps = ["AbstractMCMC", "Accessors", "DensityInterface", "Random"] +git-tree-sha1 = "6380a9a03a4207bac53ac310dd3a283bb4df54ef" +uuid = "7a57a42e-76ec-4ea3-a279-07e840d6d9cf" +version = "0.8.4" + +[[deps.AbstractTrees]] +git-tree-sha1 = "2d9c9a55f9c93e8887ad391fbae72f8ef55e1177" +uuid = "1520ce14-60c1-5f80-bbc7-55ef81b5835c" +version = "0.4.5" + +[[deps.Accessors]] +deps = ["CompositionsBase", "ConstructionBase", "Dates", "InverseFunctions", "LinearAlgebra", "MacroTools", "Markdown", "Test"] +git-tree-sha1 = "c0d491ef0b135fd7d63cbc6404286bc633329425" +uuid = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697" +version = "0.1.36" + + [deps.Accessors.extensions] + AccessorsAxisKeysExt = "AxisKeys" + AccessorsIntervalSetsExt = "IntervalSets" + AccessorsStaticArraysExt = "StaticArrays" + AccessorsStructArraysExt = "StructArrays" + AccessorsUnitfulExt = "Unitful" + + [deps.Accessors.weakdeps] + AxisKeys = "94b1ba4f-4ee9-5380-92f1-94cde586c3c5" + IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953" + Requires = "ae029012-a4dd-5104-9daa-d747884805df" + StaticArrays = "90137ffa-7385-5640-81b9-e52037218182" + StructArrays = "09ab397b-f2b6-538f-b94a-2f83cf4a842a" + Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d" + +[[deps.Adapt]] +deps = ["LinearAlgebra", "Requires"] +git-tree-sha1 = "6a55b747d1812e699320963ffde36f1ebdda4099" +uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e" +version = "4.0.4" +weakdeps = ["StaticArrays"] + + [deps.Adapt.extensions] + AdaptStaticArraysExt = "StaticArrays" + +[[deps.AdvancedHMC]] +deps = ["AbstractMCMC", "ArgCheck", "DocStringExtensions", "InplaceOps", "LinearAlgebra", "LogDensityProblems", "LogDensityProblemsAD", "ProgressMeter", "Random", "Requires", "Setfield", "SimpleUnPack", "Statistics", "StatsBase", "StatsFuns"] +git-tree-sha1 = "dfa0e3508fc3df81d28624b328f3b937c1df8bc2" +uuid = "0bf59076-c3b1-5ca4-86bd-e02cd72cde3d" +version = "0.6.1" + + [deps.AdvancedHMC.extensions] + AdvancedHMCCUDAExt = "CUDA" + AdvancedHMCMCMCChainsExt = "MCMCChains" + AdvancedHMCOrdinaryDiffEqExt = "OrdinaryDiffEq" + + [deps.AdvancedHMC.weakdeps] + CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba" + MCMCChains = "c7f686f2-ff18-58e9-bc7b-31028e88f75d" + OrdinaryDiffEq = "1dea7af3-3e70-54e6-95c3-0bf5283fa5ed" + +[[deps.AdvancedMH]] +deps = ["AbstractMCMC", "Distributions", "FillArrays", "LinearAlgebra", "LogDensityProblems", "Random", "Requires"] +git-tree-sha1 = "16589dbdd36c782ff01700908e962b303474f641" +uuid = "5b7e9947-ddc0-4b3f-9b55-0d8042f74170" +version = "0.8.1" +weakdeps = ["DiffResults", "ForwardDiff", "MCMCChains", "StructArrays"] + + [deps.AdvancedMH.extensions] + AdvancedMHForwardDiffExt = ["DiffResults", "ForwardDiff"] + AdvancedMHMCMCChainsExt = "MCMCChains" + AdvancedMHStructArraysExt = "StructArrays" + +[[deps.AdvancedPS]] +deps = ["AbstractMCMC", "Distributions", "Random", "Random123", "Requires", "SSMProblems", "StatsFuns"] +git-tree-sha1 = "5dcd3de7e7346f48739256e71a86d0f96690b8c8" +uuid = "576499cb-2369-40b2-a588-c64705576edc" +version = "0.6.0" +weakdeps = ["Libtask"] + + [deps.AdvancedPS.extensions] + AdvancedPSLibtaskExt = "Libtask" + +[[deps.AdvancedVI]] +deps = ["ADTypes", "Bijectors", "DiffResults", "Distributions", "DistributionsAD", "DocStringExtensions", "ForwardDiff", "LinearAlgebra", "ProgressMeter", "Random", "Requires", "StatsBase", "StatsFuns", "Tracker"] +git-tree-sha1 = "187f67ab998f25208651262fee9539d845016b26" +uuid = "b5ca4192-6429-45e5-a2d9-87aec30a685c" +version = "0.2.5" + + [deps.AdvancedVI.extensions] + AdvancedVIEnzymeExt = ["Enzyme"] + AdvancedVIFluxExt = ["Flux"] + AdvancedVIReverseDiffExt = ["ReverseDiff"] + AdvancedVIZygoteExt = ["Zygote"] + + [deps.AdvancedVI.weakdeps] + Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9" + Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f" + +[[deps.AliasTables]] +deps = ["PtrArrays", "Random"] +git-tree-sha1 = "9876e1e164b144ca45e9e3198d0b689cadfed9ff" +uuid = "66dad0bd-aa9a-41b7-9441-69ab47430ed8" +version = "1.1.3" + +[[deps.ArgCheck]] +git-tree-sha1 = "a3a402a35a2f7e0b87828ccabbd5ebfbebe356b4" +uuid = "dce04be8-c92d-5529-be00-80e4d2c0e197" +version = "2.3.0" + +[[deps.ArgTools]] +uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f" +version = "1.1.1" + +[[deps.ArrayInterface]] +deps = ["Adapt", "LinearAlgebra", "SparseArrays", "SuiteSparse"] +git-tree-sha1 = "133a240faec6e074e07c31ee75619c90544179cf" +uuid = "4fba245c-0d91-5ea0-9b3e-6abc04ee57a9" +version = "7.10.0" + + [deps.ArrayInterface.extensions] + ArrayInterfaceBandedMatricesExt = "BandedMatrices" + ArrayInterfaceBlockBandedMatricesExt = "BlockBandedMatrices" + ArrayInterfaceCUDAExt = "CUDA" + ArrayInterfaceCUDSSExt = "CUDSS" + ArrayInterfaceChainRulesExt = "ChainRules" + ArrayInterfaceGPUArraysCoreExt = "GPUArraysCore" + ArrayInterfaceReverseDiffExt = "ReverseDiff" + ArrayInterfaceStaticArraysCoreExt = "StaticArraysCore" + ArrayInterfaceTrackerExt = "Tracker" + + [deps.ArrayInterface.weakdeps] + BandedMatrices = "aae01518-5342-5314-be14-df237901396f" + BlockBandedMatrices = "ffab5731-97b5-5995-9138-79e8c1846df0" + CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba" + CUDSS = "45b445bb-4962-46a0-9369-b4df9d0f772e" + ChainRules = "082447d4-558c-5d27-93f4-14fc19e9eca2" + GPUArraysCore = "46192b85-c4d5-4398-a991-12ede77f4527" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + StaticArraysCore = "1e83bf80-4336-4d27-bf5d-d5a4f845583c" + Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" + +[[deps.Artifacts]] +uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" + +[[deps.Atomix]] +deps = ["UnsafeAtomics"] +git-tree-sha1 = "c06a868224ecba914baa6942988e2f2aade419be" +uuid = "a9b6321e-bd34-4604-b9c9-b65b8de01458" +version = "0.1.0" + +[[deps.AxisAlgorithms]] +deps = ["LinearAlgebra", "Random", "SparseArrays", "WoodburyMatrices"] +git-tree-sha1 = "01b8ccb13d68535d73d2b0c23e39bd23155fb712" +uuid = "13072b0f-2c55-5437-9ae7-d433b7a33950" +version = "1.1.0" + +[[deps.AxisArrays]] +deps = ["Dates", "IntervalSets", "IterTools", "RangeArrays"] +git-tree-sha1 = "16351be62963a67ac4083f748fdb3cca58bfd52f" +uuid = "39de3d68-74b9-583c-8d2d-e117c070f3a9" +version = "0.4.7" + +[[deps.BangBang]] +deps = ["Accessors", "Compat", "ConstructionBase", "InitialValues", "LinearAlgebra", "Requires"] +git-tree-sha1 = "08e5fc6620a8d83534bf6149795054f1b1e8370a" +uuid = "198e06fe-97b7-11e9-32a5-e1d131e6ad66" +version = "0.4.2" + + [deps.BangBang.extensions] + BangBangChainRulesCoreExt = "ChainRulesCore" + BangBangDataFramesExt = "DataFrames" + BangBangStaticArraysExt = "StaticArrays" + BangBangStructArraysExt = "StructArrays" + BangBangTablesExt = "Tables" + BangBangTypedTablesExt = "TypedTables" + + [deps.BangBang.weakdeps] + ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" + DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" + StaticArrays = "90137ffa-7385-5640-81b9-e52037218182" + StructArrays = "09ab397b-f2b6-538f-b94a-2f83cf4a842a" + Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" + TypedTables = "9d95f2ec-7b3d-5a63-8d20-e2491e220bb9" + +[[deps.Base64]] +uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" + +[[deps.Baselet]] +git-tree-sha1 = "aebf55e6d7795e02ca500a689d326ac979aaf89e" +uuid = "9718e550-a3fa-408a-8086-8db961cd8217" +version = "0.1.1" + +[[deps.Bijectors]] +deps = ["ArgCheck", "ChainRules", "ChainRulesCore", "ChangesOfVariables", "Compat", "Distributions", "Functors", "InverseFunctions", "IrrationalConstants", "LinearAlgebra", "LogExpFunctions", "MappedArrays", "Random", "Reexport", "Requires", "Roots", "SparseArrays", "Statistics"] +git-tree-sha1 = "49491db48b1c70eefa5115e626100dbd6c0ff4c0" +uuid = "76274a88-744f-5084-9051-94815aaf08c4" +version = "0.13.12" + + [deps.Bijectors.extensions] + BijectorsDistributionsADExt = "DistributionsAD" + BijectorsForwardDiffExt = "ForwardDiff" + BijectorsLazyArraysExt = "LazyArrays" + BijectorsReverseDiffExt = "ReverseDiff" + BijectorsTrackerExt = "Tracker" + BijectorsZygoteExt = "Zygote" + + [deps.Bijectors.weakdeps] + DistributionsAD = "ced4e74d-a319-5a8a-b0ac-84af2272839c" + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + LazyArrays = "5078a376-72f3-5289-bfd5-ec5146d43c02" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" + Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f" + +[[deps.CEnum]] +git-tree-sha1 = "389ad5c84de1ae7cf0e28e381131c98ea87d54fc" +uuid = "fa961155-64e5-5f13-b03f-caf6b980ea82" +version = "0.5.0" + +[[deps.Calculus]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad" +uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9" +version = "0.5.1" + +[[deps.ChainRules]] +deps = ["Adapt", "ChainRulesCore", "Compat", "Distributed", "GPUArraysCore", "IrrationalConstants", "LinearAlgebra", "Random", "RealDot", "SparseArrays", "SparseInverseSubset", "Statistics", "StructArrays", "SuiteSparse"] +git-tree-sha1 = "291821c1251486504f6bae435227907d734e94d2" +uuid = "082447d4-558c-5d27-93f4-14fc19e9eca2" +version = "1.66.0" + +[[deps.ChainRulesCore]] +deps = ["Compat", "LinearAlgebra"] +git-tree-sha1 = "575cd02e080939a33b6df6c5853d14924c08e35b" +uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" +version = "1.23.0" +weakdeps = ["SparseArrays"] + + [deps.ChainRulesCore.extensions] + ChainRulesCoreSparseArraysExt = "SparseArrays" + +[[deps.ChangesOfVariables]] +deps = ["LinearAlgebra", "Test"] +git-tree-sha1 = "2fba81a302a7be671aefe194f0525ef231104e7f" +uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0" +version = "0.1.8" +weakdeps = ["InverseFunctions"] + + [deps.ChangesOfVariables.extensions] + ChangesOfVariablesInverseFunctionsExt = "InverseFunctions" + +[[deps.Combinatorics]] +git-tree-sha1 = "08c8b6831dc00bfea825826be0bc8336fc369860" +uuid = "861a8166-3701-5b0c-9a16-15d98fcdc6aa" +version = "1.0.2" + +[[deps.CommonSolve]] +git-tree-sha1 = "0eee5eb66b1cf62cd6ad1b460238e60e4b09400c" +uuid = "38540f10-b2f7-11e9-35d8-d573e4eb0ff2" +version = "0.2.4" + +[[deps.CommonSubexpressions]] +deps = ["MacroTools", "Test"] +git-tree-sha1 = "7b8a93dba8af7e3b42fecabf646260105ac373f7" +uuid = "bbf7d656-a473-5ed7-a52c-81e309532950" +version = "0.3.0" + +[[deps.Compat]] +deps = ["TOML", "UUIDs"] +git-tree-sha1 = "b1c55339b7c6c350ee89f2c1604299660525b248" +uuid = "34da2185-b29b-5c13-b0c7-acf172513d20" +version = "4.15.0" +weakdeps = ["Dates", "LinearAlgebra"] + + [deps.Compat.extensions] + CompatLinearAlgebraExt = "LinearAlgebra" + +[[deps.CompilerSupportLibraries_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae" +version = "1.1.1+0" + +[[deps.CompositionsBase]] +git-tree-sha1 = "802bb88cd69dfd1509f6670416bd4434015693ad" +uuid = "a33af91c-f02d-484b-be07-31d278c5ca2b" +version = "0.1.2" +weakdeps = ["InverseFunctions"] + + [deps.CompositionsBase.extensions] + CompositionsBaseInverseFunctionsExt = "InverseFunctions" + +[[deps.ConsoleProgressMonitor]] +deps = ["Logging", "ProgressMeter"] +git-tree-sha1 = "3ab7b2136722890b9af903859afcf457fa3059e8" +uuid = "88cd18e8-d9cc-4ea6-8889-5259c0d15c8b" +version = "0.1.2" + +[[deps.ConstructionBase]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "260fd2400ed2dab602a7c15cf10c1933c59930a2" +uuid = "187b0558-2788-49d3-abe0-74a17ed4e7c9" +version = "1.5.5" +weakdeps = ["IntervalSets", "StaticArrays"] + + [deps.ConstructionBase.extensions] + ConstructionBaseIntervalSetsExt = "IntervalSets" + ConstructionBaseStaticArraysExt = "StaticArrays" + +[[deps.Crayons]] +git-tree-sha1 = "249fe38abf76d48563e2f4556bebd215aa317e15" +uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f" +version = "4.1.1" + +[[deps.DataAPI]] +git-tree-sha1 = "abe83f3a2f1b857aac70ef8b269080af17764bbe" +uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" +version = "1.16.0" + +[[deps.DataStructures]] +deps = ["Compat", "InteractiveUtils", "OrderedCollections"] +git-tree-sha1 = "1d0a14036acb104d9e89698bd408f63ab58cdc82" +uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8" +version = "0.18.20" + +[[deps.DataValueInterfaces]] +git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6" +uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464" +version = "1.0.0" + +[[deps.Dates]] +deps = ["Printf"] +uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" + +[[deps.DefineSingletons]] +git-tree-sha1 = "0fba8b706d0178b4dc7fd44a96a92382c9065c2c" +uuid = "244e2a9f-e319-4986-a169-4d1fe445cd52" +version = "0.1.2" + +[[deps.DelimitedFiles]] +deps = ["Mmap"] +git-tree-sha1 = "9e2f36d3c96a820c678f2f1f1782582fcf685bae" +uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab" +version = "1.9.1" + +[[deps.DensityInterface]] +deps = ["InverseFunctions", "Test"] +git-tree-sha1 = "80c3e8639e3353e5d2912fb3a1916b8455e2494b" +uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d" +version = "0.4.0" + +[[deps.DiffResults]] +deps = ["StaticArraysCore"] +git-tree-sha1 = "782dd5f4561f5d267313f23853baaaa4c52ea621" +uuid = "163ba53b-c6d8-5494-b064-1a9d43ac40c5" +version = "1.1.0" + +[[deps.DiffRules]] +deps = ["IrrationalConstants", "LogExpFunctions", "NaNMath", "Random", "SpecialFunctions"] +git-tree-sha1 = "23163d55f885173722d1e4cf0f6110cdbaf7e272" +uuid = "b552c78f-8df3-52c6-915a-8e097449b14b" +version = "1.15.1" + +[[deps.Distributed]] +deps = ["Random", "Serialization", "Sockets"] +uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b" + +[[deps.Distributions]] +deps = ["AliasTables", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns"] +git-tree-sha1 = "22c595ca4146c07b16bcf9c8bea86f731f7109d2" +uuid = "31c24e10-a181-5473-b8eb-7969acd0382f" +version = "0.25.108" +weakdeps = ["ChainRulesCore", "DensityInterface", "Test"] + + [deps.Distributions.extensions] + DistributionsChainRulesCoreExt = "ChainRulesCore" + DistributionsDensityInterfaceExt = "DensityInterface" + DistributionsTestExt = "Test" + +[[deps.DistributionsAD]] +deps = ["Adapt", "ChainRules", "ChainRulesCore", "Compat", "Distributions", "FillArrays", "LinearAlgebra", "PDMats", "Random", "Requires", "SpecialFunctions", "StaticArrays", "StatsFuns", "ZygoteRules"] +git-tree-sha1 = "f4dd7727b07b4b7fff5ff4149118ee06e83dfab7" +uuid = "ced4e74d-a319-5a8a-b0ac-84af2272839c" +version = "0.6.55" + + [deps.DistributionsAD.extensions] + DistributionsADForwardDiffExt = "ForwardDiff" + DistributionsADLazyArraysExt = "LazyArrays" + DistributionsADReverseDiffExt = "ReverseDiff" + DistributionsADTrackerExt = "Tracker" + + [deps.DistributionsAD.weakdeps] + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + LazyArrays = "5078a376-72f3-5289-bfd5-ec5146d43c02" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" + +[[deps.DocStringExtensions]] +deps = ["LibGit2"] +git-tree-sha1 = "2fb1e02f2b635d0845df5d7c167fec4dd739b00d" +uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae" +version = "0.9.3" + +[[deps.Downloads]] +deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"] +uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6" +version = "1.6.0" + +[[deps.DualNumbers]] +deps = ["Calculus", "NaNMath", "SpecialFunctions"] +git-tree-sha1 = "5837a837389fccf076445fce071c8ddaea35a566" +uuid = "fa6b7ba4-c1ee-5f82-b5fc-ecf0adba8f74" +version = "0.6.8" + +[[deps.DynamicPPL]] +deps = ["ADTypes", "AbstractMCMC", "AbstractPPL", "Accessors", "BangBang", "Bijectors", "Compat", "ConstructionBase", "Distributions", "DocStringExtensions", "LinearAlgebra", "LogDensityProblems", "LogDensityProblemsAD", "MacroTools", "OrderedCollections", "Random", "Requires", "Test"] +git-tree-sha1 = "882510df0da64bae763a65240b07fb4b1c462317" +uuid = "366bfd00-2699-11ea-058f-f148b4cae6d8" +version = "0.27.1" + + [deps.DynamicPPL.extensions] + DynamicPPLChainRulesCoreExt = ["ChainRulesCore"] + DynamicPPLEnzymeCoreExt = ["EnzymeCore"] + DynamicPPLForwardDiffExt = ["ForwardDiff"] + DynamicPPLMCMCChainsExt = ["MCMCChains"] + DynamicPPLReverseDiffExt = ["ReverseDiff"] + DynamicPPLZygoteRulesExt = ["ZygoteRules"] + + [deps.DynamicPPL.weakdeps] + ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" + EnzymeCore = "f151be2c-9106-41f4-ab19-57ee4f262869" + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + MCMCChains = "c7f686f2-ff18-58e9-bc7b-31028e88f75d" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + ZygoteRules = "700de1a5-db45-46bc-99cf-38207098b444" + +[[deps.EllipticalSliceSampling]] +deps = ["AbstractMCMC", "ArrayInterface", "Distributions", "Random", "Statistics"] +git-tree-sha1 = "e611b7fdfbfb5b18d5e98776c30daede41b44542" +uuid = "cad2338a-1db2-11e9-3401-43bc07c9ede2" +version = "2.0.0" + +[[deps.EnumX]] +git-tree-sha1 = "bdb1942cd4c45e3c678fd11569d5cccd80976237" +uuid = "4e289a0a-7415-4d19-859d-a7e5c4648b56" +version = "1.0.4" + +[[deps.ExprTools]] +git-tree-sha1 = "27415f162e6028e81c72b82ef756bf321213b6ec" +uuid = "e2ba6199-217a-4e67-a87a-7c52f15ade04" +version = "0.1.10" + +[[deps.FFTW]] +deps = ["AbstractFFTs", "FFTW_jll", "LinearAlgebra", "MKL_jll", "Preferences", "Reexport"] +git-tree-sha1 = "4820348781ae578893311153d69049a93d05f39d" +uuid = "7a1cc6ca-52ef-59f5-83cd-3a7055c09341" +version = "1.8.0" + +[[deps.FFTW_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "c6033cc3892d0ef5bb9cd29b7f2f0331ea5184ea" +uuid = "f5851436-0d7a-5f13-b9de-f02708fd171a" +version = "3.3.10+0" + +[[deps.FileWatching]] +uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee" + +[[deps.FillArrays]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "0653c0a2396a6da5bc4766c43041ef5fd3efbe57" +uuid = "1a297f60-69ca-5386-bcde-b61e274b549b" +version = "1.11.0" +weakdeps = ["PDMats", "SparseArrays", "Statistics"] + + [deps.FillArrays.extensions] + FillArraysPDMatsExt = "PDMats" + FillArraysSparseArraysExt = "SparseArrays" + FillArraysStatisticsExt = "Statistics" + +[[deps.FiniteDiff]] +deps = ["ArrayInterface", "LinearAlgebra", "Requires", "Setfield", "SparseArrays"] +git-tree-sha1 = "2de436b72c3422940cbe1367611d137008af7ec3" +uuid = "6a86dc24-6348-571c-b903-95158fe2bd41" +version = "2.23.1" + + [deps.FiniteDiff.extensions] + FiniteDiffBandedMatricesExt = "BandedMatrices" + FiniteDiffBlockBandedMatricesExt = "BlockBandedMatrices" + FiniteDiffStaticArraysExt = "StaticArrays" + + [deps.FiniteDiff.weakdeps] + BandedMatrices = "aae01518-5342-5314-be14-df237901396f" + BlockBandedMatrices = "ffab5731-97b5-5995-9138-79e8c1846df0" + StaticArrays = "90137ffa-7385-5640-81b9-e52037218182" + +[[deps.ForwardDiff]] +deps = ["CommonSubexpressions", "DiffResults", "DiffRules", "LinearAlgebra", "LogExpFunctions", "NaNMath", "Preferences", "Printf", "Random", "SpecialFunctions"] +git-tree-sha1 = "cf0fe81336da9fb90944683b8c41984b08793dad" +uuid = "f6369f11-7733-5829-9624-2563aa707210" +version = "0.10.36" +weakdeps = ["StaticArrays"] + + [deps.ForwardDiff.extensions] + ForwardDiffStaticArraysExt = "StaticArrays" + +[[deps.FunctionWrappers]] +git-tree-sha1 = "d62485945ce5ae9c0c48f124a84998d755bae00e" +uuid = "069b7b12-0de2-55c6-9aab-29f3d0a68a2e" +version = "1.1.3" + +[[deps.FunctionWrappersWrappers]] +deps = ["FunctionWrappers"] +git-tree-sha1 = "b104d487b34566608f8b4e1c39fb0b10aa279ff8" +uuid = "77dc65aa-8811-40c2-897b-53d922fa7daf" +version = "0.1.3" + +[[deps.Functors]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "d3e63d9fa13f8eaa2f06f64949e2afc593ff52c2" +uuid = "d9f16b24-f501-4c13-a1f2-28368ffc5196" +version = "0.4.10" + +[[deps.Future]] +deps = ["Random"] +uuid = "9fa8497b-333b-5362-9e8d-4d0656e87820" + +[[deps.GPUArraysCore]] +deps = ["Adapt"] +git-tree-sha1 = "ec632f177c0d990e64d955ccc1b8c04c485a0950" +uuid = "46192b85-c4d5-4398-a991-12ede77f4527" +version = "0.1.6" + +[[deps.HypergeometricFunctions]] +deps = ["DualNumbers", "LinearAlgebra", "OpenLibm_jll", "SpecialFunctions"] +git-tree-sha1 = "f218fe3736ddf977e0e772bc9a586b2383da2685" +uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a" +version = "0.3.23" + +[[deps.InitialValues]] +git-tree-sha1 = "4da0f88e9a39111c2fa3add390ab15f3a44f3ca3" +uuid = "22cec73e-a1b8-11e9-2c92-598750a2cf9c" +version = "0.3.1" + +[[deps.InplaceOps]] +deps = ["LinearAlgebra", "Test"] +git-tree-sha1 = "50b41d59e7164ab6fda65e71049fee9d890731ff" +uuid = "505f98c9-085e-5b2c-8e89-488be7bf1f34" +version = "0.3.0" + +[[deps.IntelOpenMP_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "be50fe8df3acbffa0274a744f1a99d29c45a57f4" +uuid = "1d5cc7b8-4909-519e-a0f8-d0f5ad9712d0" +version = "2024.1.0+0" + +[[deps.InteractiveUtils]] +deps = ["Markdown"] +uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" + +[[deps.Interpolations]] +deps = ["Adapt", "AxisAlgorithms", "ChainRulesCore", "LinearAlgebra", "OffsetArrays", "Random", "Ratios", "Requires", "SharedArrays", "SparseArrays", "StaticArrays", "WoodburyMatrices"] +git-tree-sha1 = "88a101217d7cb38a7b481ccd50d21876e1d1b0e0" +uuid = "a98d9a8b-a2ab-59e6-89dd-64a1c18fca59" +version = "0.15.1" + + [deps.Interpolations.extensions] + InterpolationsUnitfulExt = "Unitful" + + [deps.Interpolations.weakdeps] + Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d" + +[[deps.IntervalSets]] +git-tree-sha1 = "dba9ddf07f77f60450fe5d2e2beb9854d9a49bd0" +uuid = "8197267c-284f-5f27-9208-e0e47529a953" +version = "0.7.10" +weakdeps = ["Random", "RecipesBase", "Statistics"] + + [deps.IntervalSets.extensions] + IntervalSetsRandomExt = "Random" + IntervalSetsRecipesBaseExt = "RecipesBase" + IntervalSetsStatisticsExt = "Statistics" + +[[deps.InverseFunctions]] +deps = ["Test"] +git-tree-sha1 = "e7cbed5032c4c397a6ac23d1493f3289e01231c4" +uuid = "3587e190-3f89-42d0-90ee-14403ec27112" +version = "0.1.14" +weakdeps = ["Dates"] + + [deps.InverseFunctions.extensions] + DatesExt = "Dates" + +[[deps.InvertedIndices]] +git-tree-sha1 = "0dc7b50b8d436461be01300fd8cd45aa0274b038" +uuid = "41ab1584-1d38-5bbf-9106-f11c6c58b48f" +version = "1.3.0" + +[[deps.IrrationalConstants]] +git-tree-sha1 = "630b497eafcc20001bba38a4651b327dcfc491d2" +uuid = "92d709cd-6900-40b7-9082-c6be49f344b6" +version = "0.2.2" + +[[deps.IterTools]] +git-tree-sha1 = "42d5f897009e7ff2cf88db414a389e5ed1bdd023" +uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e" +version = "1.10.0" + +[[deps.IteratorInterfaceExtensions]] +git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856" +uuid = "82899510-4779-5014-852e-03e436cf321d" +version = "1.0.0" + +[[deps.JLLWrappers]] +deps = ["Artifacts", "Preferences"] +git-tree-sha1 = "7e5d6779a1e09a36db2a7b6cff50942a0a7d0fca" +uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210" +version = "1.5.0" + +[[deps.KernelAbstractions]] +deps = ["Adapt", "Atomix", "InteractiveUtils", "LinearAlgebra", "MacroTools", "PrecompileTools", "Requires", "SparseArrays", "StaticArrays", "UUIDs", "UnsafeAtomics", "UnsafeAtomicsLLVM"] +git-tree-sha1 = "db02395e4c374030c53dc28f3c1d33dec35f7272" +uuid = "63c18a36-062a-441e-b654-da1e3ab1ce7c" +version = "0.9.19" + + [deps.KernelAbstractions.extensions] + EnzymeExt = "EnzymeCore" + + [deps.KernelAbstractions.weakdeps] + EnzymeCore = "f151be2c-9106-41f4-ab19-57ee4f262869" + +[[deps.KernelDensity]] +deps = ["Distributions", "DocStringExtensions", "FFTW", "Interpolations", "StatsBase"] +git-tree-sha1 = "7d703202e65efa1369de1279c162b915e245eed1" +uuid = "5ab0869b-81aa-558d-bb23-cbf5423bbe9b" +version = "0.6.9" + +[[deps.LBFGSB]] +deps = ["L_BFGS_B_jll"] +git-tree-sha1 = "e2e6f53ee20605d0ea2be473480b7480bd5091b5" +uuid = "5be7bae1-8223-5378-bac3-9e7378a2f6e6" +version = "0.4.1" + +[[deps.LLVM]] +deps = ["CEnum", "LLVMExtra_jll", "Libdl", "Preferences", "Printf", "Requires", "Unicode"] +git-tree-sha1 = "839c82932db86740ae729779e610f07a1640be9a" +uuid = "929cbde3-209d-540e-8aea-75f648917ca0" +version = "6.6.3" + + [deps.LLVM.extensions] + BFloat16sExt = "BFloat16s" + + [deps.LLVM.weakdeps] + BFloat16s = "ab4f0b2a-ad5b-11e8-123f-65d77653426b" + +[[deps.LLVMExtra_jll]] +deps = ["Artifacts", "JLLWrappers", "LazyArtifacts", "Libdl", "TOML"] +git-tree-sha1 = "88b916503aac4fb7f701bb625cd84ca5dd1677bc" +uuid = "dad2f222-ce93-54a1-a47d-0025e8a3acab" +version = "0.0.29+0" + +[[deps.LRUCache]] +git-tree-sha1 = "b3cc6698599b10e652832c2f23db3cab99d51b59" +uuid = "8ac3fa9e-de4c-5943-b1dc-09c6b5f20637" +version = "1.6.1" +weakdeps = ["Serialization"] + + [deps.LRUCache.extensions] + SerializationExt = ["Serialization"] + +[[deps.L_BFGS_B_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "77feda930ed3f04b2b0fbb5bea89e69d3677c6b0" +uuid = "81d17ec3-03a1-5e46-b53e-bddc35a13473" +version = "3.0.1+0" + +[[deps.LaTeXStrings]] +git-tree-sha1 = "50901ebc375ed41dbf8058da26f9de442febbbec" +uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" +version = "1.3.1" + +[[deps.LazyArtifacts]] +deps = ["Artifacts", "Pkg"] +uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3" + +[[deps.LeftChildRightSiblingTrees]] +deps = ["AbstractTrees"] +git-tree-sha1 = "fb6803dafae4a5d62ea5cab204b1e657d9737e7f" +uuid = "1d6d02ad-be62-4b6b-8a6d-2f90e265016e" +version = "0.2.0" + +[[deps.LibCURL]] +deps = ["LibCURL_jll", "MozillaCACerts_jll"] +uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21" +version = "0.6.4" + +[[deps.LibCURL_jll]] +deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"] +uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0" +version = "8.4.0+0" + +[[deps.LibGit2]] +deps = ["Base64", "LibGit2_jll", "NetworkOptions", "Printf", "SHA"] +uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" + +[[deps.LibGit2_jll]] +deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll"] +uuid = "e37daf67-58a4-590a-8e99-b0245dd2ffc5" +version = "1.6.4+0" + +[[deps.LibSSH2_jll]] +deps = ["Artifacts", "Libdl", "MbedTLS_jll"] +uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8" +version = "1.11.0+1" + +[[deps.Libdl]] +uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" + +[[deps.Libtask]] +deps = ["FunctionWrappers", "LRUCache", "LinearAlgebra", "Statistics"] +git-tree-sha1 = "ed1f362b3fd13f00b65e61d98669c652c17663ab" +uuid = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f" +version = "0.8.7" + +[[deps.LineSearches]] +deps = ["LinearAlgebra", "NLSolversBase", "NaNMath", "Parameters", "Printf"] +git-tree-sha1 = "7bbea35cec17305fc70a0e5b4641477dc0789d9d" +uuid = "d3d80556-e9d4-5f37-9878-2ab0fcc64255" +version = "7.2.0" + +[[deps.LinearAlgebra]] +deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"] +uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" + +[[deps.LogDensityProblems]] +deps = ["ArgCheck", "DocStringExtensions", "Random"] +git-tree-sha1 = "f9a11237204bc137617194d79d813069838fcf61" +uuid = "6fdf6af0-433a-55f7-b3ed-c6c6e0b8df7c" +version = "2.1.1" + +[[deps.LogDensityProblemsAD]] +deps = ["DocStringExtensions", "LogDensityProblems", "Requires", "SimpleUnPack"] +git-tree-sha1 = "98cad2db1c46f2fff70a5e305fb42c97a251422a" +uuid = "996a588d-648d-4e1f-a8f0-a84b347e47b1" +version = "1.9.0" + + [deps.LogDensityProblemsAD.extensions] + LogDensityProblemsADADTypesExt = "ADTypes" + LogDensityProblemsADEnzymeExt = "Enzyme" + LogDensityProblemsADFiniteDifferencesExt = "FiniteDifferences" + LogDensityProblemsADForwardDiffBenchmarkToolsExt = ["BenchmarkTools", "ForwardDiff"] + LogDensityProblemsADForwardDiffExt = "ForwardDiff" + LogDensityProblemsADReverseDiffExt = "ReverseDiff" + LogDensityProblemsADTrackerExt = "Tracker" + LogDensityProblemsADZygoteExt = "Zygote" + + [deps.LogDensityProblemsAD.weakdeps] + ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b" + BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf" + Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9" + FiniteDifferences = "26cc04aa-876d-5657-8c51-4c34ba976000" + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" + Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f" + +[[deps.LogExpFunctions]] +deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"] +git-tree-sha1 = "18144f3e9cbe9b15b070288eef858f71b291ce37" +uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688" +version = "0.3.27" +weakdeps = ["ChainRulesCore", "ChangesOfVariables", "InverseFunctions"] + + [deps.LogExpFunctions.extensions] + LogExpFunctionsChainRulesCoreExt = "ChainRulesCore" + LogExpFunctionsChangesOfVariablesExt = "ChangesOfVariables" + LogExpFunctionsInverseFunctionsExt = "InverseFunctions" + +[[deps.Logging]] +uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" + +[[deps.LoggingExtras]] +deps = ["Dates", "Logging"] +git-tree-sha1 = "c1dd6d7978c12545b4179fb6153b9250c96b0075" +uuid = "e6f89c97-d47a-5376-807f-9c37f3926c36" +version = "1.0.3" + +[[deps.MCMCChains]] +deps = ["AbstractMCMC", "AxisArrays", "Dates", "Distributions", "IteratorInterfaceExtensions", "KernelDensity", "LinearAlgebra", "MCMCDiagnosticTools", "MLJModelInterface", "NaturalSort", "OrderedCollections", "PrettyTables", "Random", "RecipesBase", "Statistics", "StatsBase", "StatsFuns", "TableTraits", "Tables"] +git-tree-sha1 = "d28056379864318172ff4b7958710cfddd709339" +uuid = "c7f686f2-ff18-58e9-bc7b-31028e88f75d" +version = "6.0.6" + +[[deps.MCMCDiagnosticTools]] +deps = ["AbstractFFTs", "DataAPI", "DataStructures", "Distributions", "LinearAlgebra", "MLJModelInterface", "Random", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Tables"] +git-tree-sha1 = "8ba8b1840d3ab5b38e7c71c23c3193bb5cbc02b5" +uuid = "be115224-59cd-429b-ad48-344e309966f0" +version = "0.3.10" + +[[deps.MKL_jll]] +deps = ["Artifacts", "IntelOpenMP_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "oneTBB_jll"] +git-tree-sha1 = "80b2833b56d466b3858d565adcd16a4a05f2089b" +uuid = "856f044c-d86e-5d09-b602-aeab76dc8ba7" +version = "2024.1.0+0" + +[[deps.MLJModelInterface]] +deps = ["Random", "ScientificTypesBase", "StatisticalTraits"] +git-tree-sha1 = "d2a45e1b5998ba3fdfb6cfe0c81096d4c7fb40e7" +uuid = "e80e1ace-859a-464e-9ed9-23947d8ae3ea" +version = "1.9.6" + +[[deps.MacroTools]] +deps = ["Markdown", "Random"] +git-tree-sha1 = "2fa9ee3e63fd3a4f7a9a4f4744a52f4856de82df" +uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09" +version = "0.5.13" + +[[deps.MappedArrays]] +git-tree-sha1 = "2dab0221fe2b0f2cb6754eaa743cc266339f527e" +uuid = "dbb5928d-eab1-5f90-85c2-b9b0edb7c900" +version = "0.4.2" + +[[deps.Markdown]] +deps = ["Base64"] +uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" + +[[deps.MbedTLS_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" +version = "2.28.2+1" + +[[deps.MicroCollections]] +deps = ["Accessors", "BangBang", "InitialValues"] +git-tree-sha1 = "44d32db644e84c75dab479f1bc15ee76a1a3618f" +uuid = "128add7d-3638-4c79-886c-908ea0c25c34" +version = "0.2.0" + +[[deps.Missings]] +deps = ["DataAPI"] +git-tree-sha1 = "ec4f7fbeab05d7747bdf98eb74d130a2a2ed298d" +uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28" +version = "1.2.0" + +[[deps.Mmap]] +uuid = "a63ad114-7e13-5084-954f-fe012c677804" + +[[deps.MozillaCACerts_jll]] +uuid = "14a3606d-f60d-562e-9121-12d972cd8159" +version = "2023.1.10" + +[[deps.NLSolversBase]] +deps = ["DiffResults", "Distributed", "FiniteDiff", "ForwardDiff"] +git-tree-sha1 = "a0b464d183da839699f4c79e7606d9d186ec172c" +uuid = "d41bc354-129a-5804-8e4c-c37616107c6c" +version = "7.8.3" + +[[deps.NLopt]] +deps = ["NLopt_jll"] +git-tree-sha1 = "3b887e2ef56be3309e68d2546c6178e2d2fa9a60" +uuid = "76087f3c-5699-56af-9a33-bf431cd00edd" +version = "1.0.2" + + [deps.NLopt.extensions] + NLoptMathOptInterfaceExt = ["MathOptInterface"] + + [deps.NLopt.weakdeps] + MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee" + +[[deps.NLopt_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "9b1f15a08f9d00cdb2761dcfa6f453f5d0d6f973" +uuid = "079eb43e-fd8e-5478-9966-2cf3e3edb778" +version = "2.7.1+0" + +[[deps.NNlib]] +deps = ["Adapt", "Atomix", "ChainRulesCore", "GPUArraysCore", "KernelAbstractions", "LinearAlgebra", "Pkg", "Random", "Requires", "Statistics"] +git-tree-sha1 = "e0cea7ec219ada9ac80ec2e82e374ab2f154ae05" +uuid = "872c559c-99b0-510c-b3b7-b6c96a88d5cd" +version = "0.9.16" + + [deps.NNlib.extensions] + NNlibAMDGPUExt = "AMDGPU" + NNlibCUDACUDNNExt = ["CUDA", "cuDNN"] + NNlibCUDAExt = "CUDA" + NNlibEnzymeCoreExt = "EnzymeCore" + + [deps.NNlib.weakdeps] + AMDGPU = "21141c5a-9bdb-4563-92ae-f87d6854732e" + CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba" + EnzymeCore = "f151be2c-9106-41f4-ab19-57ee4f262869" + cuDNN = "02a925ec-e4fe-4b08-9a7e-0d78e3d38ccd" + +[[deps.NaNMath]] +deps = ["OpenLibm_jll"] +git-tree-sha1 = "0877504529a3e5c3343c6f8b4c0381e57e4387e4" +uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3" +version = "1.0.2" + +[[deps.NamedArrays]] +deps = ["Combinatorics", "DataStructures", "DelimitedFiles", "InvertedIndices", "LinearAlgebra", "Random", "Requires", "SparseArrays", "Statistics"] +git-tree-sha1 = "0ae91efac93c3859f5c812a24c9468bb9e50b028" +uuid = "86f7a689-2022-50b4-a561-43c23ac3c673" +version = "0.10.1" + +[[deps.NaturalSort]] +git-tree-sha1 = "eda490d06b9f7c00752ee81cfa451efe55521e21" +uuid = "c020b1a1-e9b0-503a-9c33-f039bfc54a85" +version = "1.0.0" + +[[deps.NetworkOptions]] +uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" +version = "1.2.0" + +[[deps.OffsetArrays]] +git-tree-sha1 = "e64b4f5ea6b7389f6f046d13d4896a8f9c1ba71e" +uuid = "6fe1bfb0-de20-5000-8ca7-80f57d26f881" +version = "1.14.0" +weakdeps = ["Adapt"] + + [deps.OffsetArrays.extensions] + OffsetArraysAdaptExt = "Adapt" + +[[deps.OpenBLAS_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"] +uuid = "4536629a-c528-5b80-bd46-f80d51c5b363" +version = "0.3.23+4" + +[[deps.OpenLibm_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "05823500-19ac-5b8b-9628-191a04bc5112" +version = "0.8.1+2" + +[[deps.OpenSpecFun_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1" +uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e" +version = "0.5.5+0" + +[[deps.Optim]] +deps = ["Compat", "FillArrays", "ForwardDiff", "LineSearches", "LinearAlgebra", "NLSolversBase", "NaNMath", "Parameters", "PositiveFactorizations", "Printf", "SparseArrays", "StatsBase"] +git-tree-sha1 = "d9b79c4eed437421ac4285148fcadf42e0700e89" +uuid = "429524aa-4258-5aef-a3af-852621145aeb" +version = "1.9.4" + + [deps.Optim.extensions] + OptimMOIExt = "MathOptInterface" + + [deps.Optim.weakdeps] + MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee" + +[[deps.Optimisers]] +deps = ["ChainRulesCore", "Functors", "LinearAlgebra", "Random", "Statistics"] +git-tree-sha1 = "6572fe0c5b74431aaeb0b18a4aa5ef03c84678be" +uuid = "3bd65402-5787-11e9-1adc-39752487f4e2" +version = "0.3.3" + +[[deps.Optimization]] +deps = ["ADTypes", "ArrayInterface", "ConsoleProgressMonitor", "DocStringExtensions", "LBFGSB", "LinearAlgebra", "Logging", "LoggingExtras", "OptimizationBase", "Pkg", "Printf", "ProgressLogging", "Reexport", "SciMLBase", "SparseArrays", "TerminalLoggers"] +git-tree-sha1 = "fe516248dcccd7285c0ac699eeaf32ae64d8e759" +uuid = "7f7a1694-90dd-40f0-9382-eb1efda571ba" +version = "3.25.1" + +[[deps.OptimizationBase]] +deps = ["ADTypes", "ArrayInterface", "DocStringExtensions", "LinearAlgebra", "Reexport", "Requires", "SciMLBase", "SparseArrays"] +git-tree-sha1 = "196a0a0ec2301de58a1e4cf2931211f6ae35f65f" +uuid = "bca83a33-5cc9-4baa-983d-23429ab6bcbb" +version = "1.0.2" + + [deps.OptimizationBase.extensions] + OptimizationEnzymeExt = "Enzyme" + OptimizationFiniteDiffExt = "FiniteDiff" + OptimizationForwardDiffExt = "ForwardDiff" + OptimizationMTKExt = "ModelingToolkit" + OptimizationReverseDiffExt = "ReverseDiff" + OptimizationSparseDiffExt = ["SparseDiffTools", "Symbolics", "ReverseDiff"] + OptimizationTrackerExt = "Tracker" + OptimizationZygoteExt = "Zygote" + + [deps.OptimizationBase.weakdeps] + Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9" + FiniteDiff = "6a86dc24-6348-571c-b903-95158fe2bd41" + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + SparseDiffTools = "47a9eef4-7e08-11e9-0b38-333d64bd3804" + Symbolics = "0c5d862f-8b57-4792-8d23-62f2024744c7" + Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" + Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f" + +[[deps.OptimizationNLopt]] +deps = ["NLopt", "Optimization", "Reexport"] +git-tree-sha1 = "83b7f5218c6ff83f65fff32923630fe8be9f5131" +uuid = "4e6fcdb7-1186-4e1f-a706-475e75c168bb" +version = "0.2.2" + +[[deps.OptimizationOptimJL]] +deps = ["Optim", "Optimization", "Reexport", "SparseArrays"] +git-tree-sha1 = "43870d726f883a47d158beebb1fc3c9fab1da9d6" +uuid = "36348300-93cb-4f02-beb5-3c3902f8871e" +version = "0.3.2" + +[[deps.OrderedCollections]] +git-tree-sha1 = "dfdf5519f235516220579f949664f1bf44e741c5" +uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d" +version = "1.6.3" + +[[deps.PDMats]] +deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"] +git-tree-sha1 = "949347156c25054de2db3b166c52ac4728cbad65" +uuid = "90014a1f-27ba-587c-ab20-58faa44d9150" +version = "0.11.31" + +[[deps.Parameters]] +deps = ["OrderedCollections", "UnPack"] +git-tree-sha1 = "34c0e9ad262e5f7fc75b10a9952ca7692cfc5fbe" +uuid = "d96e819e-fc66-5662-9728-84c9c7592b0a" +version = "0.12.3" + +[[deps.Pkg]] +deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"] +uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" +version = "1.10.0" + +[[deps.PositiveFactorizations]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "17275485f373e6673f7e7f97051f703ed5b15b20" +uuid = "85a6dd25-e78a-55b7-8502-1745935b8125" +version = "0.2.4" + +[[deps.PrecompileTools]] +deps = ["Preferences"] +git-tree-sha1 = "5aa36f7049a63a1528fe8f7c3f2113413ffd4e1f" +uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a" +version = "1.2.1" + +[[deps.Preferences]] +deps = ["TOML"] +git-tree-sha1 = "9306f6085165d270f7e3db02af26a400d580f5c6" +uuid = "21216c6a-2e73-6563-6e65-726566657250" +version = "1.4.3" + +[[deps.PrettyTables]] +deps = ["Crayons", "LaTeXStrings", "Markdown", "PrecompileTools", "Printf", "Reexport", "StringManipulation", "Tables"] +git-tree-sha1 = "88b895d13d53b5577fd53379d913b9ab9ac82660" +uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d" +version = "2.3.1" + +[[deps.Printf]] +deps = ["Unicode"] +uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" + +[[deps.ProgressLogging]] +deps = ["Logging", "SHA", "UUIDs"] +git-tree-sha1 = "80d919dee55b9c50e8d9e2da5eeafff3fe58b539" +uuid = "33c8b6b6-d38a-422a-b730-caa89a2f386c" +version = "0.1.4" + +[[deps.ProgressMeter]] +deps = ["Distributed", "Printf"] +git-tree-sha1 = "763a8ceb07833dd51bb9e3bbca372de32c0605ad" +uuid = "92933f4c-e287-5a05-a399-4b506db050ca" +version = "1.10.0" + +[[deps.PtrArrays]] +git-tree-sha1 = "077664975d750757f30e739c870fbbdc01db7913" +uuid = "43287f4e-b6f4-7ad1-bb20-aadabca52c3d" +version = "1.1.0" + +[[deps.QuadGK]] +deps = ["DataStructures", "LinearAlgebra"] +git-tree-sha1 = "9b23c31e76e333e6fb4c1595ae6afa74966a729e" +uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc" +version = "2.9.4" + +[[deps.REPL]] +deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"] +uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" + +[[deps.Random]] +deps = ["SHA"] +uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" + +[[deps.Random123]] +deps = ["Random", "RandomNumbers"] +git-tree-sha1 = "4743b43e5a9c4a2ede372de7061eed81795b12e7" +uuid = "74087812-796a-5b5d-8853-05524746bad3" +version = "1.7.0" + +[[deps.RandomNumbers]] +deps = ["Random", "Requires"] +git-tree-sha1 = "043da614cc7e95c703498a491e2c21f58a2b8111" +uuid = "e6cf234a-135c-5ec9-84dd-332b85af5143" +version = "1.5.3" + +[[deps.RangeArrays]] +git-tree-sha1 = "b9039e93773ddcfc828f12aadf7115b4b4d225f5" +uuid = "b3c3ace0-ae52-54e7-9d0b-2c1406fd6b9d" +version = "0.3.2" + +[[deps.Ratios]] +deps = ["Requires"] +git-tree-sha1 = "1342a47bf3260ee108163042310d26f2be5ec90b" +uuid = "c84ed2f1-dad5-54f0-aa8e-dbefe2724439" +version = "0.4.5" + + [deps.Ratios.extensions] + RatiosFixedPointNumbersExt = "FixedPointNumbers" + + [deps.Ratios.weakdeps] + FixedPointNumbers = "53c48c17-4a7d-5ca2-90c5-79b7896eea93" + +[[deps.RealDot]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "9f0a1b71baaf7650f4fa8a1d168c7fb6ee41f0c9" +uuid = "c1ae055f-0cd5-4b69-90a6-9a35b1a98df9" +version = "0.1.0" + +[[deps.RecipesBase]] +deps = ["PrecompileTools"] +git-tree-sha1 = "5c3d09cc4f31f5fc6af001c250bf1278733100ff" +uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01" +version = "1.3.4" + +[[deps.RecursiveArrayTools]] +deps = ["Adapt", "ArrayInterface", "DocStringExtensions", "GPUArraysCore", "IteratorInterfaceExtensions", "LinearAlgebra", "RecipesBase", "SparseArrays", "StaticArraysCore", "Statistics", "SymbolicIndexingInterface", "Tables"] +git-tree-sha1 = "758bc86b90e9fee2edc4af2a750b0d3f2d5c02c5" +uuid = "731186ca-8d62-57ce-b412-fbd966d074cd" +version = "3.19.0" + + [deps.RecursiveArrayTools.extensions] + RecursiveArrayToolsFastBroadcastExt = "FastBroadcast" + RecursiveArrayToolsForwardDiffExt = "ForwardDiff" + RecursiveArrayToolsMeasurementsExt = "Measurements" + RecursiveArrayToolsMonteCarloMeasurementsExt = "MonteCarloMeasurements" + RecursiveArrayToolsReverseDiffExt = ["ReverseDiff", "Zygote"] + RecursiveArrayToolsTrackerExt = "Tracker" + RecursiveArrayToolsZygoteExt = "Zygote" + + [deps.RecursiveArrayTools.weakdeps] + FastBroadcast = "7034ab61-46d4-4ed7-9d0f-46aef9175898" + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + Measurements = "eff96d63-e80a-5855-80a2-b1b0885c5ab7" + MonteCarloMeasurements = "0987c9cc-fe09-11e8-30f0-b96dd679fdca" + ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" + Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" + Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f" + +[[deps.Reexport]] +git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b" +uuid = "189a3867-3050-52da-a836-e630ba90ab69" +version = "1.2.2" + +[[deps.Requires]] +deps = ["UUIDs"] +git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7" +uuid = "ae029012-a4dd-5104-9daa-d747884805df" +version = "1.3.0" + +[[deps.ReverseDiff]] +deps = ["ChainRulesCore", "DiffResults", "DiffRules", "ForwardDiff", "FunctionWrappers", "LinearAlgebra", "LogExpFunctions", "MacroTools", "NaNMath", "Random", "SpecialFunctions", "StaticArrays", "Statistics"] +git-tree-sha1 = "cc6cd622481ea366bb9067859446a8b01d92b468" +uuid = "37e2e3b7-166d-5795-8a7a-e32c996b4267" +version = "1.15.3" + +[[deps.Rmath]] +deps = ["Random", "Rmath_jll"] +git-tree-sha1 = "f65dcb5fa46aee0cf9ed6274ccbd597adc49aa7b" +uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa" +version = "0.7.1" + +[[deps.Rmath_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "d483cd324ce5cf5d61b77930f0bbd6cb61927d21" +uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f" +version = "0.4.2+0" + +[[deps.Roots]] +deps = ["Accessors", "ChainRulesCore", "CommonSolve", "Printf"] +git-tree-sha1 = "1ab580704784260ee5f45bffac810b152922747b" +uuid = "f2b01f46-fcfa-551c-844a-d8ac1e96c665" +version = "2.1.5" + + [deps.Roots.extensions] + RootsForwardDiffExt = "ForwardDiff" + RootsIntervalRootFindingExt = "IntervalRootFinding" + RootsSymPyExt = "SymPy" + RootsSymPyPythonCallExt = "SymPyPythonCall" + + [deps.Roots.weakdeps] + ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" + IntervalRootFinding = "d2bf35a9-74e0-55ec-b149-d360ff49b807" + SymPy = "24249f21-da20-56a4-8eb1-6a02cf4ae2e6" + SymPyPythonCall = "bc8888f7-b21e-4b7c-a06a-5d9c9496438c" + +[[deps.RuntimeGeneratedFunctions]] +deps = ["ExprTools", "SHA", "Serialization"] +git-tree-sha1 = "04c968137612c4a5629fa531334bb81ad5680f00" +uuid = "7e49a35a-f44a-4d26-94aa-eba1b4ca6b47" +version = "0.5.13" + +[[deps.SHA]] +uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" +version = "0.7.0" + +[[deps.SSMProblems]] +deps = ["AbstractMCMC"] +git-tree-sha1 = "f640e4e8343c9d5f470e2f6ca6ce79f708ab6376" +uuid = "26aad666-b158-4e64-9d35-0e672562fa48" +version = "0.1.1" + +[[deps.SciMLBase]] +deps = ["ADTypes", "ArrayInterface", "CommonSolve", "ConstructionBase", "Distributed", "DocStringExtensions", "EnumX", "FunctionWrappersWrappers", "IteratorInterfaceExtensions", "LinearAlgebra", "Logging", "Markdown", "PrecompileTools", "Preferences", "Printf", "RecipesBase", "RecursiveArrayTools", "Reexport", "RuntimeGeneratedFunctions", "SciMLOperators", "SciMLStructures", "StaticArraysCore", "Statistics", "SymbolicIndexingInterface", "Tables"] +git-tree-sha1 = "265f1a7a804d8093fa0b17e33e45373a77e56ca5" +uuid = "0bca4576-84f4-4d90-8ffe-ffa030f20462" +version = "2.38.0" + + [deps.SciMLBase.extensions] + SciMLBaseChainRulesCoreExt = "ChainRulesCore" + SciMLBaseMakieExt = "Makie" + SciMLBasePartialFunctionsExt = "PartialFunctions" + SciMLBasePyCallExt = "PyCall" + SciMLBasePythonCallExt = "PythonCall" + SciMLBaseRCallExt = "RCall" + SciMLBaseZygoteExt = "Zygote" + + [deps.SciMLBase.weakdeps] + ChainRules = "082447d4-558c-5d27-93f4-14fc19e9eca2" + ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" + Makie = "ee78f7c6-11fb-53f2-987a-cfe4a2b5a57a" + PartialFunctions = "570af359-4316-4cb7-8c74-252c00c2016b" + PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0" + PythonCall = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d" + RCall = "6f49c342-dc21-5d91-9882-a32aef131414" + Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f" + +[[deps.SciMLOperators]] +deps = ["ArrayInterface", "DocStringExtensions", "LinearAlgebra", "MacroTools", "Setfield", "SparseArrays", "StaticArraysCore"] +git-tree-sha1 = "10499f619ef6e890f3f4a38914481cc868689cd5" +uuid = "c0aeaf25-5076-4817-a8d5-81caf7dfa961" +version = "0.3.8" + +[[deps.SciMLStructures]] +git-tree-sha1 = "d778a74df2f64059c38453b34abad1953b2b8722" +uuid = "53ae85a6-f571-4167-b2af-e1d143709226" +version = "1.2.0" + +[[deps.ScientificTypesBase]] +git-tree-sha1 = "a8e18eb383b5ecf1b5e6fc237eb39255044fd92b" +uuid = "30f210dd-8aff-4c5f-94ba-8e64358c1161" +version = "3.0.0" + +[[deps.Serialization]] +uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" + +[[deps.Setfield]] +deps = ["ConstructionBase", "Future", "MacroTools", "StaticArraysCore"] +git-tree-sha1 = "e2cc6d8c88613c05e1defb55170bf5ff211fbeac" +uuid = "efcf1570-3423-57d1-acb7-fd33fddbac46" +version = "1.1.1" + +[[deps.SharedArrays]] +deps = ["Distributed", "Mmap", "Random", "Serialization"] +uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383" + +[[deps.SimpleUnPack]] +git-tree-sha1 = "58e6353e72cde29b90a69527e56df1b5c3d8c437" +uuid = "ce78b400-467f-4804-87d8-8f486da07d0a" +version = "1.1.0" + +[[deps.Sockets]] +uuid = "6462fe0b-24de-5631-8697-dd941f90decc" + +[[deps.SortingAlgorithms]] +deps = ["DataStructures"] +git-tree-sha1 = "66e0a8e672a0bdfca2c3f5937efb8538b9ddc085" +uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c" +version = "1.2.1" + +[[deps.SparseArrays]] +deps = ["Libdl", "LinearAlgebra", "Random", "Serialization", "SuiteSparse_jll"] +uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" +version = "1.10.0" + +[[deps.SparseInverseSubset]] +deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"] +git-tree-sha1 = "52962839426b75b3021296f7df242e40ecfc0852" +uuid = "dc90abb0-5640-4711-901d-7e5b23a2fada" +version = "0.1.2" + +[[deps.SpecialFunctions]] +deps = ["IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"] +git-tree-sha1 = "2f5d4697f21388cbe1ff299430dd169ef97d7e14" +uuid = "276daf66-3868-5448-9aa4-cd146d93841b" +version = "2.4.0" +weakdeps = ["ChainRulesCore"] + + [deps.SpecialFunctions.extensions] + SpecialFunctionsChainRulesCoreExt = "ChainRulesCore" + +[[deps.SplittablesBase]] +deps = ["Setfield", "Test"] +git-tree-sha1 = "e08a62abc517eb79667d0a29dc08a3b589516bb5" +uuid = "171d559e-b47b-412a-8079-5efa626c420e" +version = "0.1.15" + +[[deps.StaticArrays]] +deps = ["LinearAlgebra", "PrecompileTools", "Random", "StaticArraysCore"] +git-tree-sha1 = "9ae599cd7529cfce7fea36cf00a62cfc56f0f37c" +uuid = "90137ffa-7385-5640-81b9-e52037218182" +version = "1.9.4" +weakdeps = ["ChainRulesCore", "Statistics"] + + [deps.StaticArrays.extensions] + StaticArraysChainRulesCoreExt = "ChainRulesCore" + StaticArraysStatisticsExt = "Statistics" + +[[deps.StaticArraysCore]] +git-tree-sha1 = "36b3d696ce6366023a0ea192b4cd442268995a0d" +uuid = "1e83bf80-4336-4d27-bf5d-d5a4f845583c" +version = "1.4.2" + +[[deps.StatisticalTraits]] +deps = ["ScientificTypesBase"] +git-tree-sha1 = "30b9236691858e13f167ce829490a68e1a597782" +uuid = "64bff920-2084-43da-a3e6-9bb72801c0c9" +version = "3.2.0" + +[[deps.Statistics]] +deps = ["LinearAlgebra", "SparseArrays"] +uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" +version = "1.10.0" + +[[deps.StatsAPI]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "1ff449ad350c9c4cbc756624d6f8a8c3ef56d3ed" +uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0" +version = "1.7.0" + +[[deps.StatsBase]] +deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"] +git-tree-sha1 = "5cf7606d6cef84b543b483848d4ae08ad9832b21" +uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" +version = "0.34.3" + +[[deps.StatsFuns]] +deps = ["HypergeometricFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"] +git-tree-sha1 = "cef0472124fab0695b58ca35a77c6fb942fdab8a" +uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c" +version = "1.3.1" +weakdeps = ["ChainRulesCore", "InverseFunctions"] + + [deps.StatsFuns.extensions] + StatsFunsChainRulesCoreExt = "ChainRulesCore" + StatsFunsInverseFunctionsExt = "InverseFunctions" + +[[deps.StringManipulation]] +deps = ["PrecompileTools"] +git-tree-sha1 = "a04cabe79c5f01f4d723cc6704070ada0b9d46d5" +uuid = "892a3eda-7b42-436c-8928-eab12a02cf0e" +version = "0.3.4" + +[[deps.StructArrays]] +deps = ["ConstructionBase", "DataAPI", "Tables"] +git-tree-sha1 = "f4dc295e983502292c4c3f951dbb4e985e35b3be" +uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a" +version = "0.6.18" +weakdeps = ["Adapt", "GPUArraysCore", "SparseArrays", "StaticArrays"] + + [deps.StructArrays.extensions] + StructArraysAdaptExt = "Adapt" + StructArraysGPUArraysCoreExt = "GPUArraysCore" + StructArraysSparseArraysExt = "SparseArrays" + StructArraysStaticArraysExt = "StaticArrays" + +[[deps.SuiteSparse]] +deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"] +uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9" + +[[deps.SuiteSparse_jll]] +deps = ["Artifacts", "Libdl", "libblastrampoline_jll"] +uuid = "bea87d4a-7f5b-5778-9afe-8cc45184846c" +version = "7.2.1+1" + +[[deps.SymbolicIndexingInterface]] +deps = ["Accessors", "ArrayInterface", "RuntimeGeneratedFunctions", "StaticArraysCore"] +git-tree-sha1 = "b479c7a16803f08779ac5b7f9844a42621baeeda" +uuid = "2efcf032-c050-4f8e-a9bb-153293bab1f5" +version = "0.3.21" + +[[deps.TOML]] +deps = ["Dates"] +uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76" +version = "1.0.3" + +[[deps.TableTraits]] +deps = ["IteratorInterfaceExtensions"] +git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39" +uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c" +version = "1.0.1" + +[[deps.Tables]] +deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "OrderedCollections", "TableTraits"] +git-tree-sha1 = "cb76cf677714c095e535e3501ac7954732aeea2d" +uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" +version = "1.11.1" + +[[deps.Tar]] +deps = ["ArgTools", "SHA"] +uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e" +version = "1.10.0" + +[[deps.TerminalLoggers]] +deps = ["LeftChildRightSiblingTrees", "Logging", "Markdown", "Printf", "ProgressLogging", "UUIDs"] +git-tree-sha1 = "f133fab380933d042f6796eda4e130272ba520ca" +uuid = "5d786b92-1e48-4d6f-9151-6b4477ca9bed" +version = "0.1.7" + +[[deps.Test]] +deps = ["InteractiveUtils", "Logging", "Random", "Serialization"] +uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" + +[[deps.Tracker]] +deps = ["Adapt", "ChainRulesCore", "DiffRules", "ForwardDiff", "Functors", "LinearAlgebra", "LogExpFunctions", "MacroTools", "NNlib", "NaNMath", "Optimisers", "Printf", "Random", "Requires", "SpecialFunctions", "Statistics"] +git-tree-sha1 = "5158100ed55411867674576788e710a815a0af02" +uuid = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" +version = "0.2.34" +weakdeps = ["PDMats"] + + [deps.Tracker.extensions] + TrackerPDMatsExt = "PDMats" + +[[deps.Transducers]] +deps = ["Accessors", "Adapt", "ArgCheck", "BangBang", "Baselet", "CompositionsBase", "ConstructionBase", "DefineSingletons", "Distributed", "InitialValues", "Logging", "Markdown", "MicroCollections", "Requires", "SplittablesBase", "Tables"] +git-tree-sha1 = "5215a069867476fc8e3469602006b9670e68da23" +uuid = "28d57a85-8fef-5791-bfe6-a80928e7c999" +version = "0.4.82" + + [deps.Transducers.extensions] + TransducersBlockArraysExt = "BlockArrays" + TransducersDataFramesExt = "DataFrames" + TransducersLazyArraysExt = "LazyArrays" + TransducersOnlineStatsBaseExt = "OnlineStatsBase" + TransducersReferenceablesExt = "Referenceables" + + [deps.Transducers.weakdeps] + BlockArrays = "8e7c35d0-a365-5155-bbbb-fb81a777f24e" + DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" + LazyArrays = "5078a376-72f3-5289-bfd5-ec5146d43c02" + OnlineStatsBase = "925886fa-5bf2-5e8e-b522-a9147a512338" + Referenceables = "42d2dcc6-99eb-4e98-b66c-637b7d73030e" + +[[deps.Turing]] +deps = ["ADTypes", "AbstractMCMC", "Accessors", "AdvancedHMC", "AdvancedMH", "AdvancedPS", "AdvancedVI", "BangBang", "Bijectors", "DataStructures", "Distributions", "DistributionsAD", "DocStringExtensions", "DynamicPPL", "EllipticalSliceSampling", "ForwardDiff", "Libtask", "LinearAlgebra", "LogDensityProblems", "LogDensityProblemsAD", "MCMCChains", "NamedArrays", "Printf", "Random", "Reexport", "Requires", "SciMLBase", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns"] +git-tree-sha1 = "cfb3b446a5e52e1da4cc71b77a9350c309c581f0" +uuid = "fce5fe82-541a-59a6-adf8-730c64b5f9a0" +version = "0.32.0" + + [deps.Turing.extensions] + TuringDynamicHMCExt = "DynamicHMC" + TuringOptimExt = "Optim" + + [deps.Turing.weakdeps] + DynamicHMC = "bbc10e6e-7c05-544b-b16e-64fede858acb" + Optim = "429524aa-4258-5aef-a3af-852621145aeb" + +[[deps.UUIDs]] +deps = ["Random", "SHA"] +uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" + +[[deps.UnPack]] +git-tree-sha1 = "387c1f73762231e86e0c9c5443ce3b4a0a9a0c2b" +uuid = "3a884ed6-31ef-47d7-9d2a-63182c4928ed" +version = "1.0.2" + +[[deps.Unicode]] +uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" + +[[deps.UnsafeAtomics]] +git-tree-sha1 = "6331ac3440856ea1988316b46045303bef658278" +uuid = "013be700-e6cd-48c3-b4a1-df204f14c38f" +version = "0.2.1" + +[[deps.UnsafeAtomicsLLVM]] +deps = ["LLVM", "UnsafeAtomics"] +git-tree-sha1 = "323e3d0acf5e78a56dfae7bd8928c989b4f3083e" +uuid = "d80eeb9a-aca5-4d75-85e5-170c8b632249" +version = "0.1.3" + +[[deps.WoodburyMatrices]] +deps = ["LinearAlgebra", "SparseArrays"] +git-tree-sha1 = "c1a7aa6219628fcd757dede0ca95e245c5cd9511" +uuid = "efce3f68-66dc-5838-9240-27a6d6f5f9b6" +version = "1.0.0" + +[[deps.Zlib_jll]] +deps = ["Libdl"] +uuid = "83775a58-1f1d-513f-b197-d71354ab007a" +version = "1.2.13+1" + +[[deps.ZygoteRules]] +deps = ["ChainRulesCore", "MacroTools"] +git-tree-sha1 = "27798139afc0a2afa7b1824c206d5e87ea587a00" +uuid = "700de1a5-db45-46bc-99cf-38207098b444" +version = "0.2.5" + +[[deps.libblastrampoline_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "8e850b90-86db-534c-a0d3-1478176c7d93" +version = "5.8.0+1" + +[[deps.nghttp2_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d" +version = "1.52.0+1" + +[[deps.oneTBB_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "7d0ea0f4895ef2f5cb83645fa689e52cb55cf493" +uuid = "1317d2d5-d96f-522e-a858-c73665f53c3e" +version = "2021.12.0+0" + +[[deps.p7zip_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0" +version = "17.4.0+2" diff --git a/tutorials/docs-17-mode-estimation/Project.toml b/tutorials/docs-17-mode-estimation/Project.toml new file mode 100755 index 000000000..09f58e878 --- /dev/null +++ b/tutorials/docs-17-mode-estimation/Project.toml @@ -0,0 +1,8 @@ +[deps] +ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b" +Optimization = "7f7a1694-90dd-40f0-9382-eb1efda571ba" +OptimizationNLopt = "4e6fcdb7-1186-4e1f-a706-475e75c168bb" +OptimizationOptimJL = "36348300-93cb-4f02-beb5-3c3902f8871e" +ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267" +StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" +Turing = "fce5fe82-541a-59a6-adf8-730c64b5f9a0" diff --git a/tutorials/docs-17-mode-estimation/index.qmd b/tutorials/docs-17-mode-estimation/index.qmd new file mode 100755 index 000000000..234b29415 --- /dev/null +++ b/tutorials/docs-17-mode-estimation/index.qmd @@ -0,0 +1,120 @@ +--- +title: Mode Estimation +engine: julia +--- + +```{julia} +#| echo: false +#| output: false +using Pkg; +Pkg.instantiate(); +``` + +In addition to sampling from the distributions of a statistical model, one may be interested in finding the most likely parameter values that maximise for instance the posterior distribution density function or the likelihood. This is called mode estimation. Turing provides support for two mode estimation techniques, [maximum likelihood estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) and [maximum a posterior](https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation) (MAP) estimation. + +To demonstrate mode estimation, let us load Turing and declare a model: + +```{julia} +using Turing + +@model function gdemo(x) + s² ~ InverseGamma(2, 3) + m ~ Normal(0, sqrt(s²)) + + for i in eachindex(x) + x[i] ~ Normal(m, sqrt(s²)) + end +end +``` + +Once the model is defined, we can construct a model instance as we normally would: + +```{julia} +# Instantiate the gdemo model with our data. +data = [1.5, 2.0] +model = gdemo(data) +``` + +Finding the maximum aposteriori or maximum likelihood parameters is as simple as + +```{julia} +# Generate a MLE estimate. +mle_estimate = maximum_likelihood(model) + +# Generate a MAP estimate. +map_estimate = maximum_a_posteriori(model) +``` + +The estimates are returned as instances of the `ModeResult` type. It has the fields `values` for the parameter values found and `lp` for the log probability at the optimum, as well as `f` for the objective function and `optim_result` for more detailed results of the optimisation procedure, such as convergence information. + +```{julia} +@show mle_estimate.values +@show mle_estimate.lp +``` + +## Controlling the optimisation process + +Under the hood `maximum_likelihood` and `maximum_a_posteriori` use the [Optimization.jl](https://github.com/SciML/Optimization.jl) package, which provides a unified interface to many other optimisation packages. By default Turing typically uses the [LBFGS](https://en.wikipedia.org/wiki/Limited-memory_BFGS) method from [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl), but we can easily change that, + +```{julia} +using OptimizationOptimJL: NelderMead +maximum_likelihood(model, NelderMead()) + +using OptimizationNLopt: NLopt.LD_TNEWTON_PRECOND_RESTART +maximum_likelihood(model, NLopt.LD_TNEWTON_PRECOND_RESTART()) +``` + +The above are just two examples, Optimization.jl supports [many more](https://docs.sciml.ai/Optimization/stable/). + +We can also help the optimisation by giving it a starting point we know is close to the final solution, or by specifying an automatic differentiation method + +```{julia} +using ADTypes: AutoReverseDiff +import ReverseDiff +maximum_likelihood(model, NelderMead(); initial_params=[0.1, 2], adtype=AutoReverseDiff()) +``` + +When providing values to arguments like `initial_params` the parameters are typically specified in the order in which they appear in the code of the model, so in this case first `s²` then `m`. More precisely it's the order returned by `Turing.Inference.getparams(model, Turing.VarInfo(model))`. + +We can also do constrained optimisation, by providing either intervals within which the parameters must stay, or costraint functions that they need to respect. For instance, here's how one can find the MLE with the constraint that the variance must be less than 0.01 and the mean must be between -1 and 1.: + +```{julia} +maximum_likelihood(model; lb=[0.0, -1.0], ub=[0.01, 1.0]) +``` + +The arguments for lower (`lb`) and upper (`ub`) bounds follow the arguments of `Optimization.OptimizationProblem`, as do other parameters for providing bounds such as `cons`. Any extraneous keyword arguments given to `maximum_likelihood` or `maximum_a_posteriori` are passed to `Optimization.solve`. Some often useful ones are `maxiters` for controlling the maximum number of iterations and `abstol` and `reltol` for the absolute and relative convergence tolerances: + +```{julia} +badly_converged_mle = maximum_likelihood(model, NelderMead(); maxiters=10, reltol=1e-9) +``` + +We can check whether the optimisation converged using the `optim_result` field of the result: + +```{julia} +@show badly_converged_mle.optim_result +``` + +For more details, such as a full list of possible arguments, we encourage the reader to read the docstring of the function `Turing.Optimisation.estimate_mode`, which is what `maximum_likelihood` and `maximum_a_posteriori` call, and the documentation of [Optimization.jl](https://docs.sciml.ai/Optimization/stable/). + +## Analyzing your mode estimate + +Turing extends several methods from `StatsBase` that can be used to analyze your mode estimation results. Methods implemented include `vcov`, `informationmatrix`, `coeftable`, `params`, and `coef`, among others. + +For example, let's examine our ML estimate from above using `coeftable`: + +```{julia} +using StatsBase: coeftable +coeftable(mle_estimate) +``` + +Standard errors are calculated from the Fisher information matrix (inverse Hessian of the log likelihood or log joint). Note that standard errors calculated in this way may not always be appropriate for MAP estimates, so please be cautious in interpreting them. + +## Sampling with the MAP/MLE as initial states + +You can begin sampling your chain from an MLE/MAP estimate by extracting the vector of parameter values and providing it to the `sample` function with the keyword `initial_params`. For example, here is how to sample from the full posterior using the MAP estimate as the starting point: + +```{julia} +#| eval: false +map_estimate = maximum_a_posteriori(model) +chain = sample(model, NUTS(), 1_000; initial_params=map_estimate.values.array) +``` From 303e248799e070f5b572df42c955cb5ad0798b6b Mon Sep 17 00:00:00 2001 From: Markus Hauru Date: Thu, 6 Jun 2024 12:14:12 +0100 Subject: [PATCH 2/2] Improvements to mode estimation docs --- .../docs-17-mode-estimation/Manifest.toml | 88 +++++++++---------- tutorials/docs-17-mode-estimation/index.qmd | 24 ++--- 2 files changed, 58 insertions(+), 54 deletions(-) mode change 100755 => 100644 tutorials/docs-17-mode-estimation/Manifest.toml diff --git a/tutorials/docs-17-mode-estimation/Manifest.toml b/tutorials/docs-17-mode-estimation/Manifest.toml old mode 100755 new mode 100644 index 4ba574a15..c07a7f9cc --- a/tutorials/docs-17-mode-estimation/Manifest.toml +++ b/tutorials/docs-17-mode-estimation/Manifest.toml @@ -94,9 +94,9 @@ version = "0.6.1" [[deps.AdvancedMH]] deps = ["AbstractMCMC", "Distributions", "FillArrays", "LinearAlgebra", "LogDensityProblems", "Random", "Requires"] -git-tree-sha1 = "16589dbdd36c782ff01700908e962b303474f641" +git-tree-sha1 = "fa4e8d6f9bae913aaa40224cf9407163e693d829" uuid = "5b7e9947-ddc0-4b3f-9b55-0d8042f74170" -version = "0.8.1" +version = "0.8.2" weakdeps = ["DiffResults", "ForwardDiff", "MCMCChains", "StructArrays"] [deps.AdvancedMH.extensions] @@ -116,9 +116,9 @@ weakdeps = ["Libtask"] [[deps.AdvancedVI]] deps = ["ADTypes", "Bijectors", "DiffResults", "Distributions", "DistributionsAD", "DocStringExtensions", "ForwardDiff", "LinearAlgebra", "ProgressMeter", "Random", "Requires", "StatsBase", "StatsFuns", "Tracker"] -git-tree-sha1 = "187f67ab998f25208651262fee9539d845016b26" +git-tree-sha1 = "3e97de1a2ccce08978cd80570d8cbb9ff3f08bd3" uuid = "b5ca4192-6429-45e5-a2d9-87aec30a685c" -version = "0.2.5" +version = "0.2.6" [deps.AdvancedVI.extensions] AdvancedVIEnzymeExt = ["Enzyme"] @@ -149,9 +149,9 @@ version = "1.1.1" [[deps.ArrayInterface]] deps = ["Adapt", "LinearAlgebra", "SparseArrays", "SuiteSparse"] -git-tree-sha1 = "133a240faec6e074e07c31ee75619c90544179cf" +git-tree-sha1 = "ed2ec3c9b483842ae59cd273834e5b46206d6dda" uuid = "4fba245c-0d91-5ea0-9b3e-6abc04ee57a9" -version = "7.10.0" +version = "7.11.0" [deps.ArrayInterface.extensions] ArrayInterfaceBandedMatricesExt = "BandedMatrices" @@ -228,9 +228,9 @@ version = "0.1.1" [[deps.Bijectors]] deps = ["ArgCheck", "ChainRules", "ChainRulesCore", "ChangesOfVariables", "Compat", "Distributions", "Functors", "InverseFunctions", "IrrationalConstants", "LinearAlgebra", "LogExpFunctions", "MappedArrays", "Random", "Reexport", "Requires", "Roots", "SparseArrays", "Statistics"] -git-tree-sha1 = "49491db48b1c70eefa5115e626100dbd6c0ff4c0" +git-tree-sha1 = "2173b2974d6afb2dbc72002c51c84803c08e8aa0" uuid = "76274a88-744f-5084-9051-94815aaf08c4" -version = "0.13.12" +version = "0.13.13" [deps.Bijectors.extensions] BijectorsDistributionsADExt = "DistributionsAD" @@ -261,15 +261,15 @@ version = "0.5.1" [[deps.ChainRules]] deps = ["Adapt", "ChainRulesCore", "Compat", "Distributed", "GPUArraysCore", "IrrationalConstants", "LinearAlgebra", "Random", "RealDot", "SparseArrays", "SparseInverseSubset", "Statistics", "StructArrays", "SuiteSparse"] -git-tree-sha1 = "291821c1251486504f6bae435227907d734e94d2" +git-tree-sha1 = "5ec157747036038ec70b250f578362268f0472f1" uuid = "082447d4-558c-5d27-93f4-14fc19e9eca2" -version = "1.66.0" +version = "1.68.0" [[deps.ChainRulesCore]] deps = ["Compat", "LinearAlgebra"] -git-tree-sha1 = "575cd02e080939a33b6df6c5853d14924c08e35b" +git-tree-sha1 = "71acdbf594aab5bbb2cec89b208c41b4c411e49f" uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" -version = "1.23.0" +version = "1.24.0" weakdeps = ["SparseArrays"] [deps.ChainRulesCore.extensions] @@ -402,9 +402,9 @@ uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b" [[deps.Distributions]] deps = ["AliasTables", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns"] -git-tree-sha1 = "22c595ca4146c07b16bcf9c8bea86f731f7109d2" +git-tree-sha1 = "9c405847cc7ecda2dc921ccf18b47ca150d7317e" uuid = "31c24e10-a181-5473-b8eb-7969acd0382f" -version = "0.25.108" +version = "0.25.109" weakdeps = ["ChainRulesCore", "DensityInterface", "Test"] [deps.Distributions.extensions] @@ -551,9 +551,9 @@ version = "0.1.3" [[deps.Functors]] deps = ["LinearAlgebra"] -git-tree-sha1 = "d3e63d9fa13f8eaa2f06f64949e2afc593ff52c2" +git-tree-sha1 = "8a66c07630d6428eaab3506a0eabfcf4a9edea05" uuid = "d9f16b24-f501-4c13-a1f2-28368ffc5196" -version = "0.4.10" +version = "0.4.11" [[deps.Future]] deps = ["Random"] @@ -677,9 +677,9 @@ version = "0.4.1" [[deps.LLVM]] deps = ["CEnum", "LLVMExtra_jll", "Libdl", "Preferences", "Printf", "Requires", "Unicode"] -git-tree-sha1 = "839c82932db86740ae729779e610f07a1640be9a" +git-tree-sha1 = "065c36f95709dd4a676dc6839a35d6fa6f192f24" uuid = "929cbde3-209d-540e-8aea-75f648917ca0" -version = "6.6.3" +version = "7.1.0" [deps.LLVM.extensions] BFloat16sExt = "BFloat16s" @@ -800,9 +800,9 @@ version = "1.9.0" [[deps.LogExpFunctions]] deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"] -git-tree-sha1 = "18144f3e9cbe9b15b070288eef858f71b291ce37" +git-tree-sha1 = "a2d09619db4e765091ee5c6ffe8872849de0feea" uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688" -version = "0.3.27" +version = "0.3.28" weakdeps = ["ChainRulesCore", "ChangesOfVariables", "InverseFunctions"] [deps.LogExpFunctions.extensions] @@ -839,9 +839,9 @@ version = "2024.1.0+0" [[deps.MLJModelInterface]] deps = ["Random", "ScientificTypesBase", "StatisticalTraits"] -git-tree-sha1 = "d2a45e1b5998ba3fdfb6cfe0c81096d4c7fb40e7" +git-tree-sha1 = "88ef480f46e0506143681b3fb14d86742f3cecb1" uuid = "e80e1ace-859a-464e-9ed9-23947d8ae3ea" -version = "1.9.6" +version = "1.10.0" [[deps.MacroTools]] deps = ["Markdown", "Random"] @@ -908,9 +908,9 @@ version = "2.7.1+0" [[deps.NNlib]] deps = ["Adapt", "Atomix", "ChainRulesCore", "GPUArraysCore", "KernelAbstractions", "LinearAlgebra", "Pkg", "Random", "Requires", "Statistics"] -git-tree-sha1 = "e0cea7ec219ada9ac80ec2e82e374ab2f154ae05" +git-tree-sha1 = "3d4617f943afe6410206a5294a95948c8d1b35bd" uuid = "872c559c-99b0-510c-b3b7-b6c96a88d5cd" -version = "0.9.16" +version = "0.9.17" [deps.NNlib.extensions] NNlibAMDGPUExt = "AMDGPU" @@ -932,9 +932,9 @@ version = "1.0.2" [[deps.NamedArrays]] deps = ["Combinatorics", "DataStructures", "DelimitedFiles", "InvertedIndices", "LinearAlgebra", "Random", "Requires", "SparseArrays", "Statistics"] -git-tree-sha1 = "0ae91efac93c3859f5c812a24c9468bb9e50b028" +git-tree-sha1 = "c7aab3836df3f31591a2b4167fcd87b741dacfc9" uuid = "86f7a689-2022-50b4-a561-43c23ac3c673" -version = "0.10.1" +version = "0.10.2" [[deps.NaturalSort]] git-tree-sha1 = "eda490d06b9f7c00752ee81cfa451efe55521e21" @@ -1075,9 +1075,9 @@ version = "1.4.3" [[deps.PrettyTables]] deps = ["Crayons", "LaTeXStrings", "Markdown", "PrecompileTools", "Printf", "Reexport", "StringManipulation", "Tables"] -git-tree-sha1 = "88b895d13d53b5577fd53379d913b9ab9ac82660" +git-tree-sha1 = "66b20dd35966a748321d3b2537c4584cf40387c7" uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d" -version = "2.3.1" +version = "2.3.2" [[deps.Printf]] deps = ["Unicode"] @@ -1096,9 +1096,9 @@ uuid = "92933f4c-e287-5a05-a399-4b506db050ca" version = "1.10.0" [[deps.PtrArrays]] -git-tree-sha1 = "077664975d750757f30e739c870fbbdc01db7913" +git-tree-sha1 = "f011fbb92c4d401059b2212c05c0601b70f8b759" uuid = "43287f4e-b6f4-7ad1-bb20-aadabca52c3d" -version = "1.1.0" +version = "1.2.0" [[deps.QuadGK]] deps = ["DataStructures", "LinearAlgebra"] @@ -1157,9 +1157,9 @@ version = "1.3.4" [[deps.RecursiveArrayTools]] deps = ["Adapt", "ArrayInterface", "DocStringExtensions", "GPUArraysCore", "IteratorInterfaceExtensions", "LinearAlgebra", "RecipesBase", "SparseArrays", "StaticArraysCore", "Statistics", "SymbolicIndexingInterface", "Tables"] -git-tree-sha1 = "758bc86b90e9fee2edc4af2a750b0d3f2d5c02c5" +git-tree-sha1 = "2cea01606a852c2431ded77293eb533b511b19e6" uuid = "731186ca-8d62-57ce-b412-fbd966d074cd" -version = "3.19.0" +version = "3.22.0" [deps.RecursiveArrayTools.extensions] RecursiveArrayToolsFastBroadcastExt = "FastBroadcast" @@ -1243,10 +1243,10 @@ uuid = "26aad666-b158-4e64-9d35-0e672562fa48" version = "0.1.1" [[deps.SciMLBase]] -deps = ["ADTypes", "ArrayInterface", "CommonSolve", "ConstructionBase", "Distributed", "DocStringExtensions", "EnumX", "FunctionWrappersWrappers", "IteratorInterfaceExtensions", "LinearAlgebra", "Logging", "Markdown", "PrecompileTools", "Preferences", "Printf", "RecipesBase", "RecursiveArrayTools", "Reexport", "RuntimeGeneratedFunctions", "SciMLOperators", "SciMLStructures", "StaticArraysCore", "Statistics", "SymbolicIndexingInterface", "Tables"] -git-tree-sha1 = "265f1a7a804d8093fa0b17e33e45373a77e56ca5" +deps = ["ADTypes", "Accessors", "ArrayInterface", "CommonSolve", "ConstructionBase", "Distributed", "DocStringExtensions", "EnumX", "FunctionWrappersWrappers", "IteratorInterfaceExtensions", "LinearAlgebra", "Logging", "Markdown", "PrecompileTools", "Preferences", "Printf", "RecipesBase", "RecursiveArrayTools", "Reexport", "RuntimeGeneratedFunctions", "SciMLOperators", "SciMLStructures", "StaticArraysCore", "Statistics", "SymbolicIndexingInterface", "Tables"] +git-tree-sha1 = "1d1d1ff37d2917cad263fa186cbc19ce4b587ccf" uuid = "0bca4576-84f4-4d90-8ffe-ffa030f20462" -version = "2.38.0" +version = "2.40.0" [deps.SciMLBase.extensions] SciMLBaseChainRulesCoreExt = "ChainRulesCore" @@ -1355,9 +1355,9 @@ version = "1.4.2" [[deps.StatisticalTraits]] deps = ["ScientificTypesBase"] -git-tree-sha1 = "30b9236691858e13f167ce829490a68e1a597782" +git-tree-sha1 = "983c41a0ddd6c19f5607ca87271d7c7620ab5d50" uuid = "64bff920-2084-43da-a3e6-9bb72801c0c9" -version = "3.2.0" +version = "3.3.0" [[deps.Statistics]] deps = ["LinearAlgebra", "SparseArrays"] @@ -1417,9 +1417,9 @@ version = "7.2.1+1" [[deps.SymbolicIndexingInterface]] deps = ["Accessors", "ArrayInterface", "RuntimeGeneratedFunctions", "StaticArraysCore"] -git-tree-sha1 = "b479c7a16803f08779ac5b7f9844a42621baeeda" +git-tree-sha1 = "a5f6f138b740c9d93d76f0feddd3092e6ef002b7" uuid = "2efcf032-c050-4f8e-a9bb-153293bab1f5" -version = "0.3.21" +version = "0.3.22" [[deps.TOML]] deps = ["Dates"] @@ -1484,10 +1484,10 @@ version = "0.4.82" Referenceables = "42d2dcc6-99eb-4e98-b66c-637b7d73030e" [[deps.Turing]] -deps = ["ADTypes", "AbstractMCMC", "Accessors", "AdvancedHMC", "AdvancedMH", "AdvancedPS", "AdvancedVI", "BangBang", "Bijectors", "DataStructures", "Distributions", "DistributionsAD", "DocStringExtensions", "DynamicPPL", "EllipticalSliceSampling", "ForwardDiff", "Libtask", "LinearAlgebra", "LogDensityProblems", "LogDensityProblemsAD", "MCMCChains", "NamedArrays", "Printf", "Random", "Reexport", "Requires", "SciMLBase", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns"] -git-tree-sha1 = "cfb3b446a5e52e1da4cc71b77a9350c309c581f0" +deps = ["ADTypes", "AbstractMCMC", "Accessors", "AdvancedHMC", "AdvancedMH", "AdvancedPS", "AdvancedVI", "BangBang", "Bijectors", "Compat", "DataStructures", "Distributions", "DistributionsAD", "DocStringExtensions", "DynamicPPL", "EllipticalSliceSampling", "ForwardDiff", "Libtask", "LinearAlgebra", "LogDensityProblems", "LogDensityProblemsAD", "MCMCChains", "NamedArrays", "Optimization", "OptimizationOptimJL", "OrderedCollections", "Printf", "Random", "Reexport", "Requires", "SciMLBase", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns"] +git-tree-sha1 = "6ea505cb1829868b333f9615e4049d7c83e97ce7" uuid = "fce5fe82-541a-59a6-adf8-730c64b5f9a0" -version = "0.32.0" +version = "0.33.0" [deps.Turing.extensions] TuringDynamicHMCExt = "DynamicHMC" @@ -1516,9 +1516,9 @@ version = "0.2.1" [[deps.UnsafeAtomicsLLVM]] deps = ["LLVM", "UnsafeAtomics"] -git-tree-sha1 = "323e3d0acf5e78a56dfae7bd8928c989b4f3083e" +git-tree-sha1 = "d9f5962fecd5ccece07db1ff006fb0b5271bdfdd" uuid = "d80eeb9a-aca5-4d75-85e5-170c8b632249" -version = "0.1.3" +version = "0.1.4" [[deps.WoodburyMatrices]] deps = ["LinearAlgebra", "SparseArrays"] diff --git a/tutorials/docs-17-mode-estimation/index.qmd b/tutorials/docs-17-mode-estimation/index.qmd index 234b29415..f3eba5a6b 100755 --- a/tutorials/docs-17-mode-estimation/index.qmd +++ b/tutorials/docs-17-mode-estimation/index.qmd @@ -10,7 +10,7 @@ using Pkg; Pkg.instantiate(); ``` -In addition to sampling from the distributions of a statistical model, one may be interested in finding the most likely parameter values that maximise for instance the posterior distribution density function or the likelihood. This is called mode estimation. Turing provides support for two mode estimation techniques, [maximum likelihood estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) and [maximum a posterior](https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation) (MAP) estimation. +After defining a statistical model, in addition to sampling from its distributions, one may be interested in finding the parameter values that maximise for instance the posterior distribution density function or the likelihood. This is called mode estimation. Turing provides support for two mode estimation techniques, [maximum likelihood estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) and [maximum a posterior](https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation) (MAP) estimation. To demonstrate mode estimation, let us load Turing and declare a model: @@ -45,23 +45,23 @@ mle_estimate = maximum_likelihood(model) map_estimate = maximum_a_posteriori(model) ``` -The estimates are returned as instances of the `ModeResult` type. It has the fields `values` for the parameter values found and `lp` for the log probability at the optimum, as well as `f` for the objective function and `optim_result` for more detailed results of the optimisation procedure, such as convergence information. +The estimates are returned as instances of the `ModeResult` type. It has the fields `values` for the parameter values found and `lp` for the log probability at the optimum, as well as `f` for the objective function and `optim_result` for more detailed results of the optimisation procedure. ```{julia} @show mle_estimate.values -@show mle_estimate.lp +@show mle_estimate.lp; ``` ## Controlling the optimisation process -Under the hood `maximum_likelihood` and `maximum_a_posteriori` use the [Optimization.jl](https://github.com/SciML/Optimization.jl) package, which provides a unified interface to many other optimisation packages. By default Turing typically uses the [LBFGS](https://en.wikipedia.org/wiki/Limited-memory_BFGS) method from [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl), but we can easily change that, +Under the hood `maximum_likelihood` and `maximum_a_posteriori` use the [Optimization.jl](https://github.com/SciML/Optimization.jl) package, which provides a unified interface to many other optimisation packages. By default Turing typically uses the [LBFGS](https://en.wikipedia.org/wiki/Limited-memory_BFGS) method from [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl) to find the mode estimate, but we can easily change that: ```{julia} using OptimizationOptimJL: NelderMead -maximum_likelihood(model, NelderMead()) +@show maximum_likelihood(model, NelderMead()) using OptimizationNLopt: NLopt.LD_TNEWTON_PRECOND_RESTART -maximum_likelihood(model, NLopt.LD_TNEWTON_PRECOND_RESTART()) +@show maximum_likelihood(model, LD_TNEWTON_PRECOND_RESTART()); ``` The above are just two examples, Optimization.jl supports [many more](https://docs.sciml.ai/Optimization/stable/). @@ -71,7 +71,9 @@ We can also help the optimisation by giving it a starting point we know is close ```{julia} using ADTypes: AutoReverseDiff import ReverseDiff -maximum_likelihood(model, NelderMead(); initial_params=[0.1, 2], adtype=AutoReverseDiff()) +maximum_likelihood( + model, NelderMead(); initial_params=[0.1, 2], adtype=AutoReverseDiff() +) ``` When providing values to arguments like `initial_params` the parameters are typically specified in the order in which they appear in the code of the model, so in this case first `s²` then `m`. More precisely it's the order returned by `Turing.Inference.getparams(model, Turing.VarInfo(model))`. @@ -82,16 +84,18 @@ We can also do constrained optimisation, by providing either intervals within wh maximum_likelihood(model; lb=[0.0, -1.0], ub=[0.01, 1.0]) ``` -The arguments for lower (`lb`) and upper (`ub`) bounds follow the arguments of `Optimization.OptimizationProblem`, as do other parameters for providing bounds such as `cons`. Any extraneous keyword arguments given to `maximum_likelihood` or `maximum_a_posteriori` are passed to `Optimization.solve`. Some often useful ones are `maxiters` for controlling the maximum number of iterations and `abstol` and `reltol` for the absolute and relative convergence tolerances: +The arguments for lower (`lb`) and upper (`ub`) bounds follow the arguments of `Optimization.OptimizationProblem`, as do other parameters for providing [constraints](https://docs.sciml.ai/Optimization/stable/tutorials/constraints/), such as `cons`. Any extraneous keyword arguments given to `maximum_likelihood` or `maximum_a_posteriori` are passed to `Optimization.solve`. Some often useful ones are `maxiters` for controlling the maximum number of iterations and `abstol` and `reltol` for the absolute and relative convergence tolerances: ```{julia} -badly_converged_mle = maximum_likelihood(model, NelderMead(); maxiters=10, reltol=1e-9) +badly_converged_mle = maximum_likelihood( + model, NelderMead(); maxiters=10, reltol=1e-9 +) ``` We can check whether the optimisation converged using the `optim_result` field of the result: ```{julia} -@show badly_converged_mle.optim_result +@show badly_converged_mle.optim_result; ``` For more details, such as a full list of possible arguments, we encourage the reader to read the docstring of the function `Turing.Optimisation.estimate_mode`, which is what `maximum_likelihood` and `maximum_a_posteriori` call, and the documentation of [Optimization.jl](https://docs.sciml.ai/Optimization/stable/).