Automatic Differentiation with coloring #17
Conversation
| using ForwardDiff: Dual, jacobian | ||
|
|
||
| fcalls = 0 | ||
| function f(dx,x) |
There was a problem hiding this comment.
Prefer 4 spaces for indentation
| end | ||
|
|
||
| p = Tuple.(eachrow(partials_array)) | ||
| t = zeros(Dual{Nothing, Float64, maximum(color)},0) |
There was a problem hiding this comment.
| t = zeros(Dual{Nothing, Float64, maximum(color)},0) | |
| t = zeros(Dual{typeof(f), Float64, maximum(color)},length(x)) |
It's usually better to pre-allocate. The first part here being typeof(f) will tag the Duals so they won't accidentally overlap with other differentiations.
| p = Tuple.(eachrow(partials_array)) | ||
| t = zeros(Dual{Nothing, Float64, maximum(color)},0) | ||
| for i in 1:length(x) | ||
| push!(t, Dual(x[i], p[i])) |
There was a problem hiding this comment.
| push!(t, Dual(x[i], p[i])) | |
| t[i] = Dual(x[i], p[i]) |
| end | ||
| end | ||
|
|
||
| J = Array(J) |
|
So there's quite a few things that can be improved performance-wise, but it makes sense to do that in a follow up PR. Ideally we would just allocate everything upfront, and let the user pass in a cache that will store everything. Then we just need to use those cache variables to effectively do the computation without allocating. I'll merge and at least do a partial PR to show what I mean by this. Additionally, we should be using chunk sizes. I find https://github.com/FluxML/Tracker.jl/blob/master/src/forward.jl a good reference on how to implement that. That will be easier to implement when we have a cache. So I'm going to merge and we can keep handling this. I'll fix the tab spacing because that was my doing. |
Added function
compute_jacobian!to compute Jacobian matrices using a coloring vector. Also added tests.