Use freelists instead of sync.Pool for value object recycling

[jpbetz]

Remove use of sync.Pool entirely and add a Allocator interface that works like so:

allocator := value.NewFreelistAllocator()
val := m.GetUsing(allocator, "key")
...
// when done with the object:
allocator.Free(val)

The freelists themselves are pooled. The typical usage is to get a freelist from a pool when starting a traversal (e.g. validationWalker or mergeWalker) and then returned at the end of the traversal.

Callers that don't opt in to use the allocator by calling the "Using" functions get heap allocation by default.

The eliminates the contention observed by sync.Pool particularly when concurrency is high.

Test: Compare cpu=32 RunParallel benchmark that calls ObjectToTyped and then Compare, just like updater.Update does
Baseline: master + #153

name                               old time/op    new time/op    delta
Compare/Pod/structured-32            64.6µs ± 8%    44.5µs ± 5%  -31.17%  (p=0.008 n=5+5)
Compare/Pod/unstructured-32          74.9µs ± 7%    66.0µs ±43%     ~     (p=0.151 n=5+5)
Compare/Node/structured-32            110µs ±11%      82µs ± 3%  -25.38%  (p=0.008 n=5+5)
Compare/Node/unstructured-32          126µs ± 5%     105µs ± 6%  -16.28%  (p=0.008 n=5+5)
Compare/Endpoints/structured-32      1.00ms ± 9%    0.85ms ± 6%  -14.92%  (p=0.008 n=5+5)
Compare/Endpoints/unstructured-32    2.04ms ± 5%    1.87ms ± 6%   -8.12%  (p=0.008 n=5+5)

name                               old alloc/op   new alloc/op   delta
Compare/Pod/structured-32            26.1kB ± 0%    26.5kB ± 0%   +1.42%  (p=0.008 n=5+5)
Compare/Pod/unstructured-32          59.6kB ± 0%    60.2kB ± 0%   +0.97%  (p=0.008 n=5+5)
Compare/Node/structured-32           54.3kB ± 0%    54.4kB ± 0%   +0.21%  (p=0.008 n=5+5)
Compare/Node/unstructured-32          106kB ± 0%     110kB ± 0%   +4.24%  (p=0.008 n=5+5)
Compare/Endpoints/structured-32       349kB ± 0%     332kB ± 0%   -4.84%  (p=0.008 n=5+5)
Compare/Endpoints/unstructured-32    2.04MB ± 0%    2.10MB ± 0%   +2.87%  (p=0.008 n=5+5)

name                               old allocs/op  new allocs/op  delta
Compare/Pod/structured-32               799 ± 0%       809 ± 0%   +1.25%  (p=0.008 n=5+5)
Compare/Pod/unstructured-32           1.23k ± 0%     1.26k ± 0%   +2.19%  (p=0.008 n=5+5)
Compare/Node/structured-32            1.62k ± 0%     1.63k ± 0%   +0.43%  (p=0.016 n=4+5)
Compare/Node/unstructured-32          2.45k ± 0%     2.62k ± 0%   +7.19%  (p=0.008 n=5+5)
Compare/Endpoints/structured-32       8.20k ± 0%     8.20k ± 0%     ~     (p=0.333 n=5+5)
Compare/Endpoints/unstructured-32     32.4k ± 0%     37.0k ± 0%  +14.03%  (p=0.008 n=5+5)

/sig api-machinery
cc @apelisse @jennybuckley @lavalamp

[apelisse]

This looks good, it's mostly mechanical after all. It clearly needs to be rebased/squashed though, thanks!

[jpbetz]

This looks good, it's mostly mechanical after all. It clearly needs to be rebased/squashed though, thanks!

Thanks for looking at it so quickly. I've squashed and reduced the copy-paste in the allocator code.

[jpbetz]

Rebased

[apelisse]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: apelisse, jpbetz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [apelisse]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment