Reduce the number of objects allocated by SLM when listing the snapshots to retain

[tlrx]

The SLM retention clean up task SnapshotRetentionTask lists all snapshots in order to identify the snapshots to retain and the snapshots to delete. While doing so it retrieves the full snapshot information, including shard snapshot details and shard snapshot failures, to later only use snapshot metadata and snapshot timestamp to select the snapshots to retain. When the snapshots contain thousands of shards it represents of lot of objects that are unnecessary created, putting a lot of pressure on the garbage collector.

We should improve the way SLM retrieves snapshots to reduce the huge allocations of objects. David made some interesting suggestions in comments.

• Move SLM eligibility check #100044
• Inline SnapshotRetentionConfiguration#getSnapshotDeletionPredicate #100053
• Remove unused methods from SnapshotRetentionConfiguration #100054
• SnapshotDetails-based SnapshotRetentionConfiguration #100058
• More-SnapshotDetails-based SnapshotRetentionConfiguration #100065
• Compute maps up front in SnapshotRetentionTask #100070
• Compute SLM retention from RepositoryData #100092

[elasticsearchmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[DaveCTurner]

I was wondering if we should instead introduce a new master-node transport action, to be called by SLM, which calls BlobStoreRepository#getSnapshotInfo and SnapshotsService#deleteSnapshots itself, avoiding the need to accumulate the intermediate list of SnapshotInfo objects at all. We don't need most of the logic of the get-snapshots API here, and we could do some nicer things like more intelligent throttling if we weren't working directly with the client-facing get-snapshots and delete-snapshots actions.

[DaveCTurner]

Another thing worth considering is whether we could refine SnapshotInfo#fromXContentInternal to use parser.skipChildren() on fields we aren't going to use. That's going to save a huge amount of allocation for cases where we just drop those fields later on.

[tlrx]

That makes perfect sense, thanks David.