Provide Cardano database incremental snapshots needed for Amaru bootstrap #2704
Description
Why
The Amaru team is using Mithril snapshots of the Cardano database for the bootstrap of the node. In particular, they need to use snapshots from 3 epochs in the past in order to reconstruct the stake distribution snapshots. The aggregator is not exposing enough entries in the routes that list the available snapshots.
What
Provide access to entries from 3 previous epochs in the aggregator routes.
How
- Add a route
/artifact/cardano-database/epoch/{epoch}that allows the filtering of the snapshots given an epoch number:- with a maximum number of epochs in the past (5 epochs)
- support the expansion of
latestvalue for epoch - support the expansion of
latest-{offset}value for epoch (offset >= 0)
- Update the OpenAPI specs
- Add a function to list artifacts by epoch in the client library
- Extend the
cardano-db snapshot list --epoch $EPOCHcommand in the client CLI (only for backend v2) - Test in the e2e test?
- Update the client manual workflow in GitHub actions
- Update the documentation
- Extend the retention of the ledger state archives (ancillary) to the maximum window of epochs in the past in the storage infrastructure (update default value of
google_storage_bucket_max_agein infra) - Assess the performance of the route:
-
Do we need to add an index on thebeaconfield of the signed entity type? - Or add an
epochcolumn in the store?
-
Other option (discarded)
- Extend the number of entries in the route that list the incremental Cardano database snapshots
- Assess the performance impact on the aggregator
- Extend the retention of the ledger state archives (ancillary) to at least
3epochs in the past in the storage infrastructure