ILM Actions can fail due to in-progress snapshots of indices

[talevy]

situation

Actions like: Delete, Freeze, Shrink all execute ES actions like
DeleteIndex and CloseIndex that throw IllegalArgumentExceptions in the cases
that snapshots of their respective indices are in progress.

What can ES do about it?

A SnapshotInProgressException can be thrown instead of an IAE so
that ILM can be signaled more explicitly that this is the case so that
it can retry actions appropriately. Actions should not move to an error
step.

[elasticmachine]

Pinging @elastic/es-core-features

[dakrone]

We had a discussion about this. For now we have a plan to see if we can address this:

• Change actions that can't happen while a snapshot is in progress to throw a SnapshotInProgressException instead of an IllegalArgumentException
• From the steps that perform unsafe-while-snapshotting behavior, catch this new exception
• If the exception is caught, register a ClusterStateObserver listener that attempts to re-run the step when the snapshot for the index is no longer in progress

[talevy]

• Change actions that can't happen while a snapshot is in progress to throw a SnapshotInProgressException instead of an IllegalArgumentException

I opened a PR for this here: #37550

[talevy]

I've also opened #37552 to codify the problem