Add best_compression back to ILM policy

[dakrone]

We originally had the best_compression codec option implemented in ILM,
however, we removed it at the last minute. We should revisit adding this as an
action in ILM to allow for saving disk space.

Switching to best_compression does require the index to be closed, the setting
changed, then a force_merge applied to take advantage of the new codec setting.
That's something we will have to work out in the ILM lifecycle.

[elasticmachine]

Pinging @elastic/es-core-features

[shashvat-kedia]

@gwbrown I am new to this project and would like to contribute can I start with this issue?

[Vboivin]

Is this issue still active? If yes, I'll take it

[dakrone]

@Vboivin yep, it's still active, let me know if you need any assistance with it.

[Vboivin]

I do not think I will be able to do it, it can be taken by someone else

[davidhowell-tx]

Any update on adding this feature into ILM policies?

It appears this was removed because it requires the index to be closed. Is it possible that this option is available during the shrink portion of the warm phase? Since the data is being copied into a new index I assume this is the opportune time to apply a new compression setting?

[SivagurunathanV]

Is the issue still active?

[gwbrown]

This issue is still active, we’ve just had other issues that are higher priority. If this feature is important to you, 👍 the issue. Contributions are welcome too!

This would probably be an option on the force merge action, as a force merge is required to reformat the data with the higher compression codec.

[SivagurunathanV]

@dakrone 👋 I am working on this issue, I got stuck with the quote you mentioned in #32373 This option is only settable while the index is closed, and doesn't make sense for a force merge. Can you give more insight on this?

[dakrone]

@SivagurunathanV so in order to implement this, the step would need to be split into multiple steps (actions) that:

• close the index if necessary (only if doing best_compression)
• update the index setting
• open the index
• wait for the index to be green
• perform the force merge

The reason it didn't make sense for the force merge at the time is that since it requires the index to be closed, the data is unavailable while closed, so it should not be something easily done by a user because they may not be aware that it causes data to be unavailable for a time.

Hopefully that helps, personally I think this should be a separate step from force merge, but @gwbrown talked about it being an option here, Gordon do you have strong feelings either way?

[gwbrown]

I don't have strong feelings about this being an option on the force_merge action or its own thing. Since we need to do a force merge following setting best_compression, it's a little weird that there are two distinct actions that will do a force merge... but given that force merge is idempotent for a read-only index (and the same number of segments given to the force merge call), it's probably fine as long as we schedule the force merge action after the action that sets best_compression.