Enable transport compression between nodes to reduce DTS costs

[dakrone]

Summary

In order to reduce DTS costs for cross-zone data transfer, we should investigate whether we should enable transport compression between nodes by default.

We could consider using a lightweight compression algorithm between the coordinating node and the primary shard, so that bulk requests are compressed. We would want to investigate using a lightweight compression algorithm such as LZ4, which is light on CPU usage so that we don't have a deleterious impact on indexing throughput.

7.14

• Add option to only compress requests/responses primarily composed of source documents (Add additional transport compression options #74587)
  • transport.compress can now be set to indexing_data as an experimental option
• Add fast LZ4 as a transport compression option (Add additional transport compression options #74587)
  • transport.compression_scheme can be set to lz4 (experimental) or deflate (default)

7.15

• Ensure decompression works properly with content aggregation
• Enable compression on more indexing_data operations (resync, transport bulk action) (Ensure indexing_data is compressed appropriately #76321)
• Reintroduce remote cluster specific compression scheme settings
• Make nodes to respond with the same compression scheme they received (Respond with same compression scheme received #76372)
• Default cluster.remote.*.transport.compression_scheme to deflate if compression is explicitly enabled on remote cluster (Remote compression scheme default to deflate #76580)
• Update documentation to reflect the fact that this work is not experimental (Remove experimental label for compression settings #76441)

7.16

• Remove unnecessary checksumming when LZ4 compressing (Do not checksum on LZ4 compress #76666)
  • We use deflate (not GZIP) for transport compression which does not checksum. We are not reading checksum on LZ4 uncompress.
• Implement performance improvements using Java9 (Add LZ4 compressor with Java9 perf improvements #77153)
• Ensure licensing is correct on forked code (Change lz4 licenses to correct header #80165)

Maybe Later:

• Transition to LZ4 frame format opposed to custom lz4-java block format

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)