Move compression config to ConnectionProfile

[Tim-Brooks]

This is related to #34483. It introduces a namespaced setting for
compression that allows users to configure compression on a per remote
cluster basis. The transport.tcp.compress remains as a fallback
setting. If transport.tcp.compress is set to true, then all requests
and responses are compressed. If it is set to false, only requests to
clusters based on the cluster.remote.cluster_name.transport.compress
setting are compressed. However, after this change regardless of any
local settings, responses will be compressed if the request that is
received was compressed.

[elasticmachine]

Pinging @elastic/es-distributed

[Tim-Brooks]

I want to make sure (in particular) that everyone agrees with this:

However, after this change regardless of any local settings, responses will be compressed if the request that is received was compressed.

This is because we do not really have a great to correlated incoming connections with settings based on cluster alias.

Second, I wanted to make sure everyone agrees that transport.tcp.compress being set to true means that everything (local and remote) is compressed. This is because it is being used as the fallback setting. With this change there is no way to only compress traffic to the local cluster. Is that something we want?

Additionally, it feels like transport.tcp.compress is a little outdated (from when we had multiple transport types). What do we feel about adding transport.compress setting that will slowly replace transport.tcp.compress?

@jasontedor @s1monw

[s1monw]

change LGTM. I want to clarify one thing, if we set transport.tcp.compress to true we compress all traffic in the local cluster. yet if we set it to false on a specific remote cluster we would not compress traffic to that cluster, correct?

[s1monw]

Additionally, it feels like transport.tcp.compress is a little outdated (from when we had multiple transport types). What do we feel about adding transport.compress setting that will slowly replace transport.tcp.compress?

I agree with this, lets do this in a separate change. we should get rid of the tcp part in all the places IMO.

[jasontedor]

@tbrooks8 Sharing the comment that I shared with you in another channel last night.

Additionally, it feels like transport.tcp.compress is a little outdated (from when we had multiple transport types). What do we feel about adding transport.compress setting that will slowly replace transport.tcp.compress?

I agree.

[jasontedor]

LGTM.

[Tim-Brooks]

yet if we set it to false on a specific remote cluster we would not compress traffic to that cluster, correct?

We will not compress requests to that cluster. If transport.tcp.compress we will compress all responses. As we do not have a way to correlate accepted server channels with settings for a specific cluster.