Get rid of `repositories.impl` settings totally

[dadoonet]

Today we have multiple ways to define settings when a user needs to create a repository:

• in elasticsearch.yml file using repositories.xxx prefix where xxx is the implementation name like s3, hdfs...
• when creating the repository itself with PUT _snaphot/repo

Based on discussion we have on #22762 and on #22301 and also internally, I think we should reduce the overall complexity and completely remove support for global settings like this.

Actually, the creation of a repository is most of the time a unique manual operation so providing settings at repository creation time does not bring a lot of complexity for the end user.

And that will simplify a lot our code and our tests.

The plan would be:

• Deprecate repositories.xxx settings in 5.x:

• Azure plugin (Deprecate global repositories.azure settings #22856)
• S3 plugin (Deprecate repositories.s3 settings #23278)

• Remove repositories.xxx settings in 6.x (master branch):

• Azure plugin (Remove global repositories.azure settings #23262)
• S3 plugin ([WIP] Remove repositories.s3 settings and deprecated settings #23276 has been closed. It will be done within another PR)

• Check other plugins:

• GCS plugin (not needed as already not supported)
• HDFS plugin (not needed as already not supported)

[clintongormley]

The URL and filesystem repos require whitelisting in the config file before they can be used. Are we happy with not requiring whitelisting for these repos?

[dadoonet]

Indeed. I should be more precise.

We should support settings either in config file or at the API level but not both.

[dadoonet]

Discussed during fix-it friday session, the initial reason we were supporting settings in both places was to run test and passing credentials in a specific elasticsearch.yml CI settings file which can contain the credentials for given platforms. But we are not using that actually so we can remove that support of settings on both places.

We just need to decide what are the settings which needs to stay in elasticsearch.yml file and at a API level.

For example, whitelisting paths or URLs is an administration ops responsability and should not be exposed to the developer so it should probably go only in the config file.

Note that when using config files instead of API settings for repositories must be consistent otherwise the behavior would change from one node to another which must be avoid. Wondering if that kind of global settings should go to the cluster state so same settings could shared between nodes?