Each call to the put-ingest-pipeline API causes a cluster state update

[DaveCTurner]

Elasticsearch version (bin/elasticsearch --version): 7.15.1

Plugins installed: []

JVM version (java -version): Bundled

OS version (uname -a if on a Unix-like system): Cloud

Description of the problem including expected versus actual behavior:

Each call to PUT /_ingest/pipeline/{id} results in a cluster state update (unless it times out in the master queue first). We should detect calls that don't change anything and skip the publication process in those cases. It'd also be nice to batch up multiple requests into a single cluster state update.

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[danhermann]

@DaveCTurner, I thought that was resolved in #78196. Are you seeing evidence that no-op ingest pipeline updates are still producing cluster state updates?

[DaveCTurner]

Ah. Hmm. Yes I see, that should address it in practice, although doing it like that doesn't do quite the right thing with concurrent requests, it really should be checked within the cluster state update task instead. I only looked at the cluster state update task, that's why I missed it.

However, the cluster in the linked issue has ~150 put-pipeline-filebeat-7.15.1-elasticsearch-deprecation-pipeline tasks in its queue, so there's something funny going on.

[DaveCTurner]

Oh never mind I see what's going on, the put-pipeline tasks are a red herring.