Watcher - workaround for potential deadlock (#47603)

7.3.0 fixes this issue by changing the locking strategy in #41451.
However, that change is not part of 6.x and the change here is
a minimal workaround to prevent the potential of deadlock.

This change will no longer retry failed bulk requests that go
through the BulkProcessor for Watcher. Specifically this removes
the retry logic when adding Watcher history and Triggered watches
when the Bulk request failed.

Related #47599

Author: jakelandis

x-pack/plugin/watcher/src/main/java/org/elasticsearch/xpack/watcher/Watcher.java

@@ -9,6 +9,7 @@
 import org.apache.logging.log4j.Logger;
 import org.elasticsearch.action.ActionRequest;
 import org.elasticsearch.action.ActionResponse;
+import org.elasticsearch.action.bulk.BackoffPolicy;
 import org.elasticsearch.action.bulk.BulkItemResponse;
 import org.elasticsearch.action.bulk.BulkProcessor;
 import org.elasticsearch.action.bulk.BulkRequest;
@@ -395,6 +396,7 @@ public void afterBulk(long executionId, BulkRequest request, Throwable failure)
             .setBulkActions(SETTING_BULK_ACTIONS.get(settings))
             .setBulkSize(SETTING_BULK_SIZE.get(settings))
             .setConcurrentRequests(SETTING_BULK_CONCURRENT_REQUESTS.get(settings))
+            .setBackoffPolicy(BackoffPolicy.noBackoff())
             .build();
 
         HistoryStore historyStore = new HistoryStore(bulkProcessor);