ingest: upsert with drop processor returns error

[jakelandis]

If performing an upsert with a default pipeline, and that default pipeline drops the document, an error will be returned to the client.

This is a minor bug since the end result is the same, the document is not indexed. However, it can pollute error logs, cause confusion, and have a minor performance penalty due to the exception and logging.

Both normal upsert and bulk upsert will error, but with different signatures

DELETE test
PUT test
{
  "settings": {
    "index.default_pipeline": "dropper"
  }
}
PUT _ingest/pipeline/dropper
{
  "processors": [
    {
      "drop": {}
    }
  ]
}

Non-bulk upsert:

POST test/doc/1/_update
{
  "script":{
    "source": "ctx._source.foo = 'bar'" 
  },
  "upsert" :{
    "foo" : "bar"
  }
}

Results in

{
  "error": {
    "root_cause": [
      {
        "type": "remote_transport_exception",
        "reason": "[node-0][127.0.0.1:9300][indices:data/write/update[s]]"
      }
    ],
    "type": "class_cast_exception",
    "reason": "class org.elasticsearch.action.update.UpdateResponse cannot be cast to class org.elasticsearch.action.index.IndexResponse (org.elasticsearch.action.update.UpdateResponse and org.elasticsearch.action.index.IndexResponse are in unnamed module of loader 'app')"
  },
  "status": 500
}

and bulk upsert results in

    {
      "index" : {
        "_index" : null,
        "_type" : null,
        "_id" : null,
        "status" : 500,
        "error" : {
          "type" : "null_pointer_exception",
          "reason" : null
        }
      }
    }

[elasticmachine]

Pinging @elastic/es-core-features

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[joegallo]

Here's a note for my own benefit, this is what an appropriate _bulk would look like -- it took me a couple of minutes to piece it together.

POST _bulk
{ "update" : { "_index" : "test-1", "_id" : "1" } }
{ "upsert" : { "some" : "fields" }, "script" : "ctx" }
{ "update" : { "_index" : "test-2", "_id" : "1" } }
{ "upsert" : { "some" : "fields" }, "script" : "ctx" }

[joegallo]

On more recent versions the _update API has moved, so the non-bulk upsert looks like this now:

POST test/_update/1
{
  "script":{
    "source": "ctx._source.foo = 'bar'"
  },
  "upsert" :{
    "foo" : "bar"
  }
}