Short response for Bulk API

[knocte]

When using the bulk API with a lot of documents (say 100K), it's kind of pointless to get a response for each of those documents because the response is huge! Cannot we have a way to call this API and only receive a response that is either:

• Successful insertion of all documents (count).
• Some insertion failed (count of the ones that failed).
• All insertion failed (count).

I know that I could probably use Bulk UDP (and I'll probably give it a go soon), but UDP doesn't receive any response whatsoever, so it's like having black&white, but not any shade of grey.

Thanks!

[colings86]

@knocte Would your use-case be solved if the response of the Bulk API contained a field which showed the number of failed insertions? This way you will not need to parse the whole response if all are successful, you can just check the value of this field is 0. If the value is not 0 then you can parse the rest of the response to find the cause.

[clintongormley]

@colings86 we already have an error flag in the bulk response, but there is something to be said for returning a minimal body if all requests succeeded.

I wouldn't be in favour of not reporting failures though, and given that requests are ordered, that implies the need to return all responses.

[knocte]

Sorry, I no longer use ElasticSearch

[colings86]

So maybe we should have an option in the request to send back a minimal response if there are no errors and a full response if there is even 1 error.

[astefan]

I would be in favor of such a feature.
For example, if I use the search API and I execute a query and I'm only interested in the number of hits, not the actual hits as well, I add ?search_type=count and I don't get back the possibly long list of hits which don't actually interest me.

Maybe add an attribute for _bulk - something like "response_type". With three values: "full" - to get back what we get today for _bulk, "partial_with_failures" - minimal response if there are no errors and a full response if there is at least one error, "minimal" - minimal response no matter if there were errors or not.

[clintongormley]

I don't like the minimal response in case of errors, but I do like an optional short response if there were no errors.