docs: Bulk API Reference lacking response format and other details

[schmichael]

Correctly processing the response to a Bulk API call is incredibly important to ensuring data integrity, however the Bulk API reference docs 2.x only state:

The response to a bulk action is a large JSON structure with the individual results of each action that was performed.

The only description of that "large JSON structure" is in an example curl command at the top of the page which only shows what a single action's response might look like:

{"took":7,"items":[{"create":{"_index":"test","_type":"type1","_id":"1","_version":1}}]}

There is a ton to infer from that snippet:

• items is an array of the same length and order as the sent actions+documents?
• The response to index actions will indicate how they were applied? So for a new document the response type will be create but for an existing doc it would be...?
• What indicates an error?!

Other oddity: non-HTTP API?

Same page and small, so I don't think it's worth a separate issue but:

If using the HTTP API...

This is the only hint on the entire page that there's a non-HTTP bulk API. Pretty sure this phrase should just be removed?

Update: Aha! https://www.elastic.co/guide/en/elasticsearch/reference/2.1/breaking_20_removed_features.html#_bulk_udp

Final nit pick: Write consistency means...?

The write consistency docs only say enough shards must be active to satisfy write consistency concerns.

In other distributed systems I associate write consistency with how many nodes must commit a write before the write is considered successfully applied.

Does write consistency depend solely on cluster state (the mere availability of active shards), or does it actually block on the writes to remote hosts?

(Also with replicas=1 quorum should really mean the primary+replica both must be available, not just one of them... one and quroum should not be the same, but thanks for at least documenting that they are!)

[schmichael]

Just stumbled across this getting started doc which is a less detailed version of the reference doc except one nice additional detail:

When the bulk API returns, it will provide a status for each action (in the same order it was sent in)

The reference doc lacks the ordering part.

[schmichael]

Aha! Found the bulk API section of the Guide which includes more details for the request and response formats, but lacks a lot of other details. It also has a big Out of Date warning at the top.

If nothing else the guide should link to the reference and vice versa. This would at least make it easy for readers to find all of the details spread across these two pages.

[ahuff44]

+1; I just spent an hour debugging before I realized that client.bulk was never calling my error callback and was instead returning error info in the response

[debadair]

Need to incorporate the relevant information from the definitive guide into the reference.

[henningandersen]

#36441 improves the docs wrt. ordering of results.