Improved percolation api

[felipellrocha]

Hey guys,

I was working with the percolator earlier today, and I thought of a nice way to improve its api. So, right now, if I register a query using the percolator:

curl -XPUT localhost:9200/_percolator/test/kuku -d '{
    "query" : {
        "term" : {
            "field1" : "value1"
        }
    }
}'

and I percolate a matching document

curl -XGET localhost:9200/test/type1/_percolate -d '{
    "doc" : {
        "field1" : "value1"
    }
}'

I get something of the effect back:

{"ok":true, "matches":["kuku"]}

Which a fine way of doing things, but I think it'd be nicer to be able to register metadata along with the query. So, something of the effect:

curl -XPUT localhost:9200/_percolator/test/ -d '{
    "query" : {
        "term" : {
            "field1" : "value1"
        }
    },
    "meta": {
        "campaign": 1,
        "user_id": 1,
        "user_name": kuku
    }
}'

If a document matched, I'd get something like this back:

{"ok":true, "matches":[{"_id": "BzA9dR7OSqGylX-0y80IJA", "campaign": 1, "user_id": 1, "user_name": kuku}]}

Currently, if I need to store more information about a query, I need to do hacks, such as concatenating strings with all the information I want to get back from that match (example: "campaign:1-id:1-name:kuku") to use as the id of that document, but that has drawbacks.

What do you guys think?

[martijnvg]

@felipellrocha With the current and the redesigned percolator register you can add metadata with the query just as regular fields since it is treated as a document. Since the percolator returns ids, you can just execute a subsequent get or multi get request to ES to get the documents with that metadata, which just gets what you want?

[felipellrocha]

That's fine and all, it's just a little roundabout. Why do it in two api calls, when you can in one? I just feel that would be a more elegant way of doing things. If es already knows which document matched, why not return the meta information that goes along with it, you know?

[martijnvg]

Yes, including the meta fields of a match make sense. I guess the percolate api should have the kind of fields support as is available in the search api.

[felipellrocha]

I like that. It'd save a lot of network traffic in cases you know which fields you're looking for, and it'd make for a consistent api throughout.

[julianhille]

we currently solve that through the percolator id. We store there joined by "_" the information we need. Would be awesome to return that within the response and without parsing the id.

[jeantil]

I am not sure what the "adoptme" label means, but we would definitely welcome a "fields" param on percolation requests to get additional information instead of encoding them in the id or doing additional requests to get them

[martijnvg]

@jeantil It means that this issue has been discussed and that it is going to be added to ES. This is not really evident from the comments... but I've heard this feature request often, so it should just be added to the percolator, so I'm adding the 2.0 label to the issue, just to make it more visible and likely to get it in sooner.