Incorrect pagination with unsigned long in comp agg

[bpintea]

Doing a composite terms agg on an unsigned_long field seems to exhibit some inconsistent behaviour:

• it can either not return all the terms; or
• it can return excess values and "get stuck" on the search_after key.

With test mapping:

{
  "mappings": {
    "properties": {
      "ulong": {
        "type": "unsigned_long"
      }
    }
  }
}

and data

{"index": {}}
{"ulong": 9223372036854775808}
{"index": {}}
{"ulong": 3}
{"index": {}}
{"ulong": 2}
{"index": {}}
{"ulong": 1}
{"index": {}}
{"ulong": 0}

running:

1. the following query with a narrow size window (of 2):

{
    "size": 0,
    "_source": false,
    "stored_fields": "_none_",
    "aggregations": {
        "groupby": {
            "composite": {
                "size": 2,
                "sources": [
                    {
                        "756fe9d8": {
                            "terms": {
                                "field": "ulong",
                                "missing_bucket": true,
                                "order": "asc"
                            }
                        }
                    }
                ]
            }
        }
    }
}

will correctly return the first two buckets for 0, 1 and the latter as the search after key:

            "after_key": {
                "756fe9d8": 1
            },

However, using that after value in a subsequent request returns no further buckets; vs. expected following buckets.

2. Running the same query from above, but with a large "enough" size (like 6), will correctly return all values in the index for the term, and a search after key, set to the highest value:

            "after_key": {
                "756fe9d8": 9223372036854775808
            },

Now if the search is repeated with the adjusted after value (see below), the response will contain again values (all except the first one, 0) and the same after_key; vs expected no further values.
Furthermore, if the request is repeated, the same answer is provided; which would result in an infinite pagination loop.

{
    "size": 0,
    "_source": false,
    "stored_fields": "_none_",
    "aggregations": {
        "groupby": {
            "composite": {
                "size": 6,
                "sources": [
                    {
                        "756fe9d8": {
                            "terms": {
                                "field": "ulong",
                                "missing_bucket": true,
                                "order": "asc"
                            }
                        }
                    }
                ]
                ,"after": {"756fe9d8": 9223372036854775808}
            }
        }
    }
}

[elasticmachine]

Pinging @elastic/es-search (Team:Search)