Java High Level REST Client plan for first release

[javanna]

This is a meta issue to track work that's being done on the Java High Level REST Client.

See https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients to know more about the state of the official Java clients and the plan moving forward.

The RestHighLevelClient will allow to reuse the same request objects (ActionRequest subclasses) and responses (ActionResponse subclasses) as the current transport client. The client object though will not be the same and the new one will not implement the current Client interface (see #9201 to know why).

The Java High Level REST Client label can be followed to track progress on the high level REST client, the following are the apis that we want to support for its first release:

• ping api (HEAD localhost:9200/)
• info api (GET localhost:9200/)
• get api
• index api
• update api
• delete api
• bulk api
• search api
  • hits
  • suggest
  • profile
  • aggregations
    • single bucket (Add parsing for single bucket aggregations #24564)
      • filter
      • children
      • nested
      • reverse nested
      • missing
      • global
      • sampler
    • multi bucket
      • terms (Add parsing for String/Long/Double Terms aggregations #24521)
      • histogram (Add parsing methods for InternalDateHistogram and InternalHistogram #24213)
      • date histogram (Add parsing methods for InternalDateHistogram and InternalHistogram #24213)
      • adjacency matrix (Add parsing for InternalAdjacencyMatrix aggregation #24700)
      • filters (Add parsing for InternalFilters aggregation #24648)
      • range (Add parsing methods to Range aggregations #24583)
      • date range (Add parsing methods to Range aggregations #24583)
      • binary range (Add parsing method for binary range aggregation #24706)
      • geohash grid (Add parsing method to GeoHashGrid aggregation #24589)
      • geodistance (Add parsing methods to Range aggregations #24583)
      • significant terms (Add parsing to Significant Terms aggregations #24682)
    • geo centroid
    • geo bounds
    • top hits (Add parsing method for Top Hits aggregation #24717)
    • scripted metric (Add parsing for InternalScriptedMetric aggregation #24738)
    • matrix stats (Add parsing method for Matrix Stats #24746)
    • numeric metrics
      • stats
      • extended stats
      • stats bucket
      • extended stats
      • min
      • max
      • avg
      • sum
      • value count
      • simple value
      • derivative
      • bucket metric value
      • cardinality
      • tdigest percentiles
      • hdr percentiles
      • tdigest percentile_ranks
      • hdr percentile_ranks
      • percentiles bucket
• search scroll api
• clear scroll api

[falsyvalues]

I'm not sure if its correct place for that... Does HTTP Client will have native client features like custer sniffing and other cluster related features?

[dadoonet]

@falsyvalues ideally ask on discuss.elastic.co.
HTTP Client already has sniffing with https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/sniffer.html

Not related to the high level client.

[moodysalem]

Could you make it a short term goal to pull out the model objects (e.g. SearchSourceBuilder, QueryBuilders) into their own maven module?

Right now the existing REST client, Jest, suggests adding the transport module as a dependency to use these request objects/builders, which in my case adds 20mb to my otherwise <10mb bundle-which is especially important because I'm using AWS lambda and a bigger bundle takes longer to get started.

Also, separating out these objects makes it easier to use your own HTTP client if you'd like to-IMO the request and response POJOs are much more useful than the client that abstracts making HTTP requests with those objects and deserialization/serialization, since there are already plenty of java HTTP clients and JSON serialization/deserialization libraries. Also in my case I need to configure the HTTP client to sign requests

EDIT: I am playing with separating out the model classes and it does seem like the model classes have a lot of dependencies on lucene and elasticsearch common classes which could make that a lot more difficult than I expected

2nd EDIT: I was able to get around bringing in the elasticsearch dependency building my query using JsonObjects and JsonArrays with a helper class

3rd EDIT: for anyone who runs into the same scenario as me.. here is the helper class that I used to build the query object https://gist.github.com/moodysalem/585033149bb85f1e6079f1d507f8c72d

[javanna]

hi @moodysalem I agree that ideally the Java client should not depend on Elasticsearch. We would love not to have that but we gave high priority to have the high level REST client out there sooner rather than later, although it will still depend on Elasticsearch initially. Once it's out, we may decide to work on taking those requests and response classes out to a common library, but as you found out that will require quite some work, which is why we decided not to do it straight-away but rather go step by step.

[s1monw]

one thing that I can see being a low hanging fruit is that we can move out our core analyzers into a module. This would allow us to move out the analyzers-common JAR that is ~ 11MB which would help a lot with this issue already. I don't think breaking out SearchSourceBuilder is a low hanging fruit at all it has lots of dependencies, I also would rather want to make core smaller and keep on depending on it than doing it the other way round. There is also highlighters that can be moved in to a module or maybe even into the same module for simplicity. Suggesters is a similar thing.. I think we should look into this! @rjernst WDYT

[nik9000]

I'd be happy to move the analyzers, suggesters, and highlighters into their own modules. I think it'd be faster to exclude their dependencies from the high level client, though a bit less clean.

[s1monw]

++ @nik9000