Resolving of Elasticsearch Transport Client with Apache Ivy/Ant fails (since 5.4.0)

[uschindler]

Hi,

I updated a Ivy-based Ant project that uses the Elasticsearch Transport Client a minute ago. After updating the dependency to the transport client (5.3.2 => 5.4.0) suddenly the project failed to build. Reason was, that some transitive dependencies of Elasticsearch were missing (e.g. Lucene itsself, various others).

The reason for this is #23809: Previously all the transitive dependencies in ES were excluded "one by one" in the POM file (see exclusions on each dependency): http://repo1.maven.org/maven2/org/elasticsearch/elasticsearch/5.3.2/elasticsearch-5.3.2.pom

In 5.4.0 it now has a *:* exclusion instead: http://repo1.maven.org/maven2/org/elasticsearch/elasticsearch/5.4.0/elasticsearch-5.4.0.pom

This seems to work with Maven (I built a plugin that uses Maven to refer to Elasticsearch and its transitive deps), but Apache Ivy/Ant breaks horribly. It seems that a bug in Ivy (2.3.0 and 2.4.0) causes that the * matches not only the transitives of elasticsearch, but also the declared dependency itsself. This causes that Ivy ignores all dependencies in Elasticsearch that are marked with this *:* exclusison pattern.

I am quite sure this is a bug in Ivy, but I don't think that gets fixed in time (Ant's Ivy project is almost dead...?).

As Gradle also uses Ivy internally, are we sure this works with Gradle?

[uschindler]

There is already a bug open @ Ivy that @nik9000 also took place in last December. No activity 👎
https://issues.apache.org/jira/browse/IVY-1531

[uschindler]

See also #21170

[danielmitterdorfer]

The likely cause for this is #24056. @rjernst any thoughts on how we should proceed with this ticket (also considering the fact that Ivy might not ever fix this...)?

[rjernst]

My changes in #24056 were intentional, because without them the generated pom is broken: it contains two exclusions sections, one with the wildcard exclusion added by gradle, and the other with our explicit exclusions. Unfortunately I had indeed forgot about #21170, and since we have no tests for any downstream build tools (including gradle and maven!), I did not catch this earlier. It is possible to fix this by reverting #24056, but will also require some additional change to find and remove the exclusions tag added by gradle. But I am a bit concerned we will keep hitting random issues with build tools until we add tests (which will be a bit tricky) which consume our generated poms.

[yrodiere]

Is there any chance this could get fixed in a 5.4.1? The problem not only affects Apache Ivy users, but also maven-enforcer-plugin users (see #24626).

[jasontedor]

Is there any chance this could get fixed in a 5.4.1?

This will definitely not be addressed in 5.4.1, we are still considering what to do here.