S3 Repository Plugin testingConventions task is failing Unix Compatibility Tests

[jbaiera]

Affecting master and 7.x

master:
Failure: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=amazon/458/console
Scan: https://gradle.com/s/uwbrvy32tllms
Message:

:plugins:repository-s3:testingConventions FAILED
--
Test classes are not included in any enabled task (:plugins:repository-s3:testRepositoryCreds,:plugins:repository-s3:integTestECSRunner,:plugins:repository-s3:integTestRunner,:plugins:repository-s3:test):
* org.elasticsearch.repositories.s3.S3RepositoryThirdPartyTests

7.x:
Failure: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+multijob-unix-compatibility/os=centos-6/241/console
Scan: https://gradle.com/s/gtn2gln6obqqm
Message:

:plugins:repository-s3:testingConventions FAILED
--
Test classes are not included in any enabled task (:plugins:repository-s3:testRepositoryCreds,:plugins:repository-s3:integTestECSRunner,:plugins:repository-s3:integTestRunner,:plugins:repository-s3:test):
* org.elasticsearch.repositories.s3.S3RepositoryThirdPartyTests

This seems to be related to #42586 as it is referencing the new test added in that PR.

I was unable to reproduce this locally. It seems that this fails in CI because testingConventions is not aware of the new thirdPartyTest task. In my local instance it looks like testingConventions is aware of the new task, and thus passes just fine.

Fair warning: My local env is a Mac, so if this is a Unix specific issue that would make sense that it wouldn't reproduce.

Setting this as Core/Infra/Build because I can't determine if it's an issue with how testingConventions is executing or if it's something to do with the S3 build file.

[elasticmachine]

Pinging @elastic/es-core-infra

[original-brownbear]

@atorok could we get some help here please? :) I'm not 100% sure how to correctly fix this.
The issue to me seems that the lack of docker causes a test task (that will only work with docker or third party credentials enabled) not to be enabled conditionally and fail this check. Can we somehow exclude it from the check? (if we can't I guess worst case I could move the exclude into code via an Assume but that seems like a dirty fix)

[alpar-t]

Neither OS has docker, so the test in question would indeed be disabled automatically by the test fixtures plugin for this reason, but the plugin also disable the testing conventions task in this case, and this is indeed not reproducible locally when passing -Dtests.fixture.enabled=false ( which is the same as not having docker ).
Would have to dig deeper to see what's going on.

[mark-vieira]

I've opened #42719 which would technically fix the problem but I think there is a larger issue with our Docker detection logic that needs to be addressed.

[romseygeek]

This is failing gradlew precommit --parallel for me on 7.x on my macbook, even if I pass -Dtests.fixture.enabled=false. I can provide logs or debug info if that helps fix this?

[mark-vieira]

@romseygeek This is likely because you do not have Docker installed on your macbook.