Docker packaging tests break the matrix.

[alpar-t]

The docker tests added in #46599 fail packaging tests:

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+packaging-tests-unix-next/os=ubuntu-16.04-packaging/13/console

The docker networking issue is one that we can easily deal with but these tests also have to account for the fact that some platforms ( like centos 6 ) don't have docker installed.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Packaging)

[alpar-t]

Here's a failure for missing docker: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+packaging-tests-unix-next/os=oraclelinux-6-packaging/13/console

[alpar-t]

The method used to install docker in Vagrant doesn't work on the GCP images:

[alpar@alpar-ci-centos-6-1007-0936 ~]$     # Add repository
[alpar@alpar-ci-centos-6-1007-0936 ~]$     sudo yum-config-manager -y --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
docker-ce.repo                                                                                                                                                                                                                                           | 2.4 kB     00:00     
repo saved to /etc/yum.repos.d/docker-ce.repo
[alpar@alpar-ci-centos-6-1007-0936 ~]$ 
[alpar@alpar-ci-centos-6-1007-0936 ~]$     # Install Docker
[alpar@alpar-ci-centos-6-1007-0936 ~]$     sudo yum install -y docker-ce docker-ce-cli containerd.io
Loaded plugins: fastestmirror, security
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: mirror.mobap.edu
 * centos-sclo-rh: mirror.fileplanet.com
 * centos-sclo-sclo: bay.uchicago.edu
 * epel: fedora-epel.mirror.lstn.net
 * extras: mirror.rackspace.com
 * updates: mirror.genesisadaptive.com
docker-ce-stable                                                                                                                                                                                                                                         | 3.5 kB     00:00     
docker-ce-stable/primary_db                                                                                                                                                                                                                              |  34 kB     00:00     
Resolving Dependencies
--> Running transaction check
---> Package containerd.io.x86_64 0:1.2.6-3.3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: containerd.io-1.2.6-3.3.el7.x86_64
--> Processing Dependency: systemd for package: containerd.io-1.2.6-3.3.el7.x86_64
--> Processing Dependency: systemd for package: containerd.io-1.2.6-3.3.el7.x86_64
---> Package docker-ce.x86_64 3:19.03.2-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libseccomp >= 2.3 for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libsystemd.so.0(LIBSYSTEMD_209)(64bit) for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: systemd for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libseccomp.so.2()(64bit) for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libsystemd.so.0()(64bit) for package: 3:docker-ce-19.03.2-3.el7.x86_64
---> Package docker-ce-cli.x86_64 1:19.03.2-3.el7 will be installed
--> Finished Dependency Resolution
Error: Package: containerd.io-1.2.6-3.3.el7.x86_64 (docker-ce-stable)
           Requires: container-selinux >= 2:2.74
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libsystemd.so.0()(64bit)
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libsystemd.so.0(LIBSYSTEMD_209)(64bit)
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: systemd
Error: Package: containerd.io-1.2.6-3.3.el7.x86_64 (docker-ce-stable)
           Requires: systemd
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libseccomp >= 2.3
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libseccomp.so.2()(64bit)
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: container-selinux >= 2:2.74
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

It's strange that it work son Vagrant at all given that centos 6 doesn't have systemd and we have sudo systemctl enable --now docker in there.

[alpar-t]

I also tried enabling ip forwarding on centos 7 ( which does have docker ).
The docker image build works but the test still fails:
https://gradle-enterprise.elastic.co/s/drfn2zihbbsfg

I'm muted the tests in b958467

[pugnascotia]

I'm on it.