5.0.2 debian package not able to build a docker image

[neoecos]

Elasticsearch version: 5.0.2

Plugins installed: []

JVM version:

OS version: debian jessie

Description of the problem including expected versus actual behavior:
The modification of the /proc inside the configuration of the debian package, makes docker unable to create the image.
Steps to reproduce:
Check:
docker-library/elasticsearch#144

Provide logs (if relevant):

[eht16]

A normal package upgrade in a LXC container is broken as well:

Setting up elasticsearch (5.0.2) ...
Couldn't write '262144' to 'vm/max_map_count', ignoring: Read-only file system
dpkg: error processing package elasticsearch (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 elasticsearch

[jasontedor]

@neoecos @eht16 I opened #21899; please let me know if this would suit your needs.

Also, please note the official Elasticsearch Docker image (the one built and supported by Elastic, not the one in Docker Hub that claims to be official but is not affiliated with nor supported by Elastic) does not have this problem. If you're interested, we have some docs and a GitHub repository.

[eht16]

@jasontedor thanks. Setting SKIP_SET_KERNEL_PARAMETERS would fit for me.
Even more convenient would be: /usr/lib/systemd/systemd-sysctl || true
Then setting the value would fail, the error message is shown but the installation will continue.
Maybe an additional hint would be useful in the case systemd-sysctl failed to explain to the user that the shown error message is harmless (as long as the user is aware to set vm.max_map_count herself on the host).

But I'm not really sure what is the better solution, just thinking.

[jasontedor]

Thanks for the feedback @eht16. I considered that approach, and my concern with it is a lot of users do not read console messages when nothing goes wrong (so if the install does not exit with an error status, such users will just not see that setting the kernel parameters did not take). The upside to that approach is that it is less likely to get in the way of the end-user.

That said, I'm glad to know that we have two viable approaches that would be acceptable to at least one of you and @neoecos. I will solicit feedback from others (e.g., @clintongormley) but either way we will get a fix in for you very soon.

[neoecos]

@jasontedor The setting solves the issue, thank you.

The difference between the two builds, is that the dockerhub image is build using the official debian package, meanwhile the elasticsearch image just downloads the binary .tar.gz.

For me, using a deb package is definitely more elegant way to do that.

RUN wget ${ES_DOWNLOAD_URL}/elasticsearch-${ELASTIC_VERSION}.tar.gz &&
EXPECTED_SHA=$(wget -O - ${ES_DOWNLOAD_URL}/elasticsearch-${ELASTIC_VERSION}.tar.gz.sha1) &&
test $EXPECTED_SHA == $(sha1sum elasticsearch-${ELASTIC_VERSION}.tar.gz | awk '{print $1}') &&
tar zxf elasticsearch-${ELASTIC_VERSION}.tar.gz &&
chown -R elasticsearch:elasticsearch elasticsearch-${ELASTIC_VERSION} &&
mv elasticsearch-${ELASTIC_VERSION}/* . &&
rmdir elasticsearch-${ELASTIC_VERSION} &&
rm elasticsearch-${ELASTIC_VERSION}.tar.gz

[jasontedor]

The difference between the two builds, is that the dockerhub image is build using the official debian package, meanwhile the elasticsearch image just downloads the binary .tar.gz.

Yes, I'm aware of what accounts for the difference. The points that I'm alluding to and making are the following:

• the official Elastic images will work when new releases are made, rather than having to adjust to upstream changes such as the one that led to the problem here for downstream maintainers
• the official Elastic images are supported by Elastic, the downstream images are not

[C0rn3j]

For those stumbling upon this and being confused like me- the setting merged ended up being ES_SKIP_SET_KERNEL_PARAMETERS, so set that to true if you want to ignore setting kernel parameters.

[jasontedor]

@C0rn3j Thanks. This is also documented on the deb and RPM installation pages.

[wwentland]

@jasontedor I do not quite understand why the issue was fixed in such a way that sysctl is called at all even if /proc isn't writable. It looks as if this is a common issue (cf. #8793) and should be easy to fix. For now it is rather cumbersome to use ES in containers. Requiring every user to ensure that ES_SKIP_SET_KERNEL_PARAMETERS is set in the environment is quite suboptimal and might not be easy to achieve if the containers are provisioned automatically.

I would recommend to not call sysctl if /proc isn't writable. One might want to document that users can, for example, run their docker containers in privileged mode.

[jasontedor]

I do not quite understand why the issue was fixed in such a way that sysctl is called at all even if /proc isn't writable.

We abhor silent failures. Either it succeeds, or it fails and you know about it and you take steps to correct it.

Requiring every user to ensure that ES_SKIP_SET_KERNEL_PARAMETERS is set in the environment is quite suboptimal and might not be easy to achieve if the containers are provisioned automatically.

I don't understand this, just bake it into the provisioning.