Problems with installing self-hosted sentry over http proxy

[goganchic]

Version

22.7.0

Steps to Reproduce

I have a server, which can access the Internet only through http proxy. To make Docker access the Internet I perform 2 steps.

Step 1

Create file /etc/systemd/system/docker.service.d/http-proxy.conf with such contents:

[Service]
Environment="HTTP_PROXY=http://proxy:3128"
Environment="HTTPS_PROXY=http://proxy:3128"
Environment="NO_PROXY=127.0.0.0/8"

Where proxy is my proxy host. Then I reload systemd with a command systemctl daemon-reload and restart docker: systemctl restart docker.service.

These actions allow docker to perform docker pull command using http proxy.

Step2

Create file ~/.docker/config.json with such contents:

{
 "proxies":
 {
   "default":
   {
     "httpProxy": "http://proxy:3128",
     "httpsProxy": "http://proxy:3128",
     "noProxy": "127.0.0.0/8"
   }
 }
}

Where proxy is my proxy host. It sets http_proxy, https_proxy and no_proxy env variables in the docker containers, so it makes it possible to execute curl and other commands that require Internet access.

After it I try to execute install.sh.

Expected Result

Everything works as described in the README and the documentation.

Actual Result

Installation fails.

apt-get fails for cron container. It does not use proxy env variables and requires to configure proxies in the /etc/apt/apt.conf. So it can be fixed with such apt.conf:

Acquire::http::proxy "http://proxy:3128/";
Acquire::https::proxy "http://proxy:3128/";

And cron/Dockerfile should be modified like this:

ARG BASE_IMAGE
FROM ${BASE_IMAGE}
USER 0
COPY apt.conf /etc/apt/apt.conf # <-- this line added
RUN apt-get update && apt-get install -y --no-install-recommends cron && \
    rm -r /var/lib/apt/lists/*
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

Even if I fix cron container problem - the installation fails again with such an error:

container for service "clickhouse" is unhealthy
An error occurred, caught SIGERR on line 3

In the docker-compose.yml there are such lines:

    healthcheck:
      test:
        [
          "CMD-SHELL",
          "wget -nv -t1 --spider 'http://localhost:8123/' || exit 1",
        ]

yandex/clickhouse-server:20.3.9.70 container has a wget from busybox, which ignores no_proxy env variable (take a look at this ticket for more details). So healthcheck command tries to make a request to the localhost though http proxy and fails.

It can be fixed with such command: HTTP_PROXY='' wget -nv -t1 --spider 'http://localhost:8123/' || exit 1.

[goganchic]

I can make a PR if the solutions from the message about looks OK.

[aminvakil]

Can you try the solution which has been provided in #1404 (comment) ?

[goganchic]

@aminvakil, it's just an alternative to ~/.docker/config.json. It does not solve all problems that I'd described

[aminvakil]

Honestly I don't agree having an apt.conf which we should keep an eye on its changes when there is an update to apt or distro we're using or ...

Also for the change you have mentioned for wget.

But I'm not maintainer of this repository after all :) You can wait and see maintainers' opinion about this if you want to.

[goganchic]

@aminvakil it's impossible to use some distro and do not pay attention to its package management system. apt-get does not respect http_proxy variable and it's required to set proxy with apt.conf file. E.g. apk from alpine uses http_proxy env variable and it's possible install packages through proxy.

if you know better solution for this problem, I'll be happy to discuss it.

If I'm not mistaken this repo is mentioned as an official way to setup self-hosted Sentry. I think that my setup is quite common for a lot of enterprises, so the possibility to install sentry without direct connection to the Internet is important, imho.

[chadwhitacre]

Just skimming so far ... is it possible that you could achieve what you want using the new enhance-image.sh hook?

[goganchic]

@chadwhitacre don't get me wrong, I can implement all required features with enhance-image.sh or just modifying the source code. I just want to make this feature work out of the box. As I can see there are some sort of proxy support in the source code, that is why I think that current behavior is a bug.

[aminvakil]

@chadwhitacre don't get me wrong, I can implement all required features with enhance-image.sh or just modifying the source code. I just want to make this feature work out of the box. As I can see there are some sort of proxy support in the source code, that is why I think that current behavior is a bug.

With your current solution, you should change apt.conf too and I disagree with adding an apt.conf to repository. But don't get me wrong, I don't disagree with making it easier rather than current solution.

Right now enhance-image.sh is what we think is good and easy way to do stuff in many environments, I'll be happy and open to hear if you have better ideas.

[goganchic]

I'd created PR #1543 without apt.conf in the repository. Please take a look.

[chadwhitacre]

If I'm not mistaken this repo is mentioned as an official way to setup self-hosted Sentry.

You are not mistaken. :)

I think that my setup is quite common for a lot of enterprises, so the possibility to install sentry without direct connection to the Internet is important, imho.
I just want to make this feature work out of the box. As I can see there are some sort of proxy support in the source code, that is why I think that current behavior is a bug.

Fair enough. #1026 is where we added http_proxy support to the curl container. You're right that there's precedent for this. It looks like we're finding a minimally invasive approach, will pick up on #1543 ...