[v20.10] Platform architecture detection is broken

[shaderecker]

• This is a bug report
• This is a feature request
• I searched existing issues before opening this one

Expected behavior

Docker correctly identifies and uses the image's platform architecture.
In my example: It should detect and use armhf on the Raspberry Pi.

Actual behavior

Docker complains about the image platform arch.
This happens only since docker-ce version 20.10. Up until 19.03.14 this still worked correctly (or at least there was no complaint/warning).

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested

Steps to reproduce the behavior

For details see here: pi-hole/docker-pi-hole#735
In short:

1. Install Docker 20.10 on Raspbian Buster
2. Run a test command docker run -it --rm --entrypoint pihole-FTL pihole/pihole:v5.2.1 -vv
3. You will see the warning

The image/tag from the test command is a multi-arch image, so normally it should auto-select the armhf arch from that.
I also tried directly addressing this by explicitely using an armhf image (e.g. v5.2.1-armhf-buster), but this still produces the same warning.
Also notice: Even as this really is an armhf image, in Docker Hub it gets displayed as an amd64 image for some reason.

Output of docker version:

i@pihole-2:~ $ docker version
Client: Docker Engine - Community
 Version:           20.10.1
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        831ebea
 Built:             Tue Dec 15 04:35:27 2020
 OS/Arch:           linux/arm
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       f001486
  Built:            Tue Dec 15 04:33:02 2020
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

pi@pihole-2:~ $ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.0-docker)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 20.10.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.79-v7+
 Operating System: Raspbian GNU/Linux 10 (buster)
 OSType: linux
 Architecture: armv7l
 CPUs: 4
 Total Memory: 924.8MiB
 Name: pihole-2
 ID: 3FB3:6WLC:2DV7:QUL5:7MPF:SBBL:B2MT:BJUN:37X2:G4DD:DCBN:NF3R
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support
WARNING: No blkio weight support
WARNING: No blkio weight_device support
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional environment details (AWS, VirtualBox, physical, etc.)

• Raspberry Pi 3 Model B Plus Rev 1.3
• Docker is installed with an Ansible role which follows the official guide: https://github.com/shaderecker/ansible-pihole/blob/master/roles/docker/tasks/main.yaml

[thaJeztah]

Just to double-check; there was no image present yet before the docker run, so it was freshly pulled, correct?

Wondering if this is related to containerd/containerd#4530 - looking at that image; https://hub.docker.com/r/pihole/pihole/tags?page=1&ordering=last_updated&name=v5.2.1 I see it supports linux/amd64, linux/arm/v7, and linux/arm64/v8. If it's indeed related to that pull request, it may be trying to match a linux/arm/v6 (which is not available).

Could you post the output of

• uname -a
• uname -m
• cat /proc/cpuinfo

/cc @tianon

[thaJeztah]

As a workaround, you should probably be able to run the image when explicitly passing --platform=linux/arm/v7 on docker run / docker pull.

[gerrodlop]

Same problem here:

pi@node03:~ $ docker run -it --rm balenalib/raspberrypi4-64:sid-20200801 /bin/bash                                                        
Unable to find image 'balenalib/raspberrypi4-64:sid-20200801' locally                                                                     
sid-20200801: Pulling from balenalib/raspberrypi4-64
6f4fd5193f3b: Pull complete
6fc3f88940fc: Pull complete
35c55595a169: Pull complete
6a0d4aaa43d8: Pull complete
e09b3e013617: Pull complete
611e6f942dc1: Pull complete
05b547aa64aa: Pull complete
8e1177087eb0: Pull complete
97afb9e2bf6c: Pull complete
0144ae10cee1: Pull complete
e9f1ccb76158: Pull complete
bbcbd264b7cb: Pull complete
a5f6e3179a06: Pull complete
ee061025e236: Pull complete
1047c2becd1a: Pull complete
b27718082afc: Pull complete
Digest: sha256:65ce6d8e66926134dfa0e2acfa47030f8977245b14d1d77c0860423965076292                                                           
Status: Downloaded newer image for balenalib/raspberrypi4-64:sid-20200801                                                                 
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
root@941d8be89e48:/# 

hello-world is ok:

pi@node03:~ $ docker run -it --rm hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
256ab8fe8778: Pull complete
Digest: sha256:1a523af650137b8accdaed439c17d684df61ee4d74feac151b5b337bd29e7eec                                                           
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.                                                                

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.                                                                 
    (arm64v8)
 3. The Docker daemon created a new container from that image which runs the                                                              
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it                                                            
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:                                                                    
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

[thaJeztah]

@gerrodlop looks like different issue; the balenalib/raspberrypi4-64:sid-20200801 is not a multi-arch image; looking at https://hub.docker.com/r/balenalib/raspberrypi4-64/tags?page=1&ordering=last_updated&name=sid-20200801, it's reported as linux/amd64;

I suspect the image is built incorrectly (specifying the wrong architecture)

[shaderecker]

@thaJeztah
Yes, I cleared all images before pulling during my testing.

uname -a

pi@pihole-2:~ $ uname -a
Linux pihole-2 5.4.79-v7+ #1373 SMP Mon Nov 23 13:22:33 GMT 2020 armv7l GNU/Linux

uname -m

pi@pihole-2:~ $ uname -m
armv7l

cat /proc/cpuinfo

pi@pihole-2:~ $ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 1
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 2
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 3
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

Hardware	: BCM2835
Revision	: a020d3
Serial		: 00000000f17aab07
Model		: Raspberry Pi 3 Model B Plus Rev 1.3

[shaderecker]

I just tried with:
pi@pihole-2:~ $ docker pull pihole/pihole:v5.2.1 --platform=linux/arm/v7

But then a docker image inspect pihole/pihole:v5.2.1 still shows:

"Architecture": "amd64",

[thaJeztah]

Interesting. That's odd. If I try to do the same for (e.g.) the hello-world image, it correctly pulls the variant I specify, but not for the pihole image

Thanks for the /proc/cpuinfo output #1170 (comment). Wondering if the issue is that the selection tries a case-sensitive match for Cpu architecture (therefore not matching CPU architecture) https://github.com/containerd/containerd/blob/2055e12953bb538228d8d9fe627fa545d7cf82be/platforms/cpuinfo.go#L93

That wouldn't explain why the docker pull with --platform doesn't match though 🤔

[thaJeztah]

Hmmm... so if I specify --platform=linux/arm64 I DO get a different image than when I specify --platform=linux/amd64;

docker pull --platform=linux/arm64 pihole/pihole:v5.2.1

[
    {
        "Id": "sha256:ddcd770c66339bae8f2e9f89940b7f1d3300538fff48e5443d1244ba21d63885",
        "RepoTags": [
            "pihole/pihole:v5.2.1"
        ],
        "RepoDigests": [
            "pihole/pihole@sha256:7888e89f4d435673c75896c546fefe3177379c99c8766c89be25a905f8def533"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2020-12-03T02:06:14.680174992Z",
        "Container": "35f3f315a63da09485218661dc7c1e335121dc5b3b10191c3ba2e8f3f5c718ac",
        "ContainerConfig": {
            "Hostname": "35f3f315a63d",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "443/tcp": {},
                "53/tcp": {},
                "53/udp": {},
                "67/udp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "ARCH=arm64",
                "UBUNTU_SUITE=buster",
                "DOCKER_REPO=multiarch/debian-debootstrap",
                "PIHOLE_ARCH=arm64",
                "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-aarch64.tar.gz",
                "PIHOLE_INSTALL=/root/ph_install.sh",
                "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
                "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
                "IPv6=True",
                "S6_LOGGING=0",
                "S6_KEEP_ENV=1",
                "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
                "ServerIP=0.0.0.0",
                "FTL_CMD=no-daemon",
                "DNSMASQ_USER=root",
                "VERSION=v5.2.1"
            ],
            "Cmd": [
                "/bin/bash",
                "-c",
                "#(nop) ",
                "SHELL [/bin/bash -c]"
            ],
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
                ]
            },
            "Image": "sha256:b4459ac3099e2fd70e9c2aa5ef710e853da11624d65f0b353f88da8c39dbd34f",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/s6-init"
            ],
            "OnBuild": null,
            "Labels": {
                "image": "pihole/pihole:v5.2.1_arm64",
                "maintainer": "adam@diginc.us",
                "url": "https://www.github.com/pi-hole/docker-pi-hole"
            },
            "Shell": [
                "/bin/bash",
                "-c"
            ]
        },
        "DockerVersion": "19.03.13+azure",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "443/tcp": {},
                "53/tcp": {},
                "53/udp": {},
                "67/udp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "ARCH=arm64",
                "UBUNTU_SUITE=buster",
                "DOCKER_REPO=multiarch/debian-debootstrap",
                "PIHOLE_ARCH=arm64",
                "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-aarch64.tar.gz",
                "PIHOLE_INSTALL=/root/ph_install.sh",
                "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
                "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
                "IPv6=True",
                "S6_LOGGING=0",
                "S6_KEEP_ENV=1",
                "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
                "ServerIP=0.0.0.0",
                "FTL_CMD=no-daemon",
                "DNSMASQ_USER=root",
                "VERSION=v5.2.1"
            ],
            "Cmd": null,
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
                ]
            },
            "Image": "sha256:b4459ac3099e2fd70e9c2aa5ef710e853da11624d65f0b353f88da8c39dbd34f",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/s6-init"
            ],
            "OnBuild": null,
            "Labels": {
                "image": "pihole/pihole:v5.2.1_arm64",
                "maintainer": "adam@diginc.us",
                "url": "https://www.github.com/pi-hole/docker-pi-hole"
            },
            "Shell": [
                "/bin/bash",
                "-c"
            ]
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 386958209,
        "VirtualSize": 386958209,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/5a3e83f7f79c72d97966cbf65f8667fc9ce69573b782785a65bb60f06fbf9d3e/diff:/var/lib/docker/overlay2/4513f8e22e61cf6940d46663389a823af08d99851a942effb447b94c6823918d/diff:/var/lib/docker/overlay2/71ddf41a3376105bf9cbaa86e15e824d43c690944d9517f7995105088422689e/diff:/var/lib/docker/overlay2/c4f94d77d21d3ba107a1eb733bb3c4dccfe583a8c25c078c712da75efeeec7af/diff:/var/lib/docker/overlay2/e8c92f5e4b9699403271db8bffdc008b02a7bf70fb01361687d8b3b5f8f85226/diff:/var/lib/docker/overlay2/975d9d9b0cd48b73553e80fd5ef1c111bb6614fc4d54ec4bfda90995918fa06c/diff:/var/lib/docker/overlay2/c31a030288e3818d99ba1d8fd75847d2e56c45e33af280d07850760155dfe021/diff",
                "MergedDir": "/var/lib/docker/overlay2/e180a4554865f5a757a84525b9db1c546be58d64ccdeb04ea60dda9a8da3fef5/merged",
                "UpperDir": "/var/lib/docker/overlay2/e180a4554865f5a757a84525b9db1c546be58d64ccdeb04ea60dda9a8da3fef5/diff",
                "WorkDir": "/var/lib/docker/overlay2/e180a4554865f5a757a84525b9db1c546be58d64ccdeb04ea60dda9a8da3fef5/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:cc77256f2d56e07985cba58506e433bb21ef7b3a06bd3a688ac6abfdfc010344",
                "sha256:c2c03ee92cd94dc4721023ebe9e6b1a1c6e3588a470d65bdabc589d68bd38395",
                "sha256:beb456e5587da0c7e993017a51b77dac721a6fc290b0c23b63fda47fa262488c",
                "sha256:038c127208f1ddfa51eab576f6de0f44c4c9991bcdb66e432d654d280db597f6",
                "sha256:7554d616a82bfa378acdc7ae9e5c3e885f29efb6cf06059071ca8648fbc9ec42",
                "sha256:110d37e83a24488a1565c2d8e5307f676a72a19d6779d0f1e97a7203f7aa03e2",
                "sha256:9657d8d1b438d67e784aaf3c96d95c943b9138efbd029453f6e7227d304c2553",
                "sha256:6fa10c09468998df7084f6affeb8d3d2b5e02ce2aaccc02249c48b41ee323392"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

docker pull --platform=linux/amd64 pihole/pihole:v5.2.1

[
    {
        "Id": "sha256:52e72f4b6a617b430cf14c6fa919eb477c48aee4f7f87cc1c55247c1d602a66f",
        "RepoTags": [
            "pihole/pihole:v5.2.1"
        ],
        "RepoDigests": [],
        "Parent": "",
        "Comment": "",
        "Created": "2020-12-03T01:59:33.41222977Z",
        "Container": "6c475f695cacb914ab84e7f07d978bd5180bbb6b5b102c099eebb185f54473e3",
        "ContainerConfig": {
            "Hostname": "6c475f695cac",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "443/tcp": {},
                "53/tcp": {},
                "53/udp": {},
                "67/udp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "PIHOLE_ARCH=amd64",
                "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-amd64.tar.gz",
                "PIHOLE_INSTALL=/root/ph_install.sh",
                "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
                "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
                "IPv6=True",
                "S6_LOGGING=0",
                "S6_KEEP_ENV=1",
                "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
                "ServerIP=0.0.0.0",
                "FTL_CMD=no-daemon",
                "DNSMASQ_USER=root",
                "VERSION=v5.2.1"
            ],
            "Cmd": [
                "/bin/bash",
                "-c",
                "#(nop) ",
                "SHELL [/bin/bash -c]"
            ],
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
                ]
            },
            "Image": "sha256:929daab554acb2a02ab991ec85bba37b6934eaad91adec262b35ab36b151a03c",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/s6-init"
            ],
            "OnBuild": null,
            "Labels": {
                "image": "pihole/pihole:v5.2.1_amd64",
                "maintainer": "adam@diginc.us",
                "url": "https://www.github.com/pi-hole/docker-pi-hole"
            },
            "Shell": [
                "/bin/bash",
                "-c"
            ]
        },
        "DockerVersion": "19.03.13+azure",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "443/tcp": {},
                "53/tcp": {},
                "53/udp": {},
                "67/udp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "PIHOLE_ARCH=amd64",
                "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-amd64.tar.gz",
                "PIHOLE_INSTALL=/root/ph_install.sh",
                "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
                "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
                "IPv6=True",
                "S6_LOGGING=0",
                "S6_KEEP_ENV=1",
                "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
                "ServerIP=0.0.0.0",
                "FTL_CMD=no-daemon",
                "DNSMASQ_USER=root",
                "VERSION=v5.2.1"
            ],
            "Cmd": null,
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
                ]
            },
            "Image": "sha256:929daab554acb2a02ab991ec85bba37b6934eaad91adec262b35ab36b151a03c",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/s6-init"
            ],
            "OnBuild": null,
            "Labels": {
                "image": "pihole/pihole:v5.2.1_amd64",
                "maintainer": "adam@diginc.us",
                "url": "https://www.github.com/pi-hole/docker-pi-hole"
            },
            "Shell": [
                "/bin/bash",
                "-c"
            ]
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 334142811,
        "VirtualSize": 334142811,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/c81092a13f818ccbd34d1dad0fbe0defef4b23cd71571b478dd1b2f8682002c2/diff:/var/lib/docker/overlay2/519ba27e5d31dc80a78b2dd4bb18aa8b0a459628263f5bbdcf358dd130f1c341/diff:/var/lib/docker/overlay2/122d90537c4950d38f96cbc46124301a013d2cf5d8e0b13fae4122b8589ad42a/diff:/var/lib/docker/overlay2/f0d6f913616a133ee8f0c1aba8551bc96414c456b62ae7f14759ae32471e5e10/diff:/var/lib/docker/overlay2/189819e86e683b4eb22fc53b27203935dd5bd89e5441e425f46992308a12a92c/diff:/var/lib/docker/overlay2/3de9aa9bcc4e0d3b932dd87da347058634301022fef4e5adc179440764ebdaea/diff:/var/lib/docker/overlay2/c413fbc60fc3e94ceb18205e325e519cd307680d21a8e31e37424fcca73db35f/diff:/var/lib/docker/overlay2/58416db361d34f20459d4545be1df8a312cf4d2c5a7303f8ff0687214f2171fa/diff:/var/lib/docker/overlay2/c84b697f7135ad21fc9763767e562e6cb152f591cde178c5e8aa7cb627b76840/diff:/var/lib/docker/overlay2/d72c77397d63398400b1f298f957e8f61ba99025bac1bcf2805634debbbc504e/diff:/var/lib/docker/overlay2/7df3bb0df183a896c0a13608577ba295aeb58d35e56beba520a9d2edcbea9ba6/diff:/var/lib/docker/overlay2/0699598278b65549bfc219f04617a9d4c1bd04e053899dcccd1b9454354e01e8/diff",
                "MergedDir": "/var/lib/docker/overlay2/27baecf20f210d74c6b5519b4ff58588030c500a1dc3f982a7554e0f5b308b30/merged",
                "UpperDir": "/var/lib/docker/overlay2/27baecf20f210d74c6b5519b4ff58588030c500a1dc3f982a7554e0f5b308b30/diff",
                "WorkDir": "/var/lib/docker/overlay2/27baecf20f210d74c6b5519b4ff58588030c500a1dc3f982a7554e0f5b308b30/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:f5600c6330da7bb112776ba067a32a9c20842d6ecc8ee3289f1a713b644092f8",
                "sha256:333ec1b36502156108bb75d5372e9bd01660ef212c63c6852afb719f2578a135",
                "sha256:7337aaa591164c259c17bb8de3b91c5aa856aa8b60bd7ed4029fcb340864134c",
                "sha256:2ad2d3172ff73c6009e9386e1a826bee4ebf6b6767bd275657b4269f771c8ff5",
                "sha256:a75feaf7b4ef0e0a2a9f2d1c7ea3737e34e2bfaac268ef854d9c13a4bd69bd95",
                "sha256:40821dc7d051aa9e92ce9de97b50d1ed3590e9ab7f9281ebabb3a2c631057d57",
                "sha256:4e1202651bb25616397108cc90af3940c2fd93b8d292978a6791046c97f347bc",
                "sha256:f7e072f2db4609884ad427f42b4a7ddab30c0ebe208d8e98ce2b38b8efc03f0c",
                "sha256:2a617d1b6a13f220d026a4a509ec0ff5fa536f6899a72f4350f6820ce654ac74",
                "sha256:7554d616a82bfa378acdc7ae9e5c3e885f29efb6cf06059071ca8648fbc9ec42",
                "sha256:c3e81aeb769d23e17e7bc675f016b640bc2a9bb2214cebf089521f554102f367",
                "sha256:9657d8d1b438d67e784aaf3c96d95c943b9138efbd029453f6e7227d304c2553",
                "sha256:6fa10c09468998df7084f6affeb8d3d2b5e02ce2aaccc02249c48b41ee323392"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

Could it be that these images were cross-compiled, and have the wrong architecture set in their metadata?

[thaJeztah]

Looks indeed that the image itself is the right variant;

"S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-aarch64.tar.gz",

"S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-amd64.tar.gz",

[thaJeztah]

So.. the manifest list ("multi arch") image has the correct architectures set, but the individual images either don't have the right architecture set, or have no architecture set (and the daemon sets the default linux/amd64 which could be for backward-compat), or something in the daemon code overwrites the architecture 🤔

docker manifest inspect pihole/pihole:v5.2.1

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3035,
         "digest": "sha256:5281797ed544ae438cd3c458171212d1e0cbf5c219a1cffe28b042a9f9821791",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1988,
         "digest": "sha256:d899e498cb7beb5784aef9d62e1bb7f28b4ec3306cf96f4d1bc6cd62e26bf599",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1988,
         "digest": "sha256:b2bdc30dd2024d1770f4e1358ea4fdf507ed9e63ae79d9e6fac9ececa2da00e6",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
         }
      }
   ]
}

[thaJeztah]

Narrowing down what the cause is, Inspecting the image by pulling it using https://github.com/moby/moby/blob/master/contrib/download-frozen-image-v2.sh

mkdir /arm64
./contrib/download-frozen-image-v2.sh /arm64 pihole/pihole:v5.2.1@sha256:d899e498cb7beb5784aef9d62e1bb7f28b4ec3306cf96f4d1bc6cd62e26bf599
Downloading 'pihole/pihole:v5.2.1@sha256:d899e498cb7beb5784aef9d62e1bb7f28b4ec3306cf96f4d1bc6cd62e26bf599' (8 layers)...
....

Download of images into '/arm64' complete.
Use something like the following to load the result into a Docker daemon:
  tar -cC '/arm64' . | docker load

cat /arm64/manifest.json
[
  {
    "Config": "ddcd770c66339bae8f2e9f89940b7f1d3300538fff48e5443d1244ba21d63885.json",
    "RepoTags": [
      "pihole/pihole:v5.2.1"
    ],
    "Layers": [
      "e1f5c9cf823075c9232453c9d7053867d3cf3f439168f98dd93d05cd21e44811/layer.tar",
      "ae605a65daf61f97f709b3e112a732c75059e575ec134f1dca3cf7575c5410d7/layer.tar",
      "e53dae905ccec45ab20632a393fc05011657c3fc1b2a77ca0bf17d10edcb7f12/layer.tar",
      "6e10c020957b38ca48574a9034a89bedf1b1319288f859eba7a7b927f39bab25/layer.tar",
      "1a19ac15b4e2d81665ae836dd62841f344ec34bae33664f04ff34b2371c1c7e7/layer.tar",
      "04d6b9264033223437559e633a15eabb193d85fb8e354f7ee84002fd0db67b29/layer.tar",
      "dee573760ae01fd1cca624a652d3478d771e6ff383a3f18fa519edc19f716dc6/layer.tar",
      "0fb2bd2b7ffd7c57f4c302f4d959de6edbd58358e6c1fd1c844aace30f26dc41/layer.tar"
    ]
  }
]

cat /arm64/ddcd770c66339bae8f2e9f89940b7f1d3300538fff48e5443d1244ba21d63885.json

{
  "architecture": "amd64",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "ExposedPorts": {
      "443/tcp": {},
      "53/tcp": {},
      "53/udp": {},
      "67/udp": {},
      "80/tcp": {}
    },
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "ARCH=arm64",
      "UBUNTU_SUITE=buster",
      "DOCKER_REPO=multiarch/debian-debootstrap",
      "PIHOLE_ARCH=arm64",
      "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-aarch64.tar.gz",
      "PIHOLE_INSTALL=/root/ph_install.sh",
      "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
      "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
      "IPv6=True",
      "S6_LOGGING=0",
      "S6_KEEP_ENV=1",
      "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
      "ServerIP=0.0.0.0",
      "FTL_CMD=no-daemon",
      "DNSMASQ_USER=root",
      "VERSION=v5.2.1"
    ],
    "Cmd": null,
    "Healthcheck": {
      "Test": [
        "CMD-SHELL",
        "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
      ]
    },
    "Image": "sha256:b4459ac3099e2fd70e9c2aa5ef710e853da11624d65f0b353f88da8c39dbd34f",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": [
      "/s6-init"
    ],
    "OnBuild": null,
    "Labels": {
      "image": "pihole/pihole:v5.2.1_arm64",
      "maintainer": "adam@diginc.us",
      "url": "https://www.github.com/pi-hole/docker-pi-hole"
    },
    "Shell": [
      "/bin/bash",
      "-c"
    ]
  },
  "container": "35f3f315a63da09485218661dc7c1e335121dc5b3b10191c3ba2e8f3f5c718ac",
  "container_config": {
    "Hostname": "35f3f315a63d",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "ExposedPorts": {
      "443/tcp": {},
      "53/tcp": {},
      "53/udp": {},
      "67/udp": {},
      "80/tcp": {}
    },
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "ARCH=arm64",
      "UBUNTU_SUITE=buster",
      "DOCKER_REPO=multiarch/debian-debootstrap",
      "PIHOLE_ARCH=arm64",
      "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-aarch64.tar.gz",
      "PIHOLE_INSTALL=/root/ph_install.sh",
      "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
      "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
      "IPv6=True",
      "S6_LOGGING=0",
      "S6_KEEP_ENV=1",
      "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
      "ServerIP=0.0.0.0",
      "FTL_CMD=no-daemon",
      "DNSMASQ_USER=root",
      "VERSION=v5.2.1"
    ],
    "Cmd": [
      "/bin/bash",
      "-c",
      "#(nop) ",
      "SHELL [/bin/bash -c]"
    ],
    "Healthcheck": {
      "Test": [
        "CMD-SHELL",
        "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
      ]
    },
    "Image": "sha256:b4459ac3099e2fd70e9c2aa5ef710e853da11624d65f0b353f88da8c39dbd34f",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": [
      "/s6-init"
    ],
    "OnBuild": null,
    "Labels": {
      "image": "pihole/pihole:v5.2.1_arm64",
      "maintainer": "adam@diginc.us",
      "url": "https://www.github.com/pi-hole/docker-pi-hole"
    },
    "Shell": [
      "/bin/bash",
      "-c"
    ]
  },
  "created": "2020-12-03T02:06:14.680174992Z",
  "docker_version": "19.03.13+azure",
  "history": [
    {
      "created": "2020-09-21T13:14:16.064296896Z",
      "created_by": "/bin/sh -c #(nop) ADD file:9dec6ab242477dcf4ffa603bdfd5e27075862c7686626da9d17e46d36bdfa680 in / "
    },
    {
      "created": "2020-09-21T13:14:17.245076223Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/bash\"]",
      "empty_layer": true
    },
    {
      "created": "2020-09-21T13:14:17.399948599Z",
      "created_by": "/bin/sh -c #(nop)  ENV ARCH=arm64 UBUNTU_SUITE=buster DOCKER_REPO=multiarch/debian-debootstrap",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:07.657658869Z",
      "created_by": "/bin/sh -c #(nop)  ARG PIHOLE_ARCH",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:08.366627676Z",
      "created_by": "/bin/sh -c #(nop)  ENV PIHOLE_ARCH=arm64",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:09.353406418Z",
      "created_by": "/bin/sh -c #(nop)  ARG S6_ARCH",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:10.341364661Z",
      "created_by": "/bin/sh -c #(nop)  ARG S6_VERSION",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:11.364166552Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-aarch64.tar.gz",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:13.000183821Z",
      "created_by": "/bin/sh -c #(nop) COPY file:dcb7fff06ca7f56b62b19a51693c1fbe9cab7bbdd8d7a1f2527267fd29841558 in /usr/local/bin/install.sh "
    },
    {
      "created": "2020-12-03T01:58:14.000270627Z",
      "created_by": "/bin/sh -c #(nop) COPY file:e4764efc839663d763445f0ed35e56861f7c1801ef50ac7bbc53f057b8f32a18 in /etc/docker-pi-hole-version "
    },
    {
      "created": "2020-12-03T01:58:14.354399312Z",
      "created_by": "/bin/sh -c #(nop)  ENV PIHOLE_INSTALL=/root/ph_install.sh",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:40.705887232Z",
      "created_by": "|2 S6_ARCH=aarch64 S6_VERSION=v1.22.1.0 /bin/sh -c bash -ex install.sh 2>&1 &&     rm -rf /var/cache/apt/archives /var/lib/apt/lists/*"
    },
    {
      "created": "2020-12-03T02:05:45.742997182Z",
      "created_by": "/bin/sh -c #(nop)  ENTRYPOINT [\"/s6-init\"]",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:47.000238082Z",
      "created_by": "/bin/sh -c #(nop) ADD dir:04ba41115e115e65f57dd6ee9d6f24290644d70968720eb39690830b87003cbf in / "
    },
    {
      "created": "2020-12-03T02:05:48.000235495Z",
      "created_by": "/bin/sh -c #(nop) COPY file:5f42354bc5364cecacf611938520aa2e01ab65d469aac487bcd0e27abc5de7d5 in /usr/local/bin/service "
    },
    {
      "created": "2020-12-03T02:05:48.650120484Z",
      "created_by": "/bin/sh -c #(nop)  ARG PHP_ENV_CONFIG",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:49.639148111Z",
      "created_by": "/bin/sh -c #(nop)  ENV PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:50.654615615Z",
      "created_by": "/bin/sh -c #(nop)  ARG PHP_ERROR_LOG",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:51.629011331Z",
      "created_by": "/bin/sh -c #(nop)  ENV PHP_ERROR_LOG=/var/log/lighttpd/error.log",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:53.000296372Z",
      "created_by": "/bin/sh -c #(nop) COPY file:c4d677569a4b094380c73d2adcf27b933b8c602c200b3bc617b71aa04ad68209 in / "
    },
    {
      "created": "2020-12-03T02:05:54.000189749Z",
      "created_by": "/bin/sh -c #(nop) COPY file:f14f0f49d9cfd5e91767a9f90081149a0e1603990adec0b6336267836fbffd58 in / "
    },
    {
      "created": "2020-12-03T02:05:54.732175479Z",
      "created_by": "/bin/sh -c #(nop)  ENV IPv6=True",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:55.742387817Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 53 53/udp",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:56.637736561Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 67/udp",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:57.676535684Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 80",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:58.754594269Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 443",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:05:59.718176164Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6_LOGGING=0",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:00.635321537Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6_KEEP_ENV=1",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:01.620055766Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:02.625719432Z",
      "created_by": "/bin/sh -c #(nop)  ENV ServerIP=0.0.0.0",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:03.624323345Z",
      "created_by": "/bin/sh -c #(nop)  ENV FTL_CMD=no-daemon",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:04.611857977Z",
      "created_by": "/bin/sh -c #(nop)  ENV DNSMASQ_USER=root",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:05.61224819Z",
      "created_by": "/bin/sh -c #(nop)  ARG PIHOLE_VERSION",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:06.612663998Z",
      "created_by": "/bin/sh -c #(nop)  ENV VERSION=v5.2.1",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:07.647574535Z",
      "created_by": "/bin/sh -c #(nop)  ENV PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:08.641198085Z",
      "created_by": "/bin/sh -c #(nop)  ARG NAME",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:09.657814486Z",
      "created_by": "/bin/sh -c #(nop)  LABEL image=pihole/pihole:v5.2.1_arm64",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:10.653781641Z",
      "created_by": "/bin/sh -c #(nop)  ARG MAINTAINER",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:11.669868426Z",
      "created_by": "/bin/sh -c #(nop)  LABEL maintainer=adam@diginc.us",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:12.662539748Z",
      "created_by": "/bin/sh -c #(nop)  LABEL url=https://www.github.com/pi-hole/docker-pi-hole",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:13.657085876Z",
      "created_by": "/bin/sh -c #(nop)  HEALTHCHECK &{[\"CMD-SHELL\" \"dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1\"] \"0s\" \"0s\" \"0s\" '\\x00'}",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T02:06:14.680174992Z",
      "created_by": "/bin/bash -c #(nop)  SHELL [/bin/bash -c]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:cc77256f2d56e07985cba58506e433bb21ef7b3a06bd3a688ac6abfdfc010344",
      "sha256:c2c03ee92cd94dc4721023ebe9e6b1a1c6e3588a470d65bdabc589d68bd38395",
      "sha256:beb456e5587da0c7e993017a51b77dac721a6fc290b0c23b63fda47fa262488c",
      "sha256:038c127208f1ddfa51eab576f6de0f44c4c9991bcdb66e432d654d280db597f6",
      "sha256:7554d616a82bfa378acdc7ae9e5c3e885f29efb6cf06059071ca8648fbc9ec42",
      "sha256:110d37e83a24488a1565c2d8e5307f676a72a19d6779d0f1e97a7203f7aa03e2",
      "sha256:9657d8d1b438d67e784aaf3c96d95c943b9138efbd029453f6e7227d304c2553",
      "sha256:6fa10c09468998df7084f6affeb8d3d2b5e02ce2aaccc02249c48b41ee323392"
    ]
  }
}

and for the amd64 image:

mkdir /amd64
./contrib/download-frozen-image-v2.sh /amd64 pihole/pihole:v5.2.1@sha256:5281797ed544ae438cd3c458171212d1e0cbf5c219a1cffe28b042a9f9821791
Downloading 'pihole/pihole:v5.2.1@sha256:5281797ed544ae438cd3c458171212d1e0cbf5c219a1cffe28b042a9f9821791' (13 layers)...
...

Download of images into '/amd64' complete.
Use something like the following to load the result into a Docker daemon:
  tar -cC '/amd64' . | docker load

cat /amd64/manifest.json
[
  {
    "Config": "52e72f4b6a617b430cf14c6fa919eb477c48aee4f7f87cc1c55247c1d602a66f.json",
    "RepoTags": [
      "pihole/pihole:v5.2.1"
    ],
    "Layers": [
      "484d6ff7582f78e0c8c70410fef1e167a82f68df64895a4318689bf265f323f9/layer.tar",
      "19e0f39c9873847b2b787aa59d8ad44189327a0815444ebeacd6b3afcf858587/layer.tar",
      "2ef4549fc54ab95a45c8649ce51fc9715eb6b951b3ff8d67688d1182d3f2bb5d/layer.tar",
      "25b5949f8916972ee72de299abbb1abc26e7fd530dc0ca31d206c108abcb23b1/layer.tar",
      "93370434bdba1941d53616635f25abfe3dbad4eff47cf10d76aa194c88a005d5/layer.tar",
      "84bfbf2cb68120795b9a2e1d8e07de393a60294cae1e6f6e3470b2f5fd0d4465/layer.tar",
      "0427eb918d977916581b1691b75f3e307e2147ac4a0f0de3606d50be4614c09e/layer.tar",
      "aecb639f13655a0c9fb3ebdf36c02c5dfe67dc25514df9cf460bf75dd1c2c9ec/layer.tar",
      "55da22151a39bc4c81879de27f6987bbc621973555660d75f7ac68ffca367126/layer.tar",
      "99a94529e89780bf856193d9c78a6dc561ad27ba97bbe16420375a15bfcd8fd2/layer.tar",
      "8459e6c5e2bf3da7a66ebb99e8f4f36cd93dc3ce6a03708e1f9ac02c6fe75e88/layer.tar",
      "d8a1fe5e592b7aa0242a563e7335d0ad870556fc633bc3e205071d65761550ec/layer.tar",
      "12375d48619baf76e3cbb78676c0fd96cc51432812dc58156e7695eff86cef5f/layer.tar"
    ]
  }
]

cat /amd64/52e72f4b6a617b430cf14c6fa919eb477c48aee4f7f87cc1c55247c1d602a66f.json | jq .

{
  "architecture": "amd64",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "ExposedPorts": {
      "443/tcp": {},
      "53/tcp": {},
      "53/udp": {},
      "67/udp": {},
      "80/tcp": {}
    },
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "PIHOLE_ARCH=amd64",
      "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-amd64.tar.gz",
      "PIHOLE_INSTALL=/root/ph_install.sh",
      "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
      "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
      "IPv6=True",
      "S6_LOGGING=0",
      "S6_KEEP_ENV=1",
      "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
      "ServerIP=0.0.0.0",
      "FTL_CMD=no-daemon",
      "DNSMASQ_USER=root",
      "VERSION=v5.2.1"
    ],
    "Cmd": null,
    "Healthcheck": {
      "Test": [
        "CMD-SHELL",
        "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
      ]
    },
    "Image": "sha256:929daab554acb2a02ab991ec85bba37b6934eaad91adec262b35ab36b151a03c",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": [
      "/s6-init"
    ],
    "OnBuild": null,
    "Labels": {
      "image": "pihole/pihole:v5.2.1_amd64",
      "maintainer": "adam@diginc.us",
      "url": "https://www.github.com/pi-hole/docker-pi-hole"
    },
    "Shell": [
      "/bin/bash",
      "-c"
    ]
  },
  "container": "6c475f695cacb914ab84e7f07d978bd5180bbb6b5b102c099eebb185f54473e3",
  "container_config": {
    "Hostname": "6c475f695cac",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "ExposedPorts": {
      "443/tcp": {},
      "53/tcp": {},
      "53/udp": {},
      "67/udp": {},
      "80/tcp": {}
    },
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "PIHOLE_ARCH=amd64",
      "S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-amd64.tar.gz",
      "PIHOLE_INSTALL=/root/ph_install.sh",
      "PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
      "PHP_ERROR_LOG=/var/log/lighttpd/error.log",
      "IPv6=True",
      "S6_LOGGING=0",
      "S6_KEEP_ENV=1",
      "S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
      "ServerIP=0.0.0.0",
      "FTL_CMD=no-daemon",
      "DNSMASQ_USER=root",
      "VERSION=v5.2.1"
    ],
    "Cmd": [
      "/bin/bash",
      "-c",
      "#(nop) ",
      "SHELL [/bin/bash -c]"
    ],
    "Healthcheck": {
      "Test": [
        "CMD-SHELL",
        "dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1"
      ]
    },
    "Image": "sha256:929daab554acb2a02ab991ec85bba37b6934eaad91adec262b35ab36b151a03c",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": [
      "/s6-init"
    ],
    "OnBuild": null,
    "Labels": {
      "image": "pihole/pihole:v5.2.1_amd64",
      "maintainer": "adam@diginc.us",
      "url": "https://www.github.com/pi-hole/docker-pi-hole"
    },
    "Shell": [
      "/bin/bash",
      "-c"
    ]
  },
  "created": "2020-12-03T01:59:33.41222977Z",
  "docker_version": "19.03.13+azure",
  "history": [
    {
      "created": "2020-11-17T20:21:17.570073346Z",
      "created_by": "/bin/sh -c #(nop) ADD file:d2abb0e4e7ac1773741f51f57d3a0b8ffc7907348842d773f8c341ba17f856d5 in / "
    },
    {
      "created": "2020-11-17T20:21:17.865210281Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
      "empty_layer": true
    },
    {
      "created": "2020-11-29T02:35:40.042063952Z",
      "created_by": "/bin/sh -c #(nop) COPY multi:adc45a1285c053f33230963c1773a349cd47bf9e84d05ea8bd222937ba7c5540 in / "
    },
    {
      "created": "2020-11-29T02:35:41.000158482Z",
      "created_by": "/bin/sh -c #(nop) COPY dir:cca3648dbdd7cf158510aa3f34162b26231a77144248f8be9cb4f249540fad23 in /etc/apt/ "
    },
    {
      "created": "2020-11-29T02:35:42.000180586Z",
      "created_by": "/bin/sh -c #(nop) COPY dir:b6fe344f2ba9a4fe86666c76dd73eba7aa0db7b37b1111a04138cd870fa7f7aa in /usr/local/repos/pam "
    },
    {
      "created": "2020-11-29T02:35:43.000150571Z",
      "created_by": "/bin/sh -c #(nop) COPY file:262f9221f7411cc3dedc913091495a2aedb98b1f29eee680ba84523cb2c322ab in /usr/local/bin/ "
    },
    {
      "created": "2020-11-29T02:35:44.578550307Z",
      "created_by": "/bin/sh -c bash -ex setup.sh && rm /usr/local/bin/setup.sh"
    },
    {
      "created": "2020-12-03T01:57:59.794627738Z",
      "created_by": "/bin/sh -c #(nop)  ARG PIHOLE_ARCH",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:00.221441121Z",
      "created_by": "/bin/sh -c #(nop)  ENV PIHOLE_ARCH=amd64",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:01.238142861Z",
      "created_by": "/bin/sh -c #(nop)  ARG S6_ARCH",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:02.240996623Z",
      "created_by": "/bin/sh -c #(nop)  ARG S6_VERSION",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:03.225666432Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6OVERLAY_RELEASE=https://github.com/just-containers/s6-overlay/releases/download/v1.22.1.0/s6-overlay-amd64.tar.gz",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:58:05.000102827Z",
      "created_by": "/bin/sh -c #(nop) COPY file:dcb7fff06ca7f56b62b19a51693c1fbe9cab7bbdd8d7a1f2527267fd29841558 in /usr/local/bin/install.sh "
    },
    {
      "created": "2020-12-03T01:58:06.000250421Z",
      "created_by": "/bin/sh -c #(nop) COPY file:e4764efc839663d763445f0ed35e56861f7c1801ef50ac7bbc53f057b8f32a18 in /etc/docker-pi-hole-version "
    },
    {
      "created": "2020-12-03T01:58:06.216505719Z",
      "created_by": "/bin/sh -c #(nop)  ENV PIHOLE_INSTALL=/root/ph_install.sh",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:00.888073076Z",
      "created_by": "|2 S6_ARCH=amd64 S6_VERSION=v1.22.1.0 /bin/sh -c bash -ex install.sh 2>&1 &&     rm -rf /var/cache/apt/archives /var/lib/apt/lists/*"
    },
    {
      "created": "2020-12-03T01:59:05.339708593Z",
      "created_by": "/bin/sh -c #(nop)  ENTRYPOINT [\"/s6-init\"]",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:06.00020376Z",
      "created_by": "/bin/sh -c #(nop) ADD dir:04ba41115e115e65f57dd6ee9d6f24290644d70968720eb39690830b87003cbf in / "
    },
    {
      "created": "2020-12-03T01:59:07.000145186Z",
      "created_by": "/bin/sh -c #(nop) COPY file:5f42354bc5364cecacf611938520aa2e01ab65d469aac487bcd0e27abc5de7d5 in /usr/local/bin/service "
    },
    {
      "created": "2020-12-03T01:59:07.400666373Z",
      "created_by": "/bin/sh -c #(nop)  ARG PHP_ENV_CONFIG",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:08.396185483Z",
      "created_by": "/bin/sh -c #(nop)  ENV PHP_ENV_CONFIG=/etc/lighttpd/conf-enabled/15-fastcgi-php.conf",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:09.398617479Z",
      "created_by": "/bin/sh -c #(nop)  ARG PHP_ERROR_LOG",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:10.399860424Z",
      "created_by": "/bin/sh -c #(nop)  ENV PHP_ERROR_LOG=/var/log/lighttpd/error.log",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:12.00016337Z",
      "created_by": "/bin/sh -c #(nop) COPY file:c4d677569a4b094380c73d2adcf27b933b8c602c200b3bc617b71aa04ad68209 in / "
    },
    {
      "created": "2020-12-03T01:59:13.000219618Z",
      "created_by": "/bin/sh -c #(nop) COPY file:f14f0f49d9cfd5e91767a9f90081149a0e1603990adec0b6336267836fbffd58 in / "
    },
    {
      "created": "2020-12-03T01:59:13.420108554Z",
      "created_by": "/bin/sh -c #(nop)  ENV IPv6=True",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:14.401150447Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 53 53/udp",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:15.39856308Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 67/udp",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:16.413183067Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 80",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:17.423655155Z",
      "created_by": "/bin/sh -c #(nop)  EXPOSE 443",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:18.403633214Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6_LOGGING=0",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:19.419131426Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6_KEEP_ENV=1",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:20.403816404Z",
      "created_by": "/bin/sh -c #(nop)  ENV S6_BEHAVIOUR_IF_STAGE2_FAILS=2",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:21.400794455Z",
      "created_by": "/bin/sh -c #(nop)  ENV ServerIP=0.0.0.0",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:22.400339818Z",
      "created_by": "/bin/sh -c #(nop)  ENV FTL_CMD=no-daemon",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:23.416784726Z",
      "created_by": "/bin/sh -c #(nop)  ENV DNSMASQ_USER=root",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:24.396713912Z",
      "created_by": "/bin/sh -c #(nop)  ARG PIHOLE_VERSION",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:25.411531635Z",
      "created_by": "/bin/sh -c #(nop)  ENV VERSION=v5.2.1",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:26.41904171Z",
      "created_by": "/bin/sh -c #(nop)  ENV PATH=/opt/pihole:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:27.404895307Z",
      "created_by": "/bin/sh -c #(nop)  ARG NAME",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:28.414446456Z",
      "created_by": "/bin/sh -c #(nop)  LABEL image=pihole/pihole:v5.2.1_amd64",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:29.399637486Z",
      "created_by": "/bin/sh -c #(nop)  ARG MAINTAINER",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:30.395105252Z",
      "created_by": "/bin/sh -c #(nop)  LABEL maintainer=adam@diginc.us",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:31.411684327Z",
      "created_by": "/bin/sh -c #(nop)  LABEL url=https://www.github.com/pi-hole/docker-pi-hole",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:32.41615888Z",
      "created_by": "/bin/sh -c #(nop)  HEALTHCHECK &{[\"CMD-SHELL\" \"dig +norecurse +retry=0 @127.0.0.1 pi.hole || exit 1\"] \"0s\" \"0s\" \"0s\" '\\x00'}",
      "empty_layer": true
    },
    {
      "created": "2020-12-03T01:59:33.41222977Z",
      "created_by": "/bin/bash -c #(nop)  SHELL [/bin/bash -c]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:f5600c6330da7bb112776ba067a32a9c20842d6ecc8ee3289f1a713b644092f8",
      "sha256:333ec1b36502156108bb75d5372e9bd01660ef212c63c6852afb719f2578a135",
      "sha256:7337aaa591164c259c17bb8de3b91c5aa856aa8b60bd7ed4029fcb340864134c",
      "sha256:2ad2d3172ff73c6009e9386e1a826bee4ebf6b6767bd275657b4269f771c8ff5",
      "sha256:a75feaf7b4ef0e0a2a9f2d1c7ea3737e34e2bfaac268ef854d9c13a4bd69bd95",
      "sha256:40821dc7d051aa9e92ce9de97b50d1ed3590e9ab7f9281ebabb3a2c631057d57",
      "sha256:4e1202651bb25616397108cc90af3940c2fd93b8d292978a6791046c97f347bc",
      "sha256:f7e072f2db4609884ad427f42b4a7ddab30c0ebe208d8e98ce2b38b8efc03f0c",
      "sha256:2a617d1b6a13f220d026a4a509ec0ff5fa536f6899a72f4350f6820ce654ac74",
      "sha256:7554d616a82bfa378acdc7ae9e5c3e885f29efb6cf06059071ca8648fbc9ec42",
      "sha256:c3e81aeb769d23e17e7bc675f016b640bc2a9bb2214cebf089521f554102f367",
      "sha256:9657d8d1b438d67e784aaf3c96d95c943b9138efbd029453f6e7227d304c2553",
      "sha256:6fa10c09468998df7084f6affeb8d3d2b5e02ce2aaccc02249c48b41ee323392"
    ]
  }
}

So for both images, the image that's in the registry has "amd64" set as "architecture". I suspect the image is cross-compiled, and has the incorrect architecture set.

[thaJeztah]

So concluding;

• looks like for this case, the issue is actually with the images having the incorrect architecture set in their metadata
• docker 20.10 now prints a warning ("WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested"). Older versions of docker silently tried to run the image (which in this case would work, because the image is an arm image, just "labeled" incorrectly
• docker 20.10 now produces an error if the architecture of the image isn't supported by the engine (which it bases on the architecture specified in the image's manifest)

docker run -it --rm --platform=linux/arm64 --entrypoint /bin/sh pihole/pihole:v5.2.1
Unable to find image 'pihole/pihole:v5.2.1' locally
v5.2.1: Pulling from pihole/pihole
aceecb32d1c3: Pull complete
5619849a9c12: Pull complete
e998d3f61df7: Pull complete
b1ab287d11d2: Pull complete
f45b672ae3e8: Pull complete
efe0958f28c6: Pull complete
1292910fb137: Pull complete
a54bdd0520cd: Pull complete
Digest: sha256:7888e89f4d435673c75896c546fefe3177379c99c8766c89be25a905f8def533
Status: Downloaded newer image for pihole/pihole:v5.2.1
docker: Error response from daemon: image with reference pihole/pihole:v5.2.1 was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64.
See 'docker run --help'.

Perhaps a "I know what I'm doing; ignore what the image says" flag would be needed?

[thaJeztah]

Note that the above is on Docker Desktop, which can run non-matching architectures (through qemu), but for that requires the image metadata to reflect the correct architecture.

Ignore this; see below