Failed build with `ERROR: blob sha256:*** not found`

[pjonsson]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Using docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 on Github's public CI runners, I had a build fail with ERROR: blob sha256:*** not found on a RUN --mount=type=cache .. line, and then the following lines (COPY --link pyproject.toml ... and RUN tar xf .. also failed).

The action is configured to cache to gha with mode=max, and the image usually builds fine. The typical cache hitrate is around 40%.

Expected behaviour

I thought the Github CI actions didn't have mounted cache unless doing the buildkit-cache-dance, and I'm not doing that, so not sure how a step with a fresh/empty cache could fail.

Actual behaviour

Build failed.

Buildx version

github.com/docker/buildx v0.26.1 1a8287f

Docker info

/usr/bin/docker version
  Client: Docker Engine - Community
   Version:           28.0.4
   API version:       1.48
   Go version:        go1.23.7
   Git commit:        b8034c0
   Built:             Tue Mar 25 15:07:16 2025
   OS/Arch:           linux/amd64
   Context:           default
  
  Server: Docker Engine - Community
   Engine:
    Version:          28.0.4
    API version:      1.48 (minimum version 1.24)
    Go version:       go1.23.7
    Git commit:       6430e49
    Built:            Tue Mar 25 15:07:16 2025
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          1.7.27
    GitCommit:        05044ec0a9a75232cad458027ca83437aae3f4da
   runc:
    Version:          1.2.5
    GitCommit:        v1.2.5-0-g59923ef
   docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0
  /usr/bin/docker info
  Client: Docker Engine - Community
   Version:    28.0.4
   Context:    default
   Debug Mode: false
   Plugins:
    buildx: Docker Buildx (Docker Inc.)
      Version:  v0.26.1
      Path:     /usr/libexec/docker/cli-plugins/docker-buildx
    compose: Docker Compose (Docker Inc.)
      Version:  v2.38.2
      Path:     /usr/libexec/docker/cli-plugins/docker-compose
  
  Server:
   Containers: 1
    Running: 1
    Paused: 0
    Stopped: 0
   Images: 1
   Server Version: 28.0.4
   Storage Driver: overlay2
    Backing Filesystem: extfs
    Supports d_type: true
    Using metacopy: false
    Native Overlay Diff: false
    userxattr: false
   Logging Driver: json-file
   Cgroup Driver: systemd
   Cgroup Version: 2
   Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
   Swarm: inactive
   Runtimes: io.containerd.runc.v2 runc
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
   runc version: v1.2.5-0-g59923ef
   init version: de40ad0
   Security Options:
    apparmor
    seccomp
     Profile: builtin
    cgroupns
   Kernel Version: 6.11.0-1018-azure
   Operating System: Ubuntu 24.04.2 LTS
   OSType: linux
   Architecture: x86_64
   CPUs: 4
   Total Memory: 15.62GiB
   Name: pkrvmubgrv54qmi
   ID: 2166c256-3f30-4fc2-abf1-e8d653854173
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Username: githubactions
   Experimental: false
   Insecure Registries:
    ::1/128
    127.0.0.0/8
   Live Restore Enabled: false

Builders list

{
    "nodes": [
      {
        "name": "builder-a64b89db-00e0-4a56-af32-f79cebc8e2570",
        "endpoint": "unix:///var/run/docker.sock",
        "status": "running",
        "buildkitd-flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host",
        "buildkit": "v0.22.0",
        "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/386",
        "features": {
          "Automatically load images to the Docker Engine image store": true,
          "Cache export": true,
          "Direct push": true,
          "Docker exporter": true,
          "Multi-platform build": true,
          "OCI exporter": true
        },
        "labels": {
          "org.mobyproject.buildkit.worker.executor": "oci",
          "org.mobyproject.buildkit.worker.hostname": "b1baeb740036",
          "org.mobyproject.buildkit.worker.network": "host",
          "org.mobyproject.buildkit.worker.oci.process-mode": "sandbox",
          "org.mobyproject.buildkit.worker.selinux.enabled": "false",
          "org.mobyproject.buildkit.worker.snapshotter": "overlayfs"
        },
        "gcPolicy": [
          {
            "all": false,
            "filter": [
              "type==source.local",
              "type==exec.cachemount",
              "type==source.git.checkout"
            ],
            "keepDuration": "48h0m0s",
            "maxUsedSpace": "488.3MiB"
          },
          {
            "all": false,
            "keepDuration": "1440h0m0s",
            "reservedSpace": "7.451GiB",
            "maxUsedSpace": "54.02GiB",
            "minFreeSpace": "13.97GiB"
          },
          {
            "all": false,
            "reservedSpace": "7.451GiB",
            "maxUsedSpace": "54.02GiB",
            "minFreeSpace": "13.97GiB"
          },
          {
            "all": true,
            "reservedSpace": "7.451GiB",
            "maxUsedSpace": "54.02GiB",
            "minFreeSpace": "13.97GiB"
          }
        ]
      }
    ],
    "name": "builder-a64b89db-00e0-4a56-af32-f79cebc8e257",
    "driver": "docker-container",
    "lastActivity": "2025-08-18T17:01:25.000Z"
  }

Configuration

The error is intermittent, so I can't provide a reproducer, but the first failing RUN looks like this:

COPY --link pyproject.toml uv.lock /build/

RUN --mount=type=cache,id=opendatacube-uv-cache,target=/root/.cache \
    uv sync --frozen --extra=deployment --no-install-project \
      --no-binary-package fiona \
      --no-binary-package netcdf4 \
      --no-binary-package psycopg2 \
      --no-binary-package rasterio \
      --no-binary-package shapely

Build logs

/usr/bin/docker buildx build --build-arg ENVIRONMENT=test --cache-from type=gha --cache-to type=gha,mode=max --file Dockerfile --iidfile /home/runner/work/_temp/docker-actions-toolkit-jLfLnh/build-iidfile-dd9dc9ce86.txt --tag ghcr.io/opendatacube/explorer:tests --load --metadata-file /home/runner/work/_temp/docker-actions-toolkit-jLfLnh/build-metadata-c3891fae96.json .
#0 building with "builder-a64b89db-00e0-4a56-af32-f79cebc8e257" instance using docker-container driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 3.98kB done
#1 DONE 0.0s

#2 resolve image config for docker-image://docker.io/docker/dockerfile:1
#2 ...

#3 [auth] docker/dockerfile:pull token for registry-1.docker.io
#3 DONE 0.0s

#2 resolve image config for docker-image://docker.io/docker/dockerfile:1
#2 DONE 0.9s

#4 docker-image://docker.io/docker/dockerfile:1@sha256:38387523653efa0039f8e1c89bb74a30504e76ee9f565e25c9a09841f9427b05
#4 resolve docker.io/docker/dockerfile:1@sha256:38387523653efa0039f8e1c89bb74a30504e76ee9f565e25c9a09841f9427b05 done
#4 sha256:fdbb657bc4fdee228c0c20080da3a4cc193ef22c6637ae12109c8feba99debea 4.19MB / 14.09MB 0.2s
#4 sha256:fdbb657bc4fdee228c0c20080da3a4cc193ef22c6637ae12109c8feba99debea 14.09MB / 14.09MB 0.3s done
#4 extracting sha256:fdbb657bc4fdee228c0c20080da3a4cc193ef22c6637ae12109c8feba99debea 0.1s done
#4 DONE 0.4s

#5 [internal] load metadata for ghcr.io/osgeo/gdal:ubuntu-small-3.10.3@sha256:dab45abca3ca83695d442018692f4f8a0f41955871c57e6101d7f89a92375caa
#5 DONE 0.8s

#6 [internal] load .dockerignore
#6 transferring context: 926B done
#6 DONE 0.0s

#7 [internal] load build context
#7 DONE 0.0s

#8 [builder 3/8] ADD --checksum=sha256:5429c9b96cab65198c2e5bfe83e933329aa16303a0369d5beedc71785a4a2f36 --chown=root:root --chmod=644 --link https://github.com/astral-sh/uv/releases/download/0.8.6/uv-x86_64-unknown-linux-gnu.tar.gz uv.tar.gz
#8 DONE 0.0s

#9 [base 1/1] FROM ghcr.io/osgeo/gdal:ubuntu-small-3.10.3@sha256:dab45abca3ca83695d442018692f4f8a0f41955871c57e6101d7f89a92375caa
#9 resolve ghcr.io/osgeo/gdal:ubuntu-small-3.10.3@sha256:dab45abca3ca83695d442018692f4f8a0f41955871c57e6101d7f89a92375caa done
#9 DONE 0.0s

#10 importing cache manifest from gha:4127165035014263950
#10 DONE 0.5s

#7 [internal] load build context
#7 transferring context: 84.65MB 0.6s done
#7 DONE 0.6s

#11 [builder 1/8] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked     export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get upgrade -y     && apt-get install -y --no-install-recommends         gcc         g++         git         libgeos-dev         libhdf5-dev         libnetcdf-dev         libudunits2-dev         libpq-dev         python3-dev
#11 CACHED

#12 [builder 2/8] WORKDIR /build
#12 CACHED

#13 [builder 3/8] ADD --checksum=sha256:5429c9b96cab65198c2e5bfe83e933329aa16303a0369d5beedc71785a4a2f36 --chown=root:root --chmod=644 --link https://github.com/astral-sh/uv/releases/download/0.8.6/uv-x86_64-unknown-linux-gnu.tar.gz uv.tar.gz
#13 CACHED

#14 [builder 4/8] RUN tar xf uv.tar.gz -C /usr/local/bin --strip-components=1 --no-same-owner
#14 CACHED

#15 [builder 5/8] COPY --link pyproject.toml uv.lock /build/
#15 CACHED

#16 [builder 6/8] RUN --mount=type=cache,id=opendatacube-uv-cache,target=/root/.cache     uv sync --frozen --extra=deployment --no-install-project       --no-binary-package fiona       --no-binary-package netcdf4       --no-binary-package psycopg2       --no-binary-package rasterio       --no-binary-package shapely
#16 ERROR: blob sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c not found

#15 [builder 5/8] COPY --link pyproject.toml uv.lock /build/
#15 ERROR: blob sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c not found

#14 [builder 4/8] RUN tar xf uv.tar.gz -C /usr/local/bin --strip-components=1 --no-same-owner
#14 ERROR: blob sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c not found

#13 [builder 3/8] ADD --checksum=sha256:5429c9b96cab65198c2e5bfe83e933329aa16303a0369d5beedc71785a4a2f36 --chown=root:root --chmod=644 --link https://github.com/astral-sh/uv/releases/download/0.8.6/uv-x86_64-unknown-linux-gnu.tar.gz uv.tar.gz
#13 ERROR: blob sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c not found

#8 [builder 3/8] ADD --checksum=sha256:5429c9b96cab65198c2e5bfe83e933329aa16303a0369d5beedc71785a4a2f36 --chown=root:root --chmod=644 --link https://github.com/astral-sh/uv/releases/download/0.8.6/uv-x86_64-unknown-linux-gnu.tar.gz uv.tar.gz
#8 DONE 0.2s

#12 [builder 2/8] WORKDIR /build
#12 ERROR: blob sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c not found

#11 [builder 1/8] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked     export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get upgrade -y     && apt-get install -y --no-install-recommends         gcc         g++         git         libgeos-dev         libhdf5-dev         libnetcdf-dev         libudunits2-dev         libpq-dev         python3-dev
#11 ERROR: blob sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c not found

#9 [base 1/1] FROM ghcr.io/osgeo/gdal:ubuntu-small-3.10.3@sha256:dab45abca3ca83695d442018692f4f8a0f41955871c57e6101d7f89a92375caa
#9 sha256:3b5643382fbea35a6bf4d8b87d0d02db8b3abdf19105b03489c72e63cc4d764c 0B / 1.57kB 0.2s
#9 sha256:3b5643382fbea35a6bf4d8b87d0d02db8b3abdf19105b03489c72e63cc4d764c 1.57kB / 1.57kB 0.2s done
#9 sha256:50fc4bfbbaf374c273c554992eb7e14b5e66dc6dfa59a72f46023825e5d6fcc9 0B / 13.04MB 0.2s
#9 sha256:39ad25c843b9f5e91537cbeac62b74600dc933480bf2c70bff4e86638d0055e9 4.76kB / 4.76kB 0.3s done
#9 sha256:8db866352ee9a93837649c318d4c68c7bdc139649114a6a9847566384afeb403 0B / 1.84MB 0.2s
#9 sha256:8db866352ee9a93837649c318d4c68c7bdc139649114a6a9847566384afeb403 1.84MB / 1.84MB 0.3s done
#9 sha256:a70bd6de95454a164db1d93a8460ab6486c169e1b9014fdcc04f9d29dc57840c 0B / 354.81kB 0.2s
#9 sha256:44bc6ea805ba015e19b3bc0278cbf23399852dc7d127879cf7beb91417a0a4e4 0B / 248.04kB 0.2s
#9 sha256:a70bd6de95454a164db1d93a8460ab6486c169e1b9014fdcc04f9d29dc57840c 354.81kB / 354.81kB 0.2s done
#9 sha256:44bc6ea805ba015e19b3bc0278cbf23399852dc7d127879cf7beb91417a0a4e4 248.04kB / 248.04kB 0.2s done
#9 sha256:117fe295ae589c229bf12bc2aef5912da375efb8228d2aae8e59b634cc91526c 4.78kB / 4.78kB 0.2s done
#9 sha256:a0bc7a5eb5b5127ba80d11220edf8348f976bf3135889f32df0bca6b88bd1b36 0B / 1.86MB 0.2s
#9 sha256:33aea99ffc8a88770924a15e0943120b254aacd6c6c922638b9253af4578e781 0B / 185.05kB 0.2s
#9 sha256:95d729dc196f0de01e89d7d55c288cf55804f87f9fa335c013d12698f1db48e0 0B / 86.43kB 0.2s
#9 sha256:a0bc7a5eb5b5127ba80d11220edf8348f976bf3135889f32df0bca6b88bd1b36 1.86MB / 1.86MB 0.4s done
#9 sha256:95d729dc196f0de01e89d7d55c288cf55804f87f9fa335c013d12698f1db48e0 86.43kB / 86.43kB 0.2s done
#9 sha256:2e1d5f4c431c089552361d7b87cea1ef3d39d56b80a86da97b527df140f31068 1.83MB / 1.83MB 0.2s done
#9 sha256:6126b1ee49c268328e4eb0055e4a8d6a709cae8f1f0ccd7332ba5988e86f7221 0B / 6.63MB 0.2s
#9 sha256:6126b1ee49c268328e4eb0055e4a8d6a709cae8f1f0ccd7332ba5988e86f7221 6.63MB / 6.63MB 0.3s done
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 0B / 88.87MB 0.2s
#9 sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0B / 32B 0.2s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 12.58MB / 88.87MB 0.5s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 20.97MB / 88.87MB 0.6s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 28.31MB / 88.87MB 0.8s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 36.70MB / 88.87MB 0.9s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 45.09MB / 88.87MB 1.1s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 53.48MB / 88.87MB 1.2s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 61.87MB / 88.87MB 1.4s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 70.25MB / 88.87MB 1.5s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 78.64MB / 88.87MB 1.7s
#9 sha256:da6dcbe481afd0d13e333088cf0fef365343dbc57a1f2b917683695eb07b986b 88.87MB / 88.87MB 1.8s done
#9 sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040 0B / 29.75MB 0.2s
#9 sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040 8.39MB / 29.75MB 0.3s
#9 sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040 16.78MB / 29.75MB 0.5s
#9 sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040 25.17MB / 29.75MB 0.6s
#9 sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040 29.75MB / 29.75MB 0.7s done
#9 extracting sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040
#9 extracting sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040 0.7s done
#9 sha256:50fc4bfbbaf374c273c554992eb7e14b5e66dc6dfa59a72f46023825e5d6fcc9 0B / 13.04MB 5.3s
#9 sha256:33aea99ffc8a88770924a15e0943120b254aacd6c6c922638b9253af4578e781 0B / 185.05kB 5.3s
#9 sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0B / 32B 5.3s
#9 DONE 7.4s

#11 [builder 1/8] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked     export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get upgrade -y     && apt-get install -y --no-install-recommends         gcc         g++         git         libgeos-dev         libhdf5-dev         libnetcdf-dev         libudunits2-dev         libpq-dev         python3-dev
#11 ERROR: failed to copy: httpReadSeeker: failed open: unexpected status code https://ghcr.io/v2/osgeo/gdal/blobs/sha256:50fc4bfbbaf374c273c554992eb7e14b5e66dc6dfa59a72f46023825e5d6fcc9: 502 Bad Gateway
------
 > [builder 1/8] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked     export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get upgrade -y     && apt-get install -y --no-install-recommends         gcc         g++         git         libgeos-dev         libhdf5-dev         libnetcdf-dev         libudunits2-dev         libpq-dev         python3-dev:
------
------
 > [builder 2/8] WORKDIR /build:
------
------
 > [builder 3/8] ADD --checksum=sha256:5429c9b96cab65198c2e5bfe83e933329aa16303a0369d5beedc71785a4a2f36 --chown=root:root --chmod=644 --link https://github.com/astral-sh/uv/releases/download/0.8.6/uv-x86_64-unknown-linux-gnu.tar.gz uv.tar.gz:
------
------
 > [builder 4/8] RUN tar xf uv.tar.gz -C /usr/local/bin --strip-components=1 --no-same-owner:
------
------
 > [builder 6/8] RUN --mount=type=cache,id=opendatacube-uv-cache,target=/root/.cache     uv sync --frozen --extra=deployment --no-install-project       --no-binary-package fiona       --no-binary-package netcdf4       --no-binary-package psycopg2       --no-binary-package rasterio       --no-binary-package shapely:
------
------
 > [builder 5/8] COPY --link pyproject.toml uv.lock /build/:
------
Dockerfile:18
--------------------
  17 |     
  18 | >>> RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  19 | >>>     --mount=type=cache,target=/var/lib/apt,sharing=locked \
  20 | >>>     export DEBIAN_FRONTEND=noninteractive \
  21 | >>>     && apt-get update \
  22 | >>>     && apt-get upgrade -y \
  23 | >>>     && apt-get install -y --no-install-recommends \
  24 | >>>         gcc \
  25 | >>>         g++ \
  26 | >>>         git \
  27 | >>>         # For shapely with --no-binary.
  28 | >>>         libgeos-dev \
  29 | >>>         libhdf5-dev \
  30 | >>>         libnetcdf-dev \
  31 | >>>         libudunits2-dev \
  32 | >>>         # For psycopg2.
  33 | >>>         libpq-dev \
  34 | >>>         python3-dev
  35 |     
--------------------
ERROR: failed to build: failed to solve: failed to compute cache key: failed to copy: httpReadSeeker: failed open: unexpected status code https://ghcr.io/v2/osgeo/gdal/blobs/sha256:50fc4bfbbaf374c273c554992eb7e14b5e66dc6dfa59a72f46023825e5d6fcc9: 502 Bad Gateway

Additional info

Could this be a fault attribution issue, so the error wasn't actually the RUN line that gets reported? The last lines of the log is:

ERROR: failed to build: failed to solve: failed to compute cache key: failed to copy: httpReadSeeker: failed open: unexpected status code https://ghcr.io/v2/osgeo/gdal/blobs/sha256:50fc4bfbbaf374c273c554992eb7e14b5e66dc6dfa59a72f46023825e5d6fcc9: 502 Bad Gateway

and that was the Github CI runner apparently failing to download the GDAL 3.10.3 image from ghcr.io. My builder image uses that image, so it would make sense if not getting all the layers of the FROM image makes things go haywire.

[tonistiigi]

"502 Bad Gateway" seems to suggest some intermittent internal error in the registry backend.

[pjonsson]

@tonistiigi sure, but this was from a Github CI runner to the Github registry, so these things happen even within the network of a single organization.

Can we get the build to stop on the first ERROR: blob sha256:... message, or does 6/11 happen at the same time as 7/11 and 8/11 so immediately stopping is difficult?

Orthogonal to that, would it be possible to get an error message that is more descriptive? I understand that not being able to find the blob is the error somewhere deep inside the mechanics, but as a user I would prefer Docker printing printing ERROR: downloading https://ghcr.io/v2/osgeo/gdal/blobs/sha256:50fc4bfbbaf374c273c554992eb7e14b5e66dc6dfa59a72f46023825e5d6fcc9 got '502 Bad Gateway' as response so I know it's not the regular problem where Docker has lost track of some blob in its cache.