Teamengine running from docker is not able to test a server launched with docker

[doublebyte1]

Description of the Problem

If we start a local instance of a server with docker, and attempt to test it using a docker container with teamengine, teamengine will not be able to access the server paths, despite using host.docker.internal, as described in the documentation. As a consequence of that, the cite-runner validation will also fail.

As an example, we can start a local pygeoapi server with this configuration:

server:
    bind:
        host: 0.0.0.0
        port: 80
    url: http://localhost:80

And start teamengine, as suggested here:

docker pull ogccite/teamengine-production:1.0-SNAPSHOT
docker run \
    --rm \
    --name=teamengine \
    --add-host=host.docker.internal:host-gateway \
    --publish=9080:8080 \
    ogccite/teamengine-production:1.0-SNAPSHOT

This is the result of the teamengine test session:

Results for session s0001
Test Name: ogcapi-features-1.0
Test version: 1.6
Time: 2025-06-25T13:55:17.777Z

Test INPUT:
noofcollections : 1
Tested Instance : http://host.docker.internal:80

Result:
Passed Core conformance classes (Implementations passing these classes can be certified): No
Number of conformance classes tested: 2
Number of conformance classes passed: 0
Number of conformance classes failed: 1

We can see it fails one of the core tests, which is checking the accessibility of the openapi path, and then skips subsequent tests.

openapi Document Retrieval -> Connection refused (Connection refused)

As a consequence, the cite-runner will also fail the tests:

ogc-cite-runner execute-test-suite \
    http://localhost:9080/teamengine \
    ogcapi-features-1.0 \
    --suite-input iut  http://host.docker.internal:80 \
    --suite-input noofcollections 3 \

╭───────────────────── Test suite ogcapi-features-1.0-1.6 ─────────────────────╮
│ cite-runner is not affiliated with the OGC. Having a CITE test suite be      │
│ declared as passed by cite-runner does not mean the implementation under     │
│ test is OGC certified nor does it mean that it is guaranteed to pass the     │
│ official CITE certification program.                                         │
│                                                                              │
│ Test suite has failed ❌                                                     │
│                                                                              │
│ - Ran 42 tests in 0.28 seconds                                               │
│ - 🔴 Failed 3 tests                                                          │
│ - 🟡 Skipped 36 tests                                                        │
│ - 🟢 Passed 3 tests                                                          │
│                                                                              │
│                                                                              │
│                             Conformance classes                              │
│   ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┓   │
│   ┃ Class                           ┃ 🔴 Failed ┃ 🟡 Skipped ┃ 🟢 Passed ┃   │
│   ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━┩   │
│   │ Coordinate Reference Systems by │ 0         │ 25         │ 0         │   │
│   │ Reference                       │           │            │           │   │
│   │ Core                            │ 3         │ 11         │ 3         │   │
│   └─────────────────────────────────┴───────────┴────────────┴───────────┘   │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

Workaround/ Working Solution

A workaround to this issue, is to make both containers (the local OGCAPI server and teamengine) join the same network, so that they can communicate with each other using that network, rather than the host's.

For instance, if we start the server with:

server:
    bind:
        host: 0.0.0.0
        port: 80
    url: http://pygeoapi:80

    networks:
      - my-shared-network

networks:
     my-shared-network:
      name: my-shared-network
      driver: bridge

And then launch teamengine, joining the my-shared-network created by the previous container:

docker run \
    --rm \
    --name=teamengine \
    --add-host=host.docker.internal:host-gateway \
    --publish=9080:8080 \
    --network=my-shared-network \
    ogccite/teamengine-production:1.0-SNAPSHOT

We can then run the tests, pointing them to http://pygeoapi:80, the name of the container.

Results for session s0001
Test Name: ogcapi-features-1.0
Test version: 1.6
Time: 2025-06-25T13:37:21.483Z

Test INPUT:
noofcollections : 3
Tested Instance : http://pygeoapi:80

Result:
Passed Core conformance classes (Implementations passing these classes can be certified): Yes
Number of conformance classes tested: 2
Number of conformance classes passed: 2
Number of conformance classes failed: 0

Subsequently, cite-runner will also pass the tests:

ogc-cite-runner execute-test-suite \
    http://localhost:9080/teamengine \
    ogcapi-features-1.0 \
    --suite-input iut http://pygeoapi:80 \
    --suite-input noofcollections 3 \

╭───────────────────── Test suite ogcapi-features-1.0-1.6 ─────────────────────╮
│ cite-runner is not affiliated with the OGC. Having a CITE test suite be      │
│ declared as passed by cite-runner does not mean the implementation under     │
│ test is OGC certified nor does it mean that it is guaranteed to pass the     │
│ official CITE certification program.                                         │
│                                                                              │
│ Test suite has passed 🏅                                                     │
│                                                                              │
│ - Ran 157 tests in 6.20 seconds                                              │
│ - 🔴 Failed 0 tests                                                          │
│ - 🟡 Skipped 9 tests                                                         │
│ - 🟢 Passed 148 tests                                                        │
│                                                                              │
│                                                                              │
│                             Conformance classes                              │
│   ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┓   │
│   ┃ Class                           ┃ 🔴 Failed ┃ 🟡 Skipped ┃ 🟢 Passed ┃   │
│   ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━┩   │
│   │ Coordinate Reference Systems by │ 0         │ 2          │ 34        │   │
│   │ Reference                       │           │            │           │   │
│   │ Core                            │ 0         │ 7          │ 114       │   │
│   └─────────────────────────────────┴───────────┴────────────┴───────────┘   │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

Complete Example

A complete example can be viewed in this GitHub workflow, that uses cite-runner standalone.

As it is not possible to control the behaviour of the teamengine container inside the cite-runner Guthub action, the cite-runner action cannot be used in this context. One possible solution to enable the action could be to accept a network parameter for the teamengine container.

[ricardogsilva]

@doublebyte1 There are different ways in which we can set up the communication between the service under test and the underlying TeamEngine service used by ogc-cite-runner. The one mentioned in the docs seems the simplest to me - even if it is already mildly complicated. I would like to avoid asking users to create custom docker networks and all that, but maybe under certain scenarios it can be the best option.

Having said that, this concrete issue does not seem to be a bug to me. If I understand your setup, I think this is simply a (pygeoapi) configuration issue.

In your first example you set your server's public URL like this

server:
  url: http://localhost:80

This presumably means that any links found in your server's generated responses will have hrefs that start with this public URL. For example the openapi document is likely being advertised to be at http://localhost:80/openapi. This means that TeamEngine will try to access it at that URL. However, because TeamEngine is running inside a docker container it will try to resolve to the container's own localhost, which is not where your server is running.

What if you configured your server like this instead:

server:
  url: http://host.docker.internal:80

This would make your server generate an openapi-related URL like http://host.docker.internal:80/openapi, which would cause the TeamEngine service to correctly send a request to the host's localhost, which is where your server is running.

I think this would solve your problem. If it does not, then I may have misunderstood the scenario - let me know 😉

[ricardogsilva]

Here you can find an example where I apply this same technique in a (still draft) PR that I'm hoping to have be merged in pygeoapi soon:

https://github.com/ricardogsilva/pygeoapi/blob/468971ec77b6ecbd024552e794f3e21f63563352/tests/cite/compose.test-cite.yaml#L40