Run tests locally provisioning in Kind #1567
Conversation
The existing `test-in-k8s.sh` script runs tests in Kind (`mvn test` in a pod). Sometimes it's convenient to have the controller running locally while still provisioning pods in Kind.
| name: dockerhost | ||
| subsets: | ||
| - addresses: | ||
| - ip: @GATEWAY_IP@ # this is the gateway IP in the "kind" docker network |
There was a problem hiding this comment.
This is the trick to let agents running in Kind to connect to the controller running in the host.
jglick
left a comment
jglick
left a comment
There was a problem hiding this comment.
An interesting trick, though kubectl tunnel is probably safer and more portable. What host env did you test this on—Linux with docker-ce-cli from e.g.
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable
or some other packaging or Docker Desktop on Mac OS X etc.?
| gateway_ip=$(docker network inspect kind | jq '[.[].IPAM.Config[].Gateway][0]') | ||
| sed "s/@GATEWAY_IP@/$gateway_ip/g" < dockerhost.yaml | kubectl apply -f - | ||
|
|
||
| mvn -f ../pom.xml test -Dtest="$@" -Djenkins.host.address=dockerhost -Dhudson.TcpSlaveAgentListener.hostName=dockerhost -DconnectorHost=0.0.0.0 |
There was a problem hiding this comment.
0.0.0.0 would allow connections from anywhere on the same LAN, so if you run tests that do not set up strong authn/authz then anyone in the same coffee shop or whatever could run a build on your controller that gets shell access to your laptop. See https://github.com/jenkinsci/kubernetes-plugin/blob/41b3ef14a5d89693b6ca6e4f85ca9451996fd95f/README.md#integration-tests-with-minikube for example. (Also this setup should be called out in the top-level README.)
There was a problem hiding this comment.
Can't the ip of something like docker0 network interface be used instead?
Ubuntu ( |
Coffee shops are not for work ;) But yeah... |
|
Also see jenkinsci/jenkins-test-harness#289 (comment) & JENKINS-73138. |
|
I successfully prototyped a different approach in #1593 which seems simpler, avoids a special |
|
Tested #1593 locally and it works as expected. Closing this. |
3 participants
The existing
test-in-k8s.shscript runs tests in Kind (mvn testin a pod). Sometimes it's convenient to have the controller running locally while still provisioning pods in Kind.Just in case it's useful for others.
Testing done
Submitter checklist