build: multi-stage builds and testing configuration

- Introduced a new `compose.yml` file for running tests in an isolated Docker environment.
- Updated `Dockerfile` to support multi-stage builds, including a dedicated test stage with Bitcoin Core.
- Enhanced `install.sh` to allow installation of dependencies only, without building JoinMarket.
- Updated documentation to include Docker usage instructions for building images and running tests.

build: add version arguments

build: update python image tag to 3.13-slim-trixie

Author: roshii

Dockerfile

@@ -1,16 +1,31 @@
-FROM debian:bookworm-slim
+ARG BITCOIN_VERSION=29.2
+ARG PYTHON_IMAGE_TAG=3.13-slim-trixie 
+FROM bitcoin/bitcoin:${BITCOIN_VERSION} AS bitcoin
+FROM python:${PYTHON_IMAGE_TAG} AS joinmarket
 
-RUN mkdir -p /jm/clientserver
 WORKDIR /jm/clientserver
+COPY ./pubkeys ./pubkeys
+COPY ./install.sh ./install.sh
+
+RUN apt-get update \
+  && apt-get install -y --no-install-recommends \
+    gnupg \
+    ca-certificates \
+    curl \
+  && ./install.sh --docker-install --deps-only \
+  && apt-get clean \
+  && rm -rf /var/lib/apt/lists/*
 
 COPY . .
 
-RUN apt-get update && apt-get install -y --no-install-recommends gnupg ca-certificates=* curl=* \
-  python3-pip=* python3=* \
-  && pip3 config set global.break-system-packages true \
-  && pip3 install 'wheel>=0.35.1' \
-  && ./install.sh --docker-install \
-  && apt-get purge -y --autoremove python3-pip \
+RUN apt-get update \
+  && ./install.sh --docker-install --no-deps \
   && apt-get clean \
   && rm -rf /var/lib/apt/lists/*
 
+FROM joinmarket AS test
+ARG BITCOIN_VERSION
+COPY --from=bitcoin /opt/bitcoin-${BITCOIN_VERSION}/bin /usr/local/bin/
+RUN ./install.sh --docker-install --no-deps --develop
+
+FROM joinmarket AS final

README.md

@@ -62,6 +62,18 @@ Alternative to this "quickstart": follow the [install guide](docs/INSTALL.md).
 * [Installation guide for Qubes+Whonix](https://github.com/qubenix/qubes-whonix-bitcoin/blob/master/1_joinmarket.md).
 * [Youtube video installation tutorial for Ubuntu](https://www.youtube.com/watch?v=zTCC86IUzWo).
 
+### Docker
+
+JoinMarket can be built and run using Docker. To build the Docker image:
+
+    docker build -t joinmarket .
+
+To run tests using Docker Compose:
+
+    docker compose up test
+
+This will run the full test suite in an isolated environment with all necessary dependencies including Bitcoin Core.
+
 ### Usage
 
 If you are new, follow and read the links in the [usage guide](docs/USAGE.md).

compose.yml

@@ -0,0 +1,15 @@
+services:
+  test:
+    image: joinmarket
+    build:
+      context: .
+      target: test
+      args:
+        BITCOIN_VERSION: 29.2
+        PYTHON_IMAGE_TAG: 3.13-slim-trixie
+    volumes:
+      - ./src:/jm/clientserver/src
+      - ./test:/jm/clientserver/test
+    shm_size: '2gb'
+    entrypoint: ["/bin/sh", "-c"]
+    command: [". ./jmvenv/bin/activate && exec ./test/run_tests.sh -v"]

docs/INSTALL.md

@@ -265,14 +265,47 @@ There, you need to install the client code (without Joinmarket's bitcoin):
 
 #### Docker Installation
 
-The [Dockerfile](../Dockerfile) provided builds a minimal Docker image which can help in getting started with a custom Docker setup. An example of building and running the [wallet-tool.py](../scripts/wallet-tool.py) script:
+The [Dockerfile](../Dockerfile) provided builds an optimized multi-stage Docker image for JoinMarket. The build process is optimized for layer caching and includes all necessary dependencies.
 
-```
-docker build -t joinmarket-test ./
-docker run --rm -it joinmarket-test bash -c "cd scripts && python3 wallet-tool.py --help"
-```
+##### Building the Docker image
+
+To build the production image:
+
+    docker build -t joinmarket .
+
+This creates a minimal production image with JoinMarket installed.
+
+##### Running JoinMarket scripts in Docker
+
+Example of running the [wallet-tool.py](../scripts/wallet-tool.py) script:
+
+    docker run --rm -it joinmarket bash -c "source jmvenv/bin/activate && cd scripts && python wallet-tool.py --help"
+
+##### Running tests with Docker Compose
+
+The repository includes a `compose.yml` file for easy testing:
+
+    docker compose up test
+
+This will:
+
+* Build a test image that includes Bitcoin Core 29.2
+* Mount the `src` and `test` directories for development
+* Run the full test suite with all dependencies configured automatically
+* Use shared memory for improved test performance
+
+##### Building custom images
+
+The Dockerfile uses multi-stage builds with the following targets:
+
+* `joinmarket` (default) - Production image with JoinMarket installed
+* `test` - Testing image that includes Bitcoin Core binaries
+
+You can build a specific target:
+
+    docker build --target test -t joinmarket:test .
 
-A new Docker image can be built using `joinmarket-test` as a base using `FROM joinmarket-test`. See [Docker documentation](https://docs.docker.com/engine/reference/builder/) for more details.
+See [Docker documentation](https://docs.docker.com/engine/reference/builder/) for more details on multi-stage builds.
 
 #### Development (or making other changes to the code)
 

docs/TESTING.md

@@ -1,5 +1,22 @@
 ### Test instructions (for developers):
 
+#### Quick start: Running tests with Docker
+
+The easiest way to run the full test suite is using Docker Compose, which handles all dependencies automatically:
+
+    docker compose run --rm test
+
+This will:
+
+* Build a Docker image with all dependencies (Python, Bitcoin Core 29.2, miniircd)
+* Run the complete test suite in an isolated environment
+* Automatically download and set up miniircd
+* No need to install bitcoind or other dependencies on your host machine
+
+For development, the Docker setup mounts the `src` and `test` directories, so you can make changes locally and re-run tests without rebuilding the image.
+
+#### Manual setup
+
 Work in your `jmvenv` virtual environment as for all Joinmarket work. Make sure to have [bitcoind](https://bitcoin.org/en/full-node) 29.0 or newer installed. Also need miniircd installed to the root (i.e. in your `joinmarket-clientserver` directory):
 
     (jmvenv)$ cd /path/to/joinmarket-clientserver

install.sh

@@ -415,6 +415,9 @@ CookieAuthentication 1
 
 joinmarket_install ()
 {
+    if [[ ${deps_only} == 1 ]]; then
+        return 0
+    fi
     reqs='services'
 
     if [[ ${with_qt} == "1" ]]; then
@@ -424,8 +427,7 @@ joinmarket_install ()
         reqs+=',test'
     fi
 
-    if [ "$with_jmvenv" == 1 ]; then pip_command=pip; else pip_command=pip3; fi
-    $pip_command install -e ".[${reqs}]" || return 1
+    pip install -e ".[${reqs}]" || return 1
 
     if [[ ${with_qt} == "1" ]]; then
         if [[ -d ~/.local/share/icons ]] && [[ -d ~/.local/share/applications ]]; then
@@ -484,7 +486,12 @@ parse_flags ()
                 ;;
             --docker-install)
                 with_sudo='0'
-                with_jmvenv='0'
+                ;;
+            --deps-only)
+                deps_only='1'
+                ;;
+            --no-deps)
+                no_deps='1'
                 ;;
             "")
                 break
@@ -507,6 +514,8 @@ Options:
 --with-local-tor            build Tor locally and autostart when needed
 --with-qt                   build the Qt GUI
 --without-qt                don't build the Qt GUI
+--deps-only                 only install dependencies, do not build or install Joinmarket
+--no-deps                   do not install dependencies
 "
                 return 1
                 ;;
@@ -568,22 +577,19 @@ main ()
     use_os_deps_check='1'
     use_secp_check='1'
     with_qt=''
-    with_jmvenv='1'
     with_sudo='1'
     reinstall='false'
+    deps_only='0'
+    no_deps='0'
     if ! parse_flags "${@}"; then
         return 1
     fi
 
     jm_source="$PWD"
-    if [ "$with_jmvenv" == 1 ]; then
-        jm_root="${jm_source}/jmvenv"
-        export PKG_CONFIG_PATH="${jm_root}/lib/pkgconfig:${PKG_CONFIG_PATH}"
-        export LD_LIBRARY_PATH="${jm_root}/lib:${LD_LIBRARY_PATH}"
-        export C_INCLUDE_PATH="${jm_root}/include:${C_INCLUDE_PATH}"
-    else
-        jm_root=""
-    fi
+    jm_root="${jm_source}/jmvenv"
+    export PKG_CONFIG_PATH="${jm_root}/lib/pkgconfig:${PKG_CONFIG_PATH}"
+    export LD_LIBRARY_PATH="${jm_root}/lib:${LD_LIBRARY_PATH}"
+    export C_INCLUDE_PATH="${jm_root}/include:${C_INCLUDE_PATH}"
 
     # os check
     install_os="$( install_get_os )"
@@ -595,56 +601,53 @@ main ()
 
     MAKEFLAGS="-j $(num_cores)" && export MAKEFLAGS
 
-    if [ "$with_jmvenv" == 1 ]; then
-        if ! venv_setup; then
-            echo "Joinmarket Python virtual environment could not be setup. Exiting."
-            return 1
-        fi
-        # shellcheck source=/dev/null
-        source "${jm_root}/bin/activate"
-    else
-        upgrade_setuptools
+    if ! venv_setup; then
+        echo "Joinmarket Python virtual environment could not be setup. Exiting."
+        return 1
     fi
+    # shellcheck source=/dev/null
+    source "${jm_root}/bin/activate"
     if [[ ${build_local_tor} == "1" ]]; then
         if ! tor_deps_install; then
             echo "Tor dependencies could not be installed. Exiting."
             return 1
         fi
     fi
-    mkdir -p "deps/cache"
-    pushd deps || return 1
-    if ! libsecp256k1_install; then
-        echo "libsecp256k1 was not built. Exiting."
-        return 1
-    fi
-    if ! libffi_install; then
-        echo "Libffi was not built. Exiting."
-        return 1
-    fi
-    if ! libsodium_install; then
-        echo "Libsodium was not built. Exiting."
-        return 1
-    fi
-    if [[ ${build_local_tor} == "1" ]]; then
-        if ! tor_install; then
-            echo "Building local Tor was requested, but not built. Exiting."
+
+    if [[ ${no_deps} == 0 ]]; then
+        mkdir -p "deps/cache"
+        pushd deps || return 1
+        if ! libsecp256k1_install; then
+            echo "libsecp256k1 was not built. Exiting."
+            return 1
+        fi
+        if ! libffi_install; then
+            echo "Libffi was not built. Exiting."
+            return 1
+        fi
+        if ! libsodium_install; then
+            echo "Libsodium was not built. Exiting."
             return 1
         fi
+        if [[ ${build_local_tor} == "1" ]]; then
+            if ! tor_install; then
+                echo "Building local Tor was requested, but not built. Exiting."
+                return 1
+            fi
+        fi
+        popd || return 1
     fi
-    popd || return 1
     if ! joinmarket_install; then
         echo "Joinmarket was not installed. Exiting."
-        if [ "$with_jmvenv" == 1 ]; then deactivate; fi
+        deactivate
         return 1
     fi
-    if [ "$with_jmvenv" == 1 ]; then
-        deactivate
-        echo "Joinmarket successfully installed
-        Before executing scripts or tests, run:
+    deactivate
+    echo "Joinmarket successfully installed
+    Before executing scripts or tests, run:
 
-        \`source jmvenv/bin/activate\`
+    \`source jmvenv/bin/activate\`
 
-        from this directory, to activate the virtual environment."
-    fi
+    from this directory, to activate the virtual environment."
 }
 main "${@}"