Skip to content

feat: dockerize evm-prover service and add to compose network#171

Open
blasrodri wants to merge 23 commits intomainfrom
blas/dockerize-evm-prover
Open

feat: dockerize evm-prover service and add to compose network#171
blasrodri wants to merge 23 commits intomainfrom
blas/dockerize-evm-prover

Conversation

@blasrodri
Copy link
Contributor

Add complete Docker infrastructure for evm-prover service including multi-stage build, SP1 toolchain, health checks, and docker-compose integration.

Resolves #81

Overview

Add complete Docker infrastructure for evm-prover service including
multi-stage build, SP1 toolchain, health checks, and docker-compose
integration.

Resolves #81
@blasrodri blasrodri self-assigned this Sep 2, 2025
@blasrodri blasrodri marked this pull request as ready for review September 4, 2025 11:46
Copy link
Collaborator

@damiannolan damiannolan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @blasrodri! 🙏🏻

I left some comments/feedback, lemme know what you think!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly if we want some custom docker-entrypoint.sh for our envirnoment we can put it inside testnet/evm-prover

Add evm-prover image build step to GitHub Actions docker workflow.
The workflow now builds and pushes both hyperlane-init and evm-prover
images to GHCR on pushes to main branch.
Copy link
Collaborator

@damiannolan damiannolan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you run the docker compose network since adding the prover service?
I'm curious how the block prover loop will work when we haven't specified the SP1_PROVER env variable, since we instantiate the prover client with from_env().

It's possible we may need to add the SP1_PROVER=mock as an env variable to the container

@blasrodri blasrodri added the enhancement New feature or request label Sep 10, 2025
@damiannolan
Copy link
Collaborator

The image fails to build for me on make start with:

=> ERROR [evm-prover builder 31/31] RUN cargo build --release --bin evm-prover                                                                                                                              22.5s
------
 > [evm-prover builder 31/31] RUN cargo build --release --bin evm-prover:
4.376    Compiling base64ct v1.8.0
4.376    Compiling base16ct v0.2.0
4.376    Compiling crypto-bigint v0.5.5
4.376    Compiling ff v0.12.1
4.385    Compiling memuse v0.2.2
4.389    Compiling hmac v0.12.1
4.392    Compiling parity-scale-codec v3.7.5
4.393    Compiling const_format v0.2.34
4.393    Compiling signature v2.2.0
4.399    Compiling byte-slice-cast v1.2.3
4.401    Compiling derive_more v1.0.0
4.402    Compiling blake2b_simd v1.0.3
4.408    Compiling sp1-core-machine v5.2.1
4.936    Compiling rfc6979 v0.4.0 (https://github.com/sp1-patches/signatures.git?tag=patch-16.9-sp1-4.1.0#1880299a)
4.953    Compiling group v0.12.1
4.956    Compiling matchers v0.2.0
4.968    Compiling sharded-slab v0.1.7
4.972    Compiling tracing-log v0.2.0
4.974    Compiling num-rational v0.4.2
4.981    Compiling num-iter v0.1.45
5.056    Compiling pem-rfc7468 v0.7.0
5.068    Compiling num-complex v0.4.6
5.082    Compiling thread_local v1.1.9
5.092    Compiling nu-ansi-term v0.50.1
5.105    Compiling pairing v0.22.0
5.149    Compiling pasta_curves v0.4.1
5.176    Compiling der v0.7.10
5.187    Compiling num_enum v0.5.11
5.282    Compiling dashu v0.4.2
5.317    Compiling range-set-blaze v0.1.16
5.325    Compiling eyre v0.6.12
5.325    Compiling ark-std v0.4.0
5.344    Compiling generic-array v1.1.0
5.351    Compiling downcast-rs v1.2.1
5.390    Compiling num v0.4.3
5.443    Compiling indenter v0.3.4
5.455    Compiling tracing-subscriber v0.3.20
5.478    Compiling gen_ops v0.4.0
5.495    Compiling ark-serialize v0.4.2
5.500    Compiling rrs-succinct v0.1.0
5.600    Compiling scale-info v2.11.6
5.608    Compiling bls12_381 v0.7.1
5.617    Compiling object v0.36.7
5.641    Compiling enum-map v2.7.3
5.653    Compiling keccak v0.1.5
5.734    Compiling itertools v0.10.5
5.752    Compiling halo2_proofs v0.1.0
5.776    Compiling vec_map v0.8.2
5.779    Compiling nohash-hasher v0.2.0
5.850    Compiling gimli v0.31.1
5.862    Compiling adler2 v2.0.1
5.917    Compiling ansi_term v0.12.1
6.019    Compiling elf v0.7.4
6.046    Compiling bytemuck v1.23.2
6.087    Compiling spki v0.7.3
6.092    Compiling snowbridge-amcl v1.0.2
6.141    Compiling miniz_oxide v0.8.9
6.151    Compiling jubjub v0.9.0
6.196    Compiling pkcs8 v0.10.2
6.337    Compiling sec1 v0.7.3
6.345    Compiling ark-ff v0.4.2
6.353    Compiling sha3 v0.10.8 (https://github.com/sp1-patches/RustCrypto-hashes?tag=patch-sha3-0.10.8-sp1-4.0.0#8f6d303c)
6.380    Compiling pasta_curves v0.5.1
6.437    Compiling p3-keccak-air v0.2.3-succinct
6.488    Compiling elliptic-curve v0.13.8
6.515    Compiling tracing-forest v0.1.6
6.545    Compiling blake2 v0.10.6
6.604    Compiling num_cpus v1.17.0
6.720    Compiling sp1-recursion-core v5.2.1
6.775    Compiling ecdsa v0.16.9 (https://github.com/sp1-patches/signatures.git?tag=patch-16.9-sp1-4.1.0#1880299a)
6.792    Compiling primeorder v0.13.1 (https://github.com/sp1-patches/elliptic-curves?tag=patch-p256-13.2-sp1-5.0.0#10cca2ef)
6.845    Compiling web-time v1.1.0
6.896    Compiling size v0.4.1
6.943    Compiling rustc-demangle v0.1.26
6.992    Compiling p3-bn254-fr v0.2.3-succinct
7.008    Compiling lock_api v0.4.13
7.053    Compiling p256 v0.13.2 (https://github.com/sp1-patches/elliptic-curves?tag=patch-p256-13.2-sp1-5.0.0#10cca2ef)
7.054    Compiling k256 v0.13.4 (https://github.com/sp1-patches/elliptic-curves?tag=patch-k256-13.4-sp1-5.0.0#f7d8998e)
7.060    Compiling parking_lot_core v0.9.11
7.068    Compiling scopeguard v1.2.0
7.069    Compiling powerfmt v0.2.0
7.171    Compiling deranged v0.5.3
7.243    Compiling sp1-recursion-gnark-ffi v5.2.1
7.257    Compiling sdd v3.0.10
7.277    Compiling option-ext v0.2.0
7.304    Compiling addr2line v0.24.2
7.353    Compiling sp1-curves v5.2.1
7.414    Compiling scc v2.4.0
7.420    Compiling parking_lot v0.12.4
7.557    Compiling dirs-sys v0.4.1
7.616    Compiling camino v1.1.12
7.629    Compiling sp1-core-executor v5.2.1
7.675    Compiling crossbeam-channel v0.5.15
7.708    Compiling halo2 v0.1.0-beta.2
7.731    Compiling dirs v5.0.1
7.794    Compiling time v0.3.43
7.807    Compiling lru v0.12.5
7.906    Compiling cargo-platform v0.1.9
7.921    Compiling iana-time-zone v0.1.63
7.960 warning: function `main` is never used
7.960  --> crates/sp1/evm-exec-types/src/lib.rs:1:4
7.960   |
7.960 1 | fn main() {}
7.960   |    ^^^^
7.960   |
7.960   = note: `#[warn(dead_code)]` on by default
7.960 
7.960 warning: `evm-exec-types` (lib) generated 1 warning
8.050    Compiling cargo_metadata v0.18.1
8.062    Compiling chrono v0.4.41
8.188    Compiling serial_test v3.2.0
8.326    Compiling sp1-prover v5.2.1
8.404    Compiling backtrace v0.3.75
8.577    Compiling tracing-appender v0.2.3
8.742    Compiling zkhash v0.2.0
11.58    Compiling sp1-recursion-compiler v5.2.1
12.09    Compiling sp1-recursion-circuit v5.2.1
14.80    Compiling sp1-build v5.2.1
17.28    Compiling evm-prover v0.1.0 (/app/crates/evm-prover)
22.02 error: failed to run custom build command for `evm-prover v0.1.0 (/app/crates/evm-prover)`
22.02 
22.02 Caused by:
22.02   process didn't exit successfully: `/app/target/release/build/evm-prover-786f94bd335b7c58/build-script-build` (exit status: 101)
22.02   --- stdout
22.02   cargo:rerun-if-changed=../../proto/prover/v1/prover.proto
22.02   cargo:rerun-if-changed=../../proto
22.02   cargo::rerun-if-changed=/app/crates/sp1/evm-exec/program/src
22.02   cargo::rerun-if-changed=/app/crates/sp1/evm-exec/program/Cargo.toml
22.02   cargo:rerun-if-changed=/app/Cargo.lock
22.02   cargo:rerun-if-changed=/app/crates/sp1/evm-exec-types
22.02   cargo:rerun-if-changed=/app/crates/sp1/evm-exec-types
22.02   cargo:rerun-if-changed=/app/crates/sp1/evm-exec-types
22.02   cargo:rerun-if-changed=/app/crates/sp1/evm-exec-types
22.02   cargo:rerun-if-changed=/app/crates/sp1/evm-exec-types
22.02   cargo:rerun-if-changed=/app/crates/evm-state-types
22.02   cargo:rerun-if-changed=/app/crates/storage
22.02   cargo:rerun-if-changed=/app/crates/evm-state-types
22.02 
22.02   --- stderr
22.02 
22.02   thread 'main' panicked at /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/sp1-build-5.2.1/src/command/local.rs:42:13:
22.02   Failed to run rustc --version ""
22.02   "rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2\n "
22.02   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
------
failed to solve: process "/bin/sh -c cargo build --release --bin evm-prover" did not complete successfully: exit code: 101
make: *** [start] Error 1

any ideas?

@damiannolan
Copy link
Collaborator

I'm still getting:

 => ERROR [evm-prover builder 29/33] RUN cargo build --release --bin evm-prover                                                                                                                               0.4s
------
 > [evm-prover builder 29/33] RUN cargo build --release --bin evm-prover:
0.349 error: failed to load manifest for workspace member `/app/crates/sp1/evm-hyperlane-types`
0.349 referenced by workspace at `/app/Cargo.toml`
0.349 
0.349 Caused by:
0.349   failed to read `/app/crates/sp1/evm-hyperlane-types/Cargo.toml`
0.350 
0.350 Caused by:
0.350   No such file or directory (os error 2)
------
failed to solve: process "/bin/sh -c cargo build --release --bin evm-prover" did not complete successfully: exit code: 101
make: *** [start] Error 1

Not sure if the PR is intended to be ready for review or not, should we convert it to draft?

@blasrodri
Copy link
Contributor Author

I'm still getting:

 => ERROR [evm-prover builder 29/33] RUN cargo build --release --bin evm-prover                                                                                                                               0.4s
------
 > [evm-prover builder 29/33] RUN cargo build --release --bin evm-prover:
0.349 error: failed to load manifest for workspace member `/app/crates/sp1/evm-hyperlane-types`
0.349 referenced by workspace at `/app/Cargo.toml`
0.349 
0.349 Caused by:
0.349   failed to read `/app/crates/sp1/evm-hyperlane-types/Cargo.toml`
0.350 
0.350 Caused by:
0.350   No such file or directory (os error 2)
------
failed to solve: process "/bin/sh -c cargo build --release --bin evm-prover" did not complete successfully: exit code: 101
make: *** [start] Error 1

Not sure if the PR is intended to be ready for review or not, should we convert it to draft?

Yes, I will put it as draft until I'm fully able to run it locally first.

@blasrodri blasrodri marked this pull request as draft September 15, 2025 15:48
@blasrodri blasrodri marked this pull request as ready for review September 16, 2025 18:50
@blasrodri
Copy link
Contributor Author

Finally, I removed all the caching optimizations because they were giving some errors due to certain structs not being found when they were there. I rather focus on having a working image first, and then improve it later on if we deem the build times too long.

Improved the Dockerfile caching by adding cargo fetch between copying
  the manifests and copying the source code
Copy link
Collaborator

@damiannolan damiannolan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to merge if we add a profile to have it disabled in default mode 🙏🏻

- Resolved conflicts in .github/workflows/docker.yml by using v6 of docker/build-push-action and adding arm64 platform support
- Re-applied cargo-sort CI check from issue #164
Makes the ev-prover service optional by adding a 'prover' profile.
This allows the service to be excluded from default docker compose up
and only started explicitly when needed.

Usage:
- docker compose --profile prover up
- docker compose up ev-prover
- docker compose run ev-prover

This keeps the service optional for the e2e demo flow while still
allowing it to be deployed when needed.
@blasrodri blasrodri requested a review from damiannolan October 13, 2025 12:24
tags: ghcr.io/${{ github.repository_owner }}/hyperlane-init:latest

- name: Build and push ev-prover Docker image
uses: docker/build-push-action@v6

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker Images' step
Uses Step
uses 'docker/build-push-action' with ref 'v6', not a pinned commit hash
Copy link
Collaborator

@damiannolan damiannolan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should address the docker workflow and remove the unneeded bind mount before we merge, otherwise lgtm

platforms: linux/amd64,linux/arm64
tags: ghcr.io/${{ github.repository_owner }}/hyperlane-init:latest

- name: Build and push ev-prover Docker image
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it would be better to create a separate workflow job for this so that we can have different build triggers.

Right now this workflow triggers a rebuild on merge to main when only the following the paths are changed, which doesn't make sense.

paths:
      - '**/Dockerfile'
      - '.github/workflows/docker.yml'
      - 'hyperlane/**'

We should likely only trigger builds of ev-prover when those files are changed and separate these

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes - now it's only on changes on this repo

# Function to initialize the service
init_service() {
echo "Initializing ev-prover service..."
/app/ev-prover init
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you run init in the entrypoint here I think you don't need to actually mount the genesis file at all, as there is a default genesis in ev-prover/resources which I think is included in the binary.

Should be possible to omit that bind mount

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right - i have addressed it

- Split docker.yml into separate jobs for hyperlane-init and ev-prover
builds
- Add conditional execution to only trigger builds when relevant files
change
- Remove unnecessary genesis.json bind mount from ev-prover service
  (genesis is embedded in binary and written by init command)
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
install: true

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker Images' step
Uses Step
uses 'docker/setup-qemu-action' with ref 'v3', not a pinned commit hash
install: true

- name: Log in to GitHub Container Registry
uses: docker/login-action@v3

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker Images' step
Uses Step
uses 'docker/setup-buildx-action' with ref 'v3', not a pinned commit hash
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker Images' step
Uses Step
uses 'docker/login-action' with ref 'v3', not a pinned commit hash
@blasrodri blasrodri requested a review from damiannolan October 16, 2025 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dockerize evm-prover and add to compose network

2 participants