Skip to content

Install rustup and cargo in CI and prod build images#64535

Closed
potiuk wants to merge 9 commits intoapache:mainfrom
potiuk:install-rustup-cargo-in-images
Closed

Install rustup and cargo in CI and prod build images#64535
potiuk wants to merge 9 commits intoapache:mainfrom
potiuk:install-rustup-cargo-in-images

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented Mar 31, 2026

Add rustup/cargo installation to the CI image and production build image so that
Python packages with Rust extensions can be compiled from source during image builds.

  • Add install_rustup() function to install_os_dependencies.sh
  • Call it for both CI and DEV (prod build) installation types
  • Set RUSTUP_HOME, CARGO_HOME env vars and add cargo to PATH in both Dockerfiles
  • Remove redundant /root/.cargo/bin PATH entry in Dockerfile.ci
  • Rust toolchain is NOT installed in the runtime image (matching the pattern for build-only tools)
Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Claude Opus 4.6)

Generated-by: Claude Code (Claude Opus 4.6) following the guidelines

@boring-cyborg boring-cyborg bot added area:dev-tools area:production-image Production image improvements and fixes backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch labels Mar 31, 2026
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Mar 31, 2026

This one should fix a nasty race condition that @radu-gheorghe had in #63988 where uv built two rust packages and installed rustup / cargo to cache at about the same time.

@potiuk potiuk force-pushed the install-rustup-cargo-in-images branch from 4dbceaf to 9eed782 Compare March 31, 2026 13:58
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Mar 31, 2026

This will need fix from #64547 likely

@potiuk potiuk force-pushed the install-rustup-cargo-in-images branch from 9eed782 to b028794 Compare April 1, 2026 13:27
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 1, 2026

This one also is quite a bit too long. I will add the same retry for installation in regular image build as in Low Deps to fix the issue cc: @radu-gheorghe

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 1, 2026

Closing in favour of #64588

@potiuk potiuk closed this Apr 1, 2026
@potiuk potiuk mentioned this pull request Apr 1, 2026
Closed
1 task
@potiuk potiuk reopened this Apr 4, 2026
@potiuk potiuk force-pushed the install-rustup-cargo-in-images branch from b028794 to d4d0500 Compare April 4, 2026 11:36
@potiuk potiuk requested a review from kaxil as a code owner April 4, 2026 11:36
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 4, 2026

OK. I restore that one, add longer timeout - this timeout is not really going to impact most of the builds, that will use local or remote cache. We should rebase #63988 and it should solve all the issues with rustup parallel installs

potiuk and others added 3 commits April 4, 2026 13:43
@potiuk
Verify rustup-init binary with SHA256 checksum instead of curl-pipe-sh
733f075 
Download the rustup-init binary directly and verify its SHA256 checksum
before execution, instead of piping the shell installer script through sh.
@potiuk
Pin rustup version with SHA256 verification like cosign
e8e8252 
Pin rustup-init to version 1.29.0 with hardcoded SHA256 checksums for
amd64 and arm64, matching the existing cosign verification pattern.
This prevents a compromised server from serving a tampered binary with
a matching checksum.
@radu-gheorghe @potiuk
Add Vespa provider
85ff4f4
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 4, 2026

Added extra verification / crypto/hash check of the init script. I will also - as next PR update "upgrede important version" prek to bump this one - with cooldown - when new version is released.

@potiuk potiuk closed this Apr 4, 2026
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 4, 2026

Re-created it here #64725 as the rebase got mixed up with Vespa changes.

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 4, 2026

cc: @bugraoz93

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bugraoz93 bugraoz93 bugraoz93 approved these changes

@ashb ashb Awaiting requested review from ashb ashb is a code owner

@gopidesupavan gopidesupavan Awaiting requested review from gopidesupavan gopidesupavan is a code owner

@amoghrajesh amoghrajesh Awaiting requested review from amoghrajesh amoghrajesh is a code owner

@jscheffl jscheffl Awaiting requested review from jscheffl jscheffl is a code owner

@jason810496 jason810496 Awaiting requested review from jason810496 jason810496 is a code owner

@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner

@jedcunningham jedcunningham Awaiting requested review from jedcunningham jedcunningham is a code owner

@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy ephraimbuddy is a code owner

@choo121600 choo121600 Awaiting requested review from choo121600 choo121600 is a code owner

@vatsrahul1001 vatsrahul1001 Awaiting requested review from vatsrahul1001 vatsrahul1001 is a code owner

Assignees

No one assigned

Labels

area:dev-tools area:production-image Production image improvements and fixes backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk @bugraoz93 @radu-gheorghe