Skip to content

Retry uv pip install on cargo-related transient failures#64588

Closed
potiuk wants to merge 1 commit intoapache:mainfrom
potiuk:retry-uv-install-cargo-failures
Closed

Retry uv pip install on cargo-related transient failures#64588
potiuk wants to merge 1 commit intoapache:mainfrom
potiuk:retry-uv-install-cargo-failures

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented Apr 1, 2026

Retry uv pip install commands in install_airflow_and_providers.py when failures
are cargo-related (.cargo in output), to handle transient build failures.

Workaround for astral-sh/uv#18801


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Claude Opus 4.6)

Generated-by: Claude Code (Claude Opus 4.6) following the guidelines

@boring-cyborg boring-cyborg bot added area:dev-tools backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch labels Apr 1, 2026
@potiuk potiuk force-pushed the retry-uv-install-cargo-failures branch 2 times, most recently from e76614d to 8a40f9d Compare April 1, 2026 18:37
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 1, 2026

One more change after testing with #63988

@potiuk potiuk force-pushed the retry-uv-install-cargo-failures branch from 8a40f9d to 1786681 Compare April 1, 2026 19:55
@zanieb
Copy link
Copy Markdown

zanieb commented Apr 1, 2026

I'd recommend just installing Rust ahead of time rather than retrying the whole command? The failure modes for concurrent rustup installs are very confusing and it can leave you in a broken state that will not succeed on retry.

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 1, 2026

I'd recommend just installing Rust ahead of time rather than retrying the whole command? The failure modes for concurrent rustup installs are very confusing and it can leave you in a broken state that will not succeed on retry.

Tried it already - #64535 it however significantly increases the time of build of the images, But yeah, I might get back to that - I wanted to see if we can do it this way. Thanks for the hint @zanieb

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds retry logic around uv-based dependency installation during container/image builds to mitigate transient cargo/maturin build failures (workaround for astral-sh/uv#18801).

Changes:

  • Introduces a Python retry helper wrapping uv pip install in scripts/in_container/install_airflow_and_providers.py.
  • Adds a Bash retry wrapper used by the Docker image install scripts (and their inlined copies in Dockerfile/Dockerfile.ci).
  • Updates uv.lock (timestamp plus additional dependency metadata/extras changes).

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
uv.lock Updates lock metadata (exclude-newer) and dependency/extras entries.
scripts/in_container/install_airflow_and_providers.py Adds _run_uv_install_with_retry() and routes uv install steps through it.
scripts/docker/install_airflow_when_building_images.sh Adds run_uv_with_cargo_retry() and wraps uv sync / install invocations.
Dockerfile.ci Updates the embedded /install_airflow_when_building_images.sh script to include the retry wrapper and use it.
Dockerfile Same embedded script updates as Dockerfile.ci.

@potiuk potiuk mentioned this pull request Apr 4, 2026
1 task
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 4, 2026

Closing in favour of #64535 - we will have to bite the bullet of longer image builds.

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented Apr 4, 2026

Thanks @zanieb for the suggestion

@potiuk potiuk closed this Apr 4, 2026
@potiuk potiuk reopened this Apr 4, 2026
@potiuk potiuk closed this Apr 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants