Skip to content

[GR-71342] Test tier run through #615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
graalvmbot wants to merge 2 commits into from
Closed

[GR-71342] Test tier run through #615

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fa9d536
[GR-71342] Test tier run through
ban-mi Jan 23, 2026
a0a9c96
Applied suggestion
olyagpl Jan 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 34 additions & 25 deletions docs/user/Test-Tiers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,53 +1,62 @@
# Detailed Test Tier Breakdown

GraalPy test tiers are similar to [CPython Platform Support Tiers](https://peps.python.org/pep-0011/), but do not constitute or imply any commitment to support.
GraalPy organizes platform testing into tiers that indicate the level of testing rigor and support you can expect for different platform configurations. This tiering system helps you understand:

Generally, running pure Python code on GraalPy without JIT is compatible with any recent JDK.
However, support for native extensions, platform-specific APIs, and just-in-time compilation is more limited.
- How thoroughly your platform is tested
- What level of stability to expect
- Which features are fully supported vs. experimental

Platform testing is organized into Tiers, each with specific goals.
Tiers are identified by a "target tuple": `[CPU architecture]-[Operating System]-[libc]-[JDK]-[Java version]`.
JDK names correspond to those used by [SDKMAN!](https://sdkman.io/).
The term "graal" is used to refer to testing on both Oracle GraalVM and GraalVM Community Edition, including GraalVM Native Image.
The following tables list tested platforms by tier.
Platforms not listed are untested.
Platforms are identified using the target tuple format: `[CPU architecture]-[Operating System]-[libc]-[JDK]-[Java version]`. JDK names follow [SDKMAN!](https://sdkman.io/) conventions, and "graal" refers to both Oracle GraalVM and GraalVM Community Edition (including Native Image).

**Tier 1**
> **Important:** GraalPy test tiers are similar to [CPython Platform Support Tiers](https://peps.python.org/pep-0011/), but do not constitute or imply any commitment to support.

- CI failures block releases.
- Changes which would break the main or release branches are not allowed to be merged; any breakage should be fixed or reverted immediately.
- All core developers are responsible to keep main, and thus these platforms, working.
- Platform-specific Python APIs and Python C extensions are fully tested.
Pure Python code runs reliably on GraalPy with recent JDKs when JIT compilation is disabled. However, advanced features like native extensions, platform-specific APIs, and JIT compilation have varying support depending on your platform tier.

## Tier 1

- **Stability:** CI failures block releases. Changes which would break the main or release branches are not allowed to be merged; any breakage should be fixed or reverted immediately.
- **Responsibility:** All core developers are responsible to keep main, and thus these platforms, working.
- **Coverage:** Platform-specific Python APIs and Python C extensions are fully tested.

| Platform | Notes |
|----------------------------------|----------------------------|
| amd64-linux-glibc-graal-latest | Oracle Linux 8 or similar. |
| aarch64-linux-glibc-graal-latest | Oracle Linux 8 or similar. |

**Tier 2**
## Tier 2

- CI failures block releases.
- Changes which would break the main or release branches are not allowed to be merged; any breakage should be fixed or tests marked as skipped.
- Circa 10% of tests running on Tier 1 platforms may be skipped on Tier 2 platforms.
- Platform-specific Python APIs are fully tested; Python C extensions may have more issues than on Tier 1 platforms.
- **Stability:** CI failures block releases. Changes which would break the main or release branches are not allowed to be merged; any breakage should be fixed or tests marked as skipped.
- **Test Coverage:** Circa 10% of tests running on Tier 1 platforms may be skipped on Tier 2 platforms.
- **Feature Support:** Platform-specific Python APIs are fully tested; Python C extensions may have more issues than on Tier 1 platforms.

| Platform | Notes |
|-----------------------------------|-------------------------|
| aarch64-macos-darwin-graal-latest | macOS on M-series CPUs. |

**Tier 3**
## Tier 3

- CI failures block releases.
- Changes which would break the main or release branches are not allowed to be merged; any breakage should be fixed or tests marked as skipped.
- Circa 25% of tests running on Tier 1 platforms may be skipped on Tier 3.
- Tests for platform-specific Python APIs and Python C extension are run, but not prioritized.
- **Stability:** CI failures block releases. Changes which would break the main or release branches are not allowed to be merged; any breakage should be fixed or tests marked as skipped.
- **Test Coverage:** Circa 25% of tests running on Tier 1 platforms may be skipped on Tier 3.
- **Feature Support:** Tests for platform-specific Python APIs and Python C extension are run, but not prioritized.

| Platform | Notes |
|---------------------------------|----------------------------------------------|
| amd64-macos-darwin-graal-latest | macOS on Intel CPUs running BigSur or newer. |
| amd64-macos-darwin-graal-latest | macOS on Intel CPUs running Big Sur or newer. |
| amd64-windows-msvc-graal-latest | Windows 11, Windows Server 2025, or newer. |
| amd64-linux-glibc-oracle-21 | JDK 21 is tested without JIT compilation. |
| aarch64-linux-glibc-oracle-21 | JDK 21 is tested without JIT compilation. |
| aarch64-macos-darwin-oracle-21 | JDK 21 is tested without JIT compilation. |
| amd64-macos-darwin-oracle-21 | JDK 21 is tested without JIT compilation. |
| amd64-windows-msvc-oracle-21 | JDK 21 is tested without JIT compilation. |

## Tier 4

- **Stability:** CI failures do not block releases. Tests may be broken on the main and release branches.
- **Test Coverage:** Smoke tests with platform-agnostic pure Python workloads are run on a regular schedule.
- **Feature Support:** Only basic pure Python functionality is tested; platform-specific features and extensions are not prioritized.

| Platform | Notes |
| ------------------------------- | ----------------------------------------------|
| amd64-linux-musl-graal-latest | Ensures GraalPy can be built for and used on musl libc platforms such as Alpine Linux. |
| amd64-linux-glibc-j9-17 | Ensures that non-Oracle JVMs work for pure Python code without JIT. |
| ppc64le-linux-glibc-oracle-17 | Ensures that other architectures (ppc64le, s390x, risc-v) work for pure Python code without JIT. |
Loading