Skip to content

Less overzealous inlining #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Shnatsel wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Less overzealous inlining #183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e45eaff
drop #[inline] on vectorize from the generator
Shnatsel Jan 22, 2026
4e4c367
re-run the generator
Shnatsel Jan 22, 2026
5916f09
Reinstate the #[inline] attribute on the outer vectorize() in the gen…
Shnatsel Jan 23, 2026
c8050d2
Re-run the generator
Shnatsel Jan 23, 2026
91e2396
Add a comment documenting the rationale for the lack of #[inline]
Shnatsel Jan 26, 2026
ef5c903
Add inlining barrier for the non-dynamically-dispatched codepath
Shnatsel Jan 26, 2026
3f1c00f
re-run the generator
Shnatsel Jan 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion fearless_simd/src/generated/avx2.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,6 @@ impl Simd for Avx2 {
#[inline]
Copy link
Member

@DJMcNab DJMcNab Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function should still be #[inline] (indeed, I believe that it possibly should be inline(always)). This is because we don't want to force LLVM to change its calling convention to call this function, we just want to break unneeded inlining.

fn vectorize<F: FnOnce() -> R, R>(self, f: F) -> R {
#[target_feature(enable = "avx2,fma")]
#[inline]
unsafe fn vectorize_avx2<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
Expand Down
5 changes: 4 additions & 1 deletion fearless_simd/src/generated/fallback.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,10 @@ impl Simd for Fallback {
}
#[inline]
fn vectorize<F: FnOnce() -> R, R>(self, f: F) -> R {
f()
fn vectorize_inner<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
vectorize_inner(f)
}
#[inline(always)]
fn splat_f32x4(self, val: f32) -> f32x4<Self> {
Expand Down
1 change: 0 additions & 1 deletion fearless_simd/src/generated/neon.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ impl Simd for Neon {
#[inline]
Copy link
Member

@DJMcNab DJMcNab Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As here

fn vectorize<F: FnOnce() -> R, R>(self, f: F) -> R {
#[target_feature(enable = "neon")]
#[inline]
unsafe fn vectorize_neon<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
Expand Down
1 change: 0 additions & 1 deletion fearless_simd/src/generated/sse4_2.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,6 @@ impl Simd for Sse4_2 {
#[inline]
Copy link
Member

@DJMcNab DJMcNab Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here

fn vectorize<F: FnOnce() -> R, R>(self, f: F) -> R {
#[target_feature(enable = "sse4.2")]
#[inline]
unsafe fn vectorize_sse4_2<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
Expand Down
5 changes: 4 additions & 1 deletion fearless_simd/src/generated/wasm.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,10 @@ impl Simd for WasmSimd128 {
}
#[inline]
fn vectorize<F: FnOnce() -> R, R>(self, f: F) -> R {
f()
fn vectorize_inner<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
vectorize_inner(f)
}
#[inline(always)]
fn splat_f32x4(self, val: f32) -> f32x4<Self> {
Expand Down
15 changes: 12 additions & 3 deletions fearless_simd_gen/src/level.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,17 +101,26 @@ pub(crate) trait Level {
let vectorize_body = if let Some(target_features) = self.enabled_target_features() {
let vectorize = format_ident!("vectorize_{}", self.name().to_ascii_lowercase());
quote! {
// This function is deliberately not marked #[inline]:
// The closure passed to it is already required to be #[inline(always)],
// so this wrapper is the only opportunity for the compiler to make inlining decisions.
#[target_feature(enable = #target_features)]
#[inline]
unsafe fn #vectorize<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
unsafe { #vectorize(f) }
}
} else {
// If this SIMD level doesn't do runtime feature detection/enabling, just call the inner function as-is
// This SIMD level doesn't do runtime feature detection/enabling, so we could just call the passed closure as-is.
//
// But the inner function is required to be annotated `#[inline(always)]`,
// so we wrap it in a function that isn't `#[inline(always)]`
// to let the compiler make its own inlining decisions, as opposed to forcing it to inline everything.
quote! {
f()
fn vectorize_inner<F: FnOnce() -> R, R>(f: F) -> R {
f()
}
vectorize_inner(f)
}
};

Expand Down