Skip to content

csplit: use VecDeque instead of Vec for FIFO buffer#10925

Open
devnexen wants to merge 1 commit intouutils:mainfrom
devnexen:csplit_vecdeque_optim
Open

csplit: use VecDeque instead of Vec for FIFO buffer#10925
devnexen wants to merge 1 commit intouutils:mainfrom
devnexen:csplit_vecdeque_optim

Conversation

@devnexen
Copy link
Contributor

Replace Vec with VecDeque in InputSplitter to avoid O(n) shifts on remove(0). Use push_front/pop_front/push_back for O(1) queue operations.

@github-actions
Copy link

GNU testsuite comparison:

Skip an intermittent issue tests/shuf/shuf-reservoir (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/sort/sort-stale-thread-mem (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/cp/link-heap is now being skipped but was previously passing.
Note: The gnu test tests/rm/many-dir-entries-vs-OOM is now being skipped but was previously passing.
Congrats! The gnu test tests/expand/bounded-memory is now passing!

@sylvestre
Copy link
Contributor

please provide hyperfine output, merci!

@devnexen
Copy link
Contributor Author

however this one does not seem worth it. main wins.

@devnexen devnexen closed this Feb 14, 2026
@devnexen devnexen reopened this Feb 14, 2026
@devnexen
Copy link
Contributor Author

hyperfine   --warmup 3   --prepare 'rm -f xx*'   -L csplit /usr/bin/csplit,"./target/profiling/coreutils.main csplit","./target/profiling/coreutils.gh10925 csplit"   "{csplit} somefile.txt 100 {9999} -z"
Benchmark 1: /usr/bin/csplit somefile.txt 100 {9999} -z
  Time (mean ± σ):     110.0 ms ±   2.0 ms    [User: 43.5 ms, System: 66.2 ms]
  Range (min … max):   106.8 ms … 113.2 ms    18 runs
 
Benchmark 2: ./target/profiling/coreutils.main csplit somefile.txt 100 {9999} -z
  Time (mean ± σ):     100.9 ms ±   4.0 ms    [User: 35.0 ms, System: 65.3 ms]
  Range (min … max):    95.6 ms … 110.1 ms    20 runs
 
Benchmark 3: ./target/profiling/coreutils.gh10925 csplit somefile.txt 100 {9999} -z
  Time (mean ± σ):      98.8 ms ±   2.5 ms    [User: 34.3 ms, System: 64.2 ms]
  Range (min … max):    94.2 ms … 104.0 ms    21 runs
 
Summary
  ./target/profiling/coreutils.gh10925 csplit somefile.txt 100 {9999} -z ran
    1.02 ± 0.05 times faster than ./target/profiling/coreutils.main csplit somefile.txt 100 {9999} -z
    1.11 ± 0.04 times faster than /usr/bin/csplit somefile.txt 100 {9999} -z
seq 1 1000000 > somefile.txt

@github-actions
Copy link

GNU testsuite comparison:

GNU test failed: tests/pr/bounded-memory. tests/pr/bounded-memory is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/shuf/shuf-reservoir (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/sort/sort-stale-thread-mem (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/expand/bounded-memory is now passing!

@sylvestre
Copy link
Contributor

as we don't have benchmarks, here is one: #10927

Replace Vec with VecDeque in InputSplitter to avoid O(n) shifts
on remove(0). Use push_front/pop_front/push_back for O(1)
queue operations.
@sylvestre sylvestre force-pushed the csplit_vecdeque_optim branch from dce735f to 9f1afa3 Compare February 14, 2026 13:06
@github-actions
Copy link

GNU testsuite comparison:

GNU test failed: tests/pr/bounded-memory. tests/pr/bounded-memory is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/tail/retry. tests/tail/retry is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/follow-name (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/factor/t36 is no longer failing!
Note: The gnu test tests/dd/no-allocate is now being skipped but was previously passing.
Note: The gnu test tests/expand/bounded-memory is now being skipped but was previously passing.
Congrats! The gnu test tests/printf/printf-surprise is now passing!

@ChrisDryden
Copy link
Collaborator

It does not appear to have detected a performance difference with the benchmarks, can you confirm if the benchmarks applied to the use case here?

@devnexen
Copy link
Contributor Author

The difference is pretty small locally after several runs more or less what seen here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants