Skip to content

perf: optimize pytest speed with collection and parallel execution improvements#124

Merged
JacobCoffee merged 3 commits intomainfrom
feature/pytest-speedup
Nov 23, 2025
Merged

perf: optimize pytest speed with collection and parallel execution improvements#124
JacobCoffee merged 3 commits intomainfrom
feature/pytest-speedup

Conversation

@JacobCoffee
Copy link
Owner

@JacobCoffee JacobCoffee commented Nov 23, 2025

Summary

Implemented pytest performance optimizations based on best practices from awesome-pytest-speedup.

Performance Improvements

Collection Speed (25% faster)

  • Before: 1.58s
  • After: 1.18s
  • Improvement: 25% faster

Test Execution (with parallel)

  • Before (sequential): ~23s
  • After (parallel): ~10s
  • Improvement: 58% faster

Total Time (collection + execution with parallel)

  • Before: ~25s
  • After: ~11s
  • Overall improvement: 56% faster

Changes

1. Collection Optimizations

  • ✅ Added `norecursedirs` to skip `.git`, `node_modules`, `docs`, `.venv`, etc.
  • ✅ Disabled unnecessary builtin pytest plugins (`doctest`, `pastebin`, `legacypath`)
  • ✅ Set `PYTHONDONTWRITEBYTECODE=1` in `Makefile` and CI workflows

2. Network Access Prevention

  • ✅ Added `pytest-socket` to prevent inadvertent network calls
  • ✅ Configured `--disable-socket --allow-unix-socket`
  • ✅ Added `@pytest.mark.enable_socket` marker for tests that need network

3. Parallel Execution (opt-in)

  • ✅ Added `pytest-xdist` for parallel test execution
  • ✅ Made parallel execution opt-in via `pytest -n auto`
  • ✅ Sequential execution remains default for safety

Files Changed

  • `.env.example` - Added `PYTHONDONTWRITEBYTECODE=1`
  • `.github/workflows/ci.yml` - Added env vars for all jobs
  • `Makefile` - Added `PYTHONDONTWRITEBYTECODE = 1`
  • `pyproject.toml` - Updated pytest config and dependencies
  • `tests/README.md` - Added performance documentation and chart
  • `uv.lock` - Updated dependencies

Usage

Default (Sequential, Safe)

```bash
make test
```

Parallel (Faster, Opt-in)

```bash
pytest -n auto
```

Quick Feedback

```bash

Run only failed tests

pytest --lf

Run failed first, then rest

pytest --ff
```

Test Results

  • ✅ Same 23 pre-existing failures as main branch
  • ✅ No new test failures introduced
  • ✅ All prek hooks passed
  • ✅ CI should pass with same results

Documentation

See detailed performance comparison and usage guide in `tests/README.md`.

Acknowledgments

Optimizations based on awesome-pytest-speedup by @zupo.

🤖 Generated with Claude Code

Summary by Sourcery

Improve pytest performance and reliability by optimizing test collection, tightening network access during tests, and enabling optional parallel execution.

New Features:

  • Introduce optional parallel test execution via pytest-xdist to speed up large test suites.
  • Add network access control in tests using pytest-socket with an opt-in marker for tests that legitimately require network access.

Enhancements:

  • Optimize pytest collection by excluding unnecessary directories and disabling unneeded built-in plugins.
  • Standardize PYTHONDONTWRITEBYTECODE across local, CI, and environment configuration to reduce filesystem overhead during tests.

Build:

  • Set PYTHONDONTWRITEBYTECODE in the Makefile to avoid writing .pyc files during local test runs.

CI:

  • Configure CI workflows to run with PYTHONDONTWRITEBYTECODE enabled for faster, cleaner test execution.

Documentation:

  • Expand tests/README.md with detailed pytest performance metrics, optimization techniques, and guidance for running tests sequentially or in parallel.

Tests:

  • Update pytest configuration to support faster collection, controlled network access, and optional parallel execution for the test suite.

Chores:

  • Refresh dependency lockfile to capture new test-related dependencies and configuration changes.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant