Skip to content

Conversation

@garbas
Copy link
Contributor

@garbas garbas commented Dec 22, 2025

Summary

This PR adds intelligent upstream cache filtering and performance optimizations to avoid redundant uploads when pushing Nix store paths to custom substituters. It also migrates the development environment to Flox and adds comprehensive developer tooling.

Key Features

🎯 Configurable Upstream Cache Filtering

  • Adds new upstream-caches input parameter that accepts comma or newline-separated cache URLs
  • Defaults to checking both cache.nixos.org and cache.flox.dev
  • Automatically includes the target substituter in the check list to prevent re-uploading existing paths
  • Only pushes paths that don't exist in any configured upstream cache

⚡ Performance Optimization (14x Faster)

Replaced sequential per-path cache checking with parallel batch queries:

  • Before: 134 seconds (1000 sequential network calls for 500 paths × 2 caches)
  • After: 10 seconds (2 parallel batch calls)
  • Speedup: 14x faster

Implementation:

  • Batches all paths into a single nix path-info query per cache
  • Runs all cache queries in parallel using background jobs
  • Uses comm for efficient set difference operations

🛠️ Developer Tooling

  • Flox Migration: Replaced Nix flakes with Flox for reproducible development environments
  • Developer Documentation: Added .claude/CLAUDE.md with project architecture, development commands, and implementation details
  • Performance Benchmarking: Added scripts/benchmark-cache-check.sh to validate and measure cache checking performance
  • README Updates: Comprehensive development setup guide with Flox installation and usage

Benefits

  • ✅ Significantly reduces CI/CD pipeline time when pushing to custom substituters
  • ✅ Avoids unnecessary uploads and bandwidth usage (skips paths already in public caches)
  • ✅ Prevents duplicate storage of paths already in target substituter
  • ✅ Configurable to support additional upstream caches as needed
  • ✅ Easier developer onboarding with Flox-based environment

Example Configuration

- uses: flox/configure-nix-action@main
  with:
    substituter: s3://my-cache
    substituter-key: ${{ secrets.NIX_KEY }}
    # Optional: customize upstream caches to check
    upstream-caches: |
      https://cache.nixos.org
      https://cache.flox.dev
      https://cache.garnix.io

Benchmark Results

Testing with 500 paths against 2 caches:

Sequential approach: 153.39s
Parallel approach:   10.57s
Speedup:            14.51x faster 🚀

Both approaches verified against identical path sets (checksum validation included).

Test Plan

  • Benchmark parallel vs sequential cache checking
  • Verify paths are checked against all configured upstream caches
  • Verify target substituter is automatically included in checks
  • Test with default upstream caches (cache.nixos.org and cache.flox.dev)
  • Test with custom upstream caches specified
  • Verify that paths existing in upstream are skipped
  • Verify that new paths are still pushed correctly
  • Test Flox development environment setup on clean machine

Add support for checking multiple upstream Nix caches before pushing store paths to avoid redundant uploads. Paths that already exist in any configured upstream cache are automatically skipped.

- Add 'upstream-caches' input parameter accepting comma or newline-separated cache URLs
- Default to checking both cache.nixos.org and cache.flox.dev
- Update push script to query each upstream cache before pushing paths
- Only push paths that don't exist in any configured upstream cache

This significantly reduces unnecessary uploads and bandwidth usage by avoiding duplication of paths already available in public caches.
Automatically add the configured substituter to the list of upstream caches to check. This prevents re-pushing paths that already exist in the target substituter, avoiding unnecessary uploads and potential errors.

The base URL is extracted from CONFIGURE_NIX_SUBSTITUTER (removing query parameters) before checking.
Replace sequential per-path cache checking with parallel batch queries for dramatic performance improvement.

Changes:
- Batch all paths into a single nix path-info query per cache
- Run cache queries in parallel using background jobs
- Use comm for efficient set difference operation

Performance impact (500 paths, 2 caches):
- Before: 134 seconds (1000 sequential network calls)
- After: 10 seconds (2 parallel batch calls)
- Speedup: 13x faster

This significantly reduces CI/CD pipeline time when pushing to custom substituters.
@garbas garbas force-pushed the configure-upstream-caches branch from b7420ab to 721abdc Compare December 24, 2025 09:41
Migrate the development environment from Nix flakes to Flox for better
reproducibility and ease of use. Add comprehensive developer documentation
and performance benchmarking tools.

Added:
- .flox/ directory with Flox environment manifest (provides Node.js 20)
- .claude/CLAUDE.md with project architecture and development guidance
- scripts/benchmark-cache-check.sh for performance testing (14x speedup verification)
- .github/workflows/auto-label.yml for automatic PR labeling
- .github/workflows/update.yml for Flox environment updates
- Development setup documentation in README.md with Flox instructions

Removed:
- flake.nix and flake.lock (replaced by Flox)
- .envrc (Flox handles environment activation)
- .github/workflows/update-flake-lock.yml (no longer needed)

Modified:
- README.md: Added "Development" section with Flox setup, build commands,
  testing instructions, and performance benchmarking guide
- package-lock.json: Updated dependencies

The benchmark script validates the 14x performance improvement from parallel
batch cache checking (134s -> 10s for 500 paths against 2 caches).
Replace Nix flake development environment with Flox in CI workflows.

Changes:
- ci.yml: Update test-javascript job to use Flox
- update-dist.yml: Update dist/ check job to use Flox
- Replace 'cachix/install-nix-action' with 'flox/[email protected]'
- Update all 'nix develop --command' calls to 'flox activate --'

Note: Test jobs that verify the configure-nix-action itself (test-minimal-action,
test-all-action, test-nix-action) intentionally still use Nix as they are testing
the action's Nix configuration functionality.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants