Skip to content

🐍 Python v0.17.0 - High-Throughput Upgrade

Choose a tag to compare

View all tags
@LZL0 LZL0 released this 08 Dec 22:40
· 31 commits to master since this release
0.17.0
fb08eda

The Python runtime for L0 receives the same performance-focused overhaul as the TypeScript version targeting Nvidia Blackwell support. This release introduces incremental JSON guardrails, sliding-window drift detection, new high-throughput defaults, and a brand-new benchmark suite demonstrating Python’s ability to sustain 120K+ tokens/sec.

This update includes major internal upgrades across guardrails and drift detection.

✨ Highlights

1. ⚡ Incremental JSON Guardrails (O(delta) cost)

json_rule() has been rewritten to match the new TS architecture:

  • New IncrementalJsonState dataclass
  • Tracks braces, brackets, string/escape state incrementally
  • Only processes delta (new characters), not full content
  • Full analyze_json_structure() executed only at stream completion
  • Automatic state reset on new/shortened streams

Result: ~5–10× faster per-token guardrail checks under streaming load.

2. 🎯 Sliding Window Drift Detection

DriftConfig now includes:

sliding_window_size: int = 500

Drift detection now:

  • Analyzes only the last N characters
  • Meta commentary, repetition, markdown collapse, tone shift all run on the window
  • Reduces drift-detection cost by O(content_length) → O(window_size)
  • Matches the TS implementation for cross-platform parity

3. 🚀 New High-Throughput Default Intervals

Python now uses the same optimized defaults as TS:

Interval Old New
Guardrails 5 tokens 15
Drift 10 tokens 25
Checkpoint 10 tokens 20

Updated in ADVANCED.md and CheckIntervals (src/l0/types.py).

4. 🧪 New Benchmark Suite (BENCHMARKS.md)

Full benchmarking added (99 additions):

  • Baseline vs core vs guardrails vs drift vs full-stack
  • Measured on Apple M1 Max with Python 3.13
  • Python achieves 1.5M tokens/sec raw iteration and 120K TPS full-stack with all guardrails enabled
  • Ready for 1000+ TPS Nvidia Blackwell inference loads

Benchmarks include reproducible pytest commands.

🗑️ Targeted Deletions / Optimization Removals

  1. Removed old full-content drift detection paths
  2. Removed malformed-pattern reporting in streaming phase (now done incrementally)
  3. Removed obsolete default interval values (5/10/10)
  4. Removed non-window-based drift comparisons to last full content
Assets 2
Loading