Skip to content

Conversation

@JasonXuDeveloper
Copy link
Contributor

Summary

Add support for executing multiple specs sequentially within a single kperf runner invocation, enabling time-series replay scenarios where workload patterns vary over time.

Changes

Core Multi-Spec Functionality

  • Add MultiSpecRunnerMetricReport type for multi-spec results in api/types/metric.go
  • Implement executeSpecs() for sequential spec execution
  • Implement aggregateResults() to combine per-spec results into aggregated metrics
  • Add printMultiSpecResults() for multi-spec output format
  • Add hasCliOverrides() to detect and prevent CLI overrides with multi-spec configs
  • Update buildRunnerGroupSummary() in runner/utils.go to handle both single and multi-spec report formats

Example and Demo

  • Add timeseries replay example config with 3 phases:
    • Baseline: 10 QPS for 30s
    • Spike: 100 QPS for 20s
    • Recovery: 25 QPS for 30s
  • Add benchTimeseriesReplayCase command to demonstrate time-series replay

Technical Details

  • Multi-spec configs with CLI flag overrides are rejected with clear error message
  • Single-spec configs continue to work without changes
  • All specs share the same HTTP client pool (configured from first spec)
  • Results are aggregated with per-spec breakdown and overall summary

Use Cases

This enables realistic workload patterns useful for:

  • Stress testing with gradual load increases
  • Capacity planning with varying traffic patterns
  • Validating system behavior during load spikes and recovery
  • Time-series replay of production traffic patterns

Testing

  • ✅ Built and tested locally
  • ✅ All existing single-spec configs work unchanged
  • ✅ Multi-spec timeseries replay example validated
  • ✅ Proper aggregation of results across multiple specs

Related

This PR builds on #226 which converted the Spec field to Specs array format.

… replay example

Add support for executing multiple specs sequentially within a single kperf
runner invocation, enabling time-series replay scenarios.

Changes:
- Add MultiSpecRunnerMetricReport type for multi-spec results
- Implement executeSpecs() for sequential spec execution
- Implement aggregateResults() to combine per-spec results
- Add printMultiSpecResults() for multi-spec output format
- Add hasCliOverrides() to detect and prevent CLI overrides with multi-spec
- Update buildRunnerGroupSummary() to handle MultiSpecRunnerMetricReport format
- Add timeseries_replay example with 3 phases: baseline (10 QPS, 30s), spike (100 QPS, 20s), recovery (25 QPS, 30s)
- Add benchTimeseriesReplayCase command to demonstrate time-series replay

This enables realistic workload patterns that vary over time, useful for
stress testing, capacity planning, and validating system behavior under
changing load conditions.

Signed-off-by: JasonXuDeveloper <[email protected]>
@fuweid fuweid merged commit 6d497ef into Azure:main Jan 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants