feat: add time-series benchmark example (4/4) #223
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add ModeConfig interface to enable self-configuring execution modes and provide backward compatibility for legacy LoadProfile format. ## Changes ### New files: - api/types/mode_config.go: ModeConfig interface and helpers - api/types/weighted_random_config.go: WeightedRandomConfig implementation - api/types/timeseries_config.go: TimeSeriesConfig implementation - Split test files for better organization ### Modified files: - api/types/load_traffic.go: - Add ExecutionMode type - Update LoadProfileSpec with Mode and ModeConfig fields - Add UnmarshalYAML/UnmarshalJSON for backward compatibility - Fix typo: RequestList.Selector (was "seletor") - Update Validate() method ## Features **ModeConfig Interface:** - Self-configuring modes declare their own overridable fields - Automatic CLI override extraction via BuildOverridesFromCLI() - Mode-specific validation and client configuration - Clean separation between mode logic and CLI tools **Backward Compatibility:** - Legacy format (rate/total/requests fields) auto-migrates to weighted-random mode - Both YAML and JSON unmarshaling supported - Existing load profiles continue to work without changes ## Testing All tests pass including: - Polymorphic deserialization tests - Backward compatibility tests for legacy format - CLI override extraction tests - Mode-specific configuration tests
Implement executor pattern for mode-specific execution strategies and adapter pattern for request builders, enabling clean separation and extensibility. ## Changes ### New Executor Pattern (request/executor/) - **executor.go**: Executor interface and metadata types - **factory.go**: Registry-based factory for creating executors - **weighted_random.go**: WeightedRandomExecutor with rate limiting - **timeseries.go**: TimeSeriesExecutor for audit log replay ### Request Builder Adapters (request/builders.go) - CreateRequestBuilderFromWeighted(): Converts WeightedRequest → RESTRequestBuilder - CreateRequestBuilderFromExact(): Converts ExactRequest → RESTRequestBuilder - Centralized factory registration functions - No unnecessary type conversions ### Refactored Scheduler (request/schedule.go) - Uses executor pattern instead of hardcoded mode logic - Mode-agnostic scheduling - Executors manage their own rate limiting ### Simplified Random Module (request/random.go) - Removed WeightedRandomRequests() function - Kept individual request builder functions - Cleaner separation of concerns ## Benefits - Each executor is self-contained and testable - Easy to add new execution modes - No coupling between scheduler and mode-specific logic - Clean adapter pattern for different request types ## Depends On - PR1: mode-config-interface (ModeConfig interface must exist first)
Update CLI tools to be mode-agnostic by using BuildOverridesFromCLI() instead of hardcoded field access. ## Changes ### cmd/kperf/commands/runner/runner.go - Remove hardcoded rate/total/duration flag handling - Use BuildOverridesFromCLI() to auto-discover overridable fields - Apply overrides via ModeConfig.ApplyOverrides() ### contrib/cmd/runkperf/commands/bench/utils.go - Use BuildOverridesFromCLI() for automatic override extraction - No longer needs to know about specific mode field names ### contrib/cmd/runkperf/commands/warmup/command.go - Updated to use mode-agnostic override handling ## Benefits - CLI tools don't depend on specific mode types - Adding new modes doesn't require CLI changes - Each mode declares its own CLI-overridable fields - Cleaner, more maintainable code ## Depends On - PR1: mode-config-interface (BuildOverridesFromCLI helper) - PR2: executor-pattern-virtualization (executor pattern must exist)
Add a time-series benchmark to demonstrate and test the time-series execution mode with exact request replay from audit logs. ## Changes ### New Files - contrib/cmd/runkperf/commands/bench/timeseries_simple.go: Benchmark command - contrib/internal/manifests/loadprofile/timeseries_simple.yaml: Load profile ### Updated Files - contrib/cmd/runkperf/commands/bench/root.go: Register timeseries_simple command ## Benchmark Configuration - 10 runner pods with 100 clients each - 3 time buckets with 1-second intervals - Tests LIST operations with various parameters (limit, resourceVersion) - Tests GET operations for specific resources ## Test Pattern **Bucket 0 (t=0s):** - LIST pods with resourceVersion=0 (stale read) - LIST configmaps with limit=100 **Bucket 1 (t=1s):** - GET specific pod (test-pod-1) - LIST pods with limit=1000 **Bucket 2 (t=2s):** - LIST events with limit=500 - GET specific configmap (test-cm-1) ## Validation This benchmark validates: - Time-series mode execution with precise timing - ExactRequest → RESTRequestBuilder conversion - Proper handling of limit, resourceVersion, namespace parameters - Adapter pattern implementation correctness ## Depends On - PR1: mode-config-interface (TimeSeriesConfig) - PR2: executor-pattern-virtualization (TimeSeriesExecutor) - PR3: mode-agnostic-cli (CLI tools)
The interval field in TimeSeriesConfig is redundant since timing is already specified via StartTime in each RequestBucket. Removing it simplifies the configuration. ## Changes - Remove Interval field from TimeSeriesConfig struct - Update GetOverridableFields() to return empty array (no CLI overrides) - Update ApplyOverrides() to reject any override attempts - Update all tests to remove interval references - Update comment examples to remove interval mentions ## Benefits - Simpler configuration - Less redundant data - Timing is determined by bucket StartTime values
- Update TimeSeriesExecutor to work without interval field - Timing is handled by bucket StartTime values - Ensure PR builds independently
- Update example YAML to remove redundant interval field - Timing is determined by bucket StartTime values
Contributor
Author
|
see #219 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Part 4 of 4 PRs implementing executor pattern architecture. See issue #219 for full context.
This PR adds a time-series benchmark to demonstrate and test the time-series execution mode with exact request replay from audit logs.
Changes
New Files
contrib/cmd/runkperf/commands/bench/timeseries_simple.go: Benchmark commandcontrib/internal/manifests/loadprofile/timeseries_simple.yaml: Load profile configUpdated Files
contrib/cmd/runkperf/commands/bench/root.go: Register timeseries_simple commandBenchmark Configuration
Test Pattern
Bucket 0 (t=0s):
Bucket 1 (t=1s):
Bucket 2 (t=2s):
Validation
This benchmark validates:
Dependencies
Closes #219