Enable metric recording metrics when using ProxyTransformer
#183
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Setting
engine.transformer.metrics truewill now enable metric recording on theStoreused by aProxyTransformer. The recorded metrics are written to theproxystore/sub-directory within therun_dirof a benchmark. Theproxystore/directory will contain two file if metrics were recorded:aggregated.json: aggregated statistics for different store operationsstats.jsonl: a line-delimited JSON file where each line corresponds to the stats for a unique object that was proxiedNote: the recorded stats only reflect the performance metrics as seen by the main TaPS process. In other words, the performance stats recorded on workers are not recorded.
To learn more about metrics in ProxyStore, see the guide.
Fixes
Type of Change
Testing
Added unit tests and validated with the following command
Pull Request Checklist
Please confirm the PR meets the following requirements.
pre-commit(e.g., ruff, mypy, etc.).