Skip to content

Conversation

@gpauloski
Copy link
Contributor

Description

Setting engine.transformer.metrics true will now enable metric recording on the Store used by a ProxyTransformer. The recorded metrics are written to the proxystore/ sub-directory within the run_dir of a benchmark. The proxystore/ directory will contain two file if metrics were recorded:

  • aggregated.json: aggregated statistics for different store operations
  • stats.jsonl: a line-delimited JSON file where each line corresponds to the stats for a unique object that was proxied

Note: the recorded stats only reflect the performance metrics as seen by the main TaPS process. In other words, the performance stats recorded on workers are not recorded.

To learn more about metrics in ProxyStore, see the guide.

Fixes

Type of Change

  • Bug (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which adds or improves functionality)
  • Internal (refactoring, performance, and testing)
  • Breaking (fix or feature that would cause existing functionality to not work as expected)
  • Documentation (no changes to the code)
  • Development (CI workflows, packages, templates, etc.)
  • Package (package dependencies and versions)

Testing

Added unit tests and validated with the following command

python -m taps.run --app cholesky --app.block-size 100 --app.matrix-size 200 --engine.transformer proxystore --engine.transformer.connector.kind file --engine.transformer.connector.options '{"store_dir": "./object-cache"}' --engine.transformer.metrics true

Pull Request Checklist

Please confirm the PR meets the following requirements.

  • Relevant tags are added (breaking, bug, documentation, enhancement, package, etc.).
  • Code changes pass pre-commit (e.g., ruff, mypy, etc.).
  • Tests have been added to show the fix is effective or that the new feature works.
  • New and existing unit tests pass locally with the changes.
  • Docs have been updated and reviewed if relevant.

@gpauloski gpauloski added the enhancement New features or improvements to existing functionality label Dec 6, 2024
@gpauloski gpauloski merged commit fa22209 into main Dec 6, 2024
6 checks passed
@gpauloski gpauloski deleted the issue-182 branch December 6, 2024 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New features or improvements to existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Accessing metrics recorded by transformers

2 participants