Skip to content

Update result summarizer to utilize id_json_file for public ids#446

Merged
ShriyaRishab merged 17 commits intomlcommons:masterfrom
arav-agarwal2:feat-id-file
Apr 2, 2026
Merged

Update result summarizer to utilize id_json_file for public ids#446
ShriyaRishab merged 17 commits intomlcommons:masterfrom
arav-agarwal2:feat-id-file

Conversation

@arav-agarwal2
Copy link
Copy Markdown
Contributor

Part of resolving mlcommons/training#854

Adds a optional argument to supply id_json_path, a list of hashed ids which serves as a map from each result record to the actual repo ID.

Summaries will include both a hash column and an id column to ensure that each run is given a unique id and previous ids are respected.

@arav-agarwal2 arav-agarwal2 requested review from a team as code owners February 20, 2026 17:53
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 20, 2026

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab
ShriyaRishab previously approved these changes Mar 5, 2026
Copy link
Copy Markdown
Contributor

@ShriyaRishab ShriyaRishab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShriyaRishab
Copy link
Copy Markdown
Contributor

@arav-agarwal2 can you please rebase?

@arav-agarwal2
Copy link
Copy Markdown
Contributor Author

I believe I've done just that - let me know if I'm missing something

@ShriyaRishab
Copy link
Copy Markdown
Contributor

WG discussed on 3/5/26 and planning to keep this open another week before finalizing.

@ShriyaRishab
Copy link
Copy Markdown
Contributor

LGTM

@arav-agarwal2
Copy link
Copy Markdown
Contributor Author

I've made a pass and have resolved your issues to the best of my ability.

One thing I wanted to point out is that the ids currently are stored like "tinybox_green_privateid.json", where "tinybox_green" is the name for the system provided by the submitter. I understand this may be an issue ( as this could be changed by the submitter ), so if there's a better way to handle storage please let me know.

@ShriyaRishab
Copy link
Copy Markdown
Contributor

ShriyaRishab commented Apr 1, 2026

Thanks, Arav. It looks great!
Why not set the filename to be consistent like privateid.json so that in the future, it'll be easy to find all the privateid.json files and probably update them with simple bash commands in case we want to do that?

@arav-agarwal2
Copy link
Copy Markdown
Contributor Author

I've added a few lines to the README as suggested.

We should be A-OK for the merge.

Copy link
Copy Markdown
Contributor

@ShriyaRishab ShriyaRishab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for addressing all the concerns!
@pgmpablo157321 should be ok to merge

@ShriyaRishab ShriyaRishab merged commit 3190466 into mlcommons:master Apr 2, 2026
2 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Apr 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants