Changing _save_summary_info to an ExpArgs static method #324
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment
|
![korbit-ai[bot]](https://avatars.githubusercontent.com/in/322216?s=60&v=4){kind=small}
There was a problem hiding this comment.
I've completed my review and didn't find any issues... but I did find this kitten.
|\__/,| (`\
_.|o o |_ ) )
-(((---(((--------Files scanned
| File Path | Reviewed |
|---|---|
| browsergym/experiments/src/browsergym/experiments/loop.py | ✅ |
Explore our documentation to understand the languages and file types we support and the files we ignore.
Need a new review? Comment
/korbit-reviewon this PR and I'll review your latest changes.Korbit Guide: Usage and Customization
Interacting with Korbit
- You can manually ask Korbit to review your PR using the
/korbit-reviewcommand in a comment at the root of your PR.- You can ask Korbit to generate a new PR description using the
/korbit-generate-pr-descriptioncommand in any comment on your PR.- Too many Korbit comments? I can resolve all my comment threads if you use the
/korbit-resolvecommand in any comment on your PR.- Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
- Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.
Customizing Korbit
- Check out our docs on how you can make Korbit work best for you and your team.
- Customize Korbit for your organization through the Korbit Console.
Current Korbit Configuration
General Settings
Setting Value Review Schedule Automatic excluding drafts Max Issue Count 10 Automatic PR Descriptions ✅ Issue Categories
Category Enabled Documentation ✅ Logging ✅ Error Handling ✅ Readability ✅ Design ✅ Performance ✅ Security ✅ Functionality ✅ Feedback and Support
Note
Korbit Pro is free for open source projects 🎉
Looking to add Korbit to your team? Get started with a free 2 week trial here
TLSDC
commented
Mar 3, 2025
Comment on lines
+127
to
+284
| @dataclass | ||
| class StepTimestamps: | ||
| env_start: float = 0 | ||
| action_exec_start: float = 0 # to extract begining of visual action from video | ||
| action_exec_stop: float = 0 # to extract end of visual action from video | ||
| action_exect_after_timeout: float = 0 | ||
| env_stop: float = 0 | ||
| agent_start: float = 0 | ||
| agent_stop: float = 0 | ||
|
|
||
|
|
||
| @dataclass | ||
| class StepInfo: | ||
| """Collects information about step that will be saved and reloaded. | ||
| Helper functions only modify the dataclass attributes and helps keeping the | ||
| information organized. | ||
|
|
||
| Attributes: | ||
| ----------- | ||
| step: int | ||
| The step number of the episode. | ||
| obs: dict | ||
| The observation of the environment. | ||
| reward: float | ||
| The reward of the step. | ||
| raw_reward: float | ||
| The raw reward of the step. | ||
| terminated: bool | ||
| Whether the episode is terminated i.e. reached a terminal state. | ||
| truncated: bool | ||
| Whether the episode is truncated i.e. reached a maximum number of steps. | ||
| action: str | ||
| The action taken by the agent. | ||
| agent_info: dict | ||
| Additional information from the agent. | ||
| stats: dict | ||
| Extra statistics about the step. | ||
| profiling: StepTimestamps | ||
| Timestamps of the different events during the episode. | ||
| """ | ||
|
|
||
| step: int = None | ||
| obs: dict = None | ||
| reward: float = 0 | ||
| raw_reward: float = 0 | ||
| terminated: bool = None | ||
| truncated: bool = None | ||
| action: str = None | ||
| agent_info: dict = field(default_factory=dict) | ||
| stats: dict = None | ||
| profiling: StepTimestamps = field(default_factory=StepTimestamps) | ||
| task_info: dict = None | ||
|
|
||
| def from_step(self, env: gym.Env, action: str, obs_preprocessor: callable): | ||
| t = self.profiling | ||
| t.env_start = time.time() | ||
| self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action) | ||
| t.env_stop = time.time() | ||
|
|
||
| self.task_info = env_info.get("task_info", None) | ||
|
|
||
| self.raw_reward = env_info.get("RAW_REWARD_GLOBAL", None) | ||
|
|
||
| t.action_exec_start = env_info["action_exec_start"] # start | ||
| t.action_exect_after_timeout = env_info["action_exec_stop"] | ||
| t.action_exec_stop = env_info["action_exec_stop"] - env_info["action_exec_timeout"] | ||
|
|
||
| if obs_preprocessor: | ||
| self.obs = obs_preprocessor(self.obs) | ||
|
|
||
| def from_action(self, agent: Agent): | ||
| self.profiling.agent_start = time.time() | ||
| self.action, self.agent_info = agent.get_action(self.obs.copy()) | ||
| self.profiling.agent_stop = time.time() | ||
|
|
||
| self.make_stats() | ||
|
|
||
| return self.action | ||
|
|
||
| def from_reset(self, env: gym.Env, seed: int, obs_preprocessor: callable): | ||
| t = self.profiling | ||
| t.env_start = time.time() | ||
| self.obs, env_info = env.reset(seed=seed) | ||
| self.reward, self.terminated, self.truncated = 0, False, False | ||
| t.env_stop = time.time() | ||
|
|
||
| t.action_exec_start = env_info.get("recording_start_time", t.env_start) | ||
| t.action_exect_after_timeout = t.env_stop | ||
| t.action_exec_stop = t.env_stop | ||
|
|
||
| if obs_preprocessor: | ||
| self.obs = obs_preprocessor(self.obs) | ||
|
|
||
| @property | ||
| def is_done(self): | ||
| return self.terminated or self.truncated | ||
|
|
||
| def make_stats(self): | ||
|
|
||
| stats = { | ||
| f"n_token_{key}": count_tokens(val) | ||
| for key, val in self.obs.items() | ||
| if isinstance(val, str) | ||
| } | ||
| stats.update(self.agent_info.pop("stats", {})) | ||
|
|
||
| messages = self.agent_info.get("chat_messages", None) | ||
| if messages is not None: | ||
| stats["n_token_agent_messages"] = count_messages_token(messages) | ||
|
|
||
| t = self.profiling | ||
| stats["step_elapsed"] = t.env_stop - t.env_start | ||
| stats["agent_elapsed"] = t.agent_stop - t.agent_start | ||
|
|
||
| self.stats = stats | ||
|
|
||
| def save_step_info(self, exp_dir, save_json=False, save_screenshot=True, save_som=False): | ||
|
|
||
| # special treatment for some of the observation fields | ||
| if self.obs is not None: | ||
| # save screenshots to separate files | ||
| screenshot = self.obs.pop("screenshot", None) | ||
| screenshot_som = self.obs.pop("screenshot_som", None) | ||
|
|
||
| if save_screenshot and screenshot is not None: | ||
| img = Image.fromarray(screenshot) | ||
| img.save(exp_dir / f"screenshot_step_{self.step}.png") | ||
|
|
||
| if save_som and screenshot_som is not None: | ||
| img = Image.fromarray(screenshot_som) | ||
| img.save(exp_dir / f"screenshot_som_step_{self.step}.png") | ||
|
|
||
| # save goal object (which might contain images) to a separate file to save space | ||
| if self.obs.get("goal_object", False): | ||
| # save the goal object only once (goal should never change once setup) | ||
| goal_object_file = Path(exp_dir) / "goal_object.pkl.gz" | ||
| if not goal_object_file.exists(): | ||
| with gzip.open(goal_object_file, "wb") as f: | ||
| pickle.dump(self.obs["goal_object"], f) | ||
| # set goal_object to a special placeholder value, which indicates it should be loaded from a separate file | ||
| self.obs["goal_object"] = None | ||
|
|
||
| with gzip.open(exp_dir / f"step_{self.step}.pkl.gz", "wb") as f: | ||
| pickle.dump(self, f) | ||
|
|
||
| if save_json: | ||
| with open(exp_dir / "steps_info.json", "w") as f: | ||
| json.dump(self, f, indent=4, cls=DataclassJSONEncoder) | ||
|
|
||
| if self.obs is not None: | ||
| # add the screenshots back to the obs | ||
| # why do we need this? | ||
| if screenshot is not None: | ||
| self.obs["screenshot"] = screenshot | ||
| if screenshot_som is not None: | ||
| self.obs["screenshot_som"] = screenshot_som | ||
|
|
||
|
|
Collaborator
Author
There was a problem hiding this comment.
just moving this upwards to fix typing
Collaborator
|
Thank you so much, this will allow me to use custom save_summary_info now |
jardinetsouffleton
approved these changes
Mar 3, 2025
layahaasini
pushed a commit
to layahaasini/BrowserGym
that referenced
this pull request
Nov 21, 2025
* changing _save_summary_info to an ExpArgs static method * switching static method _save_summary_info to classic method save_summary_info
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This way one can subclass ExpArgs and customize the change summary method
Description by Korbit AI
What change is being made?
Convert
_save_summary_infointo asave_summary_infostatic method of theExpArgsclass and refactor the logic to integrate theStepInfomanagement into this method.Why are these changes being made?
This refactor centralizes the summary-saving logic within the
ExpArgsclass, improving code organization and making theExpArgsclass more self-contained regarding experiment data management. This change enhances maintainability by encapsulating related functionalities within the class and eliminating external function dependencies.