Dev/format inputinfo

[GACLove]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @GACLove, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the handling of input parameters across the seko_talk model inference pipelines. By introducing a structured default_input_info within model configurations and a new SekoTalkInputs dataclass, the changes aim to centralize input management, enhance flexibility, and improve the maintainability of the system. This allows for more dynamic control over inference parameters and simplifies the overall pipeline initialization process.

Highlights

• Refactored Input Parameter Management: Input parameters for seko_talk models (rs2v, s2v, f2v) have been centralized into a new default_input_info structure within configuration files and a dedicated SekoTalkInputs dataclass. This improves clarity and allows for more flexible overriding of default values.
• Dynamic Inference Steps Handling: The infer_steps parameter is now explicitly passed to the scheduler's prepare method, rather than being implicitly read from the scheduler's configuration, enabling dynamic control during runtime.
• Streamlined Pipeline Initialization: The ShotPipeline and its derivatives (ShotRS2VPipeline, ShotStreamPipeline) no longer rely on a monolithic ShotConfig object. Instead, they accept clip_configs directly and dynamically construct input information, simplifying the pipeline setup.
• Updated Model and LoRA Paths: Configuration files now consistently use absolute paths (prefixed with /data/temp/) for model and LoRA weights, and an assertion has been added to verify model path existence.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• configs/seko_talk/shot/rs2v/rs2v.json
  • Updated model_path to SekoTalk-v2.7_beta2-bf16-step4.
  • Removed infer_steps and resize_mode from the top-level configuration.
  • Added a new default_input_info object to encapsulate various input parameters like infer_steps, resize_mode, prompt, negative_prompt, image_path, audio_path, and save_result_path.
• configs/seko_talk/shot/stream/f2v.json
  • Updated model_path to include the /data/temp/ prefix.
  • Removed infer_steps and resize_mode from the top-level configuration.
  • Updated lora_configs path to include the /data/temp/ prefix.
  • Added a new default_input_info object containing various input parameters.
• configs/seko_talk/shot/stream/s2v.json
  • Updated model_path to include the /data/temp/ prefix.
  • Removed infer_steps and resize_mode from the top-level configuration.
  • Added a new default_input_info object containing various input parameters.
• lightx2v/models/runners/wan/wan_audio_runner.py
  • Modified the run_clip_main method to explicitly pass infer_steps as an argument to self.model.scheduler.prepare.
• lightx2v/models/schedulers/scheduler.py
  • Removed the initialization of self.infer_steps and self.caching_records from the __init__ method.
• lightx2v/models/schedulers/wan/audio/scheduler.py
  • Refactored the EulerScheduler.__init__ method to explicitly initialize all attributes instead of relying on super().__init__(config).
  • Modified the prepare method to accept infer_steps as an argument and set self.infer_steps dynamically.
• lightx2v/shot_runner/rs2v_infer.py
  • Removed the ShotConfig import and its usage.
  • Integrated init_input_info_from_args and self.check_input_info for dynamic input parameter handling.
  • Updated the ShotRS2VPipeline constructor and generate method signatures to accept clip_configs and args respectively.
  • Adjusted input parameter retrieval to use the new clip_input_info object for audio paths and save paths.
• lightx2v/shot_runner/shot_base.py
  • Removed the ShotConfig dataclass.
  • Updated imports for input information utilities (fill_input_info_from_defaults) and config setting (set_config).
  • Modified ClipConfig to expect config_json as a dictionary.
  • Refactored load_clip_configs to use set_config for processing clip configurations.
  • Updated the ShotPipeline constructor to accept clip_configs directly.
  • Added a new check_input_info method for populating input details from default configurations.
  • Removed overlap_frame and overlap_latent attributes from ShotPipeline initialization.
• lightx2v/shot_runner/stream_infer.py
  • Removed the ShotConfig import and adopted the new input information handling similar to rs2v_infer.py.
  • Updated ShotStreamPipeline constructor and generate method signatures.
  • Adjusted audio loading and saving operations to use args.audio_path and args.save_result_path respectively, along with model_fps.
• lightx2v/utils/input_info.py
  • Introduced _UnsetType and UNSET constant for explicit handling of unset fields in dataclasses.
  • Added the SekoTalkInputs dataclass to encapsulate input parameters for s2v and rs2v tasks.
  • Implemented from_args class method to construct SekoTalkInputs from argparse.Namespace.
  • Added normalize_unset_to_none method to convert UNSET values to None.
  • Introduced init_input_info_from_args and fill_input_info_from_defaults utility functions.
• lightx2v/utils/set_config.py
  • Added an assertion assert os.path.exists(config["model_path"]) to ensure the model path exists before proceeding.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a significant and beneficial refactoring of how input parameters are handled. By removing the rigid ShotConfig and introducing a flexible SekoTalkInputs dataclass with a UNSET default pattern, the code becomes more modular and easier to extend. The new approach of loading default parameters from default_input_info in the JSON configuration files is a clean way to manage defaults.

My review includes a critical fix for a duplicate field in a dataclass and a suggestion to improve maintainability by removing code duplication in a scheduler's __init__ method. Overall, this is a solid improvement to the codebase.

[GACLove]

model_path 不属于 rs2v 的， 建议放置到它的上层，clip 的 item 的配置中。