Skip to content

Conversation

@Agoniii
Copy link
Collaborator

@Agoniii Agoniii commented Jan 8, 2026

What does this PR do?

To fix the issue: #4822

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: ...
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly addresses a potential crash in scaled_fp8_blockwise by replacing a strict assertion with a padding mechanism. The code now pads input tensors to ensure their dimensions are multiples of the block size, which is necessary for blockwise FP8 quantization. The implementation is clean, saving the original shape and cropping the tensor back after processing.

A significant concern is that an almost identical function, also named scaled_fp8_blockwise, exists in verl/utils/sglang/sglang_fp8_utils.py. This function has not been updated and still contains the original assertion, which will lead to the same crash if used. To prevent this bug and improve maintainability by reducing code duplication, it is highly recommended to apply the same padding logic to the sglang utility file. This could be done in this PR or a follow-up.

@wuxibin89 wuxibin89 merged commit e75c570 into volcengine:main Jan 12, 2026
82 of 93 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants