Skip to content

Commit 3769092

Browse files
committed
fix format
1 parent 355d9d8 commit 3769092

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

deepspeed/runtime/zero/parameter_offload.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -492,7 +492,8 @@ def _run_after_backward_function(sub_module):
492492

493493
def pre_sub_module_forward_function(self, sub_module):
494494
see_memory_usage(f"Before sub module function {sub_module.__class__.__name__}", force=False)
495-
prev_grad_state = torch.is_grad_enabled() # we don't want to enable grad for sub modules fetching, yet the subfunction need to know if grad is enabled
495+
prev_grad_state = torch.is_grad_enabled(
496+
) # we don't want to enable grad for sub modules fetching, yet the subfunction need to know if grad is enabled
496497
torch.set_grad_enabled(False)
497498
global FWD_MODULE_STACK
498499
FWD_MODULE_STACK.append(sub_module)

0 commit comments

Comments
 (0)