In grpo series of experiments, should we fix the learning rate at the very beginning?

[heng380]

the first experiment of grpo is to sweep over the lr, find the best and fix it afterwards, but personally performance of following exps are highly correlated to learning rate, for example

1. for off policy grpo, we shall lower the lr as the epoch grows from 1 to some number
2. for off policy grpo, we shall lower the lr as the train batch size reduce from 256 to some number
3. for seq level loss exp, as we norm the sum by the max length of batch response, the token-loss optimal lr is not the best, too

It seems unfair to compare all settings as we have set the fixed lr at the very beginning