ThinkPRM numerical scores

[shuvom-s]

Hi,

Thank you for the great and well-organized repo! I was wondering if you have a way to provide numerical scores at each step, instead of just \boxed{correct} vs \boxed{incorrect}. The huggingface alludes to this:

"Overconfidence: Generative PRMs like ThinkPRM can sometimes produce scores clustered near 0 or 1, potentially not reflecting true uncertainty"

However, I was wondering how I can access these scores. Thanks!

[shuvom-s]

Closing this issue, as I was able to get it myself. See https://huggingface.co/peiyi9979/math-shepherd-mistral-7b-prm for an example of numerical scores.

[mukhal]

Thanks for your interest!

The method you linked is only valid for discriminative PRMs. For ThinkPRM, you can extract scores using:

from prm import ThinkPRM

# Initialize ThinkPRM
prm = ThinkPRM(
    model_name_or_path="launch/ThinkPRM-1.5B",
    max_length=4096,
    temperature=0.1,
    n=1
)

# Example question and reasoning steps
question = "What is 15% of 200?"
reasoning_steps = [
    "To find 15% of 200, I need to multiply 200 by 0.15",
    "200 × 0.15 = 30",
    "Therefore, 15% of 200 is 30"
]

# Evaluate correctness
results = prm.predict_correctness_batch([question], [reasoning_steps])

Then the score will be in results[0]['prefix_score']

Please let me know if you have any more questions

[shuvom-s]

Got it, thanks! Is there a way to get the scores for every prefix, without having to pass in each prefix separately? E.g., If I have a list like

steps = [step1, step2, step3]

Is there a way to get the prefix score for:
step1
step1-2
step1-3

without having to pass in each prefix separately.

Thanks!