refactor: params to plan prompt

[jmatejcz]

Purpose

• Parametrize Megamind plan prompt

Proposed Changes

PlanPrompts dataclass

Issues

• Links to relevant issues

Testing

Test it out with some random prompts in file rai_bench/examples/tool_calling_custom_agent.py

prompts = PlanPrompts(
        objective_template="xddddd\n",
        first_step_prompt="\ndo first step!!!\n",
        steps_done_header="Bly bly bly:\n",
        next_step_prompt="do somth lol: \n",
        completion_prompt="return when finished",
    )

  agent = create_megamind(
      megamind_llm=supervisor_llm,
      megamind_system_prompt=task.get_system_prompt(),
      executors=executors,
      anylyzer_prompt=task.get_planning_prompt(),
      plan_prompts=prompts,
  )

You should see difference in user prompt

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.95%. Comparing base (074af8e) to head (0d7bec0).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #709      +/-   ##
==========================================
+ Coverage   65.34%   67.95%   +2.61%     
==========================================
  Files          78       78              
  Lines        3388     3405      +17     
==========================================
+ Hits         2214     2314     +100     
+ Misses       1174     1091      -83     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Juliaj]

Hello @deepkarkada, thanks for your interest in contributing! Based on your interest for agentic programming, this PR relates to the long-horizon planning agent and could use your feedback. A good starting point for understanding this PR is to look into #679 which describes some initial design decisions of the agent.

One way to get hands-on experience is to set up your dev environment to run the script from PR #679. To do that, you'll need to set up RAI. Instructions can be found at https://robotecai.github.io/rai/setup/install/. Please feel free to ping technical-support or reach out to me if you run into any issues setting up your machine. Any of us will be happy to walk you through the steps.

As for timeline, since this is your first PR, I wouldn't stress about it. As long as you get started, the overall goal is achieved.

[jmatejcz]

brought back previous arg name 3409d0a
so that the backward compatibility is maintained, but it is a bit misleading, because the task_planning_prompt actually refers to prompt passed to the analyzer at the end of the megamind loop.

I m not sure if I should leave this name

[deepkarkada]

Hi @jmatejcz, Julia added me to the PR to help test new PlanPrompts. I have a few clarification questions about the agent design. From my understanding of PR #679 : the megamind agent now combines planner, replanner and supervisor. Essentially, there is no planning as such, the execution is step-by-step where given the objective, megamind comes up with first step, second step etc and binds it to the executor with the right tools. I guess my confusion/lack of context is around configuring the planning prompt. Is it configuring the subtask for each step? Is it possible to do so without knowing what step is generated. If there is a PR around more context to the PlanPrompts, please feel free to share, I would definitely be interested to learn more.
So far, I've tested the benchmark a few times (with and without the PlanPrompts). Here are the results I've seen (LLM used is gpt-4o):
| TASK | SCORE | TOTAL TIME |
| Without PlanPrompts | 0.0 | 24.150 |
| With PlanPrompts | 0.2 | ~700 (hits recursion error) |

[jmatejcz]

Hi @jmatejcz, Julia added me to the PR to help test new PlanPrompts. I have a few clarification questions about the agent design. From my understanding of PR #679 : the megamind agent now combines planner, replanner and supervisor. Essentially, there is no planning as such, the execution is step-by-step where given the objective, megamind comes up with first step, second step etc and binds it to the executor with the right tools. I guess my confusion/lack of context is around configuring the planning prompt. Is it configuring the subtask for each step? Is it possible to do so without knowing what step is generated. If there is a PR around more context to the PlanPrompts, please feel free to share, I would definitely be interested to learn more. So far, I've tested the benchmark a few times (with and without the PlanPrompts). Here are the results I've seen (LLM used is gpt-4o): | TASK | SCORE | TOTAL TIME | | Without PlanPrompts | 0.0 | 24.150 | | With PlanPrompts | 0.2 | ~700 (hits recursion error) |

About the confusion with planning this is what i mentioned in this comment above #709 (comment), where the name task_planning_prompt is misleading. Now it is essentially, as stated in docstring Prompt that helps summarize the step in a way that helps planning task.

PlanPrompts is on the other hand just customization of the main prompt

Idk if you read comments by me and @Juliaj under PR #679 - this is origin of the idea