Skip to content

feature: end-to-end agent workflow #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vaibs-d merged 53 commits into from
May 24, 2025
Merged

feature: end-to-end agent workflow #120

vaibs-d merged 53 commits into from
May 24, 2025

Conversation

@marcellodebernardi
Copy link
Contributor

@marcellodebernardi marcellodebernardi commented May 24, 2025

This PR makes several changes to introduce an "end-to-end" model building flow including data analysis, feature engineering, model testing, and more. Specifically, the changes include:

  1. Data analysis agent that reviews datasets to extract insights for feature engineering and modeling
  2. Dataset splitter agent that handles splitting the dataset into train/test/val
  3. Feature engineering agent that develops feature transformations for the task and creates a feature-engineered dataset
  4. Model testing agent that tests the performance of the final packaged model on the test set and produces a report on the model's performance

In addition to the above, this PR includes a number of smaller changes such as more comprehensive reporting, improvements to MLFlow run logging, etc.

Apologies for the long PR - it's been a long week 😂

@marcellodebernardi
fix: broken mlflow unit test
2bf027f
@marcellodebernardi
refactor: move mls agent to plexe/agents
264a43c
@marcellodebernardi
refactor: remove 'constraints' as we don't use this
1c13c39
@marcellodebernardi
refactor: remove plexe exceptions as we don't use them
1ce12af
@marcellodebernardi
fix: remove references to constraints
d4a1bc9
@marcellodebernardi
refactor: move cot callback to internal
38dc89a
@marcellodebernardi
refactor: move agent tools to plexe/tools package
91c380a
@marcellodebernardi
docs: update docs to reflect refactoring
f6bbf14
@marcellodebernardi
feat: add checkpointing and resumption
0900ba2
@marcellodebernardi
feat: mlflow logging with parent runs for resumption
880d850
@marcellodebernardi
fix: checkpointing disabled by default
076c361
@marcellodebernardi
fix: ensure mlflow exp is active
4845466
@marcellodebernardi
refactor: move mlops_engineer to plexe/agents
40076cf
@marcellodebernardi
refactor: put all agents under orchestrator agent
10ae3b7
@marcellodebernardi
fix: remove schema and eda from pre-build steps
aa9045f
@marcellodebernardi
docs: update multi-agent-system.md
1e91a58
@marcellodebernardi
fix: remove unnecessary todos from models.py
c72e133
@marcellodebernardi
fix: input schema not extracted after run
e2ab836
@marcellodebernardi
fix: dangling instruction in schema_resolver_prompt_templates.yaml
909a699
@marcellodebernardi
fix: eda agent missing from orchestrator
6646b4a
@marcellodebernardi
fix: mlflow callback labels every build failed
adb767e
@marcellodebernardi
fix: dataset analyser has no prompt template
4f5ff6d
@marcellodebernardi
fix: remove custom run names from mlflow child runs
b93c04f
@marcellodebernardi
fix: mlflow logging failing because of safe_get
5d8e3fc
@marcellodebernardi
chore: remove unnecessary comments
6a26a78
@marcellodebernardi
fix: prompt templates to improve agent behaviour
b61f98b
@marcellodebernardi
refactor: move object registry to plexe/core
36fbb6a
@marcellodebernardi
feat: add resumption to model build
4e60626
@marcellodebernardi
feat: add feature engineering agent
5c3d6b4
@marcellodebernardi
feat: introduce immutability for object registry
1db7733
@marcellodebernardi
fix: allow required imports for feature engineer
c6bdfa6
@marcellodebernardi
fix: clarify dataset analyser task
9c77763
@marcellodebernardi
fix: clarify feature engineer task
f1ab294
@marcellodebernardi
fix: incorrect registry import in dataset splitter
55b3a64
@marcellodebernardi
fix: miscellaneous fixes to dataset splitting and schema resolution
a9afc69
@marcellodebernardi
fix: miscellaneous fixes to feature engineer
7e95184
@marcellodebernardi
feat: save dataset splitting code to bundle
c8d571d
@marcellodebernardi
fix: eda report retrieving not always working
dc1b727
@marcellodebernardi
fix: ensure manager doesn't try to do work itself
a1e48c8
@marcellodebernardi
fix: dataset splitting code registered incorrectly
6cd94f3
@marcellodebernardi
fix: switch feature engineer to use ml engineer model
569552a
@marcellodebernardi
fix: add statsmodels to data analyser and feature engineer
9188b8a
@marcellodebernardi
chore: update packages to upgrade smolagents
d898c98
@marcellodebernardi
chore: bump to 0.22.0
01d750e
@marcellodebernardi
feat: add model testing agent
92432dc
@marcellodebernardi
feat: add model testing agent
fead55a
@marcellodebernardi
refactor: switch to yaml for model archive
fc48452
@marcellodebernardi
fix: clean up example scripts
b979075
@marcellodebernardi
Merge remote-tracking branch 'origin/feature/iterative-model-building…
dce15a8 
…' into feature/robust-and-repeatable-builds

# Conflicts:
#	plexe/agents/agents.py
#	plexe/agents/model_tester.py
#	plexe/core/storage.py
@marcellodebernardi
fix: disable resumption for now
55d351f
@marcellodebernardi
fix: remove unnecessary unit tests
eada7f0
@plexe-ai plexe-ai deleted a comment from jazzberry-ai bot May 24, 2025
@marcellodebernardi marcellodebernardi added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request labels May 24, 2025
vaibs-d
vaibs-d approved these changes May 24, 2025
View reviewed changes
Copy link
Contributor

@vaibs-d vaibs-d left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely cracked changes! I'm excited about the addition of the new agents 🔥

@vaibs-d vaibs-d merged commit 918e256 into main May 24, 2025
5 checks passed
@vaibs-d vaibs-d deleted the feature/end-to-end-agent-workflow branch May 24, 2025 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vaibs-d vaibs-d vaibs-d approved these changes

Assignees

No one assigned

Labels

bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@marcellodebernardi @vaibs-d