R Meetup - Real Data Science USA (formerly LA R Users Group)

Benchmarking AI Agents for Optimizing Machine Learning Pipelines: An XGBoost Case Study

by Szilard Pafka, PhD
Chief Scientist, Epoch

AI agents are increasingly used to automate data science work, and new models are often compared on a single benchmark run. But how much would the result change if you simply ran the same agent again?

In this talk I'll present experiments where autonomous AI agents get one hour to improve a baseline XGBoost model in a binary classification problem on a public dataset. Each agent is made up of a harness (e.g. Codex, Claude Code, Pi), a large language model (LLM) (e.g. GPT-6.1 Sol, Claude Opus 5.5) and a prompt, written in the spirit of Karpathy's autoresearch project, that gives the agent detailed instructions on how to optimize the ML pipeline. It runs in an isolated Docker container, where it can engineer features from the data, tune XGBoost hyperparameters and search the web for tips. Its final model is scored on a holdout set it never sees, and automatic checks verify whether it followed the rules.

Since LLMs, and therefore the agents, are non-deterministic, results are expected to vary somewhat from run to run, but surprisingly, they vary enormously. If we run each setup 20 times, the models' average scores rank them clearly. However, individual runs vary so much that comparing two agents on single runs gives the wrong ranking far too often. For example, in 20 runs each of three OpenAI models, the best run of the weakest beat half the runs of the mid-tier one, and even the strongest model lost about one in five head-to-head matchups against the runner-up.

I'll show results for several harnesses and models, ranging from OpenAI and Anthropic frontier models to open-weight models running locally on a single GPU, with some in between. We'll also look under the hood at what the agents actually do: which tricks most of them find, which features the stronger models learn to drop, and where the weaker ones go wrong.

Finally, I'll share practical lessons for anyone using AI agents to automate machine learning work: how to evaluate agents reliably, how much the choice of harness and model matters, and what all this means for interpreting single-run results from benchmarks and leaderboards. All code and materials discussed in the talk are open source and available on GitHub.

Bio:
Szilard studied physics in the 90s and obtained a PhD by using statistical methods to analyze the risk of financial portfolios. He worked in finance, then in 2006 he became Chief Scientist of a tech company in Santa Monica, California, doing everything data science (analysis, modeling, data visualization, machine learning, data infrastructure, AI). He was the founder/organizer of several meetups in the Los Angeles area (R, data science, etc.) and of the data science community website datascience.la for more than a decade, until he relocated to Texas in 2021. He is the author of a well-known machine learning benchmark on GitHub (1000+ stars) and a frequent speaker at conferences (keynote/invited talks at KDD, R/Finance, Crunch and eRum, and contributed talks at useR!, Predictive Analytics World, EARL, H2O World, Data Science Pop-up, DataWorks Summit, etc.), and has developed and taught graduate data science and machine learning courses as a visiting professor at UCLA and CEU (Central European University).