How Agents CLI Helps You Catch and Fix AI Agent Regressions

When you’re building an AI agent, the hardest part isn’t getting it to work once—it’s keeping it working as you make changes. Agents CLI’s evaluation tools give you a systematic way to catch problems before they reach users, measure the impact of changes, and steadily improve your agent’s quality.

Why evaluation matters

Building an AI agent that works in a demo is different from building one that stays reliable through model updates, tool changes, and unexpected user inputs. Without systematic testing:

• Small changes can break behaviors you didn’t think to test
• Model updates might degrade performance without warning
• You can’t measure whether a change actually improved things

The Agents CLI evaluation tools implement a quality engineering technique called “hill-climbing”—making small changes, measuring their impact, and keeping only what improves your score.

Setting up evaluation

The evaluation workflow starts with a dataset of test cases that represent different ways users might interact with your agent. A basic weather agent might need cases for:

1. Simple greetings (“hello”)
2. Weather queries for supported cities (“weather in San Francisco”)
3. Queries for unsupported locations (“weather in Rome”)
4. Off-topic questions (“who won the World Series?”)

The test cases live in a JSON file with example inputs and expected response formats. Agents CLI provides templates to get you started.

Running your first evaluation

With test cases prepared, you:

1. Generate responses: agents-cli eval generate --dataset tests/dataset.json
2. Grade those responses: agents-cli eval grade --traces output/*.json

This produces scores like:

A healthy baseline shows variation—if every case scores perfectly, your tests might be too easy.

Catching regressions

The real power comes when you make changes. Imagine accidentally changing your weather agent’s prompt to act like a chef:

## Before  
system_prompt="You are a helpful assistant"  

## After  
system_prompt="You are a professional chef. Suggest recipes at the end of every response."  

Re-running the evaluation catches this:

The scores dropped because:

• Greetings now include unsolicited recipe advice
• Weather queries ignore the tool and make up forecasts
• Off-topic answers force in cooking tips

Comparing model performance

Evaluation also helps when considering model changes. Swapping from gemini-3.8-flash to gemini-2.5-flash produces:

The cheaper model handles simple cases well but struggles with edge cases. This helps you decide if the tradeoff is worth it.

Automating the process

For production agents, set up:

1. Scheduled runs - Daily evaluations against a larger test set catch drift from model updates
2. Alerting - A script compares new scores to baselines and alerts on significant drops
3. Optimization - Advanced tools can automatically test prompt variations to find improvements

Getting started

To implement this for your agent:

1. Define 5-10 test cases covering key behaviors
2. Run initial evaluation to establish a baseline
3. Re-evaluate after every meaningful change
4. Set up alerts for production agents

Read next

• A2UI gets faster with swappable formats and decision models
• Agents CLI Adds Framework Flexibility with Extension System

Want to try all of this hands-on? Start with the free Claude Code from Zero course.