Introduction
Google published an Agents CLI evaluation walkthrough on October 28. It shows how a team can check an AI assistant after changing its instructions or model. In one example, an accidental prompt change leaves the program running but changes how it answers and uses a tool.
What the tools do
Agents CLI can run saved requests, record responses and tool calls, grade the resulting traces and compare two sets of results. Its evaluation guide separates checks for instruction following, tool use and completion across a conversation. Teams can also define their own criteria.
That gives the review a useful starting point: open the case that changed, inspect the answer and follow the actions that produced it.
A practical example for a small business
Consider a booking assistant. Before changing it, we would save examples covering an ordinary request, missing dates, a customer changing their mind and a request the business cannot handle.
For each example, write down the expected next step. A changed date should trigger a fresh availability check. Missing information should lead to a question. A request outside the service area should reach a person when that is the agreed process.
Then run those same examples against the proposed version. Keep the inputs fixed so the comparison has meaning. Check the recorded actions as well as the written reply. We would hold a release if it confirmed an unavailable booking, even if the overall score improved.
This is an example of how we would plan a test, not a report of an araNET deployment or measured customer outcome.
Availability and limits
The CLI is available in Google’s public repository under Apache 2.0. It is developer tooling for Google’s agent stack; using it involves project configuration and model access. Check the chosen model and evaluation-service costs before scheduling repeated runs.
Google’s article is a new tutorial for an existing tool, not a new model launch. Its four-case demonstration is too small to establish production reliability. The article also warns that automatic prompt optimization can fit the scoring criteria too closely. Its optimization example is limited to ADK projects, with human review before a candidate is promoted.
Our take at araNET: keep the failed examples with the project. When a real issue is corrected, add a case that would catch it next time. A repeatable check becomes part of maintaining the service.
