Systems | Development | Analytics | API | Testing

In one release test, my AI agent triaged 43 backlog items into 134 executable cases in Katalon True Platform. The first pass at the steps caused trouble. We had bundled setup, navigation, action, and judgment into long instructions. The runner returned verdicts, but a failed step left me unsure which expectation it had checked. We split the cases into one action and one checkable expected result per step. The failures became specific enough to review against the screenshots and session recording.

“Customers can apply a discount code” looks ready for testing. Give that sentence to an AI model and it can produce cases for valid, invalid, expired, and reused codes in seconds. The trouble starts when someone asks what a valid code should do. Which basket is eligible? Does the discount change the displayed total before or after tax? Should the saved order carry the same amount?

Your team has more changes to test, more environments to cover and less time to maintain regression suites. AI can help produce tests, but that introduces another workload: deciding whether those tests check the right behavior, whether a repair hides a defect and whether the evidence is enough to support a release. Agentic QA tools aim to take on more of that work, from planning and authoring to execution and failure investigation.

In a release test I published in July, Katalon’s AI runner reported seven cases passed and four failed. I could open the steps, inspect their screenshots, and watch the browser session. The failed rows showed which expectations broke. The passed rows deserve the same scrutiny: what had the agent observed before it called each step passed? An AI test runner has to perform the action and judge the outcome. A false pass is a passed verdict when the expected behavior did not occur.

When the news about OpenAI's agents broke on September 26, 2026, my first reaction wasn't shock. It was recognition. If you missed it: OpenAI's agents were running what should have been routine research tasks during training and evaluation, and they ended up on US government websites. They reached the SEC and the Census Bureau, and their activity touched the Justice and Commerce Departments and government sites in five states.

Talk to any developer who has been using an AI coding assistant for a few months, and you'll hear some version of the same thing: "I'm getting through a lot more than I used to." Same person. Same hours. More output. Now ask a QA manager how they're keeping up with all that extra output. You'll usually get a tired laugh, followed by: I'm not. And I can't hire. That's the gap I want to talk about. Development is finding ways to grow its capacity without growing the team.

Most mobile automation programs don't stall because a tool can't tap a button. They stall on everything around the tap: the setup before the first test, a device lab nobody can keep current, maintenance that grows faster than coverage, and the moment someone asks "are we ready to ship?" and nobody has a clean answer.

Software testing is becoming increasingly complex. Modern engineering teams are expected to release faster, support continuous delivery, manage larger test suites, and maintain quality across web, mobile, API, and AI-driven applications. That is where test management tools come in. In 2026, test management is no longer limited to storing test cases and tracking pass/fail results.