Systems | Development | Analytics | API | Testing

You’ve probably encountered the online conversations about GitHub’s AI scaling challenges. A lot of the chatter is about how forgiving (or not) users should be to GitHub in light of inexorable traffic increases but also GitHub’s own decision-making. There is certainly irony in the fact that it was a swashbuckling GitHub, against the objections of many open source developers, that pressed Copilot on the industry.

In a release test I published in July, Katalon’s AI runner reported seven cases passed and four failed. I could open the steps, inspect their screenshots, and watch the browser session. The failed rows showed which expectations broke. The passed rows deserve the same scrutiny: what had the agent observed before it called each step passed? An AI test runner has to perform the action and judge the outcome. A false pass is a passed verdict when the expected behavior did not occur.

Your team has more changes to test, more environments to cover and less time to maintain regression suites. AI can help produce tests, but that introduces another workload: deciding whether those tests check the right behavior, whether a repair hides a defect and whether the evidence is enough to support a release. Agentic QA tools aim to take on more of that work, from planning and authoring to execution and failure investigation.

There's a conversation happening in a lot of data teams right now. It goes something like this: the lakehouse is clearly where things are heading, Apache Iceberg is becoming the format everyone wants to land in, and the AI use cases the business is asking for need clean, open, queryable data.

Enterprise software organizations allocate most of their total IT budgets to software validation and quality maintenance. According to the Consortium for Information and Software Quality (CISQ), poor software quality drains over $2.41 trillion annually from the US economy in operational failures, technical debt, and unmitigated production incidents.