Systems | Development | Analytics | API | Testing

If you're running Node.js in production, you've likely heard the buzz around OpenTelemetry. It's the industry standard for observability, backed by major vendors, and it promises vendor-neutral telemetry collection across your entire stack. For many teams, it's a game-changer: finally, a unified way to collect traces, metrics, and logs without getting locked into a single vendor's ecosystem.

Having a build cache solution is a powerful way to speed up builds, especially at scale. Bitrise Build Cache already accelerates builds across multiple ecosystems, but to get the most out of it we also need to optimize the build cache clients themselves and ensure stability across changing network environments. In this blog post, I’ll walk through the steps we took to improve stability and performance for Bitrise Build Cache customers.

Streaming tokens is easy. Resuming cleanly is not. A user refreshes mid-response, another client joins late, a mobile connection drops for 10 seconds, and suddenly your “one answer” is 600 tiny messages that your UI has to stitch back together. Message history turns into fragments. You start building a side store just to reconstruct “the response so far”. This is not a model problem. It’s a delivery problem That’s why we developed message appends for Ably AI Transport.

Advances in large language models (LLMs) and their widespread accessibility have transformed both what software can do and how we build it. The use of LLMs has quickly evolved from simple single-turn interactions to AI agents that reason, use tools, manage state, and operate autonomously.

APIs and integrations are the foundation of the modern enterprise. Every organization needs to securely connect systems, move data, and automate workflows, all while maintaining control, visibility, and flexibility. Increasingly, those same APIs are also being consumed by AI-powered applications and agents that must interact safely with underlying business systems.

In today's digital landscape, APIs are the backbone of modern applications, and AI is the engine of innovation. As organizations increasingly rely on microservices and AI-powered features, the API gateway has become the critical control point for managing traffic. But as LLM/GenAI and MCP requests flow through these gateways, they bring a new wave of security challenges.

Here’s what you need to know: Bottom Line: Treat LLMs as untrusted clients. Secure database access with strict governance, API controls, and robust monitoring to mitigate risks without sacrificing productivity.