Author here.

The core thesis is that we are currently building "Open Loop" systems and hoping they behave reliably.

It’s not just about unit tests, better prompting, or buying more tools. It is a fundamental architectural gap. We have excellent "Actuators" (LangChain, Vector DBs) that execute actions, but we lack the "Sensors" (continuous measurement) and the "Controller" (the operational logic to correct drift).

In Control Theory, an Open Loop system with stochastic components (y ~ P(y|x)) guarantees degradation over time. The post argues that you can't "tool" your way out of this—you need to build a governance layer that acts as the system's brain.

I propose mapping the AI stack to Control Theory components: 1. Actuators (Muscles): Tools like LangChain. They execute but are blind to meaning. 2. Constraints (Skeleton): JSON Schemas/Pydantic. They fix syntax but ignore semantics. 3. Sensors (Nerves): Golden Sets & Evals. The missing feedback loop in most stacks. 4. Controller (Brain): The Operating Model that closes the loop.

Happy to discuss the mapping of Control Theory to AI Engineering.

We are seeing a crisis where engineers apply "Linear Software" tools (CI/CD, rigid assertions) to "Behavioral Software" (LLMs). If you force an LLM to have zero variance, you turn it into a slow database. This post proposes replacing "Unit Tests" with "Control Loops" based on Control Theory principles.

This is an early draft of a framework for handling uncertainty in LLM-based development. It tries to outline simple artifacts and evaluation loops that could make this work more predictable. Posting it here to understand whether the core idea makes sense and to learn how others structure their process around non-deterministic systems.

I’m the author. I wrote this piece because a lot of discussions around LLM-based systems focus on prompts or model benchmarks, while the real complexity starts at the system level: how components interact, how failures propagate, how to evaluate behavior over time, and how to keep nondeterministic elements inside a predictable architecture.

The article tries to map this landscape and highlight patterns that seem essential when building anything more serious than a single-model call: • where unpredictability actually comes from • how architecture shapes reliability • why evaluation is harder than it looks • and what guardrails or control layers help keep the system stable

If anyone is working on similar problems or wants to challenge any of the points — happy to discuss, compare approaches, or clarify specific sections.

A deep engineering guide to building reliable LLM-based systems. Covers failure modes, hallucination control, evaluation traps, system decomposition, guardrails, and architecture patterns for treating LLMs as probabilistic components rather than deterministic logic. Focused on real engineering challenges, not hype.

A deep dive into what it really takes to build dependable software on top of inherently unpredictable LLMs. Covers architectural patterns, failure modes, evaluation pitfalls, hallucination-resistant design, guardrails, system decomposition, and how to treat LLMs as probabilistic components rather than “smart APIs.” Focused on engineering practices rather than hype, with practical frameworks for building stable, observable, testable LLM-based systems.

Most AI features in delivery tools look good in demos but fail once teams try to use them. The core issue is not the models but the lack of traceability inside platforms like Jira, Monday, Asana, Azure DevOps and others. AI cannot make real decisions when it cannot see the full chain from requirement to code to test to release.

This article explains how missing links in these systems limit real AI automation and what kind of traceability layer is needed for serious AI workflows.