Agent Alignment Is a Knowledge Problem, Not a Prompt Problem

There is a recurring pattern across engineering teams running agentic coding workflows: the agent knows how to write code, but it does not reliably know what your team has decided. It will reach for an HTTP client you deprecated last quarter. It will apply an architectural pattern you retired six months ago. It will make a reasonable-looking choice, and in every fresh session it gets to make that choice again, because nothing in its context told it that your team had already settled this.

We documented what we observed in WHY.md when we started building the solution. The short version: longer prompts helped at the margins, and then they stopped being sufficient. We maintained the governing rules in AGENTS.md and CLAUDE.md, which is better than nothing, but those files are flat text. The agent reads them at session start, they blur into background context, and the important line gets buried alongside fifty others. The model has no way to distinguish a currently active rule from one the team retired a year ago, and it has no mechanism for surfacing the governing standard at the exact moment it is about to make the choice that standard was written to govern.

That is the structural failure we built data-olympus to address. It is now open source (Apache 2.0) and available for any team running agentic coding workflows.

Three modes of agent knowledge failure

Before describing the solution, naming the failure modes is useful, because the diagnosis determines the intervention.

Context loss is the most common. The team has written the standard somewhere: a CLAUDE.md file, a design doc, an AGENTS.md annotation. The agent encounters the standard at session start, processes it as context, and moves on. Twenty tool calls and three subtasks later, the governing rule has decayed in effective attention weight. The agent reaches a decision point and makes what looks like a reasonable default. It does not consult the rule. The rule is technically still in the context window, but it is not retrievable at the right moment.

Superseded-rule confusion produces the most subtle failures. When a team updates a standard, they write the new version and update the document. But if the old standard is present anywhere else in context, an older AGENTS.md commit, a system prompt that predates the change, a design doc the agent indexed, the agent has two plausible answers. There is no signal in flat text that identifies which version is currently authoritative. The agent may correctly surface the newer one, or it may act on the older one. The outcome depends on attention dynamics the team cannot predict or audit.

Autonomous default selection causes the most long-term drift. The agent reaches a decision point for which no governing rule is immediately available in context. Rather than flagging the gap, it picks a reasonable-looking default: a popular library, a common pattern, whatever training data suggests is normal practice. The choice is not wrong in the abstract. It is wrong for this codebase, these constraints, these settled team decisions. By the time the choice surfaces in a pull request, it has already been woven into other decisions that depended on it.

The agent was not misbehaving. It was doing exactly what a capable agent does in the absence of governing information: it made a locally reasonable choice. The problem was the absence of the information, not the behavior.

What governance looks like with and without the consultation step

The sequence below shows the agent decision flow for a concrete scenario: an agent implementing a new service needs to select an HTTP client. The first path shows what happens when the governing standard exists only in flat context files. The second shows what happens with the data-olympus consultation gate in place.

sequenceDiagram
    participant A as Agent
    participant KB as data-olympus
    participant C as Codebase

    Note over A,C: Without governance
    A->>A: Reaches HTTP client decision
    Note right of A: Standard exists in AGENTS.md<br/>but is not surfaced here
    A->>C: import axios (training-data default)
    Note over C: May conflict with team standard

    Note over A,C: With data-olympus
    A->>KB: kb_consult("http client selection")
    KB-->>A: STD-NS-001 active (T2): use got@14<br/>supersedes STD-NS-000 (axios, retired 2025-11)
    A->>C: import got (governed choice)
    Note over C: Aligned with current decision

The difference is not whether the agent is capable of following the standard. It is whether the governing standard is surfaced at the right decision point, with enough signal to distinguish the current decision from a retired predecessor. The consultation response includes the document’s status, tier, and supersedes chain, so the agent knows it is acting on a currently active standard rather than something the team has since moved away from.

What we caught that we would have missed before

Since adopting data-olympus in our own agentic coding workflows, a pattern became visible that was not visible when we relied on context files alone.

The drift was not random. It clustered around specific decision categories: dependency selection (where the agent’s training-data prior was at odds with our chosen standard), authentication middleware patterns (where we had made a non-obvious architectural choice), and query builder conventions (where an older version of the pattern was still present in a legacy service the agent had indexed as context). In each category, the pre-governance behavior was not a failure in any single session. Looking at any individual session, the agent produced defensible code. The problem only became visible across sessions: the choices were inconsistent. Different agent runs were making different decisions for the same decision class.

Compared with the period before governance, the difference is measurable in the most direct way available: pull requests that reached review with a standards conflict stopped arriving. The conflicts did not disappear because the agents became more capable. They stopped because the governing standard was surfaced at the moment of decision rather than buried in session-start context that the agent had already moved past.

The enforcement hooks make this auditable. When a session wrote code in a governed path without a preceding consultation for the relevant decision class, the gap was flagged. Over time, the consultation record made the pattern legible: these were not random one-off failures. They were structural, repeatable, and preventable.

How data-olympus works

The architecture is deliberately minimal. A knowledge bundle is a directory of markdown files under git, each with YAML frontmatter defining the document’s id, type, status, and tier. A T1 standard (universal, applies to all work) appears differently in queries than a T3 decision (project-scoped). Documents with status: superseded are filtered from search results by default, so the agent cannot accidentally act on a retired decision even when it appears in git history.

A standards document for HTTP client selection in a NestJS backend service looks like this:

---
id: STD-NS-001
type: standard
status: active
tier: T2
title: HTTP client for NestJS services
description: got@14 is the approved HTTP client for all NestJS backend services.
tags: [http, nestjs, backend, dependencies]
timestamp: "2026-05-20"
owner: platform-team
supersedes: STD-NS-000
---

## Decision

All NestJS backend services use `got@14` as their HTTP client.

## Rationale

axios introduces inconsistent error handling across Node versions. got@14 provides
a stable promise-based interface with consistent timeout and retry behavior under
our operating load profiles.

## Migration

Services still using axios should migrate at their next planned refactoring cycle.

The MCP server exposes the bundle via kb_consult, kb_search, kb_get, and kb_gate_check. An agent configured with the server checks the knowledge base before making a governed decision. The enforcement hooks, available for Claude Code, Codex, Gemini, and other major coding agents, make this a hard gate: a PreToolUse hook intercepts file write operations and requires a consultation record for governed paths before the write proceeds.

The entire knowledge base lives in git. There is no proprietary database, no vendor lock-in, no SDK required to read the files. A git diff on the knowledge bundle shows exactly what changed, who changed it, and why. Adding a new standard is a pull request. Retiring one is changing its status field to superseded. Human and agent readable, the same file.

Trade-offs: when this fits and when it does not

data-olympus fits when three conditions hold simultaneously: the codebase has established conventions that agents are expected to follow, multiple agent sessions operate on the same codebase over time, and the cost of invisible drift (inconsistent implementations, deprecated dependency reintroduction, retired-pattern resurgence) exceeds the overhead of maintaining a governed knowledge bundle.

It is the wrong tool when: the team is in greenfield prototyping with no settled conventions, a single agent is working a bounded task where session-to-session consistency is not a concern, or the codebase has fewer established decisions than the governance overhead would justify. The tool is not an appropriate first-day adoption for a team that has not yet developed standards worth governing. Build the standards first. The governance layer is what makes them durable across agent sessions.

Three current limitations are worth stating plainly, because they affect whether the tool fits today rather than later. The format is at v0.1 and still pre-stable, so the schema can change between releases. Retrieval is full-text rather than semantic, which means consultations depend on sensible titles and tags rather than embedding similarity. For a governance corpus that is curated by hand and deliberately small, that has been adequate in our use, but it is a trade-off we made on purpose, not an accident. And writes go through a single-writer server by design, for the safety guarantees described above; reads scale out through read replicas, so this is a constraint on the write path, not on query throughput. We would rather you know these before you adopt than discover them after.

The comparison documentation in the repository covers how this approach relates to vector memory systems, ADR tooling, agent-context conventions like llms.txt, and enterprise data catalogs. The positioning is deliberate: data-olympus governs the retrieval of human-curated decisions at the decision moment. It is not trying to be a self-mutating memory layer that ingests on its own, a code search tool, or a documentation site generator. Those are adjacent problems, and other tools address them well. Where data-olympus does keep memory, through a proposal inbox, the entries are human-reviewed before they become governing rules, which is the whole point: the knowledge base only contains decisions a person signed off on.

Four questions for evaluating whether your workflow has this problem

The quickstart at the repository runs a local demo in under five minutes. But the most useful starting point is not the quickstart itself. It is this exercise run against your own codebase and recent agent session history.

If the drift check returns results and the session consistency check shows divergence in settled decision classes, the problem is structural. Prompt engineering addresses the instruction layer. data-olympus addresses the knowledge layer. They solve different problems, and conflating them is the reason teams end up with longer and longer system prompts that still do not reliably surface the governing rule at the right moment.

Run the diagnosis with your own agent

The fastest way to find out whether this applies to you is to let your own agent tell you. Point your current coding agent at your repository and give it this:

Audit the last 10 pull requests in this repository against our documented engineering standards
(AGENTS.md, CLAUDE.md, and anything under docs/). For each pull request, list every place where
you would have chosen a different library, pattern, or dependency than our standards specify.
Then assess honestly: how many of these would a layer that surfaces the governing standard at the
moment of decision have prevented before the code was written?

Read the result as a signal, not a verdict. It will surface some false positives, and it cannot see decisions the team never wrote down. But if it returns a list, and the same decision classes keep showing up on it, the drift is real and it is already costing review cycles. That is the gap data-olympus is built to close, and running this is the most honest way we know to decide whether the tool earns a place in your workflow before you install anything.

The project is at github.com/knaisoma/data-olympus. The WHY.md file covers the original motivation and design reasoning in more depth, for anyone who missed last week’s announcement. The repository includes comparison documentation, an adoption guide for teams at different stages, and a quickstart that works with a local demo bundle or your own knowledge files.

If you are evaluating whether this approach fits your team’s agentic workflow, or if the four questions above surfaced a pattern you recognize, we have worked through this problem in our own operations and are glad to compare notes on how it applies in your context.