Writing

Where the Rules Live: AI Engineering Practice in Three Tiers

By Jay Zeng | October 2026


A few weeks ago I checked the instruction file that our AI agents read at the start of every session. It was wrong. It named three chains; the system runs on four. It described a module layout that two refactors had replaced. It pointed to a project tracker we migrated away from months ago.

Nobody noticed, because nobody human reads that file anymore. That is not a complaint — it’s the finding.

Documentation that humans need gets maintained. Onboarding docs, runbooks, the README you point a new hire at — people read them, so when they drift, someone feels it and fixes it. Documentation that only the machine reads loses its maintenance constituency entirely. The agent reads the stale file, obeys it, and confidently tells you about a world that no longer exists. Machine-readability doesn’t make docs self-maintaining. It makes them un-maintained.

This is where a new kind of debt accumulates. Every incident with an AI agent teaches the team something — don’t trust an empty RPC result, this vendor’s API changed shape, that file is load-bearing. In a traditional team, those lessons die in chat scrollback or one engineer’s head. With agents, some get captured — as private memories, as a hard-won paragraph buried in a config file, as a prompt fragment copy-pasted between sessions. But if the lesson doesn’t land in version control alongside the fix that resolved it, the organization pays to learn it again. That gap — between what the system has taught you and what your rules actually say — is cognitive debt. It compounds quietly, because nothing fails when it’s incurred.

So the question I set out to answer was structural, not stylistic: where should AI engineering practice live, so that quality is constrained by rules instead of whoever’s memory happens to be in the room?

Three tiers

I think the answer has three layers, and the boundaries between them matter more than any single layer’s content.

graph TD
    P["AI ENGINEERING PRACTICE"] --> T1["TIER 1 — ENFORCED<br/>Org-owned hooks and settings<br/>small, hard, non-negotiable"]
    P --> T2["TIER 2 — CURATED<br/>Org vets and recommends<br/>authors nothing commodity"]
    P --> T3["TIER 3 — THE REPO CONTRACT<br/>Version-controlled rules<br/>one source for engineers and agents"]
    T1 -.- T1N["prose is suggestion<br/>hooks are policy"]
    T2 -.- T2N["curation beats authorship<br/>repos adopt what's already good"]
    T3 -.- T3N["where lessons land<br/>structured by when they're needed"]

Tier 1: enforced at the org, and therefore small

The most important thing to understand about agent instruction files: the agent treats them as context, not configuration. A line that says “never commit secrets” is a suggestion to a language model — it works most of the time, which is worse than a rule that either works or visibly doesn’t. A pre-commit hook that scans for secrets and blocks the commit is policy. It cannot have an off day.

That distinction tells you what belongs at the org layer of a large engineering organization: the things you would not accept an individual engineer opting out of. Secret scanning. Guards on credential stores. Approval gates on destructive operations. Managed settings that a repo can’t quietly override. This layer should be deployed centrally by the platform or security team, and it should stay small and hard.

A useful discipline here is separating two classes of rules. Invariants are machine-checked and non-overridable — a hook, a CI gate, a managed policy. Defaults are overridable, but only through a declared, greppable exception — an explicit config change a reviewer can see, never a silent local edit. Most org rules that fail in practice fail because they were written as prose when they were actually invariants, or enforced as invariants when they should have been defaults.

Tier 1 is mandatory by nature. Everything above it is not — and that’s deliberate.

Tier 2: curated, not authored

The tempting failure mode at the org layer is content: a central team publishes grand TDD guidelines, code review checklists, a prompt library. Don’t. The commodity content already exists — agent plugin ecosystems ship test-driven-development workflows, review skills, debugging methodologies, and they iterate faster than any internal standards document will. An org re-authoring them produces a worse duplicate, and adoption fails for the same reason it always does: engineers recognize boilerplate and route around it.

What the org layer can uniquely provide is judgment about the ecosystem, plus context nobody else has. Which plugins and tool connections survived an actual evaluation against your compliance posture. Which model routing keeps metered costs sane. What your regulatory environment adds to the standard advice. That’s a recommendation list maintained by people who did the vetting — curation as a service, not authorship.

Keep CI in the same spirit: the org can publish guidelines, but the repo chooses its workflow. Standards that can’t survive repo-level selection probably weren’t standards.

Tier 3: the repo contract — where the value gets extracted

The repo is where practice actually lives, and the design goal is a single source of truth with two audiences: the engineer who joins the project and the agent that works on it. These are converging — the industry is settling on a per-repo instruction file (AGENTS.md, CLAUDE.md) that both humans and tools read. Treat that file as the project’s practice contract, and structure it by one principle: when does this need to be loaded?

The refactor this implies is real. On a project I work on, the always-loaded file went from 413 lines to 322, with four path-scoped rule files and a skill carrying the rest. Fourteen hard-won lessons moved out of an agent’s private memory and into version control in that one pass. Nothing was deleted; everything moved to the tier where it’s loaded when needed instead of always.

Truth maintenance: the part that actually prevents rot

Structure decides where rules live. Three mechanisms decide whether they stay true.

Author for contracts, not status. The litmus test for any line in the practice contract: will this still be true in six months with no edits? “The pipeline has fifteen steps and each owes its neighbors a specific contract” survives. “Migration to the new tracker is in progress” rots on schedule. Status belongs in issue trackers and design docs, which have lifecycles — not in files the agent treats as permanent truth.

Code wins, and the doc gets fixed in the same change. Every repo pays lip service to “docs may be outdated” as an excuse. Flip it into an obligation: when code and doc disagree, the code is right, and fixing the doc is part of the change that broke it — like fixing a failing test. The agent, which reads every doc in every session, becomes your drift detector instead of a silent consumer of stale instructions. Ours now flags contradictions rather than obeying them.

Lessons graduate through the PR that resolves them. The week’s hardest-won lesson — the one sitting in an incident thread — should land as a rule in the same pull request as its fix. Not in someone’s notes, not “we should write that down someday.” This was our biggest gap: an agent’s accumulated memory held fourteen rules-worthy lessons for weeks because the only promotion path was recall. The fix is a convention, not a tool: post-incident rules ship with the fix, every time.

What I’d tell a team starting Monday

  1. Audit before you add. Read your agent instruction file against reality first. Stale rules don’t just waste context — they actively teach the agent a wrong model of the system. You cannot stack good practice on a rotten base.
  2. Move the non-negotiables out of prose. Anything you wouldn’t accept an engineer opting out of becomes a hook or a gate. What remains as prose is advice, and should be honest about being advice.
  3. Budget the always-loaded file like production memory. Cap it. Cut everything derivable. Push mechanics to path-scoped rules and procedures to skills.
  4. Don’t author what the ecosystem already ships. Curate it. Your org’s value is the vetting and the context, not a duplicate TDD guide.
  5. Install the graduation rule. Lessons land as rules with the PR that resolves them. This single convention converts incidents into assets automatically.
  6. Run a hygiene pass periodically. Usage tells you what’s actually load-bearing. In one recent pass over my own setup: seventeen agent plugins installed, two in active use; four external tool connections, two with zero calls in two weeks. Everything unused was still costing context — attention budget — every single session.

The test

The structure isn’t the achievement. A three-tier diagram is a presentation, not a practice. The test is operational: when the next incident happens, does its lesson have an obvious home? If the answer is “the agent’s memory” or “the postmortem doc someone might read,” the debt is still compounding. If the answer is “the rule that ships with the fix” — you’ve turned cognitive debt into an asset that appreciates.

The agents will read whatever we write, every session, forever. That’s not a reason to write more. It’s a reason to write things that stay true.