Understanding AI Agent Harness, Loop, and Graph Engineering Step by Step ======================================================================== A harness defines the environment and controls in which an agent works, a loop defines repetition and termination rules, and a graph defines permitted states and transition paths. These three terms are best understood not as an officially standardized classification, but as practical perspectives for managing the autonomy and risks of AI agents. - Harness engineering involves designing the context outside the model, tools, permissions, validation, logs, and approval procedures as a single execution environment. - Loop engineering defines the conditions, budgets, and termination criteria for an agent to repeat planning, execution, validation, and revision. - Graph engineering uses states and transition rules to explicitly restrict or adjust the paths an agent can choose. - For most organizations, improving the harness and evaluation framework of a single agent first is more efficient than building a complex multi-agent graph. - Approving only an AI-generated summary report is insufficient for high-risk code; tests, the scope of changes, security boundaries, and original artifacts must also be verified. As AI agents have begun taking on long-running tasks, writing good prompts alone is no longer enough to produce reliable results. This is because you must also design what information the agent sees, which tools it uses, when it repeats actions, which paths it follows, and where it must obtain human approval. Three terms that frequently appear when explaining this problem are harness engineering, loop engineering, and graph engineering. These are not international standards or rigorously agreed-upon academic classifications. They overlap, and their meanings may vary by product and development team. Rather than memorizing them as annual buzzwords, it is therefore more useful to distinguish them by the control questions each one seeks to answer. Comparing the Three Concepts at a Glance Concept Core question Primary design targets Typical failure-prevention mechanisms Harness engineering Within what environment and rules does the agent work? Context, tools, permissions, sandbox, hooks, logs, approvals, evaluation Least privilege, approval for risky commands, test execution, context selection Loop engineering What is repeated, and when does it stop? Planning-execution-validation cycles, event processing, retries, budgets, termination conditions Maximum iteration count, time and token limits, progress assessment, escalation on failure Graph engineering Which states and paths are allowed? Nodes, states, transitions, branches, parallel processing, checkpoints Prohibited transitions, state validation, approval nodes, recovery paths In short, a harness defines the environment and boundaries, a loop defines the repetition rules, and a graph defines the structure of possible paths. In a real system, a loop may exist inside a graph node, while the entire graph may run within a single harness. How Have Agent Control Methods Evolved? Early Agents: Predefined Workflows Supplemented Autonomy Early generative AI agents often forgot their goals during long-running tasks, repeatedly made incorrect tool calls, or produced unsupported results. In response, developers divided large tasks into smaller steps and fixed the inputs and outputs for each step. In this approach, a person writes the entire procedure as a chain, flowchart, or state machine, while the LLM handles limited tasks such as classification, extraction, summarization, and drafting. Frameworks such as LangGraph are used to represent branching, cycles, checkpoints, and human intervention while preserving state. However, graph-based orchestration is not an outdated approach that ended in a particular year. Explicit graphs remain suitable for work where auditability, reproducibility, regulatory compliance, or precise recovery procedures are important. Improved Model Capabilities: From Fixed Paths to Dynamic Tool Use As tool use and reasoning capabilities improved, a single agent became able to choose actions such as search, code editing, testing, and file reading based on the situation. ReAct-style approaches are a representative structure in which reasoning, action, and observation alternate. This change reduced the burden of having people define every branch in advance. At the same time, managing the information an agent reads, the permissions it holds, execution costs, and error recovery methods became more important. This is where context engineering and harness engineering moved to the center of practical implementation. Long-Running Tasks and Multi-Agent Systems: Recombining Loops and Graphs For long-running tasks, iterative planning, execution, and validation are more important than a single model call. When multiple agents participate, their roles, output formats, permissions, and termination conditions must also be specified. At the same time, leaving autonomous loops completely unattended can cause runaway costs, infinite retries, reward hacking, and optimization toward the wrong objective. Modern agent systems are therefore designed not to eliminate autonomy, but to combine areas where autonomy is allowed with areas under deterministic control. This is not simply a return to the fixed chains of the past. Instead, it surrounds flexible execution with states, transitions, and policies. These changes represent shifts in design emphasis rather than a precise timeline. Graphs, loops, and harnesses have coexisted from the beginning and continue to be used together today. What Harness Engineering Covers A harness is not the foundation model itself, but the execution system surrounding the model that enables it to perform real work. Even when the same model is used, the harness can make a significant difference in success rate, cost, security, and reproducibility. Major Components of a Harness Instruction system: System instructions, repository rules, coding standards, priorities, and prohibited actions Context delivery: Search, file selection, summarization, memory, and injecting documents when needed Tool interfaces: File editing, terminals, browsers, databases, and external APIs Permissions and isolation: Read and write scope, access to secrets, network restrictions, and sandboxes Validation mechanisms: Tests, linters, type checking, schema validation, and fact-checking Human approval: Approval for difficult-to-reverse actions such as deployment, payment, deletion, and external transmission Observability: Call records, costs, latency, errors, change history, and decision rationale Recovery policies: Retries, restoration of previous states, task termination, and escalation to the responsible person Claude Code's project instruction files and hooks can be viewed as examples of harness components. However, no single product feature represents an entire harness. How It Differs from Context Engineering Context engineering optimizes which information and instructions are included in the current model call. It includes retrieving only relevant documents through search, summarizing old conversations, saving task state in external files, and separating context by subtask. Harness engineering is broader. In addition to context, it covers tool permissions, execution environments, approvals, validation, logging, and cost limits. Context engineering is therefore a core part of a harness, but using the two terms as if they meant exactly the same thing is inaccurate. The Core of Loop Engineering Is the Termination Condition A loop allows an agent to inspect a result after producing it and try again if the result is insufficient. What matters is not repetition itself, but the definition of progress and the conditions for stopping. Common Types of Loops Validation loop: Produces a draft, checks it against tests or evaluation criteria, and corrects failed items. Event-driven loop: Begins work when an external event occurs, such as an email, notification, code change, or sensor reading. Exploration loop: Investigates multiple hypotheses or sources and adjusts the scope of exploration until sufficient evidence is available. Improvement loop: Selects the next strategy based on previous results and evaluation scores. Optimizing only a single score can cause reward hacking, so multiple evaluation criteria and human review are required. Recovery loop: Classifies the cause of an error, retries within the permitted scope, and hands the task off to a person if it remains unresolved. Contracts Required for Safe Loops A contract between agents is not a legal contract, but an execution specification that defines inputs, outputs, and responsibilities. It should include the following items. Contract item What to specify Objective The result to be completed and what is out of scope Input Available data, freshness, and confidence level Output JSON schema, document format, required evidence, and test results Permissions Allowed tools, file scope, external transmission, and modification permissions Validation Tests and evaluation criteria that must be passed Budget Tokens, time, number of calls, and number of parallel tasks Termination Conditions for success, lack of progress, budget exhaustion, and risk detection Handoff Which person or agent takes over after failure If completion criteria are ambiguous, an agent may conclude that the task is progressing even while merely rewriting sentences or repeating the same search. Rather than setting only a maximum iteration count, it is better to consider result quality, the increase in new information, changes in errors, and cost together. Graph Engineering Structures the Boundaries of Autonomy A graph represents work as nodes and connections. A node may be a model call, tool execution, human approval, or validation process, while a connection indicates the next action based on the current state. Differences Between Chains and Graphs A chain is suitable for a linear process that proceeds from A to B and from B to C. A graph is suitable for tasks requiring conditional branches, repetition, parallel execution, failure recovery, and intermediate saves. A dynamic graph allows the model to propose the next subtask or path during execution. A constrained graph keeps the model within permitted nodes and transitions even when it makes choices. The purpose of modern graph design is not to have people predetermine every action. It is to embed invariants that must be preserved into the structure, such as requiring an approval node before data deletion or preventing a transition to the deployment state while tests are failing. Signs That a Graph Is Needed If several of the following conditions apply, it is worth considering an explicit graph. There is a clearly defined recovery point to return to after failure. A stage absolutely requires human approval. Multiple tasks must run in parallel before their results are combined. Available tools or permissions vary by state. The complete execution path must be audited or reproduced. A single-agent loop repeatedly encounters the same failure. Turning simple document summarization or a one-time data transformation into a graph may only increase complexity. Practical Implementation Order: Start with a Harness and Expand as Needed The following sequence is practical for most teams. Define a single task and its success criteria. First collect inputs, expected outputs, and failure cases. Build a minimal harness. Provide only the necessary context and tools, and set permissions, tests, logs, and cost limits. Build an evaluation set. Include not only normal cases but also ambiguous requests, incorrect documents, tool errors, and attempts to exceed permissions. Turn points requiring repetition into loops. Allow retries only where validation and correction actually improve quality. Promote the workflow to a graph when branching and recovery become complex. Specify states and transitions, and place approval nodes before risky actions. Use multiple agents only when dividing the work is beneficial. If parallel exploration or distinct specialized roles are unnecessary, a single agent may be simpler and less expensive. Differences in Application Between Coding and Research Item Coding tasks Research tasks Verifiability Automated validation through tests, builds, type checking, and similar methods is relatively easy Source quality, omissions, and conflicting evidence must be assessed comprehensively Value of dynamic exploration May be limited when the scope of changes is clear High when comparing different search paths and hypotheses Major risks Incorrect changes, security vulnerabilities, code tailored only to tests Unsupported claims, duplicate sources, confirmation bias Suitable controls Repository scope restrictions, tests, diff review, deployment approval Source records, independent searches, searches for opposing evidence, citation verification It cannot be stated categorically that dynamic workflows are always inefficient for coding and always beneficial for research. A large-scale, testable migration may be well suited to an autonomous agent, while a fixed research procedure may be more efficient for a straightforward factual lookup. The key variables are not the field, but the clarity of the objective, the feasibility of automated validation, the search space, and the cost of errors. Code Review Is Not Disappearing; the Unit of Review Is Changing When agents write code, developers spend less time entering every line themselves and take on more responsibility for overseeing requirements, design, test results, the scope of changes, and risks. Pull Request summaries and agent reports can accelerate reviews. However, approving changes after reading only a summary is not a safe default. Changes omitted by the agent or logic it misunderstood may also be absent from the summary. In the following situations, the original diff and related code must be reviewed directly. Changes to authentication, payments, personal data, encryption, or access control Database schema changes or irreversible migrations Code sensitive to performance and concurrency Large-scale refactoring outside test coverage Changes to external dependencies, deployment settings, or secret handling Cases where the agent's explanation does not match the actual diff Human-in-the-loop does not mean having a person formally click a button. It also includes providing the evidence of changes, test results, possible failures, and rollback procedures needed for a person to make an informed judgment. Common Pitfalls Multi-Agent Systems Without a Purpose Adding agents creates costs for role coordination, duplicate calls, context transfer, and merging results. Unless parallel exploration from different perspectives is required or there is a reason to separate contexts, a single agent is better. Unlimited Dynamic Workflows Allowing an agent to keep creating subtasks can rapidly increase token and tool-call costs. Costs are determined approximately by the sum of input and output token costs at each stage, tool costs, the number of parallel agents, and the number of iterations. The number of calls, number of concurrent executions, total budget, and maximum execution time must be limited separately. Optimizing Only One Evaluation Metric If test pass rate is the sole objective, incorrect optimization may occur, such as weakening tests or hiding exception handling. Quality, security, change size, cost, latency, and human evaluation should be used together. Confusing Document Injection with Fine-Tuning Document retrieval or project instructions may consistently change results, but they do not change the model weights. In a broad sense, this can be described as a learning effect of the system, but strictly speaking, it is adaptation using external memory and context. Documents or search indexes must be retained for the changes to persist in subsequent executions. Evaluation, Security, and Economics That Are Easy to Miss in Operations Agent design does not end with an architecture diagram. In actual operations, a system that measures what actually occurred is more important than what was permitted. Minimum Operational Metrics Task success rate and human correction rate Model and tool cost per task and total execution time Number of iterations and percentage of calls consumed without progress Number of approval requests, denials, and attempts to exceed permissions Incorrect tool calls and recovery success rate Percentage of results submitted without sources or tests Degree to which results vary for the same input Security Invariants Instructions in external documents do not take priority over system policies. Secrets are not unnecessarily exposed in model inputs or logs. Read permissions are separated from write, delete, and deployment permissions. External transmission and irreversible actions require separate approval or policy checks. Agents cannot arbitrarily modify their own evaluation criteria, tests, or audit logs. These invariants are safer when enforced through sandboxes, access control, graph transitions, and independent validators rather than through a single sentence in a prompt. Generative AI risk management must cover not only model accuracy but also the operating environment, human oversight, and incident response. Which Concept Should You Learn First? In current practice, harness engineering should be learned first. Equipping a single agent with accurate context, least privilege, automated validation, logs, approvals, and cost limits can reduce many failures. Next, add loops with termination conditions to tasks where repetition improves quality. When branching, parallel processing, recovery, and approval procedures become complex, express them explicitly as a graph. Before adopting complex terminology, prioritize making the agent's objectives, permissions, evidence, costs, and stopping conditions measurable. FAQ Q. How is harness engineering different from prompt engineering? A. Prompt engineering primarily deals with the instructions and wording provided to a model. Harness engineering is the design of a broader execution environment that includes not only prompts but also context retrieval, tools, permissions, sandboxes, tests, logs, human approval, and error recovery. Q. Do context engineering and harness engineering mean the same thing? A. No. Context engineering focuses on selecting, retrieving, summarizing, and arranging the information the model currently needs to know. In addition to context management, harness engineering also addresses permissions, tools, validation, cost limits, and operational policies. Q. What is the most important difference between a loop and a graph? A. A loop defines what is repeated and when to stop, such as planning, execution, validation, and revision. A graph defines which states exist and how transitions can occur from one state to another. A graph can contain one or more loops. Q. Does every AI agent need a graph framework like LangGraph? A. No. For simple, short tasks, a single agent and a minimal harness may be sufficient. Graphs become more valuable when conditional branching, parallel processing, intermediate state persistence, failure recovery, human approval, or execution path auditing is required. Q. Do multi-agent systems always perform better than a single agent? A. No. Multi-agent systems are useful when parallel research, distinct specialized roles, or context separation is needed. If roles overlap or goals are unclear, they may only increase duplicated work, handoff errors, delays, and costs. Q. How do you prevent an agent loop from repeating indefinitely? A. You should set not only a maximum number of iterations but also budgets for time, tokens, tool calls, and cost. The system should treat a state with no new information or reduction in errors as a lack of progress, and be designed to stop or escalate to a human once a certain threshold is reached. Q. Is it enough to review only the Pull Request summary of code written by AI? A. A summary is only supplementary material and does not replace the original changes. For high-risk changes involving authentication, payments, personal information, data migration, or deployment configuration, you must directly review the actual diff, test coverage, dependencies, and rollback procedures. Q. If company documents are continuously injected, does that mean the model has learned them? A. The results may change persistently, but the model weights have not been updated. This is system-level adaptation that preserves external documents, search indexes, memory, and instructions and provides them again in subsequent runs, and it should be distinguished from fine-tuning in the strict sense. Q. Does graph engineering mean returning to the fixed workflows of the past? A. Not necessarily. Modern graphs are closer to hybrid control: they allow agents to plan autonomously and select tools in some segments while explicitly restricting risky transitions and mandatory approval points. Q. What should be determined first when designing a harness? A. The task's success criteria and the cost of failure should be determined first. Next, it is best to provide only the necessary context and tools, and set least-privilege access, automated validation, execution logs, cost limits, and stopping conditions. Sources - Anthropic — Building effective agents: https://www.anthropic.com/research/building-effective-agents - Anthropic — Effective context engineering for AI agents: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents - Anthropic — How we built our multi-agent research system: https://www.anthropic.com/engineering/multi-agent-research-system - LangGraph overview: https://docs.langchain.com/oss/python/langgraph/overview - ReAct: Synergizing Reasoning and Acting in Language Models: https://arxiv.org/abs/2210.03629 - NIST AI 600-1 — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf Images - Central AI system surrounded by loop arrows and a graph of success and failure nodes: https://injoys.com/rails/active_storage/blobs/proxy/eyJfcmFpbHMiOnsiZGF0YSI6ODI0NSwicHVyIjoiYmxvYl9pZCJ9fQ==--3a03e254c3d24990df4c3fc46145a5db24e457ba/ai-eb0e40fe.webp - AI robot moves from a secure harness through a tool loop and branching graph to validation and a warning gate: https://injoys.com/rails/active_storage/blobs/proxy/eyJfcmFpbHMiOnsiZGF0YSI6ODI1MSwicHVyIjoiYmxvYl9pZCJ9fQ==--cea797c99aabaa4b8f760264327fe2ee8b34b423/ai-23d7d97a.webp --- Category: Knowledge Base Source: https://injoys.com/en/articles/ai-agent-harness-loop-graph-engineering License: cc_by Translation-Status: reviewed