{"id":257,"date":"2026-09-03T13:55:09","date_gmt":"2026-09-03T13:55:09","guid":{"rendered":"https:\/\/site.beachbot.dk\/?p=257"},"modified":"2026-09-03T13:55:09","modified_gmt":"2026-09-03T13:55:09","slug":"the-model-isnt-the-architecture-why-ai-agents-need-a-harness","status":"publish","type":"post","link":"https:\/\/site.beachbot.dk\/index.php\/2026\/09\/03\/the-model-isnt-the-architecture-why-ai-agents-need-a-harness\/","title":{"rendered":"The Model Isn\u2019t the Architecture: Why AI Agents Need a Harness"},"content":{"rendered":"

For much of the generative AI boom, enterprise technology strategy has revolved around one question: Which model should we use?<\/strong><\/p>\n

GPT, Claude, Gemini, Llama and an expanding field of specialist models are compared on reasoning ability, context windows, benchmark performance and cost.<\/p>\n

But as organizations move from AI experiments to production systems, that question is becoming less important.<\/p>\n

The more consequential architectural question is:\u00a0What surrounds the model?<\/strong><\/p>\n

An AI model can reason, generate text and interpret information. But it cannot, by itself, reliably operate an enterprise process. For that, organizations need an additional architectural layer responsible for context, memory, tools, permissions, workflow, monitoring and recovery.<\/p>\n

That layer is increasingly described as the agent harness<\/strong>.<\/p>\n

IBM’s explanation of models and harnesses highlights an important distinction between model inference and the orchestration required to turn model intelligence into useful agent behavior.<\/p>\n

For enterprise architects, that distinction has significant consequences.<\/p>\n

The model is becoming a component, not the architecture<\/h2>\n

Consider a conventional AI application, where a user submits a prompt. The application sends it to a language model and the model generates an answer.<\/p>\n

The architecture is comparatively simple:\u00a0User \u2192 Application \u2192 Model \u2192 Response<\/strong><\/p>\n

An enterprise AI agent is different.<\/p>\n

Suppose an organization wants an AI agent that can investigate an invoice discrepancy.<\/p>\n

The agent may need to:<\/p>\n

\"\"<\/p>\n

The language model performs only part of this process.<\/p>\n

The surrounding system determines what the model knows, what it can do and what happens when something goes wrong<\/strong>.<\/p>\n

That surrounding system is the harness.<\/p>\n

What belongs in an enterprise agent harness?<\/h2>\n

There is no single universal definition, but architecturally an enterprise-grade harness can be thought of as several cooperating capabilities.<\/p>\n

Context management<\/h3>\n

The harness determines what information should be supplied to the model for a particular task.<\/p>\n

That might include customer records, documents, policies, previous conversations, application state or dynamically retrieved enterprise knowledge.<\/p>\n

This makes context engineering an architectural concern rather than merely a prompting technique.<\/p>\n

Memory<\/h3>\n

Agents may need information that persists beyond a single inference request.<\/p>\n

But enterprise memory cannot simply mean “remember everything.”<\/p>\n

Architects must decide:<\/p>\n

    \n
  • what should be remembered;<\/li>\n
  • where it should be stored;<\/li>\n
  • how long it should survive;<\/li>\n
  • who can access it;<\/li>\n
  • whether it contains regulated information;<\/li>\n
  • how it can be corrected or deleted.<\/li>\n<\/ul>\n

    Memory therefore intersects with information architecture, security and records management.<\/p>\n

    Tool orchestration<\/h3>\n

    Useful agents need to interact with systems.\u00a0Those tools might include APIs, databases, search services, SaaS applications, workflow engines or emerging interfaces such as MCP.\u00a0The harness determines which tools are available and when they should be invoked.\u00a0This creates a new integration layer in the enterprise architecture.<\/p>\n

    Identity and authorization<\/h3>\n

    An agent capable of calling enterprise systems requires an identity.\u00a0More importantly, the architecture must determine whose authority the agent is exercising<\/strong>.\u00a0If an employee asks an agent to retrieve salary information, approve an invoice or modify a customer account, the model cannot be allowed to decide whether the operation is permitted.\u00a0Authorization must remain deterministic and enforceable outside the model.<\/p>\n

    Workflow and state<\/h3>\n

    Many enterprise tasks cannot be completed in one model invocation.\u00a0Agents may need to execute sequences such as:<\/p>\n

    reason \u2192 retrieve \u2192 evaluate \u2192 call tool \u2192 inspect result \u2192 reason again \u2192 act<\/strong><\/p>\n

    The harness maintains that state and controls the loop.<\/p>\n

    Error recovery<\/h3>\n

    Enterprise systems fail.\u00a0APIs time out. Documents are missing. Permissions change. Models misunderstand instructions. Tools return unexpected results.\u00a0Production agent architecture therefore needs explicit handling for retries, fallbacks, alternative tools, human escalation and safe termination.<\/p>\n

    A clever model without recovery mechanisms is still a fragile system.<\/p>\n

    Observability<\/h3>\n

    Traditional applications generate logs and metrics.\u00a0Agentic systems need something richer.\u00a0Architects may need visibility into:<\/p>\n

      \n
    • prompts and context;<\/li>\n
    • model selection;<\/li>\n
    • tool calls;<\/li>\n
    • intermediate decisions;<\/li>\n
    • token consumption;<\/li>\n
    • latency;<\/li>\n
    • failures;<\/li>\n
    • policy violations;<\/li>\n
    • confidence and evaluation results.<\/li>\n<\/ul>\n

      The objective is not simply application monitoring.\u00a0It is decision-path observability<\/strong>.<\/p>\n

      The architectural shift: intelligence becomes replaceable<\/h2>\n

      This leads to one of the more interesting consequences for enterprise architecture.\u00a0The model does not necessarily have to be permanent.\u00a0A well-designed harness could potentially route different tasks to different models.<\/p>\n

      For example:<\/p>\n

      \n
      \n
      \n
      \n
      \n
      \n
      \n
      \n
      <\/div>\n
      \n
      \n
      \n
      \n
      \n
                       Enterprise Agent\r\n                       \u2502\r\n                Agent Harness\r\n                       \u2502\r\n       \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\r\n       \u2502               \u2502               \u2502\r\n     Memory          Tools          Policies\r\n       \u2502               \u2502               \u2502\r\n       \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\r\n                       \u2502\r\n                 Model Router\r\n                \/      |      \\\r\n          Model A   Model B   Model C<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      A relatively inexpensive model might classify requests and a stronger reasoning model might handle complex analysis.\u00a0A specialist model could process code or images and a\u00a0private model might handle particularly sensitive information.<\/p>\n
      The architecture stops treating “the AI” as a single model.\u00a0Instead, models become interchangeable computational resources behind a stable orchestration layer<\/strong>.<\/p>\n
      That is a much more familiar enterprise architecture pattern.<\/p>\n
      Don’t benchmark the model. Benchmark the system.<\/h2>\n
      This distinction also changes how organizations should evaluate AI.<\/p>\n
      A model can achieve excellent benchmark scores while the application built around it performs poorly.<\/p>\n
      Consider an agent that uses a highly capable model but:<\/p>\n
        \n
      • retrieves irrelevant documents;<\/li>\n
      • repeatedly calls expensive tools;<\/li>\n
      • loses state between steps;<\/li>\n
      • has excessive latency;<\/li>\n
      • cannot recover from API failures;<\/li>\n
      • consumes enormous numbers of tokens;<\/li>\n
      • or regularly requires human intervention.<\/li>\n<\/ul>\n
        The model may be excellent but the system is not. Production performance depends heavily on orchestration capabilities such as memory, tool routing and error recovery rather than model capability alone.\u00a0Enterprise AI therefore needs system-level metrics.<\/p>\n
        Instead of asking only:\u00a0How accurate is the model? <\/strong>-organizations should increasingly measure:<\/p>\n
          \n
        • Cost per successfully completed task<\/strong><\/li>\n
        • Time per completed task<\/strong><\/li>\n
        • Tool-call success rate<\/strong><\/li>\n
        • Human escalation rate<\/strong><\/li>\n
        • Recovery rate<\/strong><\/li>\n
        • Policy violation rate<\/strong><\/li>\n
        • End-to-end task accuracy<\/strong><\/li>\n<\/ul>\n
          Those measurements tell architects considerably more about production viability than a model benchmark alone.<\/p>\n
          The harness may become the strategic asset<\/h2>\n
          There is another important implication:\u00a0Models are evolving extremely quickly.<\/p>\n
          Today’s preferred model may not be the preferred model twelve months from now and an enterprise architecture tightly coupled to one model provider therefore creates unnecessary architectural risk.\u00a0The organization’s more durable investment may instead be its agent infrastructure<\/strong>:<\/p>\n
            \n
          • its connectors;<\/li>\n
          • its semantic layer;<\/li>\n
          • its knowledge architecture;<\/li>\n
          • its identity model;<\/li>\n
          • its policies;<\/li>\n
          • its observability;<\/li>\n
          • its evaluation framework;<\/li>\n
          • its workflow definitions;<\/li>\n
          • and its integration with enterprise systems.<\/li>\n<\/ul>\n
            Models can then evolve underneath that architecture and this resembles previous shifts in enterprise computing.\u00a0Applications became less dependent on particular servers and services became less dependent on particular infrastructure.\u00a0Cloud-native applications became less dependent on individual machines.<\/p>\n
            Agentic architecture may produce a similar abstraction:\u00a0enterprise capabilities become less dependent on individual AI models.<\/strong><\/p>\n
            Enterprise architects should design the harness first<\/h2>\n
            This suggests a practical change in how organizations approach agentic AI projects.<\/p>\n
            Rather than beginning architecture discussions with: Which LLM should we standardize on? <\/strong>–\u00a0start with: What capabilities must an AI agent have to operate safely inside our enterprise?<\/strong><\/p>\n
            That conversation naturally leads to architecture.<\/p>\n
              \n
            • What identity does an agent have?<\/li>\n
            • What can it access?<\/li>\n
            • How does it discover tools?<\/li>\n
            • Where does its context originate?<\/li>\n
            • How is enterprise knowledge represented?<\/li>\n
            • What actions require human approval?<\/li>\n
            • How are decisions audited?<\/li>\n
            • How does an agent recover when a tool fails?<\/li>\n
            • Can models be substituted without redesigning the application?<\/li>\n
            • How do we measure the cost of completing a business outcome?<\/li>\n<\/ul>\n
              Those questions are much harder than choosing an LLM and they are also where enterprise architecture creates value.<\/p>\n
              From model-centric AI to system-centric AI<\/h2>\n
              The first phase of generative AI encouraged organizations to think about models.\u00a0The next phase will force them to think about systems.\u00a0Models provide intelligence whereas harnesses turn that intelligence into controlled behavior and in an enterprise, controlled behavior is ultimately what matters.<\/p>\n
              The winning architecture may therefore not be the organization with access to the most powerful model but\u00a0 the organization that builds the best environment around whatever model it chooses.<\/p>\n
              For enterprise architects, that means the center of gravity is moving away from model selection<\/strong> and toward orchestration, identity, integration, context, governance and observability<\/strong>. The model may be the brain.\u00a0But the harness is increasingly becoming the enterprise architecture around it.<\/p>\n","protected":false},"excerpt":{"rendered":"
              For much of the generative AI boom, enterprise technology strategy has revolved around one question: Which model should we use? GPT, Claude, Gemini, Llama and an […]<\/p>\n","protected":false},"author":2,"featured_media":262,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[148,145,1],"tags":[],"class_list":["post-257","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-agents-and-automation","category-ai-architecture","category-uncategorized"],"blog_post_layout_featured_media_urls":{"thumbnail":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-150x150.png",150,150,true],"full":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07.png",1536,1024,false]},"categories_names":{"148":{"name":"AI Agents & Automation","link":"https:\/\/site.beachbot.dk\/index.php\/category\/ai-agents-and-automation\/"},"145":{"name":"AI Architecture","link":"https:\/\/site.beachbot.dk\/index.php\/category\/ai-architecture\/"},"1":{"name":"Uncategorized","link":"https:\/\/site.beachbot.dk\/index.php\/category\/uncategorized\/"}},"tags_names":[],"comments_number":"0","wpmagazine_modules_lite_featured_media_urls":{"thumbnail":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-150x150.png",150,150,true],"cvmm-medium":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-300x300.png",300,300,true],"cvmm-medium-plus":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-305x207.png",305,207,true],"cvmm-portrait":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-400x600.png",400,600,true],"cvmm-medium-square":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-600x600.png",600,600,true],"cvmm-large":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-1024x1024.png",1024,1024,true],"cvmm-small":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07-130x95.png",130,95,true],"full":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/09\/ChatGPT-Image-3.-sep.-2026-15.16.07.png",1536,1024,false]},"_links":{"self":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts\/257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/comments?post=257"}],"version-history":[{"count":7,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts\/257\/revisions"}],"predecessor-version":[{"id":266,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts\/257\/revisions\/266"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/media\/262"}],"wp:attachment":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/media?parent=257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/categories?post=257"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/tags?post=257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}