{"id":334,"date":"2026-11-16T14:13:28","date_gmt":"2026-11-16T14:13:28","guid":{"rendered":"https:\/\/site.beachbot.dk\/?p=334"},"modified":"2026-11-16T14:13:28","modified_gmt":"2026-11-16T14:13:28","slug":"ai-gateway-logging-sensitive-data","status":"publish","type":"post","link":"https:\/\/site.beachbot.dk\/index.php\/2026\/11\/16\/ai-gateway-logging-sensitive-data\/","title":{"rendered":"What Should Your AI Gateway Remember?"},"content":{"rendered":"<p><!-- run-marker: ai-enterprise-x-editorial-188d6fc3-9820-478c-906e-dc0dc5908850 --><\/p>\n<p>An insurer\u2019s claims assistant reads an adjuster\u2019s notes and drafts a request for missing documents. During a rollout, the claims tool starts returning a timeout. The platform team can see the failed call in its shared trace viewer. What it shouldn\u2019t see there is the claimant\u2019s medical note, copied into an exception by the client that handled the timeout.<\/p>\n<p>That\u2019s the distinction to settle before rolling out an AI gateway: the shared trace should show enough to locate and investigate a failure. Sensitive content should stay with the team that owns the workload, available through a separate, authorized investigation path when there\u2019s a reason to see it.<\/p>\n<p>Two November 9 announcements on X put different sides of this design in view. <a href=\"https:\/\/x.com\/langfuse\/status\/2119837547555618872\">Langfuse<\/a> presented a gateway for model calls and tracing. <a href=\"https:\/\/x.com\/appwrite\/status\/2119679207836131488\">Appwrite<\/a> presented MCP1 as a shared entry point for agent tools. These are launch posts from two vendors. Public X visibility is incomplete, and the posts tell us what the vendors are promoting, not how widely enterprises have adopted the pattern.<\/p>\n<h2>One run, two places to leak<\/h2>\n<p>Langfuse\u2019s <a href=\"https:\/\/langfuse.com\/blog\/introducing-gateway\">beta announcement<\/a> says its gateway can record usage and cost without prompt or response content; full capture is opt-in. <a href=\"https:\/\/mcp1.com\/docs\">MCP1\u2019s documentation<\/a> describes a separate kind of gateway, one that checks and records tool calls. Those announcements describe different traffic. In a claims workflow, the same medical note can start as a tool result, become input to a model and show up again in an application error if the client copies part of the request into its diagnostic message.<\/p>\n<p>That\u2019s why switching off prompt logging at the model gateway is only one part of the work. The tool client, application logger and trace exporter can each emit a copy of the same data. Even a read-only tool can create a durable copy elsewhere through its instrumentation. If the platform team can search that copy, a successful access check at the claims system no longer tells you who can read the note.<\/p>\n<h2>Give platform operators a trail to follow<\/h2>\n<p>For this workflow, let the central trace carry an opaque run ID, the approved tool and model route, the operation\u2019s status and duration, a bounded error code and usage where available. On a timeout, that should be enough to tell whether the model stalled or the claims lookup failed, and whether other calls on the same route are failing. Keep free-text exceptions, claim identifiers, prompts and tool payloads out of that shared record. A random run ID can link events without giving the platform operator a search key for a person\u2019s file.<\/p>\n<p>Define those fields where telemetry is emitted, then enforce the contract at export. <a href=\"https:\/\/opentelemetry.io\/docs\/security\/handling-sensitive-data\/\">OpenTelemetry\u2019s guidance<\/a> recommends minimizing collection and describes Collector processors for filtering and redaction. Use those as a second line of defense. An attribute allowlist alone won\u2019t save a pipeline if an allowed message field or log body still carries raw text. Have the instrumented client emit a known error code instead of forwarding an upstream error verbatim.<\/p>\n<p>The claims team should be able to follow the run ID inside its own restricted environment and review the original record under its normal access rules. If it needs the exact assembled prompt to diagnose a formatting bug, start with a synthetic reproduction. When that fails and real content is necessary, scope any separate capture to a named incident, a small set of requests and a defined expiry. Give it its own access and retention rules, including any copies made for debugging or evaluation.<\/p>\n<h2>What you give up by keeping traces lean<\/h2>\n<p>Full capture makes some failures easier to reproduce. With a slim trace, an operator can spot a bad deployment or a failing tool, but may need the claims team to work out why the model took a particular note as an instruction. That handoff adds time, and the exact input may be gone by the time the incident is reported. Set an escalation path and practice the handoff before the first incident. If only the claims team needs the original text to debug a case, keep that text out of the viewer everyone uses to monitor the rollout.<\/p>\n<h2>Test the timeout, then search the exports<\/h2>\n<p>Before rollout, put distinct synthetic markers in a claim note and an upstream error response. Run one successful request, force a tool timeout and retry, then exercise the denied-call path. Search the shared trace store, exported logs and any enabled evaluation sink for both markers. Pass only if neither marker appears and an operator with ordinary platform access can still identify the failing component and hand off the run to the claims team. Have an authorized claims reviewer verify that the run can be located there. If you can preserve privacy only by making the incident impossible to investigate, the trace contract still needs work.<\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/x.com\/langfuse\/status\/2119837547555618872\">Langfuse on X: AI Gateway public beta<\/a>, November 9, 2026.<\/li>\n<li><a href=\"https:\/\/x.com\/appwrite\/status\/2119679207836131488\">Appwrite on X: MCP1 announcement<\/a>, November 9, 2026.<\/li>\n<li><a href=\"https:\/\/langfuse.com\/blog\/introducing-gateway\">Langfuse: AI Gateway launch and logging modes<\/a>.<\/li>\n<li><a href=\"https:\/\/mcp1.com\/docs\">MCP1: gateway, policy and audit documentation<\/a>.<\/li>\n<li><a href=\"https:\/\/opentelemetry.io\/docs\/security\/handling-sensitive-data\/\">OpenTelemetry: Handling sensitive data<\/a>.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>AI gateway logs should help operators investigate failures without turning shared traces into a second store for sensitive case data. Here\u2019s how to test that boundary.<\/p>\n","protected":false},"author":2,"featured_media":333,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[131],"tags":[160,158,164],"class_list":["post-334","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-ai-governance","tag-enterprise-architecture","tag-observability"],"blog_post_layout_featured_media_urls":{"thumbnail":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-150x150.png",150,150,true],"full":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary.png",1672,941,false]},"categories_names":{"131":{"name":"AI","link":"https:\/\/site.beachbot.dk\/index.php\/category\/ai\/"}},"tags_names":{"160":{"name":"AI Governance","link":"https:\/\/site.beachbot.dk\/index.php\/tag\/ai-governance\/"},"158":{"name":"Enterprise Architecture","link":"https:\/\/site.beachbot.dk\/index.php\/tag\/enterprise-architecture\/"},"164":{"name":"Observability","link":"https:\/\/site.beachbot.dk\/index.php\/tag\/observability\/"}},"comments_number":"0","wpmagazine_modules_lite_featured_media_urls":{"thumbnail":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-150x150.png",150,150,true],"cvmm-medium":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-300x300.png",300,300,true],"cvmm-medium-plus":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-305x207.png",305,207,true],"cvmm-portrait":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-400x600.png",400,600,true],"cvmm-medium-square":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-600x600.png",600,600,true],"cvmm-large":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-1024x941.png",1024,941,true],"cvmm-small":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary-130x95.png",130,95,true],"full":["https:\/\/site.beachbot.dk\/wp-content\/uploads\/2026\/11\/ai-gateway-telemetry-boundary.png",1672,941,false]},"_links":{"self":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts\/334","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/comments?post=334"}],"version-history":[{"count":2,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts\/334\/revisions"}],"predecessor-version":[{"id":345,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/posts\/334\/revisions\/345"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/media\/333"}],"wp:attachment":[{"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/media?parent=334"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/categories?post=334"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/site.beachbot.dk\/index.php\/wp-json\/wp\/v2\/tags?post=334"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}