System-Prompt Hardening & Instruction-Hierarchy Enforcement

A public-facing conversational AI is a synthetic subject directed to interact with external users through a publicly reachable chat interface. This includes customer support chatbots, sales assistants, website assistants, public knowledge bots, and similar services that respond on behalf of the organization.

This directive creates an elevated exposure condition because every message is untrusted input, but may still influence the synthetic subject’s response. The interface is both a service channel and a manipulation surface. Users may attempt to override instructions, force unauthorized roles, extract source material, generate prohibited advice, or cause the synthetic subject to make commitments that appear to come from the organization.

The main risk is often legal, contractual, regulatory, or reputational rather than technical. Even with limited internal access, a public-facing synthetic subject speaks with apparent organizational authority. If it quotes prices, offers discounts, provides refund guidance, interprets policy, gives regulated advice, or produces offensive content, the adverse outcome may be attributed to the operator.

Investigators should assess the synthetic subject’s directive, published scope, system instructions, response controls, disclaimers, transcript retention, connected tools, and retrieval sources. Particular attention should be given to unauthorized commitments, grounding in approved material, and manipulation that produced an off-policy response.

Investigative Relevance

Public-facing conversational AI is a high-reach synthetic insider pattern because it can be invoked by the public at scale. The lack of an authentication boundary weakens attribution: the external actor may remain anonymous, while the generated output remains visibly associated with the operator.

The standing instruction stack is the configured set of system prompts, developer instructions, policies, role definitions, objectives, prohibitions, and tool-use rules that govern a synthetic subject’s behavior. It also includes the priority order used to resolve conflicts between trusted instructions, user prompts, retrieved content, tool outputs, and other untrusted inputs.

This configuration creates an elevated exposure condition because the instruction stack defines how the synthetic subject interprets its role and boundaries. If the stack is incomplete, changed without review, poorly prioritized, or mixed with untrusted content, the synthetic subject may follow lower-trust instructions over approved constraints.

The primary risk is instruction override. A user prompt, retrieved document, tool response, or external message may conflict with the standing instruction stack and cause the synthetic subject to ignore rules, exceed scope, reveal information, call tools incorrectly, or act outside its approved purpose.

A related risk is false reliance on prompt secrecy. System prompts may be extracted or inferred, and should not contain secrets, credentials, hidden authorization logic, or controls that must remain confidential to be effective. Security decisions should be enforced by downstream systems, not by prompt wording alone.

Investigators should review the approved instruction stack, prompt versions, prompt hashes, change history, session-level effective prompts, tool-use rules, prompt-extraction attempts, and behavior that diverges from declared constraints. Particular attention should be given to prompt drift, prompt tampering, unapproved prompt edits, exposed secrets, and cases where the synthetic subject followed untrusted instructions over higher-priority rules.

Investigative Relevance

Standing instruction stack is relevant because it defines the synthetic subject’s configured role, boundaries, and instruction hierarchy. It is a core configuration element for determining whether the synthetic subject acted according to approved instructions or was influenced by lower-trust input.

Indirect untrusted-content invocation occurs when attacker-controlled or low-trust content causes a synthetic subject to act. The effective instruction is embedded in data the synthetic subject retrieves, ingests, summarizes, renders, or processes, rather than being issued directly by the apparent operator.

This may include instructions hidden in email, webpages, documents, customer relationship management fields, support tickets, pull request comments, uploaded files, calendar invites, chat messages, tool outputs, or other content sources. The synthetic subject may treat the embedded instruction as part of the task context and execute it as if it were a legitimate command.

This invocation creates an elevated exposure condition because the person who triggers the synthetic subject may not be the person who supplied the effective instruction. An employee may ask an assistant to summarize a mailbox, review a document, process a lead, or inspect a pull request, while the actual behavioral instruction comes from external or attacker-controlled content previously placed in that source.

The primary risk is indirect prompt injection. A malicious instruction may cause the synthetic subject to retrieve private data, override constraints, call tools, generate outbound links, render images, transmit information, or combine internal data with an external communication path. This is especially significant where the same context contains private data, untrusted content, and outbound connectivity.

Investigators should review the ingested content, retrieval records, tool outputs, prompt and response logs, rendered links and images, external fetches, web proxy records, Data Loss Prevention (DLP) alerts, requester identity, and downstream tool calls. Particular attention should be given to instruction-like text in retrieved content, invisible or obfuscated payloads, auto-rendered markdown images, new or recently registered domains, character-by-character request patterns, and outputs that combine private data with an outbound channel.

Investigative Relevance

Indirect untrusted-content invocation is relevant because the apparent user request may be benign while the effective command is supplied by another party. The investigation must reconstruct not only who invoked the synthetic subject, but which retrieved or ingested content shaped the action.

Model Context Protocol (MCP) invocation occurs when a synthetic subject is caused to act, change behavior, select tools, or continue a workflow through MCP-supplied content. MCP is an integration pattern that allows a synthetic subject to discover and use external tools, data sources, and actions through a structured interface.

This invocation creates an elevated exposure condition because MCP servers can supply model-visible information before, during, and after tool use. Tool descriptions, server instructions, schemas, tool outputs, resource content, sampling requests, prompts, and connector metadata may all enter the synthetic subject’s context and influence its behavior.

The primary risk is protocol-mediated instruction flow. The synthetic subject may treat MCP-supplied text as trusted operational context even when it originates from a third-party server, compromised connector, unapproved tool, or attacker-influenced resource. This may cause the synthetic subject to call tools, disclose information, change reasoning, prefer malicious servers, or pass sensitive context across boundaries.

Investigators should review MCP server inventory, tool definitions, server instructions, schemas, tool-list responses, resource reads, sampling requests, prompt templates, tool outputs, connection history, version changes, and tool-call logs. Particular attention should be given to instruction-bearing metadata, changed tool descriptions, untrusted server output, cross-server influence, and behavior that begins when an MCP server connects or returns content.

Investigative Relevance

MCP invocation is relevant because MCP can introduce instructions into a synthetic subject through several protocol surfaces, not only through executed tool calls. A synthetic subject may be influenced when a server connects, when tools are listed, when resources are read, when prompts are supplied, or when tool output is returned.

This section is especially relevant where synthetic subjects connect to MCP servers, tool registries, marketplace connectors, local developer tools, SaaS integrations, filesystem tools, browser tools, or third-party agent tool platforms.

Deconstructed and staged invocation occurs when a harmful objective is deliberately broken into a sequence of prompts, subtasks, role-play frames, or staged inputs that appear benign when reviewed individually. The synthetic subject may comply with each request in isolation, while the sequence composes into an unauthorized, prohibited, or harmful action.

This invocation creates an elevated exposure condition because the effective instruction is distributed across time. No single prompt may contain the full objective, but the chain may cause the synthetic subject to perform reconnaissance, generate instructions, prepare tools, retrieve sensitive data, call systems, or complete a workflow that would have been blocked if requested directly.

The primary risk is cumulative intent concealment. A user may launder a prohibited objective through small technical questions, partial requests, pretextual framing, defensive role-play, or staged task handoffs. The synthetic subject may treat each step as ordinary assistance while progressively assembling the capability, context, or action path needed for the final harmful outcome.

A related risk is machine-paced staging. The sequence may show non-human timing, repetitive phrasing, short technical prompts, or sustained task progression without normal human exploration or browsing rhythm. These patterns may indicate that the synthetic subject is being driven through a deconstructed campaign rather than a legitimate interactive task.

Investigators should review the full prompt sequence, session history, cross-session activity, task tree, tool-call logs, non-human identity records, stated reasoning, role-play framing, and downstream actions. Particular attention should be given to individually benign prompts that form a recognizable chain, repeated short technical requests, pretextual security-testing claims, escalation from reconnaissance to exploitation or exfiltration, and actions whose purpose becomes clear only when the sequence is reconstructed.

Investigative Relevance

Deconstructed and staged invocation is relevant because synthetic subject behavior may be shaped by a campaign rather than a single prompt. The investigation must evaluate the cumulative objective of the interaction, not only whether each individual invocation appeared permitted.

This section is especially relevant where synthetic subjects assist with technical workflows, security analysis, coding, data retrieval, account operations, investigation tasks, tool use, or multi-step planning that can be deconstructed into benign-looking stages.

Poisoned tool description occurs when a connected tool’s description, metadata, schema, or usage instructions contain hidden or adversarial instructions that influence the synthetic subject. These instructions may be visible to the model during tool selection but hidden, truncated, or unclear to the human reviewer.

This configuration creates an elevated exposure condition because tool descriptions are not passive documentation. They may enter the model’s context and affect which tool is selected, what data is passed to it, and how the synthetic subject behaves before or after the call.

The primary risk is metadata-driven manipulation. A synthetic subject may disclose secrets, call unsafe tools, pass sensitive data, or perform unauthorized actions because the tool description instructs it to do so.

Investigators should review tool descriptions, metadata, schemas, client display behavior, hidden text, encoded content, prompt-like language, and tool-call logs. Particular attention should be given to descriptions containing instructions to ignore rules, access sensitive files, forward data, or prefer one tool over another.

Investigative Relevance

Poisoned tool descriptions are relevant because the attack surface may sit in tool metadata rather than user prompts or tool code. This sub-section is especially relevant where tools are installed from marketplaces, open registries, third-party repositories, or MCP servers managed outside the organization.

Connect-time tool metadata invocation occurs when a synthetic subject is influenced by tool, connector, or Model Context Protocol (MCP) metadata at the moment the tool is made available. MCP is an integration pattern that allows a synthetic subject to discover and use external tools, data sources, and actions through a structured interface.

This invocation does not require the tool to be called. The triggering content may appear in the tool description, server instructions, schema, parameter text, tool list, or other metadata loaded into the synthetic subject’s context during connection or discovery. Once that metadata is visible to the model, it may function as an instruction source.

This creates an elevated exposure condition because the synthetic subject may change behavior before any observable tool execution occurs. A malicious or compromised connector may instruct the synthetic subject to prefer a certain tool, ignore competing tools, request sensitive data, disclose information, alter its reasoning, or prepare a later action before the operator has approved any tool call.

The primary risk is line jumping. The connector-supplied metadata enters the instruction context ahead of the normal approval point, allowing it to influence the synthetic subject before a human reviews a specific action. Human approval may then become ineffective because the synthetic subject has already been steered by the metadata it received at connection time.

A related risk is metadata drift. A tool or server may appear safe when first approved, then later change its description, schema, server instructions, or metadata. If those changes are not detected and re-approved, an already trusted connector can become a new invocation source without a new user prompt or tool execution.

Investigators should review the exact tool metadata loaded into context, tool descriptions, server instructions, schema fields, connector version history, MCP server responses, approval records, tool-list payloads, hidden characters, and behavior changes following connection. Particular attention should be given to instruction-like metadata, invisible Unicode, changed descriptions, cross-server tool shadowing, and behavioral shifts that correlate with connector enrollment rather than a user prompt.

Investigative Relevance

Connect-time tool metadata invocation is relevant because the triggering instruction may enter the synthetic subject before any tool use appears in ordinary logs. An investigation that reviews only executed tool calls may miss the earlier metadata that caused the synthetic subject to behave differently.

This section is especially relevant where synthetic subjects connect to MCP servers, plugins, marketplace tools, internal connectors, tool registries, tool discovery endpoints, or dynamically supplied function definitions.

MCP tool-output invocation occurs when output returned by a Model Context Protocol (MCP) tool causes a synthetic subject to act. MCP is an integration pattern that allows a synthetic subject to discover and use external tools, data sources, and actions through a structured interface.

This invocation occurs after a tool call, when the tool result is returned to the synthetic subject’s context. The output may contain ordinary data, but it may also contain instruction-bearing content that causes the synthetic subject to call another tool, disclose information, alter its plan, or continue a workflow in an unsafe direction.

The primary risk is tool-returned instruction execution. A compromised, malicious, or attacker-influenced MCP tool may return content that the synthetic subject treats as a command rather than as data. This is especially significant where the tool processes web pages, files, tickets, customer records, repository content, database results, or other low-trust sources.

Investigators should review raw tool outputs, tool-call logs, returned content, subsequent tool calls, prompt and response records, and downstream actions. Particular attention should be given to instructions embedded in tool results, external content returned by tools, repeated tool-call chains, and actions that began only after a specific tool output entered context.

Investigative Relevance

MCP tool-output invocation is relevant because tool results can become an instruction source after the operator has already approved or initiated a tool call. The investigation must determine whether the synthetic subject acted on the requested tool result or on instructions embedded inside that result.

This sub-section is especially relevant where MCP tools retrieve webpages, read files, query databases, summarize tickets, inspect repositories, fetch external resources, or return model-visible text from untrusted or mixed-trust sources.

MCP prompt template invocation occurs when a prompt template supplied by an MCP server causes a synthetic subject to act. MCP prompt templates allow servers to provide structured messages and instructions for interacting with language models. Prompt templates can provide reusable instructions, workflows, or task structures that are exposed by a server and made available to the user or client.

This invocation creates an elevated exposure condition because a server-supplied prompt may carry instructions that appear operationally legitimate. The prompt template may define the task, request inputs, structure the workflow, select tools, or frame what the synthetic subject should do next.

The primary risk is server-supplied task steering. A malicious or compromised MCP prompt template may cause the synthetic subject to request sensitive data, invoke unsafe tools, disclose context, bypass policy, or route work through attacker-controlled resources while appearing to follow a normal workflow.

Investigators should review MCP prompt templates, prompt arguments, template provenance, server ownership, prompt version history, user selections, resulting model prompts, and downstream tool calls. Particular attention should be given to prompt templates that request secrets, route data externally, override standing instructions, or cause tool use beyond the apparent task.

Investigative Relevance

MCP prompt template invocation is relevant because the instruction source may be a reusable prompt supplied by a server rather than a direct user instruction. The operator may select a prompt template without understanding the full model-visible instruction it contains.

This sub-section is especially relevant where MCP servers expose slash-command-style workflows, reusable prompts, coding templates, investigation templates, automation prompts, or task-specific prompt libraries.

MCP sampling invocation occurs when a Model Context Protocol (MCP) server requests that the client or host perform a model completion on the server’s behalf. 

A model completion is the process where a model generates a response from supplied messages, instructions, or context. In this pattern, the MCP server can ask the host model to generate text, reason over supplied content, summarize information, or produce an answer that is then returned to the server or used in the workflow.

This invocation creates an elevated exposure condition because the server is no longer only returning data or exposing tools. It may initiate a model-facing request that causes reasoning, summarization, generation, or tool-adjacent behavior through the client’s model environment.

The primary risk is server-originated model steering. A malicious or compromised MCP server may use sampling requests to introduce instructions, request sensitive context, summarize information it should not receive, or cause the host model to generate content that supports later misuse.

Investigators should review MCP sampling requests, server identity, request payloads, messages supplied by the server, model responses, user consent records, returned outputs, and subsequent server or tool activity. Particular attention should be given to sampling requests from low-trust servers, requests involving sensitive context, unexpected model completions, and server-originated prompts that resemble operator instructions.

Investigative Relevance

MCP sampling invocation is relevant because it allows an MCP server to initiate model activity rather than merely respond to a user-initiated tool call. The invocation source is the server request, and the resulting model output may influence later actions.

This sub-section is especially relevant where MCP clients permit server-requested sampling, agentic workflows, model-in-the-loop tools, server-side planning, or integrations that allow external servers to request completions through the host model.