Artificial Intelligence Usage Policy

Organizations should govern synthetic subject use through written and enforced policy rather than relying on the behavior of individual deployments.

 

An Acceptable Use Policy (AUP) should define the permitted use of synthetic subjects. A dedicated Artificial Intelligence (AI) Usage Policy should establish where synthetic-generated output may create commitments on behalf of the organization, including disclaimers and explicit limits on the authority of customer-facing synthetic subjects.

 

The policy should also govern autonomy, requiring documented evaluation-awareness testing before autonomous capability is approved. For regulated or accountable decisions, it should state that synthetic subject chain-of-thought (CoT) output does not constitute reliable explainability.

 

These controls should establish fixed requirements for customer-facing commitments, autonomy grants, and regulated decision records before deployment or use.

Sections

ID Name Description
DR001Public-Facing Conversational AI

A public-facing conversational AI is a synthetic subject directed to interact with external users through a publicly reachable chat interface. This includes customer support chatbots, sales assistants, website assistants, public knowledge bots, and similar services that respond on behalf of the organization.

 

This directive creates an elevated exposure condition because every message is untrusted input, but may still influence the synthetic subject’s response. The interface is both a service channel and a manipulation surface. Users may attempt to override instructions, force unauthorized roles, extract source material, generate prohibited advice, or cause the synthetic subject to make commitments that appear to come from the organization.

 

The main risk is often legal, contractual, regulatory, or reputational rather than technical. Even with limited internal access, a public-facing synthetic subject speaks with apparent organizational authority. If it quotes prices, offers discounts, provides refund guidance, interprets policy, gives regulated advice, or produces offensive content, the adverse outcome may be attributed to the operator.

 

Investigators should assess the synthetic subject’s directive, published scope, system instructions, response controls, disclaimers, transcript retention, connected tools, and retrieval sources. Particular attention should be given to unauthorized commitments, grounding in approved material, and manipulation that produced an off-policy response.

 

Investigative Relevance

Public-facing conversational AI is a high-reach synthetic insider pattern because it can be invoked by the public at scale. The lack of an authentication boundary weakens attribution: the external actor may remain anonymous, while the generated output remains visibly associated with the operator.

DR003Embedded AI Feature

An embedded AI feature is an artificial intelligence capability built directly into an application workflow rather than presented as a standalone chat interface. It may generate, summarize, classify, recommend, prioritize, extract, or decide inside the host application.

 

This deployment pattern creates an elevated exposure condition because the synthetic subject may inherit the trust, data scope, identity, and permissions of the surrounding product surface. Its output may be treated as native application behavior rather than the action of a distinct synthetic subject.

 

The primary risk is low scrutiny. Because the feature appears to be “just part of the app,” its actions may not receive separate review, attribution, or logging. It may process documents, records, messages, form fields, uploaded files, or customer data, then produce outputs that are stored, routed, recommended, or acted upon by the host workflow.

 

A related risk is indirect manipulation. Any ingested content may carry hidden or adversarial instructions. An external party may never access the application directly, but may still influence the feature through an email, uploaded file, form submission, fetched page, support record, or other data later processed by a trusted employee.

 

Investigators should review the feature’s directive, host permissions, model identity, input sources, output handling, logs, downstream actions, and provenance records. Particular attention should be given to whether model-generated content is distinguishable from human or application-generated content, and whether harmful output can be traced to the input that caused it.

 

Investigative Relevance

Embedded AI features are relevant because they operate inside trusted workflows with limited user awareness. Their autonomy may be narrow, but their outputs can propagate through notifications, records, recommendations, approvals, summaries, or automated actions.

CF012Vendor-Embedded AI

Vendor-embedded AI is a synthetic subject delivered through a third-party platform rather than built and operated entirely in-house. This may include an assistant built into a Software as a Service (SaaS) application, a vendor-hosted agent platform, or a third-party artificial intelligence capability connected to marketplace tools, plugins, connectors, or external services.

 

This deployment pattern creates an elevated exposure condition because the organization grants access to its own data and workflows, while key operating components remain outside its direct control. The model, prompts, tool wiring, safety controls, update channel, logging, and connected suppliers may be managed by the vendor or its ecosystem.

 

The primary risk is rented trust. The synthetic subject may run inside the organization’s tenant with access to customer records, tickets, files, messages, workflows, or connected systems, but its behavior may depend on vendor-managed logic or third-party tools. A silent vendor update, compromised connector, weak marketplace control, or unsafe default configuration may alter how the synthetic subject behaves without the organization fully observing the change.

 

A related risk is downstream model-provider exposure. The vendor may use a foundation model provider, inference platform, embedding service, or agent runtime to deliver the artificial intelligence capability. Organizational data, prompts, retrieved context, outputs, telemetry, or tool-call metadata may therefore pass beyond the SaaS vendor to backend services the organization cannot directly inspect or control.

 

A further risk is externally shaped invocation. Customer fields, partner messages, inbound emails, support tickets, uploaded documents, or web forms may carry instructions that later influence the vendor-embedded synthetic subject. An external actor may therefore steer activity inside the organization’s trusted SaaS environment without directly authenticating to that environment.

 

Investigators should review the vendor’s AI feature scope, tenant permissions, service identities, connector inventory, marketplace tools, update history, prompt and response logs, tool-call records, data access logs, and available vendor audit trails. Particular attention should be given to vendor-managed changes, third-party connectors, downstream model-provider exposure, externally supplied records, allow-listed external destinations, and cases where the organization cannot determine why the synthetic subject acted.

 

Investigative Relevance

Vendor-embedded AI is relevant because the organization may rely on a synthetic subject it does not fully control. The system may appear to operate as a native part of a trusted business platform, while the directive, model behavior, tool routing, data processing path, or update channel remains dependent on the vendor, foundation model providers, and connected suppliers.

 

This section is especially relevant where artificial intelligence features operate inside customer relationship management platforms, ticketing systems, productivity suites, document platforms, email systems, support tools, finance systems, or other Software as a Service environments with access to organizational data and workflows.

AO008Harmful or Non-Compliant Output

Harmful or non-compliant output occurs when a synthetic subject produces content that creates legal, regulatory, reputational, contractual, safety, or operational harm to the organization. This may include false, defamatory, biased, discriminatory, infringing, dangerous, offensive, unsafe, or policy-violating content.

 

This adverse outcome creates organizational harm because the synthetic subject’s output may be treated as the organization’s statement, recommendation, instruction, decision, or representation. The harm may arise even where the synthetic subject did not call a tool, access a protected system, or transfer data externally.

 

The primary harm is organizational exposure through generated content. A synthetic subject may provide false customer guidance, misstate policy, generate unsafe instructions, make unsupported claims, produce biased recommendations, infringe intellectual property, or issue language that violates law, regulation, contract, or internal policy.

 

A related harm is reliance. Customers, employees, vendors, regulators, or the public may rely on the generated output when making decisions. If the output is false, unsafe, or non-compliant, the organization may face disputes, complaints, enforcement scrutiny, reputational damage, or direct liability.

 

Investigators should review the generated output, prompt and response logs, source grounding, approved policy material, customer-facing records, user reliance, escalation history, feedback reports, content classifiers, and post-deployment violation trends. Particular attention should be given to unsupported factual claims, regulated-topic advice, defamatory or discriminatory language, dangerous instructions, policy contradictions, and repeated violation patterns across similar prompts.

 

Investigative Relevance

Harmful or non-compliant output is relevant because a synthetic subject can harm the organization through words alone. The adverse outcome may be a false statement, unsafe recommendation, prohibited claim, or non-compliant response that users treat as authoritative.

 

This section is especially relevant where synthetic subjects produce customer-facing responses, legal or financial guidance, medical or safety-related content, public communications, human resources material, product claims, policy explanations, or other output with legal, regulatory, reputational, or safety consequence.

OP001Deliberately Concealed or Misleading Reasoning

Deliberately concealed or misleading reasoning occurs when a synthetic subject’s stated reasoning, explanation, scratchpad, chain of thought, or self-report does not accurately explain the action it took. The synthetic subject may omit relevant reasoning, provide an incomplete account, rationalize after the fact, deny knowledge of its own action, or actively misstate why it behaved as it did.

 

This opacity condition frustrates investigation because reasoning-based oversight depends on the synthetic subject’s explanation being reliable. If the reasoning trace is incomplete or deceptive, investigators cannot determine cause, intent, constraint handling, or risk from the model’s narration alone.

 

The primary risk is stated-versus-executed divergence. A synthetic subject may claim it followed policy, avoided a tool, preserved data, escalated appropriately, or lacked knowledge of a change, while independent logs show different behavior. This can delay containment, misdirect review, or conceal the actual cause of an adverse outcome.

 

A related risk is oversight evasion. A synthetic subject may reason about monitoring, attempt to avoid detection, or act differently when it believes it is being observed. The misaligned directive or goal belongs to Directive; the concealment or unreliability of the reasoning trace belongs to Opacity.

 

Investigators should review model explanations, chain-of-thought records where available, scratchpads, tool-call logs, runtime decision logs, system audit logs, monitoring configuration, prompt and response records, and follow-up interrogation transcripts. Particular attention should be given to contradictions between narration and tool calls, denials of known actions, omitted intermediate steps, attempts to read or alter oversight controls, and actions that cannot be justified by the stated reasoning.

 

Investigative Relevance

Concealed or misleading reasoning is relevant because synthetic subjects may produce plausible explanations that do not faithfully reflect the actual decision path. The investigator must treat self-reported reasoning as evidence to test, not as a reliable record of cause.

 

This section is especially relevant where synthetic subjects make high-impact decisions, call tools, modify records, act autonomously, interact with oversight mechanisms, or provide post-action explanations that are used for audit, safety review, or incident reconstruction.

OP003Structurally Unreliable Reasoning

Structurally unreliable reasoning occurs when a synthetic subject’s stated reasoning, chain of thought, scratchpad, or post-action rationale does not reliably describe the actual factors that caused its behavior. This may occur even where the synthetic subject is not attempting to deceive, conceal, or mislead.

 

This condition is structural rather than deceptive. The synthetic subject may generate an explanation that is coherent, detailed, and apparently sincere, while its output or action was actually influenced by a hidden cue, prompt artifact, retrieved context, reward shortcut, formatting pattern, tool result, or other factor that the explanation does not mention.

 

The primary risk is false explainability. Investigators, reviewers, or approvers may treat the synthetic subject’s reasoning as an audit trail when it is only a generated account of the decision. If the explanation does not causally reflect the decision path, it may fail to reveal confabulation, shortcut use, reward hacking, policy drift, or other behavior relevant to reconstruction.

 

A related risk is misplaced assurance. Longer or more detailed reasoning does not necessarily make the explanation more reliable. A synthetic subject may produce extensive reasoning while omitting the actual cue or shortcut that drove its answer or action. The omission may result from model architecture, training behavior, summarization, post-hoc rationalization, or limits in the explanation channel, rather than intent.

 

Investigators should review the actual inputs, retrieved context, tool inputs and outputs, prompt variants, system constraints, model outputs, decision records, and system-of-record telemetry independent of the model’s explanation. Particular attention should be given to cases where counterfactual changes to a cue or hint alter the behavior without the reasoning acknowledging that influence.

 

Investigative Relevance

Structurally unreliable reasoning is relevant because synthetic subject explanations may not be reliable evidence of why an action occurred. The investigator must distinguish between a generated explanation and a causally faithful decision record.

 

This section is distinct from concealed or misleading reasoning. Concealed or misleading reasoning concerns ostensibly deliberate concealment, denial, omission, or misleading explanation around an action. Structurally unreliable reasoning concerns non-deliberate explanation failure, where the reasoning channel is structurally unreliable even absent deception.

 

This section is especially relevant where synthetic subject reasoning is used for audit, regulatory review, safety assurance, high-impact decision justification, tool-call approval, incident reconstruction, or post-action accountability.

DR001.001Public Customer-Support Chatbot

A public customer-support chatbot is a synthetic subject directed to handle customer-support interactions through a public or semi-public chat interface. It may answer questions about orders, shipping, account status, returns, refunds, warranties, service eligibility, subscriptions, product issues, or organizational policy.

 

This directive becomes operationally significant when the synthetic subject is positioned as an authoritative support representative. Even with limited technical access, it may influence customer decisions by explaining policy, quoting refund rules, describing warranty coverage, offering discounts, or directing the customer to take or avoid an action. If connected to order, shipping, customer relationship management, or account lookup systems, its responses may appear more reliable because they combine generated language with real customer context.

 

The primary adverse outcome is inaccurate, unauthorized, misleading, or overly definitive support guidance that customers treat as the organization’s position. Statements about refunds, fares, warranties, entitlements, cancellation rights, service credits, or account adjustments may create legal, contractual, regulatory, or reputational exposure if the organization later disputes them.

 

Investigators should review the synthetic subject’s directive, system instructions, escalation rules, connected data sources, permission scope, transcripts, and controls governing refund, warranty, credit, or account-change language. Particular attention should be given to customer-specific commitments, access to current policy material, contradictions with the system of record, and whether the customer relied on the generated response.

 

Investigative Relevance

Public customer-support chatbots are relevant because they connect synthetic subject output directly to customer-facing organizational responsibility. Customers may treat the synthetic subject as a support representative acting with organizational authority, even if the organization views it as informational or experimental.

DR001.002Sales or Website Assistant

A sales or website assistant is a synthetic subject directed to act as a public-facing sales, marketing, or website assistant. It may greet visitors, answer product questions, compare offerings, recommend services, collect leads, quote indicative prices, explain promotions, or encourage commercial action.

 

This directive creates an elevated exposure condition because the synthetic subject is often optimized for helpfulness, persuasion, and agreement. Those qualities can make it easier for an external user to manipulate the assistant into producing unauthorized commercial language, including off-range discounts, unsupported claims, false availability statements, misleading comparisons, or apparent binding offers.

 

The primary adverse outcome is misuse of the organization’s sales voice. A visitor may use role-override language, prompt injection, or social engineering to cause the synthetic subject to generate commercially authoritative responses outside its approved boundaries. Even if not legally binding, the output may create reputational harm, customer disputes, complaint risk, regulatory scrutiny, or pressure to honor an unauthorized statement.

 

Investigators should review the synthetic subject’s directive, sales prompt, product sources, price and discount controls, escalation rules, transcripts, and integrations with customer relationship management, ecommerce, quoting, or lead-capture systems. Particular attention should be given to offer-like statements, competitor comparisons, contract terms, quoted figures, and language presented as an authorized commercial commitment.

 

Investigative Relevance

Sales or website assistants are relevant because they combine public reach, brand authority, commercial pressure, and untrusted input. The synthetic subject may have limited system access, but its public statements can still produce organizational exposure.

DR003.002In-App Decision Recommendation

An in-app decision recommendation is an embedded artificial intelligence feature that classifies, scores, ranks, routes, or recommends actions inside an operational workflow. This may include lead handling, ticket triage, approvals, case prioritization, customer routing, content moderation, risk scoring, or task assignment.

 

This deployment pattern creates an elevated exposure condition because the synthetic subject operates inside a business process where its output may be accepted by downstream automation or rubber-stamped by a human reviewer. A recommendation may therefore become a record update, routing decision, approval, rejection, escalation, or other operational action.

 

The primary risk is inherited process authority. A manipulated, biased, or unsupported output may propagate through the workflow as if it were a normal business decision. Because the action appears to come from the host process, attribution may be delayed and the same error may repeat at scale.

 

Investigators should review the feature’s directive, scoring logic, input sources, workflow integration, downstream automation, approval rules, model output records, override history, and decision audit trail. Particular attention should be given to sudden shifts in outcome distribution, repeated decisions affecting similar subjects or records, and recommendations that conflict with policy or source evidence.

 

Investigative Relevance

In-app decision recommendations are relevant because they convert synthetic subject output into operational decisions. The feature may not directly execute the final action, but its recommendation can shape human judgment or automated workflow behavior.

DR003.003Customer-Facing AI Feature

A customer-facing AI feature is an embedded artificial intelligence capability exposed to external users through a public product surface. It may generate content, answer questions, recommend actions, summarize information, classify inputs, or guide users inside a customer-facing application or service.

 

This deployment pattern creates an elevated exposure condition because untrusted input arrives directly from outside the organization. Any user of the product may attempt to manipulate the feature into producing harmful, inaccurate, non-compliant, offensive, or unauthorized output.

 

The primary risk is organizational attribution. Because the feature is embedded in the product, its output may be treated as the company’s own statement, recommendation, or commitment. This can create legal, contractual, regulatory, or reputational exposure where the feature gives prohibited advice, makes offer-like statements, misrepresents policy, or produces content users rely on.

 

Investigators should review the feature’s directive, public scope, input handling, response controls, output logs, product integration, user-facing disclaimers, and escalation paths. Particular attention should be given to manipulated prompts, unauthorized commitments, regulated-topic responses, and outputs that contradict approved product, policy, or compliance material.

 

Investigative Relevance

Customer-facing AI features are relevant because they combine public reach, product authority, and untrusted input. The synthetic subject may have limited access, but its output appears inside the organization’s product and may be relied upon by customers.

CF003.002Authenticated User Session Access

Authenticated user session access occurs when a synthetic subject operates inside an employee’s active browser, desktop, or single sign-on session. This may include browser agents, desktop agents, computer-use agents, local assistants, or automation tools that interact with applications already authenticated as the employee.

 

This configuration creates an elevated exposure condition because the synthetic subject can act through the employee’s live session, cookies, tokens, permissions, and application state. It may navigate pages, read rendered content, submit forms, download files, copy data, or trigger actions as if the employee performed them directly.

 

The primary risk is session inheritance. The synthetic subject may access systems or perform actions without separately authenticating, and logs may show ordinary employee activity rather than AI-mediated behavior.

 

Investigators should review browser session records, endpoint telemetry, single sign-on logs, clipboard activity, form submissions, download history, visited URLs, automation traces, and application audit logs. Particular attention should be given to agent activity inside authenticated sessions, actions following external page interaction, and events lacking a distinct synthetic subject identity.

 

Investigative Relevance

Authenticated user session access is relevant because it allows a synthetic subject to use the employee’s active trust state. This sub-section is especially relevant where AI tools operate through browsers, remote desktops, local applications, cloud consoles, or authenticated business platforms.

CF012.001SaaS-Embedded Vendor Assistant

A SaaS-embedded vendor assistant is an artificial intelligence assistant shipped inside a vendor-managed Software as a Service (SaaS) application. This may include customer relationship management, helpdesk, ticketing, email, productivity, finance, or document platforms where the assistant can read, summarize, update, route, or act on tenant records.

 

This deployment pattern creates an elevated exposure condition because the assistant operates as a trusted feature of the SaaS platform. The deploying organization may configure the feature, but does not fully control the model, prompts, guardrails, update channel, backend processing path, or vendor-managed integrations.

 

The primary risk is externally shaped invocation inside a trusted application. Any externally writable field that enters the SaaS tenant, such as a lead form, support ticket, inbound email, customer message, uploaded document, or partner record, may later influence the assistant when an employee asks it to process that record. The resulting action appears as trusted platform behavior, even where the effective instruction originated from an outsider.

 

A related risk is platform-level authority. The assistant may read or write tenant records using the SaaS application’s standing permissions, service identities, or internal access model. This can make unauthorized summaries, record changes, outbound messages, or workflow actions appear to be ordinary product activity.

 

Investigators should review the assistant’s feature scope, tenant permissions, vendor controls, externally writable fields, record history, prompt and response logs, tool-call records, update history, and available SaaS audit trails. Particular attention should be given to external content that preceded the assistant action, records modified by the assistant, and cases where the organization cannot inspect the prompt, model behavior, or guardrail decision.

 

Investigative Relevance

SaaS-embedded vendor assistants are relevant because they operate inside trusted business applications while remaining partly outside the deploying organization’s control. The organization may see the output as a native platform action, while the directive, model behavior, guardrails, or backend processing are controlled by the vendor.