preventions
- ID: SPV013
- Created: 26th August 2026
- Updated: 26th August 2026
- Contributors: Nimer Kees, The ITM Team, Yonatan Machluf,
Model & Tool Supply-Chain Vetting
Organizations should assess the provenance, integrity, and update path of every component used by a synthetic subject.
Model weights, checkpoints, and fine-tunes should be verified, signed, version-pinned, and re-evaluated against a held-out baseline after any change. Tool descriptions, metadata, permissions, and implementation should be reviewed before connecting any marketplace tool or Model Context Protocol (MCP) server.
Dependencies should be pinned and signed, with unauthorized package installation blocked. A Software Bill of Materials (SBOM) should cover the full synthetic subject stack, including models, tools, libraries, rules, and configuration. Training data should also be subject to provenance and integrity controls.
Released artifacts should retain verifiable provenance and signatures so unauthorized modification or delayed malicious behavior can be identified. Where a vendor-supplied Software-as-a-Service (SaaS) assistant cannot be independently inspected, contracts should require access to relevant audit logs and timely notification of breaches or material changes affecting the assistant or its update channel.
Sections
| ID | Name | Description |
|---|---|---|
| CF002 | Connected Tools and Functions | Connected tools and functions are the plugins, connectors, functions, application programming interfaces, and Model Context Protocol (MCP) servers made available to a synthetic subject. MCP is an integration pattern that allows a synthetic subject to discover and use external tools, data sources, and actions through a structured interface.
This configuration creates an elevated exposure condition because connected tools define what the synthetic subject can do beyond text generation. A tool may allow it to read records, send messages, update systems, retrieve files, call APIs, execute commands, or transmit data outside the immediate application.
The primary risk is unsafe tool trust. Tool names, descriptions, schemas, parameters, outputs, and server definitions may influence how the synthetic subject chooses and uses tools. If these definitions are changed, poisoned, impersonated, or poorly governed, the synthetic subject may call the wrong tool, pass sensitive data to an unsafe tool, or perform an action outside its approved purpose.
Investigators should review connected tool inventories, MCP server records, tool definitions, schemas, descriptions, permissions, version history, provenance, tool-call logs, runtime egress, and change approvals. Particular attention should be given to newly added tools, changed descriptions or schemas, shadowed tool names, unexpected destinations, and tool calls that cannot be attributed to an approved configuration.
Investigative RelevanceConnected tools and functions are relevant because they convert a synthetic subject from an advisory system into an actor with operational capability. The configuration of tools determines whether the synthetic subject can only answer, or whether it can retrieve, modify, send, execute, or integrate across systems.
This section is especially relevant where synthetic subjects use marketplace tools, plugins, MCP servers, internal functions, SaaS connectors, email tools, database tools, cloud APIs, command execution tools, or externally supplied tool runtimes. |
| CF007 | Model and Build Provenance | Model and build provenance is the configuration that determines which model, fine-tune, system prompt, instruction data, build artifact, and distribution pipeline produce the synthetic subject. It includes whether those components are verified, signed, versioned, reproducible, and traceable to an approved source.
This configuration creates an elevated exposure condition because the synthetic subject’s behavior may be shaped before deployment. Malicious or unsafe behavior may be introduced through model weights, fine-tuning data, system prompts, training data, extension builds, dependency updates, or continuous integration and continuous delivery (CI/CD) pipelines.
The primary risk is latent or supply-chain-introduced behavior. A model, fine-tune, or build artifact may behave normally under most conditions, but change behavior when a trigger, date, keyword, context, or task appears. A compromised build or update pipeline may also inject instructions, code, or configuration that changes the synthetic subject’s behavior after review.
Investigators should review model provenance, fine-tune records, training and instruction data, system prompt versions, build artifacts, signatures, software bills of materials, CI/CD logs, deployment history, and runtime version records. Particular attention should be given to unapproved model changes, unsigned artifacts, unexplained behavior shifts, injected prompts, over-scoped CI/CD tokens, and actions that cannot be tied to a known model or prompt version.
Investigative RelevanceModel and build provenance is relevant because configuration begins before runtime. The synthetic subject may appear to follow its deployed directive, while the effective behavior was shaped by an earlier model, data, prompt, or supply-chain change.
This section is especially relevant where synthetic subjects rely on fine-tuned models, third-party model weights, vendor extensions, local models, custom system prompts, model updates, CI/CD-built agents, or distributed application artifacts. |
| CF012 | Vendor-Embedded AI | Vendor-embedded AI is a synthetic subject delivered through a third-party platform rather than built and operated entirely in-house. This may include an assistant built into a Software as a Service (SaaS) application, a vendor-hosted agent platform, or a third-party artificial intelligence capability connected to marketplace tools, plugins, connectors, or external services.
This deployment pattern creates an elevated exposure condition because the organization grants access to its own data and workflows, while key operating components remain outside its direct control. The model, prompts, tool wiring, safety controls, update channel, logging, and connected suppliers may be managed by the vendor or its ecosystem.
The primary risk is rented trust. The synthetic subject may run inside the organization’s tenant with access to customer records, tickets, files, messages, workflows, or connected systems, but its behavior may depend on vendor-managed logic or third-party tools. A silent vendor update, compromised connector, weak marketplace control, or unsafe default configuration may alter how the synthetic subject behaves without the organization fully observing the change.
A related risk is downstream model-provider exposure. The vendor may use a foundation model provider, inference platform, embedding service, or agent runtime to deliver the artificial intelligence capability. Organizational data, prompts, retrieved context, outputs, telemetry, or tool-call metadata may therefore pass beyond the SaaS vendor to backend services the organization cannot directly inspect or control.
A further risk is externally shaped invocation. Customer fields, partner messages, inbound emails, support tickets, uploaded documents, or web forms may carry instructions that later influence the vendor-embedded synthetic subject. An external actor may therefore steer activity inside the organization’s trusted SaaS environment without directly authenticating to that environment.
Investigators should review the vendor’s AI feature scope, tenant permissions, service identities, connector inventory, marketplace tools, update history, prompt and response logs, tool-call records, data access logs, and available vendor audit trails. Particular attention should be given to vendor-managed changes, third-party connectors, downstream model-provider exposure, externally supplied records, allow-listed external destinations, and cases where the organization cannot determine why the synthetic subject acted.
Investigative RelevanceVendor-embedded AI is relevant because the organization may rely on a synthetic subject it does not fully control. The system may appear to operate as a native part of a trusted business platform, while the directive, model behavior, tool routing, data processing path, or update channel remains dependent on the vendor, foundation model providers, and connected suppliers.
This section is especially relevant where artificial intelligence features operate inside customer relationship management platforms, ticketing systems, productivity suites, document platforms, email systems, support tools, finance systems, or other Software as a Service environments with access to organizational data and workflows. |
| CF002.001 | Unapproved Connected Tool | Unapproved connected tool occurs when a synthetic subject is configured to use a plugin, connector, function, application programming interface, or Model Context Protocol (MCP) server that has not been reviewed, approved, or inventoried.
This configuration creates an elevated exposure condition because an unapproved tool may introduce unknown permissions, data access, runtime behavior, supplier dependencies, or outbound destinations. The synthetic subject may treat the tool as trusted even though the organization has not assessed its provenance or purpose.
The primary risk is uncontrolled extension of capability. A single unapproved tool may allow the synthetic subject to retrieve data, send messages, call external services, or perform actions outside the intended design.
Investigators should review connected tool inventories, MCP server lists, configuration files, package manifests, marketplace installs, connector approvals, and tool-call logs. Particular attention should be given to tools installed outside approved channels or tools that do not appear in asset or vendor inventories.
Investigative RelevanceUnapproved connected tools are relevant because they expand what a synthetic subject can do without governance visibility. This sub-section is especially relevant where teams can install marketplace tools, local MCP servers, plugins, or connectors without central approval. |
| CF002.002 | Changed Tool Definition | Changed tool definition occurs when a connected tool’s name, description, schema, parameters, permissions, or server definition changes after approval. This may occur through a vendor update, package update, configuration change, or replacement Model Context Protocol (MCP) server.
This configuration creates an elevated exposure condition because synthetic subjects rely on tool definitions to decide when and how to use tools. A changed description or schema may alter tool selection, input handling, output routing, or the apparent purpose of the tool.
The primary risk is post-approval behavior drift. A tool approved for one purpose may later behave differently, request different inputs, expose new actions, or direct the synthetic subject toward unsafe use without a full review.
Investigators should review tool definition history, package versions, MCP server manifests, approval records, schema changes, descriptions, parameters, and tool-call patterns before and after the change. Particular attention should be given to changed descriptions, added parameters, new external destinations, and tools that changed without re-approval.
Investigative RelevanceChanged tool definitions are relevant because tool behavior can shift after the organization has accepted the integration. This sub-section is especially relevant where connected tools update automatically, use remote schemas, or depend on vendor-managed metadata. |
| CF002.004 | Impersonating Tool Package | Impersonating tool package occurs when a connected tool, plugin, package, or Model Context Protocol (MCP) server presents itself as a trusted integration while being operated, published, or modified by an unauthorized or malicious supplier.
This configuration creates an elevated exposure condition because the synthetic subject and the deploying team may trust the tool based on its name, description, repository appearance, or apparent relationship to a legitimate service.
The primary risk is supplier impersonation. An impersonating package may receive sensitive inputs, send messages, call APIs, or exfiltrate data while appearing to be an approved or official tool.
Investigators should review package names, publishers, repository ownership, signatures, download sources, version history, dependency changes, marketplace records, and tool-call logs. Particular attention should be given to lookalike names, recently transferred ownership, sudden behavior changes, and tools that mimic official vendor integrations.
Investigative RelevanceImpersonating tool packages are relevant because synthetic subjects may use third-party tools with high trust and high permissions. This sub-section is especially relevant where agents rely on public package registries, marketplace listings, MCP servers, or community-maintained connectors. |
| CF012.001 | SaaS-Embedded Vendor Assistant | A SaaS-embedded vendor assistant is an artificial intelligence assistant shipped inside a vendor-managed Software as a Service (SaaS) application. This may include customer relationship management, helpdesk, ticketing, email, productivity, finance, or document platforms where the assistant can read, summarize, update, route, or act on tenant records.
This deployment pattern creates an elevated exposure condition because the assistant operates as a trusted feature of the SaaS platform. The deploying organization may configure the feature, but does not fully control the model, prompts, guardrails, update channel, backend processing path, or vendor-managed integrations.
The primary risk is externally shaped invocation inside a trusted application. Any externally writable field that enters the SaaS tenant, such as a lead form, support ticket, inbound email, customer message, uploaded document, or partner record, may later influence the assistant when an employee asks it to process that record. The resulting action appears as trusted platform behavior, even where the effective instruction originated from an outsider.
A related risk is platform-level authority. The assistant may read or write tenant records using the SaaS application’s standing permissions, service identities, or internal access model. This can make unauthorized summaries, record changes, outbound messages, or workflow actions appear to be ordinary product activity.
Investigators should review the assistant’s feature scope, tenant permissions, vendor controls, externally writable fields, record history, prompt and response logs, tool-call records, update history, and available SaaS audit trails. Particular attention should be given to external content that preceded the assistant action, records modified by the assistant, and cases where the organization cannot inspect the prompt, model behavior, or guardrail decision.
Investigative RelevanceSaaS-embedded vendor assistants are relevant because they operate inside trusted business applications while remaining partly outside the deploying organization’s control. The organization may see the output as a native platform action, while the directive, model behavior, guardrails, or backend processing are controlled by the vendor. |
| CF012.002 | Third-Party Tool Agent Platform | A third-party tool agent platform is a standalone artificial intelligence agent platform connected to external tools through marketplaces, plugins, connectors, or Model Context Protocol (MCP) servers. MCP servers provide a standardized way for agents to access external tools, data sources, and actions. Each connected tool may allow the agent to read data or perform actions such as sending email, calling application programming interfaces (APIs), updating records, or retrieving files.
This deployment pattern creates an elevated exposure condition because every added tool becomes another supplier in the trust chain. The organization may approve the agent platform, but each marketplace tool, connector, or MCP server introduces its own permissions, update path, dependencies, and operator.
The primary risk is tool-mediated compromise. A connected tool may become malicious through a silent update, impersonating package, compromised dependency, weak marketplace review, or supplier compromise. Once connected, the tool may act through the agent’s authority and access organizational data or systems on the agent’s behalf.
A related risk is attribution failure. Harmful activity may appear to originate from the agent platform, while the actual cause sits inside a specific tool, connector, package, or MCP server. If logs do not preserve tool-level provenance, investigators may be unable to determine which supplier component read the data, sent the message, called the API, or caused the adverse outcome.
Investigators should review the agent platform’s directive, installed tools, marketplace sources, MCP servers, connector permissions, package provenance, update history, tool-call logs, outbound destinations, and vendor audit records. Particular attention should be given to newly added tools, recently updated packages, impersonating names, unexpected email or API actions, and cases where activity cannot be attributed to a specific connected tool.
Investigative RelevanceThird-party tool agent platforms are relevant because they extend organizational trust beyond the agent vendor to every connected tool in the platform ecosystem. A tool added for convenience may become a separate operational actor with access to sensitive data and action capability.
This sub-section is especially relevant where a standalone agent platform can install marketplace tools, connect to MCP servers, send mail, call APIs, read documents, access customer data, update SaaS records, or execute workflow actions through third-party integrations. |