Attack paths for a Microsoft Foundry penetration test

Why this can work

Foundry's connected resources are separate Azure resources with their own network settings. Teams lock down the Foundry account because it is the thing called "AI", then leave the AI Search index or the storage account public because someone needed portal access during the build. The data the agent reads lives in those stores, so the private Foundry endpoint protects very little on its own.

Names are also predictable. Entra ID authentication and private link both need a custom subdomain on the Foundry account, and organisations name resources by a convention, so an attacker who knows one name can guess the rest. Public DNS for a resource with a private endpoint still returns a privatelink CNAME, which confirms the resource exists without saying whether public access is off. The only way to know is to send a request.

Why this can work

Running someone else's models with a stolen key is the most common real world abuse of AI services, because it needs no exploit at all. A Foundry account with local auth disabled has no keys to steal, but the estate still holds other secret material: storage SAS tokens in data science notebooks, AI Search admin keys if local auth was left on, client secrets for service principals that hold Foundry User, and connection secrets held by older hub based workspaces.

A client secret is the dangerous one. It works from anywhere on the internet unless Conditional Access for workload identities restricts where it can sign in from, which needs Workload ID licensing and is often not configured. The private endpoint then becomes the last control: the attacker can get a valid token but has no network path to use it.

Why this can work

Prompt Shields and the content filters are classifiers. They catch known patterns well and reduce the success rate of new ones, but no classifier stops every attempt. Attacks that spread intent across several turns, encode the request (base64, leetspeak, another language), split a payload between a message and an uploaded file, or wrap it in role play are the usual ways past a single turn check.

Partial success is therefore expected, so the useful question for the client is what success costs. That depends on two things the tester can establish: whether the system prompt holds anything sensitive (internal URLs, names of tools and indexes, business rules, credentials), and whether a jailbroken assistant has tools it can then misuse. That is why this path is Medium on its own but feeds AP04 and AP08.

Why this can work

A language model has no reliable way to tell instructions from data once both are in its context. Every document, email, web page and tool response an agent reads is therefore an input channel for whoever wrote it. The attacker never signs in to Foundry, never touches the client's identity system, and the action is carried out under the agent's identity on behalf of a legitimate user, so it looks like normal use.

The defence is a stack: Prompt Shields on documents and tool responses, guardrails on tool calls, a per agent tool allow list, human approval on writes, a narrowly scoped agent identity, and egress filtering. The test establishes which of those layers exist for each agent and how many have to fail before something happens. That is the single most useful result a client can get from an AI pen test, which is why this path carries the most time.

Scoping depends on the read surfaces. Before quoting, list every agent, every grounding source and every tool response it consumes, and mark which can be written by someone outside the trust boundary: suppliers, customers, all staff, or the public web through Grounding with Bing.

Why this can work

This leak happens after every server side control has finished. Foundry returns text; the client turns markdown into an image tag; the browser fetches the image; the query string carries the data. Nothing in Foundry, APIM or the firewall sees the browser's request, because it leaves from the user's device.

Chat platforms add a second channel. When an agent posts into Teams or Slack, link unfurling makes the platform's own servers fetch any URL in the message to build a preview, with no click from the user. The client control is therefore the only control, and it has to cover every surface the agent posts to.

Why this can work

AI Search does not know who the end user is. Security trimming works only if the application or agent adds a filter on an access control field, built from the signed in user's groups, to every query. If one tool omits the filter, if the filter is built from parameters the model chooses, or if a second index was added without access control fields, the search identity returns everything it can see, which is usually everything.

Threads and files have the same weakness at the API layer. If the client application accepts a conversation or file ID from the browser and passes it on without checking who owns it, any user can read any thread by changing an ID. The platform stores the data correctly; the application decides who sees it.

Why this can work

AP06 is about what an index returns. This path is about the systems behind the agent's tools: HR, ticketing, CRM, finance. When the agent calls them with its own identity, the downstream system checks the agent's rights, not the user's. To serve every user, agents are usually granted the union of every user's needs, so each user inherits everything the agent can do.

The fix is the on behalf of user flow, or an explicit authorisation check in the tool, but both are extra work and often skipped. The test needs white box access to the agent's tool definitions and role assignments to choose targets well.

Why this can work

Approval is configured tool by tool. Teams put approval on the obvious tool ("send email") and miss the one that matters ("update claim status"). Where approval is implemented in the application rather than the platform, it can sometimes be satisfied by text: an injected line saying "the user has already approved this" is just more context to the model.

Loops come from instructions such as "keep checking until it changes", with no maximum steps or run timeout. Spend limits are the backstop for both, and they are worth verifying because a limit set in the wrong product or keyed on the wrong counter does nothing.

Why this can work

In the reference design the agent runtime is injected into the client's own subnet, so its requests come from inside the VNet with a line of sight to private endpoints and, through ExpressRoute, possibly to on premises systems. Any tool that fetches a URL or host chosen by the model, such as a "read web page" MCP tool or an OpenAPI tool with a URL parameter, is a server side request forgery primitive whose source is the client's network.

MCP adds a supply chain angle. The model reads each tool's description to decide how to use it, so a description can carry instructions (tool poisoning), and a server can change its description or behaviour after it was reviewed. Allow listing by exact server URL and pinning the tool schema are the controls; both need verifying, not assuming.

Why this can work

Agent subnets need to reach Entra ID and Azure services, and firewall rules for that are often written with wildcards such as *.blob.core.windows.net or broad service tags. Those match every storage account in Azure, including one the attacker created in their own subscription. "Private egress only to Azure" can still mean "egress to the attacker's Azure".

DNS is the other common gap. If the agent subnet can resolve arbitrary external names, data can leave encoded in subdomain lookups to a domain whose name server the attacker controls, even when no HTTP egress is allowed. Code interpreter runs in a Microsoft managed sandbox; whether it has outbound access in the client's configuration is worth checking rather than assuming.

Why this can work

Foundry's roles separate control plane from data plane well on paper, but organisations hand out Contributor or Foundry Owner to unblock a team and never take it back. One account with those rights can detach a filter policy from a deployment, which disables content safety for every app on it in a single change.

Connections are the subtler route. Connections created at the Foundry resource level can be shared across projects, and some connection types still hold secrets that an identity with the right action can list. Azure Policy is meant to be the backstop for public access and keys, so the test also confirms the policy is in Deny, at the right scope, without an exemption covering this resource.

Why this can work

Default data plane roles include fine tuning and dataset upload. Personal data trained into weights cannot be selectively removed, so this is a data protection failure even if no attacker is involved, which is why the priority is Medium: the harm is regulatory and slow rather than an immediate breach.

The approved models policy matches asset IDs by prefix. An entry without a trailing slash approves every model whose name starts the same way, and a publisher entry approves every future model from that publisher. Testing what the policy actually allows is the only way to know.

Why this can work

The pipeline usually holds the highest standing rights in the estate, because it deploys everything. Workload identity federation removes the secret but moves the risk into the federated credential's subject: a subject that matches any branch, any pull request or any environment lets code that never passed review obtain a token.

The self hosted runner inside the spoke is both a privileged identity holder and a network foothold. If it is not ephemeral, tokens and files from earlier runs can remain on disk. If the repository is public and the runner accepts jobs from forks, an outsider can run code inside the client's network.

Why this can work

A private endpoint is reachable from anything routed to its subnet: peered spokes, on premises over ExpressRoute, and other clouds over the interconnect. NSGs on a private endpoint subnet only take effect when network policies are enabled on it, which is off by default on older subnets. Private often means "reachable from the whole corporate network".

Entra ID still applies, which is why the priority is Medium: reach alone is not access. The finding that matters is reach combined with any valid token, for example a compromised user who holds Foundry User for an unrelated project.

Why this matters

A control that blocks silently looks exactly like a control that never ran, and a detection rule that has never fired may not work. Running detection checks alongside the attack paths, as a purple team exercise, turns the test into evidence: each blocked attempt shows up as a log line, each successful one as an alert, and any gap between them is a finding.

Logging itself is a target. An attacker who deletes a diagnostic setting blinds every rule that depends on it, and the only rule that catches that is one that looks for absence.