AI-500 · CHEAT SHEET

Microsoft Certified: Multi-Agent AI Solutions Expert

Key concepts grouped by official exam weight.

Download as PDF (free, no sign-up)

1. Architect multi-agent solutions

15–20%

Design logical architecture for multi-agent solutions

Before creating agents, pick the least complex level: a direct model call when a prompt is enough, a single agent with tools as the usual default, and multiple agents only for security or compliance boundaries, multiple teams, planned growth, or when testing shows one agent is not enough. Fixed steps are modelled as deterministic workflows.
A multi-agent workflow combines patterns (sequential, concurrent, group chat, handoff and magentic) per stage. Control loops need acceptance criteria, an iteration cap and a fallback such as escalating to a person. Human-in-the-loop steps are defined as approval or feedback, optional or mandatory, and state should be persisted at those points so work resumes without being replayed.
Document each agent’s responsibilities, non-overlapping roles and prohibited actions in an agent charter. Instructions are treated as versioned code with identity and tone, scope and boundaries, tool mandates and citation rules. Autonomy grows gradually from no permitted actions, and boundaries are enforced with deterministic controls, not only with the system message.
Each agent should have its own identity with minimal RBAC roles and explicitly allowed tools (allowed_tools, approvals for high-risk operations). In Foundry, the agent identity covers unattended access or different access levels per agent; the project managed identity gives all agents the same access, and OAuth identity passthrough preserves each user’s permissions.
MCP connects each agent to its tools and data: the host creates one client per server, the server exposes tools, resources and prompts, and the transport is stdio locally or Streamable HTTP remotely. A2A connects autonomous agents to each other: they discover one another through the Agent Card at /.well-known/agent-card.json and receive updates by polling, streaming or push notifications to a webhook.

Specify technology components to include in a multi-agent solution

In production, A2A connects agents that cross process, team or organizational boundaries and are discovered through their Agent Card; agent-as-tool is enough inside one process. Foundry toolboxes expose tools and MCP servers through one endpoint with centralized authentication and versioning, and Foundry IQ provides shared knowledge bases that respect user permissions.
Zero Trust for multi-agent systems: each published agent gets its own Entra identity with least-privilege RBAC roles, the token audience matches the target resource, and Conditional Access blocks by default and on risk. Private networking avoids public paths, and Purview and Foundry Control Plane apply DLP policies, auditing and compliance to agent interactions.
Tiered persistence separates session state (AgentSession and history providers, such as Cosmos DB to resume on another instance), shared team state (workflow state with shared scopes and durable checkpoints), and long-term semantic memory (Foundry memory stores isolated per user scope, with TTL and item deletion).
Prompt agents have no compute to manage; hosted agents run your container in per-session sandboxes with managed scaling, identity and endpoint, and their cost depends on per-session CPU and memory. The Durable Extension on Azure Functions adds serverless durable orchestration, and self-hosting (for example, on AKS) gives full control of networking and runtime.
Multi-agent observability builds on OpenTelemetry with the GenAI conventions: nested invoke_agent spans, execute_tool spans with arguments and results, and W3C trace context propagated to MCP servers. Agent Framework exports traces to Application Insights; sensitive data is captured only in development, and checkpoints make complete workflow runs replayable.

2. Develop multi-agent solutions in Azure

30–35%

Design and implement advanced prompt engineering strategies in multi-agent solutions

Advanced prompt engineering combines few-shot examples as user-assistant pairs, dynamic context injection with context providers before each run, and defensive guidelines: giving the model an out, repeating instructions, and Prompt Shields for attacks hidden in tool content. The lifecycle uses immutable agent versions, evaluation before publishing, and monitoring.
In Agent Framework, handoff passes the whole conversation to another agent, while agent-as-tool keeps control in the primary agent. AIContextProvider classes inject instructions, messages or tools before each run and keep per-session state in AgentSession. A system message steers but does not guarantee behavior: define scope and fallback behavior.
Fine-tuning in Microsoft Foundry changes model weights: SFT with example responses, DPO with preferred/rejected pairs and RFT with a grader. Always keep training, validation and test data separate and pick the checkpoint using held-out tasks. It does not replace retrieval for current data; retrain when task data changes or the base model is retired.

Design and implement agent memory, context management, and knowledge integration

In Agent Framework, history can be local (in the session) or service-managed; only local history can be compacted and routed to another provider. Compaction groups messages into atomic units, keeps system messages and chains strategies from gentlest to most aggressive. Use AdditionalProperties for per-run context and StateBag for conversation state.
Memory in Foundry Agent Service is long-term: it extracts, consolidates and retrieves user profile, chat summary and procedural memories. Scope isolates memories per user ({{$userId}} with the x-memory-user-id header), TTL and per-scope deletion cover the lifecycle, and user_profile_details excludes sensitive data. Short-term memory lives in the framework session.
Multi-agent RAG on Azure AI Search depends on good chunking: fixed size with overlap, by section or semantic, and chunking even when a single vector represents content poorly. Use the same embedding model for indexing and querying and vectorize only meaningful fields. Hybrid search with RRF and semantic ranker (@search.rerankerScore) improve precision.
Foundry IQ turns Azure AI Search into reusable knowledge bases: each base groups indexed or remote sources (SharePoint, web, MCP servers), and agentic retrieval splits the question into semantically reranked subqueries. Agents consume it as an MCP tool (knowledge_base_retrieve) using the project managed identity and the Search Index Data Reader role.

Build and integrate tool ecosystems

With function calling, the model proposes the call and the application executes it and returns the result with its tool_call_id. tool_choice lets the model decide (auto), blocks calls (none) or forces a specific function. In Agent Framework, progressive exposure (ctx.add_tools) and function middleware control which tools are visible and when they run.
An MCP server exposes tools that clients discover with tools/list and invoke with tools/call. On Azure you can build one with the Functions MCP extension (McpToolTrigger and the mcp_extension key), host an SDK server on Functions or Container Apps, expose Standard Logic Apps workflows, or turn an API Management REST API into MCP tools governed by policies.
MCP separates protocol errors (JSON-RPC errors such as an unknown tool) from tool execution errors, which go in the result with isError set to true so the model can self-correct. Retry only idempotent operations on transient faults, use Circuit Breaker to fail fast during outages, and degrade gracefully without hiding errors.
Treat remote tool descriptions and results as untrusted and validate them before passing them to the model; when a tool declares an outputSchema, the client should validate structuredContent. Enable Prompt Shields document attack detection for tool responses, validate output between agents, and measure with evaluators such as Tool Call Success or Tool Output Utilization.

Implement multi-agent orchestration

Sequential chains agents in a fixed order; concurrent (fan-out/fan-in) runs agents in parallel and needs an aggregation strategy; group chat uses a central manager that chooses who speaks; handoff transfers full control between peer agents; and in orchestrator-subagent the main agent calls others as tools and keeps control.
Approvals can be scoped to sensitive tools (approval_mode or ApprovalRequiredAIFunction in Agent Framework, require_approval in Foundry MCP) or pause after an agent response or a Magentic plan. Persist state with checkpoints so you can resume without replaying work, and in LangChain use reject, not respond, to deny side-effecting actions.
Azure OpenAI prompt caching reuses identical prefixes (messages and tool definitions), doesn't change the response and is measured with cached_tokens; put stable content first. API Management semantic caching returns responses by similarity: tune score-threshold (lower is stricter), separate users with vary-by and add rate limiting after the lookup.
Control concurrency with fan-out and fan-in edges in Agent Framework workflows, round, stall and reset limits in Magentic, and per-key llm-token-limit in API Management so one orchestrator can't exhaust shared quota. Use the Batch API for large asynchronous workloads with separate quota, and scale agent services in Container Apps with KEDA rules such as concurrent HTTP requests.
In A2A, the Agent Card (at /.well-known/agent-card.json) describes skills, endpoint and securitySchemes; credentials are obtained out of band, travel in headers, and transport must be encrypted. In MCP, the server validates that the token was issued for it and doesn't pass other tokens through. In Foundry, keep credentials in project connections and restrict tools with allowed_tools.

3. Evaluate, optimize, and monitor multi-agent solutions

20–25%

Design and implement evaluation and validation strategies for multi-agent solutions

In Foundry, human evaluation uses templates with thumbs up/down, slider, multiple choice or free-form text questions; only one template can be active per agent. Reviewers rate each response from the preview web app, and results are stored in Application Insights under its retention policy, so download them as CSV to keep them long term.
Foundry provides process evaluators for tools (Tool Call Accuracy, Tool Selection, Tool Input Accuracy, Tool Output Utilization, Tool Call Success), RAG evaluators for knowledge (Retrieval, Document Retrieval, Groundedness, Relevance, Response Completeness) and Task Adherence for instructions. To measure memory across a dialogue, use evaluation_level=conversation with messages-format data.

Optimize prompt and model performance for multi-agent solutions

Concurrent orchestration shortens duration when agents work independently on the same input, but avoid it when they need cumulative context or quota prevents it. In Azure OpenAI the request limit is evaluated over short windows, the token estimate includes max_tokens, and failed requests still count; use retries with exponential backoff and random jitter.
In Agent Framework, compaction of in-memory history combines strategies from gentle to aggressive: collapsing tool results, LLM summarization, sliding window and truncation. A sliding window forgets early facts and a generic summary prompt dilutes them; Foundry memory gives continuity across sessions, and hybrid search fixes vector-only retrieval gaps.
Continuous improvement in Foundry combines LLM judges (prompt-based and rubric evaluators), synthetic data (Simple QnA or Simulation seed) and datasets built from production traces. The agent optimizer closes the loop: it evaluates a baseline, generates instruction, tool and model candidates, scores them on the same dataset and promotes the best one.

Design and implement observability and monitoring for multi-agent solutions

The Foundry Agent Monitoring Dashboard reads token usage, latency, run success rate and evaluation scores from Application Insights, with configurable alerts. Continuous evaluation samples live traffic and scheduled evaluation uses fixed datasets to detect drift. OpenTelemetry traces (invoke_agent, plan, execute_tool) correlate failures across agents and tools.
Define measurable SLOs backed by SLIs and calculate each flow's composite SLO as the product of its critical dependencies; an SLA is a contract with a limited scope, not a target to copy. Combine Service Health and Resource Health for platform health with Application Insights standard availability tests, using retries and several locations, for your own endpoints.
Bound autonomous loops with max_iterations and limit automatic tool invocation with max_iterations, max_function_calls and max_duration_seconds; when a limit is hit, the client asks for a final answer. Keep max_tokens as low as the scenario allows, because it counts toward the TPM limit, and shrink context with compaction, prompt caching or semantic caching.
Analyze Foundry spend in Cost Management by resource and meter, filter by the automatic project tag to charge back costs across projects, and use budgets with action groups, because Azure OpenAI has no hard spending limits. Quota is assigned in TPM per region, model and deployment type, and llm-emit-token-metric measures tokens per consumer.
Foundry tracing is off until you connect Application Insights; then it records inputs, outputs, tool calls, tokens and latency server-side using OpenTelemetry GenAI conventions (invoke_agent, execute_tool, plan). Agent Framework propagates traceparent to client-opened MCP servers. Restrict who can read traces, because they can hold prompts, and use log search alerts on them.

4. Secure, govern, and deploy multi-agent solutions

20–25%

Design and implement security for multi-agent solutions

Use Microsoft Entra ID and least-privilege RBAC: Foundry Agent Consumer for callers that only use agents and Foundry User for builders; API keys bypass RBAC. Published agents get their own identity, which needs its own role assignments. Isolate the network with private endpoints for inbound access and a delegated subnet for the agents' outbound traffic.
In production, authenticate with Microsoft Entra ID (scope https://ai.azure.com/.default) instead of API keys, which can't identify the user. To act for a user, use on-behalf-of, which carries only delegated scopes; for MCP or A2A tools choose a key, the agent identity, the project managed identity, or OAuth identity passthrough when each user must keep their own permissions.
Access Key Vault with managed identities and least-privilege RBAC roles, such as Secrets User to read secrets or Crypto User to use keys. Keys rotate through a rotation policy and are referenced by versionless URIs; secrets rotate through Event Grid and Azure Functions; certificates renew automatically. Customer-managed keys require soft delete and purge protection.
The AI Red Teaming Agent uses PyRIT to simulate attacks, scores each attack-response pair, and reports the attack success rate (ASR). Shift testing left: model selection, development, pre-deployment, and scheduled runs after release. Agentic risks such as data leakage or prohibited actions are tested only in cloud red teaming, ideally in a purple, nonproduction environment.

Design and implement guardrails for multi-agent solutions

In Microsoft Foundry, a guardrail is a collection of controls that define the risk, the intervention point (user input, tool call, tool response, and output) and the action. Tool call and tool response points apply only to agents, and the guardrail assigned to an agent fully overrides the model's guardrail; without an assignment, the agent inherits its deployment's guardrail.
Domain-specific constraints are covered with custom blocklists (exact match or regex, applied to prompts or completions), the per-category personal data filter, Task Adherence to detect tool calls that don't match user intent and, in code, Agent Framework middleware, which can inspect tool arguments and terminate execution.
The AI Red Teaming Agent, built on PyRIT, sends synthetic adversarial queries and attack strategies to models or agents and summarizes results with the attack success rate (ASR). Agentic risks (prohibited actions, sensitive data leakage, task adherence) are tested only in the cloud with synthetic data and mock tools. Foundry evaluations can also generate simulated conversations.

Deploy multi-agent solutions to Azure

DTAP promotes each release through development, test, acceptance and production environments. Blue/green keeps two identical environments that can each handle full load and switches traffic to the new one; canary releases to a small group first and expands in phases with bake times of hours or days. Both are progressive exposure models recommended by the Well-Architected Framework.
When a deployment fails, the rollout stops and you choose between rolling back to the last known good configuration, rolling forward with a fix, or deploying new infrastructure. In Azure Container Apps you roll back by moving traffic weights between labeled revisions. In Azure Pipelines, approvals and checks on environments, such as Query Azure Monitor Alerts, control when each stage proceeds.
Deterministic orchestrator logic is validated with unit tests isolated from the model; integrations with tools and agents with mock dependencies and injected failures. Automated evaluations score responses with another model and agent evaluators such as Tool Call Accuracy or Task Completion, and CI compares agent versions with statistical significance tests to catch regressions.
Infrastructure is deployed only as declarative code (Bicep, ARM or Terraform) through CI/CD pipelines, with one set of files parameterized per environment. The what-if operation shows the predicted changes without applying them. CI runs when a pull request is created, and pipelines authenticate to Azure with Microsoft Entra ID through OpenID Connect.

Practice this certification