Architecture
Enterprise Agent Governance Fabric: The only enterprise governance framework that spans all 7 architectural planes of the Microsoft AI Agent Platform Stack — from infrastructure and MCP fleet governance through execution and observability to the Agent 365 control plane. Every plane is secured by 17 Zero Trust controls that enforce verify-explicitly, least-privilege, and assume-breach at every boundary.
For an at-a-glance visual overview, see the Reference Architecture page with 3 ASCII reference diagrams covering high-level topology, security zones, and data flow.
Architecture Overview
The diagram below shows the 7-layer architecture spanning from the Agent 365 Control Plane through Infrastructure Services to the Business Value Observability Plane — with 17 Zero Trust controls woven across every boundary.
Click any architectural layer to explore its components and Zero Trust controls.
Figure 1 — Azure Foundry Enterprise Kit: 7-layer architecture with 17 Zero Trust controls
Zero Trust Security — The Foundation
Zero Trust by Architectural Layer
| Layer | ZT Controls | What's Enforced | Value |
|---|---|---|---|
| Control Plane (Agent 365) | ZT-01, ZT-09 | Entra Agent Identity, RBAC role assignments, Azure Policy governance initiative | Every agent has a verified identity; no anonymous agents reach production |
| Execution Plane (Foundry + Agent Framework) | ZT-02, ZT-03, ZT-05, ZT-06, ZT-07 | API Gateway JWT validation, AI Gateway content safety, runtime interceptor kill switch, model gateway RBAC, A2A AgenticIdentity auth | Every request is authenticated, every response is filtered, every tool call passes through the interceptor |
| Governance Plane (Observability) | ZT-08, ZT-09, ZT-13, ZT-14 | OTel telemetry with PII masking, Azure Policy enforcement, evaluation dataset protection, continuous anomaly detection | Compliance evidence is tamper-proof; anomalies trigger automatic remediation |
| Trust Plane (Certification) | ZT-04, ZT-10, ZT-11 | Key Vault CMK encryption, Cosmos DB geo-fencing + data residency, memory retention isolation | Audit trails are cryptographically signed; data never leaves approved regions |
| Infrastructure (Shared Services) | ZT-03, ZT-04 | Private endpoints for all PaaS services, VNet isolation with 5 subnets, NSG rules, private DNS zones | No public internet exposure; all traffic stays within the enterprise network boundary |
| MCP Fleet (WS-9) | ZT-07, ZT-12, ZT-15 | MCP server auth (5 methods), transport security (TLS + VNet), tool-level authorization per agent class, APIM rate limiting, supply chain validation | Every MCP tool call is authenticated, authorized, rate-limited, and auditable |
17 Zero Trust Controls
| Control | Scope | Enforcement |
|---|---|---|
| ZT-01 | Identity | Managed Identity hierarchy, Entra Agent Identity, per-agent RBAC, no shared credentials |
| ZT-02 | API Gateway | APIM JWT validation, rate limiting (60 RPM/tool, 200 RPM/agent), IP filtering (RFC 1918), header sanitization |
| ZT-03 | AI Gateway | Content Safety filtering, AI Search private endpoint, prompt injection detection, DLP integration |
| ZT-04 | Secrets & Keys | Key Vault with CMK encryption, certificate rotation, private endpoint, soft-delete protection |
| ZT-05 | Runtime Interceptor | Kill switch with policy triggers, token budget enforcement, anomaly-based halt, Decision Ledger logging |
| ZT-06 | Model Gateway | Model deployment RBAC, endpoint protection, inference rate limiting, model version pinning |
| ZT-07 | Agent Communication | A2A AgenticIdentity auth, MCP gateway authorization, identity propagation, egress control |
| ZT-08 | Telemetry | OTel pipeline with PII masking, field-level redaction, App Insights private endpoint |
| ZT-09 | Azure Policy | Governance initiative with 12+ policy definitions, tag enforcement, resource compliance |
| ZT-10 | Data Boundary | Cosmos DB geo-fencing, data residency enforcement, cross-region replication controls |
| ZT-11 | Memory Governance | Retention policies (operational vs. compliance), user-scoped isolation, memory purge on session end |
| ZT-12 | Toolbox | Tool allowlisting per agent class, version governance, approval tiers (read=auto, write=require) |
| ZT-13 | Evaluation Security | Eval dataset protection, result integrity validation, ground truth access control |
| ZT-14 | Continuous Monitoring | Anomaly detection with configurable thresholds, alert-triggered remediation workflows |
| ZT-15 | MCP Security | Server auth (5 methods), TLS 1.2+ transport, VNet-only ingress, per-agent session limits, supply chain validation |
| ZT-16 | Compliance Automation | Policy engine integration, automated evidence collection, continuous compliance scoring, framework mapping (NIST/EU AI Act/SOC 2) |
| ZT-17 | Data Residency | Sovereign data boundary enforcement, geo-fencing validation, cross-region replication controls, data-residency module with jurisdiction-specific policies |
Zero Trust Request Flow
Watch a request traverse 9 security boundaries — from identity verification through tool execution to the tamper-evident Decision Ledger.
Azure Services in the Request Flow
Each step in the Zero Trust flow is powered by specific Azure services. Click a layer to see the services and their roles.
7-Plane Architecture
The architecture follows a layered governance model with 7 distinct planes. Each plane has clear responsibilities, security boundaries, and integration points. The diagram below maps to these planes top-to-bottom.
Plane 1 — Agent 365: Enterprise Governance Control Plane
Agent 365 (GA May 2025) is Microsoft's enterprise governance control plane for all AI agents. The kit enhances Agent 365 with enterprise-grade controls that Microsoft does not provide natively — cryptographic audit trails, automated certification pipelines, and cross-agent policy topology.
| Component | Purpose | Kit Integration |
|---|---|---|
| Agent Registry (4 types) | Central catalog of all AI agents across the organization | agent365_publisher.py auto-registers agents with governance metadata |
| Work IQ MCP Catalog | Calendar, Mail, Teams signals as MCP tools | a365-mcp-permissions.yaml configures Work IQ MCP access per agent class |
| A365 SDK | Runtime observability + notifications | a365-bootstrap.sh initializes A365 config + observability bridge |
| Cross-Cloud Registry | AWS/GCP agent sync for multi-cloud governance | Planned (WS-1.7) — config schema defined, implementation in progress |
Right-side endpoints: Foundry Agent Applications (ARM-native) → Agent Publishing & Lifecycle → Versioned Deployments / RBAC → Responses API → Activity Log → A2A Endpoint
Plane 2 — Agent Execution Plane
The runtime layer where AI agents execute. Built on Azure AI Foundry Agent Service and Microsoft Agent Framework with the Foundry Toolbox, Agent Memory, Guardrails, and A2A Protocol.
| Component | Key Features | Kit Assets |
|---|---|---|
| Azure AI Foundry Agent Service (Responses API) | Entra RBAC, private endpoints, model deployments | agent_client.py, foundry-project.bicep |
| Microsoft Agent Framework | Graph Workflows, Session State, Middleware, MCP Clients | Python + TypeScript SDK starters with OTel middleware |
| Foundry Toolbox (Preview) | MCP-compatible, versioned registry, Entra auth | toolbox_manager.py, toolbox_lifecycle_manager.py, 5 YAML configs |
| Agent Memory (Preview) | Chat Summary Store, User Profile Store, user-scoped isolation | memory-store-definitions.yaml (20-msg trigger, retention policies) |
| Foundry Guardrails | Content Safety, PII Detection, Prompt Attacks, Task Adherence | guardrails_manager.py, guardrails.bicep, content-safety.bicep |
| A2A Protocol (v0.3) | Agent Cards, Skill Discovery, AgenticIdentity Auth, Delegation | a2a_manager.py, A2A tool template, test suite |
Security: ZT-02 (API Gateway), ZT-03 (AI Gateway), ZT-05 (Runtime Interceptor), ZT-06 (Model Gateway), ZT-07 (A2A Auth)
Plane 3 — Governance & Observability Plane
Fleet-wide monitoring, compliance enforcement, and security integration. OTel traces, metrics, and logs flow from the execution plane into this layer.
| Component | Purpose | Kit Assets |
|---|---|---|
| App Insights + Dashboards | Distributed tracing, custom dashboards, alert rules | app-insights.bicep, 2 Azure Monitor workbooks |
| Log Analytics + KQL | Fleet-wide query and analysis | 6 KQL packs: agent monitoring, tracing, MCP fleet, gateway analytics, multi-agent correlation, tool call analytics |
| Foundry Control Plane | Fleet monitoring, compliance enforcement, bulk remediation, Defender/Purview/Entra integration | 14 remediation workflows, kill switch queries, observability bridge |
| Microsoft Defender for AI Agents | Threat detection, security alerts, incident response | Planned (WS-5.1) — Defender integration for AI agent threat surface |
| Azure Purview | Data catalog, DLP, sensitivity labels | Data boundary policy, analyst toolbox Purview search, DLP provider config |
Security: ZT-08 (OTel + PII masking), ZT-09 (Azure Policy), ZT-13 (Eval security), ZT-14 (Continuous monitoring)
Plane 4 — Trust & Certification Plane (Proprietary)
proprietary governance overlay that converts runtime evidence into auditable, certifiable artefacts. Evidence streams flow from the observability plane into the trust plane for compliance processing.
| Component | Purpose | Compliance Mapping |
|---|---|---|
| Decision Narrative Ledger | Hash-chained audit trail in Cosmos DB — every agent decision recorded with SHA-256 chain | NIST AI RMF: Manage |
| Agent Scorecard Compiler | PDF release dossier combining eval results, policy compliance, trust metrics | EU AI Act: Technical Documentation |
| Certification Pipeline | Bronze → Silver → Gold automated promotion with scorecard-based gate logic | Internal: Release Governance |
| Cross-Agent Policy Graph | Inter-agent permission topology with over-privilege detection and cascade violation analysis | Responsible AI: Transparency |
| Agent App Publisher | ARM-native agent publishing with governance metadata, container registry push | Azure CAF: Resource Management |
| Guardrails Manager | Enterprise presets for Foundry Guardrails content safety configuration | Content Safety: Risk Management |
| PII Masker | Telemetry + log sanitizer — Presidio-based detection with field-level redaction | GDPR / Data Protection |
Security: ZT-04 (Key Vault CMK signing), ZT-10 (Data boundary geo-fencing), ZT-11 (Memory retention isolation)
Plane 5 — Shared Infrastructure Services
Foundational Azure PaaS services shared across all planes. Each service is provisioned with private endpoints, managed identities, and encryption at rest.
| Service | Purpose | Security |
|---|---|---|
| Azure AI Search | RAG grounding with semantic ranker, configurable replicas and partitions | Private endpoint, managed identity access, ZT-03 |
| Cosmos DB | Decision Ledger storage + Agent Memory persistence (serverless) | Geo-fencing, CMK encryption, private endpoint, ZT-04/10 |
| Key Vault | Secrets management + MCP auth credential rotation + CMK keys | Soft-delete, purge protection, private endpoint, ZT-04 |
| Entra ID | RBAC + 3 User-Assigned Managed Identities (agent, deploy, monitor) | Entra Agent Identity blueprints, per-agent RBAC, ZT-01 |
Plane 6 — MCP Fleet Governance Plane (WS-9)
End-to-end lifecycle management for MCP servers — the only enterprise governance framework that treats MCP infrastructure as a governed fleet with the same rigor applied to agents themselves.
| Component | Purpose | Kit Asset |
|---|---|---|
| Toolbox Lifecycle Manager | Version create → test → promote → deprecate → cleanup with approval gates | trust-extensions/toolbox_lifecycle_manager.py |
| Private Tool Catalog | Azure API Center-backed registry for MCP server discovery and governance | infra/bicep/modules/api-center.bicep |
| MCP Auth Provider | 5 Foundry MCP auth methods with token caching and Key Vault rotation | sdk-starters/python/mcp_auth_provider.py |
| MCP Server Hosting | Azure Functions (event-driven) + Container Apps (stateful/VNet) templates | adapters/hosting/azure-functions/, adapters/hosting/container-apps/ |
| Toolbox Endpoint Resolver | Consumer vs. developer endpoint routing with APIM gateway rewriting | sdk-starters/python/toolbox_endpoint_resolver.py |
| MCP Fleet Dashboard | 12 KQL queries: fleet health, tool usage, rate limiting, error classification, latency heatmap | observability/kql/mcp-fleet-dashboard.kql |
Security: ZT-07 (MCP gateway auth), ZT-12 (Toolbox governance), ZT-15 (MCP security — auth, transport, tool-level authz, supply chain)
Plane 7 — Business Value Observability Plane (AI Value Realization)
The control plane for AI value realization — answers “is the agent delivering measurable business value?” through per-interaction ROI computation, business scorecards, drift detection, and automated remediation.
| Component | Purpose | Kit Asset |
|---|---|---|
| Business Value Schema | Canonical schema for value metrics, scorecard templates, drift rules, maturity model | observability/schema/business-value-schema.json |
| ROI Computation Engine | Per-interaction cost_saved, time_saved, efficiency_ratio from manifest baselines | sdk-starters/python/business_value_tracker.py |
| Scorecard Compiler | 5 templates (service, sales, analyst, compliance, workflow) with A–F grading | observability/schema/business-value-schema.json |
| Drift Detection | 8 rules monitoring containment, cost, satisfaction, ROI, and adoption trends | observability/remediation-workflows.yaml |
| Executive Dashboard | Azure Monitor Workbook with 15+ tiles for business stakeholders | dashboards/business-value-dashboard.json |
| KQL Query Library | 16 production-ready queries for value analytics in App Insights / Log Analytics | observability/kql/business-value-queries.kql |
Data flow: Governance Plane (OTel spans) → Value Plane (ROI + scorecards) → Trust Plane (certification gates)
Cross-Cutting Components
These components span multiple planes and are not confined to a single architectural layer:
7 Enterprise MCP Adapters
| Adapter | System | Auth |
|---|---|---|
| ServiceNow | ITSM — incidents, changes, knowledge | OAuth 2.0 + Redis cache |
| SAP S/4HANA | ERP — purchase orders, materials, goods receipt | OAuth 2.0 client credentials |
| Salesforce | CRM — records, opportunities, contacts | OAuth 2.0 JWT bearer |
| Microsoft Graph | M365 — users, mail, Teams, calendar | Entra Managed Identity |
| Workday | HCM — employee records, org structure | OAuth 2.0 + SAML |
| Internal API | Custom REST APIs via APIM gateway | Managed Identity / API key |
| MCP Server Hosting | Azure Functions + Container Apps templates | Managed Identity / Entra Agent Identity |
6 Agent Templates
| Blueprint | Use Case | Toolbox Approval |
|---|---|---|
| Knowledge Agent | RAG-based Q&A, document search, knowledge synthesis | Auto |
| Operations Agent | Incident management, monitoring, system health | Conditional |
| Policy Agent | Compliance checking, policy evaluation, audit | Require |
| Workflow Agent | Cross-system orchestration, approvals, data sync | Require |
| Analyst Agent | Data analysis, report generation, trend detection | Auto |
Evaluation Harness
- Built-in Evaluators — Accuracy, groundedness, relevance, coherence, fluency via
azure-ai-evaluation - Custom Evaluators — Domain-specific evaluation plugins with YAML configuration
- CI/CD Integration — GitHub Actions + Azure DevOps pipelines with release gates (pre_release, canary, regression)
- Continuous Evaluation — Scheduled evaluation with sampling and cron triggers (
continuous-eval.bicep)
Infrastructure Modules (28 Bicep)
Subscription-scoped Bicep deployment with 19 composable modules following Azure CAF Landing Zone patterns. Each module maps to one or more Zero Trust controls:
| # | Module | Resources | ZT |
|---|---|---|---|
| 1 | managed-identities.bicep | 3 User-Assigned MI + RBAC assignments | ZT-01 |
| 2 | network.bicep | VNet, 5 subnets (incl. MCP), NSGs, 6 Private DNS Zones | ZT-03/15 |
| 3 | key-vault.bicep | Key Vault + CMK encryption + private endpoint | ZT-04 |
| 4 | app-insights.bicep | Application Insights + Log Analytics Workspace | ZT-08 |
| 5 | ai-search.bicep | Azure AI Search with semantic ranker | ZT-03 |
| 6 | cosmos-db.bicep | Cosmos DB (Decision Ledger + Memory databases) | ZT-04 |
| 7 | foundry-project.bicep | AI Foundry Hub + Project + 3 model deployments | ZT-01 |
| 8 | agent-application.bicep | Foundry Agent Applications (ARM resource) | ZT-04 |
| 9 | guardrails.bicep | Foundry Guardrails content safety configuration | ZT-04 |
| 10 | azure-policy.bicep | Azure Policy governance initiative | ZT-09 |
| 11 | entra-agent-identity.bicep | Entra Agent Identity Blueprint | ZT-01 |
| 12 | api-management.bicep | APIM AI Gateway (JWT, token limits, caching) | ZT-02/06/07 |
| 13 | content-safety.bicep | Azure AI Content Safety resource | ZT-03 |
| 14 | runtime-interceptor.bicep | Runtime Interceptor Container App + kill switch | ZT-05 |
| 15 | toolbox.bicep | Foundry Toolbox governance (tool allowlisting) | ZT-12 |
| 16 | continuous-eval.bicep | Continuous Evaluation pipeline scheduling | ZT-14 |
| 17 | api-center.bicep | API Center — Private MCP Tool Catalog | ZT-12/15 |
| 18 | function-mcp.bicep | Azure Functions MCP server hosting | ZT-15 |
| 19 | container-app-mcp.bicep | Container Apps MCP server hosting | ZT-15 |
Data Flow
- User or system triggers agent via Azure AI Foundry Agent API (Responses API)
- Runtime Interceptor (ZT-05) validates identity, policy, and content safety before execution
- Agent selects tools from Foundry Toolbox (filtered by per-agent-class allowlist policy)
- MCP adapter calls enterprise system (ServiceNow, SAP, etc.) via APIM AI Gateway (ZT-02) with rate limiting
- Response passes through Foundry Guardrails (content safety + PII masking + prompt attack defense)
- Decision is recorded in the Decision Ledger (Cosmos DB) with SHA-256 hash chain
- Telemetry emitted to App Insights via OTel with PII masking (ZT-08)
- Agent registered in Agent 365 for cross-platform governance and fleet observability
- Continuous Evaluation runs scheduled assessments; scorecard compiler updates trust score
- Certification pipeline evaluates gate criteria — Bronze → Silver → Gold promotion
9 Innovation Differentiators
proprietary innovations that go beyond what Azure provides natively:
- Unified Governance — Single fabric across Foundry, Copilot Studio, Agent 365, third-party, and cross-cloud agents
- Cryptographic Decision Ledger — Tamper-evident, hash-chained audit trail (NIST/EU AI Act compliant)
- Cross-Agent Policy Graph — Compliance topology with cascade violation detection and over-privilege analysis
- Autonomous Certification Pipeline — Bronze/Silver/Gold gates for automated environment promotion
- Kill Switch with Policy Triggers — Runtime interceptor halts agents on token budget, policy violation, or anomaly
- Three-IQ Convergence — Foundry IQ + Work IQ + Fabric IQ intelligence integration
- Dual-Mode Memory Governance — Operational vs. compliance memory with separate retention policies
- Production-Readiness Scorecard — PDF release dossiers as automated release gates
- Enterprise MCP Fleet Governance — End-to-end lifecycle: versioning, auth, hosting, catalog, observability
Extension Framework (Layer 2)
The kit's extension framework provides modular governance capabilities. Every extension follows a common contract: ExtensionBase subclass, validate_config(),health_check(), zero hardcoded secrets (DefaultAzureCredential), and OTel spans under afek.extensions.<name>.
| Extension | Purpose | Standards |
|---|---|---|
| Compliance Mapper | Framework mapping, gap analysis, evidence collection | NIST AI RMF, EU AI Act, SOC 2 |
| Data Residency | Sovereignty validation, boundary enforcement | GDPR, PIPEDA, OSFI B-13 |
| Drift Detection | Model output drift analysis, statistical tests | NIST MEASURE 2.5, EU AI Act Art.9 |
| Incident Response | Automated incident triage, escalation, remediation | NIST MANAGE 4.1, Azure WAF OE:08 |
| Explainability | Decision audit trails, fairness analysis | EU AI Act Art.13, NIST MAP 2.3 |
| Third-Party Risk | Vendor assessment, dependency scanning | SOC 2 CC9.2, OSFI B-10 |
| DR Failover | RTO/RPO definitions, recovery playbooks | Azure WAF RE:09 |
| Reporting | Compliance reports, evidence packaging | SOC 2, ISO 27001 |
| Sovereign Keys | Key rotation automation, sovereignty policies | NIST SP 800-57 |
| Confidential Compute | TEE attestation, confidential agent deployment | Azure CC, SEV-SNP |
| Fine-Tuning | Safety evaluation, training data audit | NIST MEASURE 2.6 |
IQ Accelerators (Layer 2)
| Accelerator | Purpose |
|---|---|
| Foundry Control Plane | Risk rating, deployment gating, policy enforcement |
| Foundry IQ | Intelligence queries, trend analysis, anomaly detection |
| Work IQ | Task analysis, workload optimization, capacity forecasting |
| Fabric IQ | Data pipeline analysis, lineage checking, quality assessment |
| Red Teaming | Adversarial attack simulation, resilience scoring |
| Lifecycle Engine | Agent lifecycle state machine (dev → staging → prod → retired) |
Industry Packs (Layer 3)
Jurisdiction-specific compliance mappings, guardrails, agent templates, and data residency configurations. Each pack follows a standard structure with a pack-manifest.yaml.
| Pack | Jurisdiction | Frameworks | Status |
|---|---|---|---|
| Banking Canada | Canada | OSFI B-13, PIPEDA, AML/ATF | ✅ Complete |
| Banking EU | European Union | EBA, DORA, GDPR, MiCA | 📋 Planned |
| Banking US | United States | OCC, FFIEC, SOX, GLBA | 📋 Planned |
| Healthcare | Multi-region | HIPAA, FDA 21 CFR Part 11 | 📋 Planned |
| Government | Multi-region | FedRAMP, ITAR, Protected B | 📋 Planned |
Standards Alignment
| Framework | How the Kit Aligns |
|---|---|
| Azure CAF | Deploy → Build → Govern lifecycle; subscription-scoped IaC; Landing Zone patterns |
| Azure WAF | Security (Zero Trust), Reliability (geo-redundancy), Cost (SKU profiles), Ops (IaC/CI) |
| Microsoft Responsible AI v2 | Foundry Guardrails, PII Masker, Decision Ledger, Eval Packs |
| Zero Trust for AI | 17 controls: verify explicitly, least-privilege, assume breach — mapped to every plane |
| NIST AI RMF 1.0 | Map (manifest) → Measure (eval) → Manage (policy) → Govern (ledger) |
| EU AI Act (High-Risk) | Audit trail, human oversight, bias monitoring, technical documentation |
Multi-LOB Platform Engineering
Enterprise-scale isolation: The kit supports multi-LOB (Line of Business) deployment with a hub-spoke architecture. Each LOB gets its own resource group, spoke VNet, identity, Key Vault, Cosmos containers, AI Search index, and Foundry Project — all sharing the centralized platform hub for cost efficiency.
Hub-Spoke Architecture
| Component | Hub (Shared) | Spoke (Per-LOB) |
|---|---|---|
| Resource Group | rg-foundry-platform-prod | rg-foundry-banking-prod |
| Networking | Hub VNet (10.0.0.0/16) + Firewall | Spoke VNet (10.x.0.0/16) + NSGs |
| Identity | Platform UAMI | LOB UAMI + 6 RBAC roles |
| Key Vault | Shared platform KV | LOB-isolated KV + PE |
| Data | Shared Cosmos account | LOB-prefixed containers |
| AI Search | Shared search service | LOB-specific index |
| AI Foundry | Foundry Hub | LOB Foundry Project |
| Cost | Platform budget | LOB-scoped budget + alerts |
Isolation Guarantees
- Network: Spoke-to-spoke traffic blocked by NSG deny-all rules; all cross-spoke routing forced through hub firewall
- Identity: Per-LOB UAMI scoped to its own resource group; Entra group-based RBAC for Owner, Developer, and Viewer roles
- Data: LOB Cosmos containers use RBAC (no shared keys); AI Search indexes are LOB-scoped with data contributor restricted to LOB UAMI
- Secrets: Dedicated Key Vault per LOB with purge protection; cross-LOB access impossible by RBAC design
Self-Service Onboarding
New LOBs are onboarded in under 30 minutes via scripts/onboard_lob.py or the afek lob onboard CLI command. The process automates CIDR allocation, Bicep deployment, RBAC assignment, and LOB registry updates. See the Deployment Guide for detailed steps.