Python SDK Starter

Build agents in Python. Production-ready modules for creating, evaluating, and managing AI agents on Azure AI Foundry — including toolbox management, PII masking, agent-to-agent delegation, and MCP authentication.

Modules

Module	Purpose	Entry Point
`agent_client.py`	Create and run agents from YAML manifests	`AgentFactory.from_manifest()`
`eval_runner.py`	Run evaluation suites against agents	`python eval_runner.py --manifest ...`
`toolbox_manager.py`	Manage Foundry Toolbox connections	`ToolboxManager`
`a2a_manager.py`	Agent-to-Agent delegation (A2A v0.3)	`A2AManager`
`pii_masker.py`	PII detection and masking	`PIIMasker`
`mcp_auth_provider.py`	MCP authentication (Entra, OAuth, API key)	`MCPAuthProvider`
`toolbox_endpoint_resolver.py`	Resolve tool endpoints from API Center	`ToolboxEndpointResolver`

Setup

# Install dependencies
pip install -r requirements.txt

# Copy and configure environment
cp .env.template .env
# Edit .env with your Azure AI Foundry connection string

Required Environment Variables

AZURE_AI_PROJECT_CONNECTION_STRING="<your-foundry-project-connection-string>"
# Optional — only needed if not using DefaultAzureCredential
AZURE_TENANT_ID="<tenant-id>"
AZURE_CLIENT_ID="<client-id>"

Usage

Create and Run an Agent

from agent_client import AgentFactory

# Load agent from manifest
factory = AgentFactory.from_manifest(
    "agent-templates/knowledge-agent/agent-manifest.yaml"
)

# Create a conversation thread
thread = factory.create_thread()

# Run the agent
response = factory.run(
    thread_id=thread.id,
    user_message="What is the company PTO policy?"
)
print(response.content)

Evaluate an Agent

# Run built-in evaluators
python eval_runner.py \
  --manifest agent-templates/knowledge-agent/agent-manifest.yaml \
  --eval-pack eval-packs/builtin/builtin-evaluators.yaml \
  --output scorecard.json

# Run with custom dataset
python eval_runner.py \
  --manifest agent-templates/analyst-agent/agent-manifest.yaml \
  --dataset eval-packs/datasets/analyst-test-cases.jsonl

Agent-to-Agent Delegation

from a2a_manager import A2AManager

a2a = A2AManager(project_client=factory.client)

# Delegate a sub-task to another agent
result = await a2a.delegate(
    target_agent="analyst-agent",
    task="Summarize Q3 revenue trends",
    context={"department": "Sales"}
)

PII Masking

from pii_masker import PIIMasker

masker = PIIMasker()
safe_text = masker.mask("Call John at 555-0123 or john@acme.com")
# Output: "Call [NAME] at [PHONE] or [EMAIL]"

Module Details

agent_client.py — AgentFactory

The core module. Reads a YAML agent manifest and creates a fully configured Foundry agent with:

Foundry Guardrails — Content safety, PII detection, prompt attack defense
Toolbox SDK — MCP tools, OpenAPI tools, Azure Functions, SharePoint, Fabric
Memory — User-scoped long-term memory via Foundry Memory service
Telemetry — OpenTelemetry spans + Azure Monitor export
HITL — Human-in-the-loop escalation via Durable Functions

eval_runner.py — Evaluation Runner

Runs evaluation suites using the azure-ai-evaluation SDK:

Built-in evaluators: groundedness, coherence, relevance, fluency, safety
Custom evaluators: business intent, policy compliance
Score normalization across 0–5, 0–10, and 0–100 scales
JSON scorecard output for CI/CD gating

Related Resources

Agent Manifest Spec — YAML manifest reference
Agent Blueprints — Pre-built agent templates
Evaluation Framework — Evaluation guide and CI integration