Telemetry Schema

Overview

All agent telemetry is emitted as OpenTelemetry spans and events through the azure-monitor-opentelemetry exporter to Azure Application Insights. The schema ensures consistent, queryable data across all agents, adapters, and trust extensions — stored in Azure Log Analytics and visualized via pre-built Azure Monitor Workbooks.

Telemetry Pipeline

Layer	Azure Service	Purpose
SDK	OpenTelemetry + azure-monitor-opentelemetry	Span/event generation & export
Ingestion	Azure Application Insights	Custom events, dependencies, traces
Storage	Azure Log Analytics Workspace	Long-term retention, KQL queries
Visualization	Azure Monitor Workbooks	Pre-built agent monitoring dashboards
Alerting	Azure Monitor Action Groups	Threshold-based alerts, remediation triggers

Event Types

Event Type	Description	Emitter
`agent_invocation`	Agent received and processed a request	Agent runtime
`tool_call`	Agent called an MCP tool	Tool server
`policy_evaluation`	Policy engine evaluated a constraint	Policy engine
`eval_result`	Evaluation completed for an agent	Eval runner
`certification_gate`	Agent passed/failed a certification check	Cert pipeline
`decision_record`	Decision recorded in ledger	Decision ledger
`guardrail_trigger`	Content safety filter triggered	Guardrails manager

Common Fields

{
  "event_type": "agent_invocation",
  "timestamp": "2025-01-15T10:30:00.000Z",
  "correlation_id": "uuid-v4",
  "agent_id": "knowledge-agent-v1.0.0",
  "agent_class": "retrieval",
  "environment": "production",
  "tenant_id": "uuid",
  "subscription_id": "uuid"
}

Agent Invocation Event

{
  "event_type": "agent_invocation",
  "duration_ms": 2340,
  "model_deployment": "gpt-4o",
  "input_tokens": 1250,
  "output_tokens": 480,
  "total_tokens": 1730,
  "tools_called": ["azure_ai_search", "file_search"],
  "tool_call_count": 3,
  "status": "success",
  "trust_score": 0.92
}

Tool Call Event

{
  "event_type": "tool_call",
  "tool_name": "create_incident",
  "adapter": "servicenow",
  "duration_ms": 890,
  "status": "success",
  "http_status": 201,
  "retry_count": 0
}

KQL Queries (Azure Log Analytics)

Pre-built KQL queries are available in observability/kql/agent-monitoring-queries.kql for Azure Log Analytics:

Agent invocation latency percentiles
Tool call error rates by adapter
Token consumption trends
Trust score distribution
Policy violation timeline
Guardrail trigger frequency

Azure Monitor Workbook

A pre-built Azure Monitor Workbook is included at observability/appinsights/agent-monitoring-workbook.json. Import it into your Azure Application Insights resource for a comprehensive agent monitoring dashboard covering latency, token consumption, trust scores, policy violations, and guardrail triggers.