IBM's Azure Foundry Enterprise Kit

Business Value Observability

The control plane for AI value realization

From “Is the agent running?” to “Is the agent delivering value?”

Traditional AI monitoring tells you whether agents are healthy. Business Value Observability tells you whether they’re worth it. Every agent interaction is measured against real business baselines — cost saved, time saved, containment rate, user satisfaction — and aggregated into executive-ready scorecards with automated drift detection.

Holistic Approach

Three Pillars × Three Levels

Enterprise agentic AI requires three observability pillars — Technical, Business, and Compliance — captured at three levels: Platform, Application, and Agent.

Foundry / Infrastructure

Technical
  • Fleet availability & uptime
  • Model deployment health & throttle rate
  • Aggregate token consumption & cost/region
  • Error rates by agent type
  • Throughput (req/sec)
  • Guardrail fleet trigger rate
Business
  • Portfolio ROI (total cost saved)
  • Fleet value realization rate
  • FTE equivalent displacement
  • Projected annual savings
  • Adoption DAU/WAU/MAU
  • Department penetration
Compliance
  • Fleet policy compliance
  • Certification tier distribution
  • Policy violation trends
  • Audit trail integrity
  • Data boundary compliance
  • Identity posture (MI usage)
Configurable Business Metrics

Three-Tier Business Observability

Business metrics vary per use case. An invoice agent tracks dollar amounts; a banking agent tracks accounts opened. The three-tier model standardizes what's universal while allowing per-use-case configuration.

Tier 1: Universal Business Metrics
Platform standard — always captured, non-configurable
Cost Saved (USD)Time Saved (min)ROI RatioContainment RateTask Completion RateUser SatisfactionQuality ScoreEfficiency RatioFTE EquivalentCycle TimeAdoption DAU/WAU/MAURework RateValue Realization RateSLA AdherenceTokens Per Outcome
Tier 2: Process-Type Metrics
Configurable per use case via agent manifest customKpis
Invoice Processing
  • invoices_processed
  • amount_validated_usd
  • three_way_match_rate
  • exception_rate
  • avg_processing_time_seconds
Account Opening
  • accounts_opened
  • kyc_aml_pass_rate
  • abandonment_rate
  • onboarding_completion_minutes
  • document_verification_accuracy
Claims Processing
  • claims_adjudicated
  • payout_usd
  • fraud_flags_raised
  • first_contact_resolution
  • avg_adjudication_hours
Incident Resolution
  • tickets_resolved
  • mttr_minutes
  • escalation_rate
  • customer_effort_score
  • reopen_rate
Order Fulfillment
  • orders_processed
  • fulfillment_time_hours
  • backorder_rate
  • shipping_accuracy
  • return_rate
Compliance Check
  • checks_performed
  • violation_rate
  • false_positive_rate
  • remediation_time_hours
  • coverage_pct
Tier 3: Custom Business Events
App-code emitted via SDK — arbitrary domain events
tracker.emit_business_event(
    name="invoice_validated",
    entity_type="invoice",
    entity_id="INV-2026-44821",
    value_numeric=47500.00,
    metadata={"vendor": "Acme Corp", "po_number": "PO-9921"}
)

tracker.emit_business_event(
    name="account_opened",
    entity_type="bank_account",
    entity_id="ACC-2026-77291",
    value_numeric=1,
    metadata={"account_type": "checking", "segment": "enterprise"}
)
Compliance Observability

Compliance Narrative Patterns

For patterns like worker-auditor (maker-checker), observability must tell a compliance story — who did what, who reviewed it, what policies were enforced, and what the audit trail shows.

Worker-Auditor (Maker-Checker)
One agent generates output, a separate agent reviews it. Observability proves separation of duties through A2A span correlation and Decision Ledger hash-chain linkage.
  1. Separation verified: source_agent ≠ target_agent with different agent.type
  2. Review performed: auditor agent.run span exists with parent trace from worker
  3. Review outcome recorded: business.outcome_category = compliance_validated
  4. Ledger linkage: auditor record references worker recordHash
  5. Override trail: business.human_override captured if auditor overrides
Escalation Chain (HITL)
Agent identifies risk threshold requiring human approval. Observability captures trigger, routing, SLA, and resolution.
  1. Trigger condition logged with business context
  2. Routing audit: escalation.routing_group shows who was notified
  3. SLA tracking: resolution_latency_ms vs timeout_minutes
  4. Resolution outcome: approved / rejected / timeout recorded
  5. Post-resolution: agent outcome_category updated after HITL resolution
Regulatory Filing Agent
Agent prepares and submits regulatory filings. Observability provides end-to-end audit trail from data gathering to confirmation.
  1. Filing initiated: business.event with entity_type = regulatory_filing
  2. Data sources audited: all agent.tool_call spans show queried sources
  3. Compliance check: agent.policy_check spans show regulations evaluated
  4. Filing submitted: business.event with filing reference ID
  5. Confirmation received: business.event with confirmation status
Maturity

Value Realization Maturity Model

L1
Basic

Cost tracking, basic containment metrics

L2
Measured

Full value metrics, scorecard grading, drift detection

L3
Optimized

Predictive analytics, automated remediation, cross-agent benchmarks

L4
Strategic

Portfolio optimization, capacity planning, business case automation

Quick Start

Get Started in 4 Steps

01
Add Baselines

Add businessMetrics.baseline to your agent manifest with manual cost, time, and error rate.

02
Instrument SDK

Wrap agent runs with BusinessValueTracker.track_run() and call set_outcome().

03
Deploy Dashboard

Import business-value-dashboard.json as an Azure Monitor Workbook.

04
Set Targets

Configure valueTargets for drift detection — containment, ROI, satisfaction thresholds.

View Architecture Operations Guide