Troubleshooting

Infrastructure Deployment

Deployment fails with "InvalidTemplate"

Run a what-if deployment first to catch template errors before execution:

az deployment sub what-if \
  --location eastus2 \
  --template-file infra/bicep/main.bicep \
  --parameters @infra/bicep/parameters/dev.bicepparam

Pre-register all required resource providers:

az provider register --namespace Microsoft.CognitiveServices
az provider register --namespace Microsoft.Search
az provider register --namespace Microsoft.DocumentDB
az provider register --namespace Microsoft.KeyVault
az provider register --namespace Microsoft.Network
az provider register --namespace Microsoft.ManagedIdentity
az provider register --namespace Microsoft.OperationalInsights

Quota exceeded

Check regional quotas before deployment:

az vm list-usage --location eastus2 --output table
az cognitiveservices usage list --location eastus2

Networking & Private Endpoints

Private endpoint DNS resolution fails

Verify DNS zone VNet links and test resolution:

az network private-dns zone list --resource-group <rg> --output table
az network private-dns link vnet list \
  --zone-name privatelink.vaultcore.azure.net \
  --resource-group <rg>

Cannot access resources from local machine

Private endpoint resources are not accessible from the public internet by design. Use Azure Bastion, a jump box inside the VNet, or az network bastion tunnel for secure access.

Identity & RBAC

Managed identity returns 403

RBAC role assignments can take up to 10 minutes to propagate. Verify:

az identity show --name <identity-name> --resource-group <rg>
az role assignment list --assignee <principal-id> --output table

OIDC federated credential errors in CI/CD

Ensure the federated credential subject matches your repository pattern:

# Verify configuration
az ad app federated-credential list --id <app-id>

# Expected subject format:
# repo:<org>/<repo>:ref:refs/heads/main
# repo:<org>/<repo>:environment:<env-name>

Agent Runtime

Model deployment returns "ModelNotFound"

Verify the model is deployed in your Foundry project:

az cognitiveservices account deployment list \
  --name <account> --resource-group <rg>

Tool execution timeout

Check adapter health endpoints:

curl -s https://<adapter-url>/health | jq .

All adapters retry 3 times with exponential backoff for transient errors (429, 500, 502, 503, 504). If retries exhaust, check the target system's availability and network connectivity.

Cosmos DB & Decision Ledger

Writes fail with 429 (throttling)

Enable autoscale on the Cosmos container:

az cosmosdb sql container throughput migrate \
  --account-name <account> --database-name <db> --name <container> \
  --resource-group <rg> --throughput-type autoscale

Hash chain validation fails

A broken audit chain indicates a record was manually modified or deleted. Investigate affected records for tampering — the ledger is designed to be append-only and immutable.

Observability

No telemetry in Application Insights

• Verify APPLICATIONINSIGHTS_CONNECTION_STRING is set in the agent runtime environment.
• Allow 2–5 minutes for telemetry ingestion lag.
• Check the KQL queries in observability/kql/ for structured investigation.

CI/CD Pipeline

Evaluation gate blocks deployment

1. Check the eval threshold in eval-packs/ci-wiring/eval-ci-integration.py.
2. Review the evaluation report artifact uploaded to the GitHub Actions run.
3. Adjust thresholds if needed — but document the rationale.

Zero Trust Policy

Policy blocks legitimate requests

1. Check which policy is blocking in the decision ledger.
2. Review the policy configuration in policy-packs/.
3. Verify the agent's Zero Trust profile assignment in zero-trust/zt-profiles.yaml.

For the full troubleshooting guide with additional scenarios, see docs/TROUBLESHOOTING.md.