Azure OpenAI Security: Networking, Identity, and Compliance Controls

Every new Azure OpenAI resource accepts API keys over the public internet. Here are the four layers that fix it, and the script that proves they hold.

Microsoft's own documentation is blunt about the starting point: a new Azure AI resource is created with key based authentication enabled, and every published hardening guide opens by telling you to turn it off. The gap between a working demo and a production deployment is usually not a missing product, it is a default nobody flipped. That gap has a documented consequence. CVE-2025-53767, a server side request forgery flaw in Azure OpenAI, let an attacker steer requests at the internal instance metadata endpoint on 169.254.169.254 and retrieve bearer tokens for the resource's managed identity, which becomes full tenant compromise when that identity carries Owner or Contributor at subscription scope.

This tutorial closes those gaps in four layers. You will disable key authentication and switch to managed identity, take the endpoint off the public internet with a private endpoint, put a gateway in front for quotas and logging, and pin down the data and compliance settings that auditors ask about. At the end you get an audit script that verifies all of it, because a control you have not tested is a control you do not have.

What securing an Azure OpenAI deployment means

Azure OpenAI security is the practice of removing shared key access in favor of scoped Microsoft Entra ID identities, isolating the service endpoint inside a virtual network with private endpoints, mediating access through a gateway that enforces quotas and logging, and configuring data handling so that residency, retention, and content filtering match your compliance obligations.

Four layers, in that order. Identity first, because it is one property and it removes the most common leak. Network second, because it eliminates the entire class of remote attackers. Gateway third, because it gives you the controls the resource does not have. Data and compliance last, because those settings depend on the first three being right.

Layer 1: Turn off the keys

Every Azure OpenAI resource exposes two authentication paths. The key path accepts a static secret in the api-key header. The identity path accepts a Microsoft Entra ID bearer token. The key path has no identity behind it, which means no per consumer audit trail and no way to revoke one caller without rotating the secret for everyone.

Set disableLocalAuth to true and the key path disappears.

# Disable key based authentication on an existing resource
az rest --method patch \
  --url "https://management.azure.com/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.CognitiveServices/accounts/$NAME?api-version=2023-05-01" \
  --body '{"properties": {"disableLocalAuth": true}}'

# Confirm it took effect
az cognitiveservices account show \
  --name "$NAME" --resource-group "$RG" \
  --query "properties.disableLocalAuth"

Once that flag is set, az cognitiveservices account keys list returns an error stating that key listing failed because local auth is disabled. That error is the control working, not a misconfiguration, and it is worth telling your team before someone opens a support ticket about it.

Enforce it across the estate with the built in Azure Policy that requires key access to be disabled on AI resources. Start the assignment in Audit mode to find every resource that would break, then move it to Deny once the migrations land. Skipping that intermediate step is the most reliable way to take down a working application on a Friday.

Assign the right RBAC role, then make it smaller

Turning off keys only helps if the identities that replace them are scoped. These are the data plane roles that matter.

That last row is the one that turns a service flaw into a tenant incident. Scope the assignment to the individual Cognitive Services account, never to the resource group or subscription, and give the runtime workload the User role only.

Call the service with a managed identity

On the application side the change is small. Replace the key with a token provider and the SDK handles the rest.

from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI

# On Azure compute this resolves to the attached managed identity.
# Locally it falls back to your az login session, so the code is identical
# in both places and no secret ever exists.
token_provider = get_bearer_token_provider(
    DefaultAzureCredential(),
    "https://cognitiveservices.azure.com/.default",
)

client = AzureOpenAI(
    azure_endpoint="https://my-openai-weu.openai.azure.com/",
    azure_ad_token_provider=token_provider,
    api_version="2024-10-21",
)

response = client.chat.completions.create(
    model="gpt-4o-prod",          # your deployment name, not the model name
    messages=[{"role": "user", "content": "Summarize this incident report."}],
)

Nothing in that file is a secret, which means nothing in it can leak through a repository, a container image layer, or a log line.

Layer 2: Take the endpoint off the internet

With keys gone, the endpoint is still publicly reachable and still accepts tokens from any network. Network isolation fixes that.

Two settings do the work. Setting publicNetworkAccess to Disabled turns off the public path entirely. Setting networkAcls.defaultAction to Deny handles the middle ground where you want selected networks rather than full isolation. A hardened resource shows Disabled and Deny together.

# 1. Create the private endpoint in your application subnet
az network private-endpoint create \
  --name pe-openai --resource-group "$RG" \
  --vnet-name vnet-app --subnet snet-privatelink \
  --private-connection-resource-id "$OPENAI_RESOURCE_ID" \
  --group-id account --connection-name openai-connection

# 2. Create the private DNS zone. This exact name matters.
az network private-dns zone create \
  --name privatelink.openai.azure.com --resource-group "$RG"

# 3. Link the zone to the VNet so workloads resolve against it
az network private-dns link vnet create \
  --name openai-dns-link --resource-group "$RG" \
  --zone-name privatelink.openai.azure.com \
  --virtual-network vnet-app --registration-enabled false

# 4. Let the private endpoint register its own A record
az network private-endpoint dns-zone-group create \
  --name openai-dns-group --resource-group "$RG" \
  --endpoint-name pe-openai \
  --private-dns-zone privatelink.openai.azure.com --zone-name openai

# 5. Close the public door
az resource update --ids "$OPENAI_RESOURCE_ID" \
  --set properties.publicNetworkAccess=Disabled \
        properties.networkAcls.defaultAction=Deny

Both of the common failure modes here are worth knowing before you open a support case.

The DNS zone name is service specific. Azure OpenAI resources register in privatelink.openai.azure.com, while Azure AI Foundry deployments may also need privatelink.cognitiveservices.azure.com and privatelink.services.ai.azure.com depending on which capabilities you use. The built in Azure Policy that configures private DNS zones for Cognitive Services accounts takes a single zone parameter and has historically registered OpenAI resources in the Cognitive Services zone, which resolves to nothing useful. If your calls fail with a name resolution error after everything looks correct in the portal, check which zone the A record actually landed in.

Public DNS resolution is not an access control. The privatelink CNAME chain stays resolvable from the internet on purpose so hybrid and migration scenarios keep working. A successful lookup from your laptop only proves the resource name exists in Azure's global namespace, and it reveals neither the private IP nor any data plane access. When publicNetworkAccess is Disabled, the request is refused at the service boundary no matter what DNS handed back.

Pick the topology that matches your constraint

Want a safe place to practice these controls?

Reading a network isolation walkthrough is one thing. Watching a call fail with a name resolution error, tracing it to the wrong private DNS zone, and fixing it is what makes the pattern stick. The Azure learning paths on KodeKloud cover identity, networking, and RBAC in hands on labs, and the KodeKloud playgrounds give you disposable cloud environments where a broken configuration costs nothing and teaches everything.

Layer 3: Put a gateway in front

The Azure OpenAI resource itself has no concept of per consumer quotas. Every application sharing a deployment competes for the same tokens per minute, and one runaway retry loop starves everyone else. Azure API Management in front of the service closes that gap and gives you the AI gateway pattern.

<policies>
  <inbound>
    <base />
    <!-- Per consumer token budget. Exceeding the rate returns 429,
         exceeding the quota returns 403. -->
    <azure-openai-token-limit
        counter-key="@(context.Subscription.Id)"
        tokens-per-minute="5000"
        estimate-prompt-tokens="true"
        remaining-tokens-header-name="x-remaining-tokens" />

    <!-- Emit per consumer token metrics for chargeback and anomaly alerts -->
    <azure-openai-emit-token-metric namespace="openai">
      <dimension name="Subscription ID" value="@(context.Subscription.Id)" />
      <dimension name="API ID" value="@(context.Api.Id)" />
    </azure-openai-emit-token-metric>

    <!-- APIM authenticates to the backend with its own managed identity -->
    <authentication-managed-identity
        resource="https://cognitiveservices.azure.com"
        output-token-variable-name="msi-access-token" />
    <set-header name="Authorization" exists-action="override">
      <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value>
    </set-header>
  </inbound>
</policies>

Three benefits come from that block. Each consumer gets a token budget rather than a shared free for all, which turns a noisy neighbour outage into a 429 for one team. Token consumption becomes a metric with dimensions, so you can alert on a consumer whose spend triples overnight, which is the earliest signal of both a bug and a stolen credential. Backend authentication happens once, at the gateway, with a single managed identity that holds the User role.

Layer 4: Data handling and compliance

The first three layers control who reaches the model. This one controls what happens to the data once it arrives, and it is where most compliance questionnaires actually land.

Deployment type decides residency. A Global deployment can process requests in any region Microsoft operates, a DataZone deployment keeps processing within a geography such as the EU or the US, and a Regional deployment keeps it in the single region you chose. Throughput and price differ across the three, so this is a real trade off rather than a checkbox. If your answer to a data residency question needs to be specific, choose Regional or DataZone deliberately and record why.

Abuse monitoring stores prompts, unless you apply otherwise. Microsoft may retain samples of flagged prompts and completions for a limited window so that authorized reviewers can investigate misuse, with access controlled through restricted workstations and just in time approval. Customers with regulated data can apply for modified abuse monitoring and content logging exemptions. Read the current terms before you promise an auditor that nothing is retained, because the default and the exception are different answers.

Fine tuning moves data. A March 2025 terms update disclosed that a fine tuning run can temporarily move data beyond the geography you selected. If you fine tune on sensitive data, verify the current policy first, keep training sets in Blob Storage behind tight RBAC, log every write to that storage, and record each fine tuning job start in Azure Monitor.

Content filtering is configurable, and Prompt Shields are separate. The default filters cover hate, sexual, violence, and self harm categories at a medium severity threshold. Prompt Shields target direct jailbreak attempts and indirect prompt injection, and indirect injection detection matters specifically for retrieval augmented generation, where the untrusted content arrives inside a document rather than the user message.

Map the controls to what they evidence, because that mapping is what turns an engineering change into an audit artifact.

Confirm the exact control identifiers with your compliance team, since frameworks such as ISO 27001, SOC 2, and the EU AI Act each phrase these obligations differently.

Layer 5: Prove you would notice

Send diagnostic logs somewhere queryable, then write the alert before you need it.

az monitor diagnostic-settings create \
  --name openai-diagnostics \
  --resource "$OPENAI_RESOURCE_ID" \
  --workspace "$LOG_ANALYTICS_WORKSPACE_ID" \
  --logs    '[{"category":"RequestResponse","enabled":true},
              {"category":"Audit","enabled":true}]' \
  --metrics '[{"category":"AllMetrics","enabled":true}]'

The single most useful detection is a spend anomaly, because token theft and a broken retry loop look identical on a bill and both need someone to look.

// Callers whose hourly request volume is far above their own baseline
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| summarize calls = count() by bin(TimeGenerated, 1h), CallerIPAddress, identity_claim_appid_g
| join kind=inner (
    AzureDiagnostics
    | where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
    | summarize baseline = avg(1.0) * count() / 24.0 by identity_claim_appid_g
  ) on identity_claim_appid_g
| where calls > baseline * 5
| order by calls desc

Adjust the field names to match your workspace schema, since diagnostic categories evolve. Pair this with Microsoft Defender for Cloud AI posture management, which flags exactly the misconfigurations this tutorial removes.

Deploy it with secure defaults

Every control above belongs in the template, not in a runbook someone follows by hand.

resource openai 'Microsoft.CognitiveServices/accounts@2024-10-01' = {
  name: 'openai-prod-weu'
  location: 'westeurope'
  kind: 'OpenAI'
  sku: { name: 'S0' }
  identity: { type: 'SystemAssigned' }
  properties: {
    customSubDomainName: 'openai-prod-weu'   // required for private link
    disableLocalAuth: true                    // no API keys, ever
    publicNetworkAccess: 'Disabled'           // private endpoint only
    networkAcls: {
      defaultAction: 'Deny'
      ipRules: []
      virtualNetworkRules: []
    }
  }
}

// Scope the app identity to this account only, with inference rights only
resource inferenceRole 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: openai
  name: guid(openai.id, appPrincipalId, 'openai-user')
  properties: {
    // Cognitive Services OpenAI User
    roleDefinitionId: subscriptionResourceId(
      'Microsoft.Authorization/roleDefinitions',
      '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd')
    principalId: appPrincipalId
    principalType: 'ServicePrincipal'
  }
}

The customSubDomainName line is not optional. Private link requires a custom subdomain, and adding one later means changing every endpoint URL your applications use.

Verify it: the posture audit

Here is the part most guides leave out. Run this against any resource and it tells you which controls are actually on.

#!/usr/bin/env bash
# audit-openai.sh <resource-group> <account-name>
set -euo pipefail
RG="$1"; NAME="$2"; PASS=0; FAIL=0

check() {  # check <description> <expected> <actual>
  if [[ "$2" == "$3" ]]; then
    printf '  PASS  %-42s %s\n' "$1" "$3"; PASS=$((PASS+1))
  else
    printf '  FAIL  %-42s got:%s want:%s\n' "$1" "$3" "$2"; FAIL=$((FAIL+1))
  fi
}

P=$(az cognitiveservices account show -g "$RG" -n "$NAME" --query properties)

check "Key authentication disabled"  "true"     "$(jq -r '.disableLocalAuth // false' <<<"$P")"
check "Public network access off"    "Disabled" "$(jq -r '.publicNetworkAccess' <<<"$P")"
check "Network default action deny"  "Deny"     "$(jq -r '.networkAcls.defaultAction // "Allow"' <<<"$P")"
check "Custom subdomain set"         "true"     "$([[ $(jq -r '.customSubDomainName // ""' <<<"$P") != "" ]] && echo true || echo false)"

PE=$(az network private-endpoint-connection list --id \
      "$(az cognitiveservices account show -g "$RG" -n "$NAME" --query id -o tsv)" -o json 2>/dev/null || echo '[]')
check "Private endpoint approved" "true" \
  "$(jq -r 'any(.[]; .properties.privateLinkServiceConnectionState.status=="Approved")' <<<"$PE")"

DS=$(az monitor diagnostic-settings list --resource \
      "$(az cognitiveservices account show -g "$RG" -n "$NAME" --query id -o tsv)" -o json)
check "Diagnostic logging enabled" "true" "$(jq -r '(.value // []) | length > 0' <<<"$DS")"

# Subscription scoped assignments for this identity are an escalation path
MI=$(az cognitiveservices account show -g "$RG" -n "$NAME" --query identity.principalId -o tsv)
if [[ -n "$MI" && "$MI" != "null" ]]; then
  BROAD=$(az role assignment list --assignee "$MI" --query \
    "[?scope=='/subscriptions/$(az account show --query id -o tsv)'] | length(@)" -o tsv)
  check "No subscription scoped role for MI" "0" "$BROAD"
fi

echo "  ----"
echo "  $PASS passed, $FAIL failed"
[[ "$FAIL" -eq 0 ]]

Run it in CI against every AI resource on a schedule and the output becomes evidence. A failing exit code in a pipeline is a far better compliance artifact than a screenshot in a wiki.

Azure OpenAI security controls at a glance

Where to start

1. Inventory every Cognitive Services and Foundry account in your subscriptions, then record disableLocalAuth, publicNetworkAccess, and the assigned roles for each.
2. Assign the built in key access policy in Audit mode, so you learn how many resources and applications would break before anything does.
3. Migrate one non production application to managed identity using the token provider pattern, and delete its key from wherever it lives today.
4. Add the private endpoint and private DNS zone for that same resource, then confirm resolution from inside the VNet before disabling public access.
5. Put API Management in front of your highest volume deployment and set a token limit per consumer, starting well above current usage.
6. Turn on diagnostic settings everywhere and build the spend anomaly alert, because detection is what covers the controls you have not added yet.
7. Run the audit script in CI on a schedule and treat a failing check as a defect with an owner, not a dashboard item.

Conclusion

The uncomfortable part of Azure OpenAI security is that almost none of it is exotic. Key authentication is one property. Network isolation is a private endpoint and a correctly named DNS zone. Least privilege is choosing the User role and scoping it to one account instead of a subscription. What makes these deployments risky is not that the controls are hard, it is that the defaults are permissive and a working demo gives you no signal that anything is missing.

Pick one production resource this week and walk the four layers on it. Disable the keys, attach the private endpoint, scope the identity, and turn on the logs. Then run the audit script and keep running it, because the deployment you hardened in July will drift by October unless something checks.

Ready to Build Real Azure Skills?

Cloud security concepts land properly when you break something and fix it. The Azure learning paths on KodeKloud cover identity, RBAC, networking, and governance through hands on labs rather than slides, and the KodeKloud playgrounds give you live cloud environments to test private endpoints, policy assignments, and managed identity flows without touching a production subscription. Start with one lab today.

FAQs