# Azure Kubernetes Service (AKS)

Source: /azure/services/aks/

## Introduction

Azure Kubernetes Service (AKS) is Azure's managed Kubernetes offering. Azure operates the control
plane while you manage node pools of worker machines that run your workloads. For more information,
see [What is Azure Kubernetes Service?](https://learn.microsoft.com/en-us/azure/aks/what-is-aks).

LocalStack for Azure creates real, working Kubernetes clusters on your machine. `az aks create`
produces a cluster backed by [k3d](https://k3d.io/) that you can reach with `kubectl`, so manifests,
Helm charts, and operators behave as they would against a cluster in the cloud. The supported APIs
are available on our [API Coverage section](#api-coverage), which provides information on the extent
of AKS's integration with LocalStack.

## Getting started

This guide is designed for users new to AKS and assumes basic knowledge of the Azure CLI, `kubectl`,
and our `lstk az` proxy.

Launch LocalStack using your preferred method. For more information, see
[Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running,
enable Azure CLI interception by running:

```bash
lstk az start-interception
```

This command points the `az` CLI away from the public Azure management REST API and toward the
LocalStack for Azure emulator API. To revert this configuration, run:

```bash
lstk az stop-interception
```

This reconfigures the `az` CLI to send commands to the official Azure management REST API.

### Create a resource group

Create a resource group to hold the cluster:

```bash
az group create \
  --name rg-aks-demo \
  --location westeurope
```

```bash title="Output"
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aks-demo",
  "location": "westeurope",
  "managedBy": null,
  "name": "rg-aks-demo",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
```

### Create a cluster

Create a cluster with a single node in its system node pool:

```bash
az aks create \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --node-count 1 \
  --generate-ssh-keys
```

The command returns when the cluster is ready to use. Locally that takes a couple of minutes: the
emulator provisions a k3d cluster, so what you get back is a live API server, not a mock.

```bash title="Output"
{
  "currentKubernetesVersion": "1.34.4",
  "fqdn": "aks-demo-rg-aks-demo-000000-oq7mpqgx.hcp.westeurope.azmk8s.io",
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rg-aks-demo/providers/Microsoft.ContainerService/managedClusters/aks-demo",
  "kubernetesVersion": "1.34",
  "location": "westeurope",
  "name": "aks-demo",
  "nodeResourceGroup": "MC_rg-aks-demo_aks-demo_westeurope",
  "powerState": {
    "code": "Running"
  },
  "provisioningState": "Succeeded",
  ...
}
```

### Show and list clusters

Retrieve the details of a single cluster:

```bash
az aks show \
  --resource-group rg-aks-demo \
  --name aks-demo
```

```bash title="Output"
{
  "currentKubernetesVersion": "1.34.4",
  "dnsPrefix": "aks-demo-rg-aks-demo-000000",
  "kubernetesVersion": "1.34",
  "location": "westeurope",
  "name": "aks-demo",
  "nodeResourceGroup": "MC_rg-aks-demo_aks-demo_westeurope",
  "provisioningState": "Succeeded",
  ...
}
```

List the clusters in a resource group:

```bash
az aks list \
  --resource-group rg-aks-demo \
  --output table
```

```bash title="Output"
Name      Location    ResourceGroup    KubernetesVersion    CurrentKubernetesVersion    ProvisioningState    Fqdn
--------  ----------  ---------------  -------------------  --------------------------  -------------------  -------------------------------------------------------------
aks-demo  westeurope  rg-aks-demo      1.34                 1.34.4                      Succeeded            aks-demo-rg-aks-demo-000000-oq7mpqgx.hcp.westeurope.azmk8s.io
```

### Update a cluster

`az aks update` changes the properties of an existing cluster. The following example sets resource
tags:

```bash
az aks update \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --tags environment=local team=platform
```

```bash title="Output"
{
  "name": "aks-demo",
  "provisioningState": "Succeeded",
  "tags": {
    "environment": "local",
    "team": "platform"
  },
  ...
}
```

### Connect with kubectl

Merge the cluster credentials into your local kubeconfig:

```bash
az aks get-credentials \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --overwrite-existing
```

```bash title="Output"
Merged "aks-demo" as current context in /home/user/.kube/config
```

Query the nodes:

```bash
kubectl get nodes
```

```bash title="Output"
NAME                               STATUS   ROLES           AGE    VERSION
aks-nodepool1-5829393-vmss000000   Ready    <none>          99s    v1.36.2+k3s1
k3d-aks-demo-7da4c24d-server-0     Ready    control-plane   2m1s   v1.36.2+k3s1
```

:::note
Unlike in the cloud, where the control plane is hidden, the local cluster also lists its k3d
control-plane node. Agent nodes carry the same `aks-<pool>-...-vmss` naming scheme as real AKS
nodes, and the `VERSION` column reflects the underlying k3s runtime rather than the cluster's
`kubernetesVersion`.
:::

### Manage node pools

Add a user node pool with two nodes:

```bash
az aks nodepool add \
  --resource-group rg-aks-demo \
  --cluster-name aks-demo \
  --name workers \
  --mode User \
  --node-count 2
```

```bash title="Output"
{
  "count": 2,
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rg-aks-demo/providers/Microsoft.ContainerService/managedClusters/aks-demo/agentPools/workers",
  "mode": "User",
  "name": "workers",
  "orchestratorVersion": "1.34",
  "osType": "Linux",
  "provisioningState": "Succeeded",
  ...
}
```

List the node pools of the cluster:

```bash
az aks nodepool list \
  --resource-group rg-aks-demo \
  --cluster-name aks-demo \
  --output table
```

```bash title="Output"
Name       OsType    VmSize    Count    MaxPods    ProvisioningState    Mode
---------  --------  --------  -------  ---------  -------------------  ------
nodepool1  Linux               1        250        Succeeded            System
workers    Linux               2        250        Succeeded            User
```

Inspect a single node pool:

```bash
az aks nodepool show \
  --resource-group rg-aks-demo \
  --cluster-name aks-demo \
  --name workers
```

```bash title="Output"
{
  "count": 2,
  "mode": "User",
  "name": "workers",
  "orchestratorVersion": "1.34",
  "osType": "Linux",
  "powerState": {
    "code": "Running"
  },
  "provisioningState": "Succeeded",
  ...
}
```

Delete the node pool when you no longer need it:

```bash
az aks nodepool delete \
  --resource-group rg-aks-demo \
  --cluster-name aks-demo \
  --name workers
```

### Stop, start, and delete

Stop the cluster to free local resources. Unlike AKS, the emulator does not keep the cluster's
Kubernetes objects while it is stopped (see [Limitations](#limitations)):

```bash
az aks stop \
  --resource-group rg-aks-demo \
  --name aks-demo
```

Verify that the cluster has stopped by confirming that `powerState` reports `Stopped`:

```bash
az aks show \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --query powerState.code \
  --output tsv
```

Start the cluster again. It comes back with its node pools, their number of agent nodes, and its
add-ons, and its API server can move to a different port, so run `az aks get-credentials` again
before you use `kubectl`:

```bash
az aks start \
  --resource-group rg-aks-demo \
  --name aks-demo
```

Delete the cluster once you are done. A deleted cluster cannot be recovered:

```bash
az aks delete \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --yes
```

### Teardown

Remove the resource group and any resources it still contains:

```bash
az group delete \
  --name rg-aks-demo \
  --yes
```

Disable Azure CLI interception to point the `az` CLI back to the official Azure management REST API:

```bash
lstk az stop-interception
```

## Autoscale workloads with KEDA

[Kubernetes Event-driven Autoscaling (KEDA)](https://learn.microsoft.com/en-us/azure/aks/keda-about)
scales workloads based on events from external sources, such as the number of messages waiting in
a queue, and can scale them down to zero when there is no work. KEDA extends the [Kubernetes
Horizontal Pod Autoscaler (HPA)](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/) rather than replacing it.

The Service Bus and Storage Queue scalers authenticate with a managed identity, which needs the
OIDC issuer and Workload Identity. The emulator enables both only when a cluster is created (see
[Limitations](#limitations)), and the [Teardown](#teardown) above removed the demo resources, so
start the interception again and create a cluster with the managed KEDA add-on, the OIDC issuer,
and Workload Identity:

```bash
lstk az start-interception

az group create \
  --name rg-aks-demo \
  --location westeurope

az aks create \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --node-count 1 \
  --enable-keda \
  --enable-oidc-issuer \
  --enable-workload-identity \
  --generate-ssh-keys

az aks get-credentials \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --overwrite-existing
```

Verify that the add-on is enabled and its deployments are ready:

```bash
az aks show \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --query workloadAutoScalerProfile.keda.enabled \
  --output tsv

kubectl get deployments \
  --namespace kube-system \
  --selector app.kubernetes.io/part-of=keda-operator
```

```bash title="Output"
true

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
keda-admission             1/1     1            1           1m
keda-metrics-apiserver     1/1     1            1           1m
keda-operator              1/1     1            1           1m
```

### Configure Workload Identity

KEDA 2.15 and later use
[Microsoft Entra Workload ID](https://learn.microsoft.com/en-us/azure/aks/keda-workload-identity)
instead of Azure AD Pod Identity. Create a user-assigned managed identity and federate it to the
KEDA operator service account:

```bash
KEDA_IDENTITY=aks-keda-identity

az identity create \
  --name "$KEDA_IDENTITY" \
  --resource-group rg-aks-demo \
  --location westeurope

KEDA_CLIENT_ID=$(az identity show \
  --name "$KEDA_IDENTITY" \
  --resource-group rg-aks-demo \
  --query clientId \
  --output tsv)

OIDC_ISSUER=$(az aks show \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --query oidcIssuerProfile.issuerUrl \
  --output tsv)

az identity federated-credential create \
  --name keda-operator \
  --identity-name "$KEDA_IDENTITY" \
  --resource-group rg-aks-demo \
  --issuer "$OIDC_ISSUER" \
  --subject system:serviceaccount:kube-system:keda-operator \
  --audiences api://AzureADTokenExchange
```

Annotate the operator service account with the identity's client ID. Then restart the operator so
that the Workload Identity webhook injects the federated token into its pods:

```bash
kubectl annotate serviceaccount keda-operator \
  --namespace kube-system \
  "azure.workload.identity/client-id=${KEDA_CLIENT_ID}" \
  --overwrite

kubectl rollout restart deployment keda-operator \
  --namespace kube-system

kubectl rollout status deployment keda-operator \
  --namespace kube-system \
  --timeout=300s
```

Grant the identity the data plane role required by the scaler. This Service Bus example creates a
queue and grants **Azure Service Bus Data Owner** on the namespace:

```bash
SERVICE_BUS_NAMESPACE=sbkedademo
SERVICE_BUS_QUEUE=work-items

az servicebus namespace create \
  --name "$SERVICE_BUS_NAMESPACE" \
  --resource-group rg-aks-demo \
  --location westeurope \
  --sku Standard

az servicebus queue create \
  --name "$SERVICE_BUS_QUEUE" \
  --namespace-name "$SERVICE_BUS_NAMESPACE" \
  --resource-group rg-aks-demo

KEDA_PRINCIPAL_ID=$(az identity show \
  --name "$KEDA_IDENTITY" \
  --resource-group rg-aks-demo \
  --query principalId \
  --output tsv)

SERVICE_BUS_ID=$(az servicebus namespace show \
  --name "$SERVICE_BUS_NAMESPACE" \
  --resource-group rg-aks-demo \
  --query id \
  --output tsv)

az role assignment create \
  --role "Azure Service Bus Data Owner" \
  --assignee-object-id "$KEDA_PRINCIPAL_ID" \
  --assignee-principal-type ServicePrincipal \
  --scope "$SERVICE_BUS_ID"
```

### Configure private scaler endpoints

Service Bus and Storage Queue scalers use `cloud: Private` when targeting the emulator. Derive the
`endpointSuffix` from the endpoints returned by Azure Resource Manager instead of hardcoding it:

```bash
SERVICE_BUS_ENDPOINT=$(az servicebus namespace show \
  --name "$SERVICE_BUS_NAMESPACE" \
  --resource-group rg-aks-demo \
  --query serviceBusEndpoint \
  --output tsv)

ARM_ENDPOINT=$(az cloud show \
  --query endpoints.resourceManager \
  --output tsv)

SERVICE_BUS_HOST="${SERVICE_BUS_ENDPOINT#*://}"
SERVICE_BUS_HOST="${SERVICE_BUS_HOST%%/*}"
SERVICE_BUS_HOST="${SERVICE_BUS_HOST%%:*}"
ARM_ENDPOINT="${ARM_ENDPOINT%/}"
ARM_PORT="${ARM_ENDPOINT##*:}"
if [[ "$ARM_PORT" == "$ARM_ENDPOINT" || "$ARM_PORT" == *"/"* ]]; then
  ARM_PORT=443
fi

SERVICE_BUS_SUFFIX="${SERVICE_BUS_HOST#${SERVICE_BUS_NAMESPACE}.}:${ARM_PORT}"
printf '%s\n' "$SERVICE_BUS_SUFFIX"
```

```bash title="Output"
servicebus.azure.localhost.localstack.cloud:4566
```

Use the derived suffix in the `ScaledObject`. The target `Deployment` can start with zero replicas;
KEDA creates and manages its Horizontal Pod Autoscaler:

```yaml title="service-bus-scaler.yaml"
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: service-bus-auth
spec:
  podIdentity:
    provider: azure-workload
    identityId: "<managed-identity-client-id>"
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: service-bus-scaler
spec:
  scaleTargetRef:
    name: service-bus-consumer
  pollingInterval: 5
  cooldownPeriod: 30
  minReplicaCount: 0
  maxReplicaCount: 4
  triggers:
    - type: azure-servicebus
      metadata:
        queueName: work-items
        namespace: sbkedademo
        messageCount: "5"
        cloud: Private
        endpointSuffix: "<service-bus-endpoint-suffix>"
      authenticationRef:
        name: service-bus-auth
```

For an Azure Storage Queue scaler, derive the suffix from the storage account's
`primaryEndpoints.queue` property and use the same private-cloud pattern:

```yaml
triggers:
  - type: azure-queue
    metadata:
      queueName: jobs
      accountName: stkedademo
      queueLength: "5"
      cloud: Private
      endpointSuffix: "<storage-queue-endpoint-suffix>"
```

The Event Hubs scaler doesn't support Workload Identity in the emulator, so it authenticates with
connection strings. Connection strings returned by LocalStack use an `sb://` endpoint and include
`UseDevelopmentEmulator=true`, which KEDA and the Azure SDKs use to connect to the emulator without
TLS:

```yaml
triggers:
  - type: azure-eventhub
    metadata:
      consumerGroup: keda-consumer
      unprocessedEventThreshold: "5"
      blobContainer: eh-checkpoints
      checkpointStrategy: blobMetadata
      connectionFromEnv: EVENTHUB_CONNECTION
      storageConnectionFromEnv: STORAGE_CONNECTION
```

The [AKS KEDA tutorials](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda)
contain complete producer and consumer applications for Service Bus, Storage Queue, and Event Hubs.
Each tutorial verifies that a backlog scales its consumer from zero to multiple replicas, drains
without dead-lettering messages, and scales back to zero.

```bash title="Service Bus tutorial output"
Observed scale-out: 0 -> 2 -> 3 -> 4
PASS: the consumer drained the work-items queue with no dead-lettered messages
PASS: the sb-consumer deployment scaled back to zero replicas
SUCCESS: KEDA scaled the sb-consumer deployment from zero to 4 replicas and back to zero
```

### Disable KEDA

Delete your KEDA custom resources before disabling the add-on:

```bash
kubectl delete \
  scaledobject/service-bus-scaler \
  triggerauthentication/service-bus-auth
kubectl wait \
  --for=delete \
  scaledobject/service-bus-scaler \
  triggerauthentication/service-bus-auth \
  --timeout=60s
```

Then disable the managed add-on with `az aks update`:

```bash
az aks update \
  --resource-group rg-aks-demo \
  --name aks-demo \
  --disable-keda \
  --query workloadAutoScalerProfile.keda.enabled \
  --output tsv
```

```bash title="Output"
false
```

:::caution
Delete all `ScaledObject`, `ScaledJob`, and `TriggerAuthentication` resources, or their containing
namespaces, before disabling KEDA.
:::

### Clean up

Delete the resource group, which removes the cluster, the Service Bus namespace, and the managed
identity, then disable Azure CLI interception:

```bash
az group delete \
  --name rg-aks-demo \
  --yes

lstk az stop-interception
```

## Features

The local control plane implements the following capabilities:

- **Cluster lifecycle**: You can create, show, list, update, stop, start, and delete clusters, and
  `az aks get-credentials` returns user and admin kubeconfigs that work with `kubectl` and Helm.
  The node resource group holds the scale sets, VM instances, network interfaces, load balancers,
  and kubelet identity that AKS creates for a cluster, along with the managed identities of the
  Key Vault secrets provider and of cluster extensions.
- **Node pools**: System and User node pools run as k3d agent nodes that follow the AKS naming
  scheme. Adding, scaling, and deleting a pool adds and removes real nodes, and the pool's node
  labels and taints are applied to its Kubernetes nodes and updated in place.
- **Scaling**: The cluster autoscaler adds nodes to a pool while pods are unschedulable and removes
  a node once one is idle, within the pool's minimum and maximum counts. Node auto-provisioning
  runs the AKS Karpenter provider, which creates and removes nodes through the AKS Machines API.
  The Kubernetes Event-driven Autoscaling (KEDA) and Vertical Pod Autoscaler add-ons can be enabled
  and disabled on a running cluster.
- **Networking**: The Cilium data plane and the Calico network policy engine run on the cluster
  and enforce Kubernetes network policies as well as their own policy resources, and Advanced
  Container Networking Services observability enables Hubble on the Cilium data plane. The managed
  Gateway API installs the standard Gateway API CRDs, which you pair with an implementation such
  as NGINX Gateway Fabric.
- **Load balancing**: The Azure cloud controller manager reconciles public and internal
  `LoadBalancer` services into the `kubernetes` and `kubernetes-internal` load balancers, public IP
  addresses, and network security group rules of the node resource group. Internal load balancers
  take their frontend IP address from the cluster's subnet, including a subnet you bring with
  `--vnet-subnet-id`.
- **Storage**: The Azure Disk CSI driver is enabled by default and gives persistent volumes the
  behavior of managed disks: data survives a detach and a reattach on another node, and volumes
  can be expanded online. The Azure Files CSI driver provisions SMB and NFS file shares, the CSI
  snapshot controller is installed, and the built-in storage classes have the same names as on AKS.
- **Secrets**: The Azure Key Vault provider for the Secrets Store CSI driver mounts Key Vault
  secrets and certificates into pods, authenticating with the add-on's managed identity or with
  Microsoft Entra Workload ID.
- **Identity**: Microsoft Entra Workload ID works with a real OIDC issuer, so pods can exchange
  service account tokens for Azure credentials without secrets. The control plane can use a
  system-assigned or a user-assigned managed identity, and each cluster gets a kubelet identity in
  its node resource group.
- **Run Command**: `az aks command invoke` runs a command, with any files attached through
  `--file`, in a transient pod on the cluster and returns its output and exit code, and
  `az aks command result` retrieves a result by its ID. `az aks update --disable-run-command`
  turns the feature off.
- **Managed namespaces**: `az aks namespace add` creates the namespace on the cluster with its
  default resource quota and network policy, and applies the adoption and delete policies.
- **Cluster extensions**: The `Microsoft.AppConfiguration` extension, created with
  `az k8s-extension create`, installs the Azure App Configuration Kubernetes Provider with its own
  managed identity.
- **Management APIs**: Maintenance configurations, agent pool snapshots, trusted access role
  bindings, upgrade profiles, the supported Kubernetes versions, and Azure Kubernetes Fleet Manager
  resources are available through Azure Resource Manager, so scripts and templates that create or
  read them run unchanged.
- **Tooling**: The same clusters can be provisioned with the Azure CLI, Terraform, or Bicep.

## Limitations

- **Stopping a cluster discards its workloads**: `az aks stop` deletes the backing k3d cluster, and
  `az aks start` creates a new one from the stored configuration. AKS keeps every object except
  standalone pods while a cluster is stopped, whereas the restarted emulator cluster has none of
  the workloads or other Kubernetes objects you created. Managed namespaces and cluster extensions
  are not applied again either, although Azure Resource Manager still lists them, so recreate them
  after a start. Its API server can also move to a different port, so run `az aks get-credentials`
  again after a start.
- **Clusters do not survive an emulator restart**: LocalStack deletes the k3d clusters when it
  shuts down and does not recreate them when it starts again.
- **Upgrades and maintenance operations have no effect**: Every cluster runs the same k3s release,
  whatever `kubernetesVersion` or `orchestratorVersion` you request, and the emulator also accepts
  versions that AKS does not offer. `az aks upgrade`, `az aks nodepool upgrade`, and node image
  upgrades do not replace any node or change the Kubernetes version that the cluster runs, and
  `az aks rotate-certs` and `az aks operation-abort` are accepted without rotating certificates or
  cancelling operations.
- **The network profile is stored, not applied**: The network plugin and its mode, such as Azure
  CNI Overlay or kubenet, and the pod and service address ranges are recorded without changing how
  the cluster is networked. Pods get their addresses from the cluster's own CNI, which is flannel
  unless you select Cilium or Calico, so `--network-plugin none` does not let you install your own
  CNI. Of the Advanced Container Networking Services features, only observability is configured,
  and only on the Cilium data plane. `az aks update` ignores network profile changes, such as
  `--enable-acns`: they are neither stored nor applied.
- **Network policies are always enforced**: Every cluster created without Cilium or Calico runs the
  k3s network policy controller, so `NetworkPolicy` objects take effect even without a network
  policy engine, where AKS does not enforce them.
- **Load balancer IP addresses are not reachable**: The external IP address that the cloud
  controller manager assigns to a `LoadBalancer` service exists only in the emulated network, so
  use `kubectl port-forward` to reach the service from your machine. The cloud controller manager
  also needs the cluster's managed identity: on a cluster that uses a service principal,
  `LoadBalancer` services stay pending.
- **API server access settings are ignored**: Private clusters, authorized IP ranges, and API
  server VNet integration are not emulated. The API server is always reachable from your machine,
  and `disableRunCommand` is the only `apiServerAccessProfile` setting that the emulator stores.
- **Microsoft Entra ID and Azure RBAC are not enforced**: The Microsoft Entra ID settings, such as
  `--enable-azure-rbac`, are stored, and `--disable-local-accounts` is ignored. Every kubeconfig,
  with or without `--admin`, carries the same cluster administrator credentials and needs no
  `kubelogin`, and Azure role assignments do not control access to Kubernetes resources.
- **OIDC issuer and Workload ID must be enabled at creation**: Every cluster reports an OIDC issuer
  URL, as AKS does for new clusters on Kubernetes 1.34 and later, but the API server issues tokens
  from it only on clusters created with `--enable-oidc-issuer` or `--enable-workload-identity`, and
  the Workload ID webhook is installed only with `--enable-workload-identity`. `az aks update` does
  not enable either feature on an existing cluster, so pass both flags to `az aks create`.
- **Most add-ons are not installed**: The Azure Key Vault secrets provider is the only entry of
  `addonProfiles` that the emulator installs, and only if it was enabled when the cluster was
  created. Other add-ons, such as monitoring and Azure Policy, are stored without installing
  anything, and `az aks enable-addons` and `az aks disable-addons` do not change an existing
  cluster. The application routing add-on, the Istio-based service mesh add-on, Azure Monitor
  managed Prometheus, and the Azure Blob CSI driver are not supported.
- **Add-on installation failures are only logged**: Add-ons are installed on a best-effort basis.
  If an add-on fails to install, the cluster still reports `Succeeded` and the add-on as enabled,
  so check the LocalStack logs when a component does not come up. A cluster extension that fails
  to install reports `Failed`.
- **The cluster autoscaler is emulated**: An emulator control loop takes the place of the upstream
  cluster autoscaler. It adds a node to every autoscaling pool while any pod is unschedulable, and
  once one of a pool's nodes is idle it removes the pool's newest node, which is not necessarily
  the idle one, without draining it. It ignores `autoScalerProfile`. Set
  `LS_AZURE_AKS_SCALING_INTERVAL` to change how often it checks the cluster (15 seconds by default)
  and `LS_AZURE_AKS_SCALE_DOWN_AFTER` to change how long a node must stay idle before the pool
  shrinks (30 seconds by default).
- **KEDA differs from the managed add-on**: The metrics server and admission webhooks run as the
  `keda-metrics-apiserver` and `keda-admission` deployments of the upstream KEDA release, where
  the AKS add-on names them `keda-operator-metrics-apiserver` and `keda-admission-webhooks`. The
  Event Hubs scaler cannot authenticate with Workload Identity against the emulator, so it uses
  connection strings.
- **Node auto-provisioning creates no default node pools**: AKS creates the `default` and
  `system-surge` Karpenter `NodePool` resources, while the emulator creates none, so define your
  own `NodePool` and `AKSNodeClass` before Karpenter provisions any node. Karpenter is installed
  only when the cluster starts, so enable node auto-provisioning with `az aks create`:
  `az aks update --node-provisioning-mode Auto` stores the setting without installing Karpenter.
- **Node sizes and operating systems are approximated**: For the VM sizes in the emulator's
  catalog, a pool's size shapes the allocatable CPU and memory of its nodes, and any other size is
  shaped as `Standard_D2s_v5`. A pool created without a VM size reports none, as the
  `az aks nodepool list` output in Getting started shows, and its nodes are not shaped, whereas
  AKS picks a default size. Node capacity is that of your machine, and the GPU sizes in the catalog
  advertise a GPU that does not exist. Every node runs Linux on your machine's CPU architecture,
  including the nodes of a Windows node pool.
- **Most node pool settings have no effect**: Settings such as `maxPods`, the OS SKU, the OS disk
  options, FIPS, node public IP addresses, and the `maxSurge` and `maxUnavailable` upgrade settings
  are stored without changing the nodes. Others, such as pod subnets, custom kubelet or Linux OS
  configuration, and the drain timeout, node soak duration, and undrainable node behavior upgrade
  settings, are neither applied nor returned by the API. Spot priority is reflected only on the
  pool's scale set, so spot nodes are never evicted. `az aks nodepool stop` and
  `az aks nodepool start` leave the pool's nodes running, and a pool created with a node count of
  `0` starts with one node.
- **Run Command runs as cluster administrator**: The command pod runs with a cluster-admin service
  account rather than your Microsoft Entra identity, each cluster keeps only its latest 100
  results, and a command times out after the number of seconds set in
  `LS_AZURE_AKS_RUN_COMMAND_TIMEOUT` (600 by default).
- **Managed namespaces are not locked**: AKS blocks changes made through the Kubernetes API to a
  managed namespace and to its `defaultresourcequota` and `defaultnetworkpolicy` objects, while
  the emulator does not, so they stay editable with `kubectl`.
- **Some resources are stored only**: Maintenance configurations and the auto-upgrade channels
  never schedule or run an upgrade, and the pricing tier is always reported as `Free`. The emulator
  ignores the `creationData` of an agent pool: `az aks nodepool add --snapshot-id` still gets the
  snapshot's Kubernetes version, VM size, OS type, and OS SKU because the Azure CLI copies them into
  the request, but a template that sets only `creationData.sourceResourceId` gets the default
  configuration. Trusted access role bindings grant no permissions inside the cluster, and Azure
  Kubernetes Fleet Manager update runs do not upgrade member clusters.
- **Azure Files NFS volumes depend on the host**: The NFS backend image has no `linux/arm64` build,
  and its NFS server does not start on every Docker host, so an NFS share can be created and still
  fail to mount, for example on Apple Silicon.

## Cluster-creation scripts

The [aks-samples](https://github.com/localstack-samples/aks-samples) repository provides two
interchangeable scripts that provision a production-shaped cluster, complete with a virtual
network, a container registry, a Log Analytics workspace, and system and user node pools. Both
scripts run unchanged against real Azure and the emulator; they differ only in the cluster
identity:

| Script | Cluster identity | When to use |
| ------ | ---------------- | ----------- |
| [01-system-assigned-managed-identity.sh](https://github.com/localstack-samples/aks-samples/blob/main/scripts/01-system-assigned-managed-identity.sh) | System-assigned managed identity | Simplest option: Azure creates and manages the identity lifecycle together with the cluster. |
| [01-user-assigned-managed-identity.sh](https://github.com/localstack-samples/aks-samples/blob/main/scripts/01-user-assigned-managed-identity.sh) | User-assigned managed identity | Use when you need a stable, pre-created identity that can be reused across resources and granted role assignments ahead of time. |

Both scripts are idempotent and safe to re-run. The same folder also contains optional add-on
installers for [Prometheus](https://github.com/localstack-samples/aks-samples/blob/main/scripts/02-install-prometheus.sh),
the [NGINX ingress controller](https://github.com/localstack-samples/aks-samples/blob/main/scripts/03-install-nginx-ingress-controller.sh),
the [Gateway API CRDs](https://github.com/localstack-samples/aks-samples/blob/main/scripts/04-install-gateway-api.sh),
[NGINX Gateway Fabric](https://github.com/localstack-samples/aks-samples/blob/main/scripts/05-install-nginx-gateway-fabric.sh),
and [cert-manager](https://github.com/localstack-samples/aks-samples/blob/main/scripts/06-install-cert-manager.sh).

## Samples

Every sample deploys the same Vacation Planner single-page web application, available in two
versions: a Python [Flask](https://flask.palletsprojects.com/) app and an
[ASP.NET Core Razor Pages](https://learn.microsoft.com/en-us/aspnet/core/razor-pages/) app. Only
the data service, its provisioning, and the way the app authenticates to it change from one sample
to the next.

| Sample | Description | Languages |
| ------ | ----------- | --------- |
| web-app-sql-database | Stores activities in an Azure SQL Database, connecting with a SQL login over TDS. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-sql-database/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-sql-database/dotnet/README.md) |
| web-app-mysql-flexible-server | Stores activities in an Azure Database for MySQL flexible server. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-mysql-flexible-server/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-mysql-flexible-server/dotnet/README.md) |
| web-app-postgresql-flexible-server | Stores activities in an Azure Database for PostgreSQL flexible server. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-postgresql-flexible-server/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-postgresql-flexible-server/dotnet/README.md) |
| web-app-in-cluster-postgresql | Stores activities in an in-cluster PostgreSQL database deployed as a Kubernetes StatefulSet, with a primary and two streaming replicas. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-in-cluster-postgresql/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-in-cluster-postgresql/dotnet/README.md) |
| web-app-cosmosdb-mongodb-api | Stores activities in a collection of an Azure Cosmos DB for MongoDB account. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-cosmosdb-mongodb-api/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-cosmosdb-mongodb-api/dotnet/README.md) |
| web-app-cosmosdb-nosql-api | Stores activities in a container of an Azure Cosmos DB for NoSQL account. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-cosmosdb-nosql-api/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-cosmosdb-nosql-api/dotnet/README.md) |
| web-app-blob-storage | Stores activities in an Azure Blob Storage container, using a connection string. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-blob-storage/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-blob-storage/dotnet/README.md) |
| web-app-file-storage | Stores activities as text files on an Azure Files share mounted by the Azure Files CSI driver, over either SMB or NFS. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-file-storage/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-file-storage/dotnet/README.md) |
| web-app-managed-identity | Stores activities in Azure Blob Storage, authenticating with Microsoft Entra Workload ID instead of a secret, and optionally exposes the app through the Gateway API with a managed TLS certificate. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-managed-identity/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-managed-identity/dotnet/README.md) |
| web-app-app-configuration | Stores activities in an Azure Database for PostgreSQL flexible server, reading every connection setting from an Azure App Configuration store through the `Microsoft.AppConfiguration` cluster extension, with the credentials kept in Azure Key Vault as Key Vault references. | [Python](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-app-configuration/python/README.md), [.NET](https://github.com/localstack-samples/aks-samples/blob/main/samples/web-app-app-configuration/dotnet/README.md) |

## Tutorials

The same repository includes standalone tutorials that exercise individual AKS capabilities. Unlike
the samples, they do not deploy the web application:

| Tutorial | Description |
| -------- | ----------- |
| [policies](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/policies) | Kubernetes network policies that enforce zero-trust traffic control with Calico and Cilium: cluster-wide default-deny, DNS-aware egress, and L3/L4/L7 ingress. |
| [ccm](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/ccm) | Exercises the Azure cloud controller manager: public and internal `LoadBalancer` services, source ranges, the nodeIP backend-pool variant, and an NGINX ingress controller. |
| [gateway-api](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/gateway-api) | Enables the managed Gateway API CRDs, installs NGINX Gateway Fabric, and routes traffic to a backend through a `Gateway` and an `HTTPRoute`. |
| [keda](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda) | Event-driven autoscaling with the KEDA add-on: a producer creates a backlog on an Azure event source, and a `ScaledObject` scales a consumer from zero to four replicas and back. |
| [keda/service-bus](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda/service-bus) | Scales a consumer on an Azure Service Bus queue with the `azure-servicebus` scaler, authenticating with Microsoft Entra Workload ID. Start here if you are new to KEDA. |
| [keda/queue-storage](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda/queue-storage) | Scales a consumer on an Azure Storage queue with the `azure-queue` scaler. Workload identity is used end to end, so no data-plane secret exists anywhere. |
| [keda/event-hubs](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda/event-hubs) | Scales a consumer on an Azure Event Hubs hub with the `azure-eventhub` scaler, whose backlog is the distance between the last enqueued event and the consumer group's blob checkpoints. |
| [key-vault-csi-driver](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/key-vault-csi-driver) | Mounts secrets from Azure Key Vault into a pod with the Secrets Store CSI driver, in both the workload identity and the user-assigned managed identity access modes. |
| [terraform/tags-labels-taints](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/terraform/tags-labels-taints) | Deploys a modular, feature-rich AKS stack with Terraform, steering workloads across agent pools with Azure resource tags, node labels, and taints, then validates them through both the ARM and Kubernetes APIs. |
| [bicep/tags-labels-taints](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/bicep/tags-labels-taints) | The same modular AKS stack built with Bicep, with parameters and outputs that mirror the Terraform tutorial one-to-one. |

## API Coverage


### Azure Kubernetes Service (AKS) API coverage

Source service: `aks`. 77 of 126 tracked operations are implemented.

Service documentation: /azure/services/aks/

| Operation | Status |
| --- | --- |
| AIManagerNamespaces.CreateOrUpdate | Not implemented |
| AIManagerNamespaces.Delete | Not implemented |
| AIManagerNamespaces.Get | Not implemented |
| AIManagerNamespaces.ListAccessKeys | Not implemented |
| AIManagerNamespaces.ListByAiManager | Not implemented |
| AIManagerNamespaces.ListCredential | Not implemented |
| AIManagerNamespaces.RotateKeys | Not implemented |
| AIManagers.CreateOrUpdate | Not implemented |
| AIManagers.Delete | Not implemented |
| AIManagers.Get | Not implemented |
| AIManagers.ListByResourceGroup | Not implemented |
| AIManagers.ListBySubscription | Not implemented |
| AIManagers.ListCredential | Not implemented |
| AIManagers.Update | Not implemented |
| AIModels.CalculateCost | Not implemented |
| AIModels.Get | Not implemented |
| AIModels.List | Not implemented |
| AgentPools.AbortLatestOperation | Implemented |
| AgentPools.CreateOrUpdate | Implemented |
| AgentPools.Delete | Implemented |
| AgentPools.DeleteMachines | Implemented |
| AgentPools.Get | Implemented |
| AgentPools.GetAvailableAgentPoolVersions | Implemented |
| AgentPools.GetUpgradeProfile | Implemented |
| AgentPools.List | Implemented |
| AgentPools.UpgradeNodeImageVersion | Implemented |
| ContainerService.ListNodeImageVersions | Implemented |
| ContainerServices.ListOrchestrators | Not implemented |
| CustomAiModels.CalculateCost | Not implemented |
| CustomAiModels.CreateOrUpdate | Not implemented |
| CustomAiModels.Delete | Not implemented |
| CustomAiModels.Get | Not implemented |
| CustomAiModels.List | Not implemented |
| DeploymentSafeguards.Create | Implemented |
| DeploymentSafeguards.Delete | Implemented |
| DeploymentSafeguards.Get | Implemented |
| DeploymentSafeguards.List | Implemented |
| IdentityBindings.CreateOrUpdate | Implemented |
| IdentityBindings.Delete | Implemented |
| IdentityBindings.Get | Implemented |
| IdentityBindings.ListByManagedCluster | Implemented |
| Machines.CreateOrUpdate | Implemented |
| Machines.Get | Implemented |
| Machines.List | Implemented |
| MaintenanceConfigurations.CreateOrUpdate | Implemented |
| MaintenanceConfigurations.Delete | Implemented |
| MaintenanceConfigurations.Get | Implemented |
| MaintenanceConfigurations.ListByManagedCluster | Implemented |
| ManagedClusters.AbortLatestOperation | Implemented |
| ManagedClusters.CreateOrUpdate | Implemented |
| ManagedClusters.Delete | Implemented |
| ManagedClusters.Get | Implemented |
| ManagedClusters.GetAccessProfile | Implemented |
| ManagedClusters.GetAccessProfiles | Implemented |
| ManagedClusters.GetCommandResult | Implemented |
| ManagedClusters.GetMeshRevisionProfile | Implemented |
| ManagedClusters.GetMeshUpgradeProfile | Implemented |
| ManagedClusters.GetOsOptions | Implemented |
| ManagedClusters.GetUpgradeProfile | Implemented |
| ManagedClusters.List | Implemented |
| ManagedClusters.ListByResourceGroup | Implemented |
| ManagedClusters.ListClusterAdminCredentials | Implemented |
| ManagedClusters.ListClusterMonitoringUserCredentials | Implemented |
| ManagedClusters.ListClusterUserCredentials | Implemented |
| ManagedClusters.ListKubernetesVersions | Implemented |
| ManagedClusters.ListMeshRevisionProfiles | Implemented |
| ManagedClusters.ListMeshUpgradeProfiles | Implemented |
| ManagedClusters.ListOutboundNetworkDependenciesEndpoints | Implemented |
| ManagedClusters.ResetAadProfile | Implemented |
| ManagedClusters.ResetServicePrincipalProfile | Implemented |
| ManagedClusters.RotateClusterCertificates | Implemented |
| ManagedClusters.RotateServiceAccountSigningKeys | Implemented |
| ManagedClusters.RunCommand | Implemented |
| ManagedClusters.Start | Implemented |
| ManagedClusters.Stop | Implemented |
| ManagedClusters.UpdateTags | Implemented |
| ManagedNamespaces.CreateOrUpdate | Implemented |
| ManagedNamespaces.Delete | Implemented |
| ManagedNamespaces.Get | Implemented |
| ManagedNamespaces.ListByManagedCluster | Implemented |
| ManagedNamespaces.ListCredential | Implemented |
| ManagedNamespaces.Update | Implemented |
| ModelDeployments.CreateOrUpdate | Not implemented |
| ModelDeployments.Delete | Not implemented |
| ModelDeployments.Get | Not implemented |
| ModelDeployments.ListByAiManagerNamespace | Not implemented |
| ModelSources.CreateOrUpdate | Not implemented |
| ModelSources.Delete | Not implemented |
| ModelSources.Get | Not implemented |
| ModelSources.List | Not implemented |
| NodeCustomizations.CreateOrUpdate | Not implemented |
| NodeCustomizations.Delete | Not implemented |
| NodeCustomizations.DeleteVersion | Not implemented |
| NodeCustomizations.Get | Not implemented |
| NodeCustomizations.GetVersion | Not implemented |
| NodeCustomizations.ListByResourceGroup | Not implemented |
| NodeCustomizations.ListBySubscription | Not implemented |
| NodeCustomizations.ListVersions | Not implemented |
| NodeCustomizations.Update | Not implemented |
| Operations.List | Implemented |
| PreparedImageSpecifications.CreateOrUpdate | Not implemented |
| PreparedImageSpecifications.Delete | Not implemented |
| PreparedImageSpecifications.DeleteVersion | Not implemented |
| PreparedImageSpecifications.Get | Not implemented |
| PreparedImageSpecifications.GetVersion | Not implemented |
| PreparedImageSpecifications.ListByResourceGroup | Not implemented |
| PreparedImageSpecifications.ListBySubscription | Not implemented |
| PreparedImageSpecifications.ListVersions | Not implemented |
| PreparedImageSpecifications.Update | Not implemented |
| PrivateEndpointConnections.Delete | Implemented |
| PrivateEndpointConnections.Get | Implemented |
| PrivateEndpointConnections.List | Implemented |
| PrivateEndpointConnections.Update | Implemented |
| PrivateLinkResources.List | Implemented |
| ResolvePrivateLinkServiceId.Post | Implemented |
| Snapshots.CreateOrUpdate | Implemented |
| Snapshots.Delete | Implemented |
| Snapshots.Get | Implemented |
| Snapshots.List | Implemented |
| Snapshots.ListByResourceGroup | Implemented |
| Snapshots.UpdateTags | Implemented |
| TrustedAccessRoleBindings.CreateOrUpdate | Implemented |
| TrustedAccessRoleBindings.Delete | Implemented |
| TrustedAccessRoleBindings.Get | Implemented |
| TrustedAccessRoleBindings.List | Implemented |
| TrustedAccessRoles.List | Implemented |