Skip to main content

Azure Kubernetes Service (AKS)

AKS is Azure’s managed Kubernetes service. Master AKS to deploy modern, cloud-native applications at scale. Azure AKS Architecture

What You’ll Learn

By the end of this chapter, you’ll understand:
  • What containers are and why they exist
  • Why Kubernetes is needed for managing containers
  • How AKS simplifies Kubernetes management
  • How to deploy and scale applications with containers
  • When to use containers vs VMs vs serverless

Introduction: What Are Containers? (Start Here if You’re Completely New)

The Problem Containers Solve

Have you ever heard this?
“But it works on my machine!” 😤
This is one of software development’s biggest headaches. Let me explain why:

Before Containers: The “Works on My Machine” Problem

Scenario: You built an amazing web application. Your development laptop:
  • Node.js version 18.15.0
  • Python 3.10
  • PostgreSQL 14
  • Ubuntu 22.04
  • 16 GB RAM
Your colleague’s laptop:
  • Node.js version 16.14.0 ← Different!
  • Python 3.9 ← Different!
  • PostgreSQL 13 ← Different!
  • Windows 11 ← Different OS!
  • 8 GB RAM ← Different!
What happens?
The root cause: Your app depends on specific versions of libraries, tools, and the operating system. When any of these change, things break.

Real-World Analogy: Shipping Containers

Think about physical shipping containers: Before Shipping Containers (1950s):
  • Pack goods in boxes, crates, barrels
  • Different ships need different loading methods
  • Goods get damaged during transfer
  • Loading/unloading takes weeks
  • Ship → Train → Truck = repack everything each time
After Shipping Containers:
  • Standard 20-foot or 40-foot metal boxes
  • Works on ships, trains, trucks
  • Contents protected and isolated
  • Loading/unloading takes hours, not weeks
  • Ship → Train → Truck = same container, no repacking
Software containers work the same way:
  • Package your app + all dependencies in one “box”
  • Runs identically on any computer with Docker
  • Your laptop → Colleague’s laptop → Production server = same container

What is a Container?

Container = A lightweight, standalone package that includes:
  • Your application code
  • Runtime (Node.js, Python, Java, etc.)
  • Libraries and dependencies
  • Configuration files
  • Operating system files (just what your app needs)
Think of it as: A fully-furnished apartment in a box. Everything your app needs to run is inside.

Container Example: Blog Website

Without Containers (Traditional Setup):
With Containers (Modern Setup):

Containers vs Virtual Machines

Visual Comparison:
Example:
  • VM: You want to run Windows software on a Mac → Use VM
  • Container: You want to deploy 100 copies of your web app → Use containers

Why Use Containers?

1. Consistency:
2. Speed:
3. Efficiency:
4. Portability:
5. Isolation:

Real-World Example: E-Commerce Website

Traditional Deployment (No Containers):
Container Deployment:

Common Mistakes Beginners Make

Mistake 1: Thinking containers are just lightweight VMs ✅ Reality: Containers share the host OS kernel, VMs don’t Mistake 2: Storing data inside containers ✅ Reality: Containers are ephemeral (temporary). Use volumes for persistent data. Mistake 3: Running multiple apps in one container ✅ Reality: One container = one process (web server OR database, not both). Think of it like the single responsibility principle in software design — each container does one thing well. If you need a web server and a database, run two containers. This lets them scale independently (you might need 10 web containers but only 1 database container). Mistake 4: Using containers for everything ✅ Reality: Sometimes VMs are better (need different OS, strong isolation)

When to Use Containers vs VMs

Use Containers When: ✅ You want fast deployment (seconds) ✅ You need to run many copies of the same app ✅ You want consistent environments (dev = production) ✅ Your app runs on Linux Use VMs When: ✅ You need complete isolation (security, compliance) ✅ You need different operating systems (Windows + Linux on same hardware) ✅ You have legacy apps that can’t be containerized ✅ You need full control over the operating system

What is Kubernetes? (The Next Step After Containers)

The Problem Kubernetes Solves

You’ve learned containers solve the “works on my machine” problem. But… Scenario: Your blog got popular! 🎉 Month 1:
  • 100 visitors/day
  • 1 container handles it easily
  • Cost: $10/month
Month 6:
  • 50,000 visitors/day
  • Need 20 containers to handle traffic
  • Multiple servers needed
New problems arise:
  1. Which server should run which container?
    • Server 1 has 20 GB RAM free, Server 2 has 4 GB free
    • Manual placement = nightmare
  2. What if a container crashes?
    • Who restarts it? How do you know it crashed?
    • Manual monitoring = 24/7 job
  3. How do users reach the right container?
    • 20 containers, each with different IP address
    • Users need one URL: myblog.com
  4. How to update without downtime?
    • Stop all 20 containers = website down
    • Update one-by-one manually = takes hours, error-prone
  5. How to handle traffic spikes?
    • Black Friday: need 50 containers
    • Tuesday at 3am: need 5 containers
    • Manual scaling = expensive or too slow
Kubernetes solves ALL these problems automatically.

Real-World Analogy: Shipping Port

Without Kubernetes (Manual Container Management):
With Kubernetes (Automated):

What is Kubernetes?

Kubernetes (K8s) = An open-source container orchestration platform that automates deployment, scaling, and management of containerized applications. Think of Kubernetes as: An operating system for your containers across many servers. Core Features:
  1. Self-Healing: Container crashed? Kubernetes restarts it automatically
  2. Load Balancing: Distributes traffic across containers
  3. Auto-Scaling: More traffic? Kubernetes adds containers. Less traffic? Removes them.
  4. Rolling Updates: Update app without downtime
  5. Service Discovery: Containers find each other automatically
  6. Storage Orchestration: Attach storage to containers automatically

Kubernetes in Simple Terms

Transportation Analogy:

Real Example: Online Shopping Website

Black Friday Sale (No Kubernetes):
Black Friday Sale (With Kubernetes):

What is Azure Kubernetes Service (AKS)?

Plain Kubernetes (DIY):
Azure Kubernetes Service (AKS) (Managed):
AKS = Kubernetes without the operational headache

Under the Hood: The AKS Control Plane

In AKS, the cluster is split into two distinct lives:

1. The Control Plane (Managed by Azure)

You never see these VMs, but they are there. They run:
  • kube-apiserver: The front door. Every kubectl command hits this.
  • etcd: The source of truth. A distributed database that stores the cluster state.
  • kube-scheduler: Decides which node should run your pod based on resources.
  • kube-controller-manager: Watches for deviations (e.g., “I need 3 pods, but only 2 are running”) and fixes them.

2. The Data Plane (Your Worker Nodes)

These are the VMs in your subscription. They run:
  • kubelet: The agent that takes orders from the control plane and starts containers.
  • kube-proxy: Handles networking and load balancing between pods.
  • Container Runtime: Usually containerd (Docker’s core).
[!IMPORTANT] Pro Insight: The ‘Free’ Control Plane In the standard AKS tier, the control plane is free. However, if you have a massive cluster (100+ nodes), you should upgrade to the Uptime SLA tier (0.10/hour= 0.10/hour = ~73/month). This gives you a guaranteed 99.95% availability for the API server itself, backed by financially-backed credits. Without this tier, the control plane has no SLA — meaning Microsoft makes no guarantees about API server uptime. For production workloads, this $73/month is cheap insurance: if kubectl commands fail during an incident because the API server is down, you cannot scale, deploy, or debug your cluster.
Practical Tip: AKS Node Pool Strategy for Cost Optimization Most teams overspend on AKS by using a single, oversized node pool. A better approach:
This three-pool strategy can save 40-60% compared to a single pool sized for peak traffic.

Cost Comparison Example

Running 20 Containers for a Web App: Option 1: Traditional VMs (No Containers):
Option 2: Plain Kubernetes (DIY):
Option 3: Azure Kubernetes Service (AKS):
Winner: AKS saves $150/month + hundreds of hours of management time

1. Why Kubernetes?

Before Kubernetes

  • Manual container orchestration
  • No auto-scaling
  • Complex networking
  • Manual load balancing
  • No self-healing

With Kubernetes

  • Automated orchestration
  • Auto-scaling (HPA, VPA, Cluster Autoscaler)
  • Service discovery
  • Built-in load balancing
  • Self-healing (restart failed pods)
[!WARNING] Gotcha: System Node Pools Every AKS cluster needs at least one “System Node Pool” to run Kubernetes itself (CoreDNS, Metrics Server). You cannot delete this pool or scale it to 0. It will always cost you money (usually 1-3 VMs).
[!TIP] Jargon Alert: Pod vs Node Node: A Virtual Machine (The house). Pod: A running process/container (The tenant living in the house). A single Node (VM) usually hosts many Pods.

2. AKS Architecture


3. Create AKS Cluster


4. AKS Networking


5. Deploy Application


6. Autoscaling


7. Best Practices

Resource Limits

Always set CPU/memory requests and limits to prevent noisy neighbors

Health Checks

Configure liveness and readiness probes for self-healing

Use Namespaces

Separate environments (dev, staging, prod) with namespaces

Security

Use Azure AD pod identity, network policies, and Pod Security Standards

Monitoring

Enable Container Insights for observability

GitOps

Use Flux or ArgoCD for declarative deployments

8. Interview Questions

Beginner Level

Answer:
  • Node: A worker machine (VM) in Kubernetes. It runs pods.
  • Pod: The smallest deployable unit. Usually contains one container (but can have sidecars).
Analogy: Node = House, Pod = Room, Container = Person in the room.
Answer:
  • ClusterIP: Internal IP only. Not accessible from outside. Default type.
  • NodePort: Exposes service on a static port on each Node IP.
  • LoadBalancer: Provisions an external Azure Load Balancer to expose service publicly.
Answer:
  • Scheduling pods (kube-scheduler)
  • Detecting and responding to cluster events (kube-controller-manager)
  • Storing cluster state (etcd)
  • Exposing the Kubernetes API (kube-apiserver)
Note: In AKS, Azure manages the control plane for you (free).

Intermediate Level

Answer:
  • Load Balancer: Layer 4 (TCP/UDP). One IP per service. Expensive for many services.
  • Ingress Controller: Layer 7 (HTTP/HTTPS). Single IP for multiple services. Supports path-based routing (/api, /web), SSL termination, and rewriting.
Answer:
  1. The Kubelet on the node detects the crash.
  2. Based on restartPolicy (default: Always), it restarts the container.
  3. If the pod is part of a Deployment/ReplicaSet, if the Node dies, the Scheduler creates a new Pod on a healthy Node.

Advanced Level

Answer: AKS handles this via Surge Upgrades:
  1. Cordon a node (prevent new pods).
  2. Drain the node (move existing pods to other nodes).
  3. Delete the node.
  4. Create a new node with the updated version.
  5. Repeat for all nodes (one by one or in batches).
Requirement: PodDisruptionBudgets must be configured to ensure minAvailable replicas during the process.
Answer: A helper container running alongside the main application container in the same Pod.Uses:
  • Logging (sending logs to Splunk/Log Analytics)
  • Proxying (Service Mesh like Istio/Linkerd)
  • Config watching (reloading configuration)
  • Security (TLS termination)

9. Helm: Kubernetes Package Manager

Helm Architecture - Kubernetes Package Manager
Helm is the package manager for Kubernetes. It simplifies deploying complex applications with reusable charts.

Why Helm?

Without Helm:
  • Manage 20+ YAML files manually
  • Copy-paste configurations for dev/staging/prod
  • Hard to version and rollback deployments
With Helm:
  • Single command deployment: helm install myapp ./chart
  • Templated configurations with values
  • Easy rollbacks: helm rollback myapp 1
  • Reusable charts from public repositories

Helm Architecture

Creating a Helm Chart

Chart.yaml (Metadata)

values.yaml (Configuration)

templates/deployment.yaml (Templated Manifest)

Deploying with Helm

Helm Repositories

Multi-Environment Strategy

values-dev.yaml:
values-prod.yaml:
[!TIP] Best Practice: Chart Versioning
  • Chart version (version in Chart.yaml): Increment when chart structure changes
  • App version (appVersion): Tracks the application version being deployed
  • Use semantic versioning: 1.2.3 (MAJOR.MINOR.PATCH)
[!WARNING] Gotcha: Helm Secrets Never commit secrets to values.yaml! Use:
  • Azure Key Vault: Inject secrets via CSI driver
  • Sealed Secrets: Encrypt secrets in Git
  • helm-secrets plugin: Encrypt values files with SOPS

10. GitOps with ArgoCD

GitOps with ArgoCD Workflow
GitOps = Git as the single source of truth for declarative infrastructure and applications.

GitOps Principles

  1. Declarative: Entire system described declaratively (YAML in Git)
  2. Versioned: Git history = deployment history
  3. Automated: Changes in Git automatically deployed
  4. Reconciled: Cluster state continuously reconciled with Git

ArgoCD Architecture

Installing ArgoCD on AKS

Creating an Application

Git Repository Structure:
ArgoCD Application Manifest:

GitOps Workflow

[!IMPORTANT] Recommendation: Separate Repos
  • Application code repo: Source code, Dockerfile
  • GitOps repo: Kubernetes manifests, Helm charts
  • CI updates GitOps repo after building image

Sync Strategies


11. Service Mesh Basics (Istio)

Istio Service Mesh Architecture
Service Mesh = Infrastructure layer for service-to-service communication with observability, security, and traffic management.

Why Service Mesh?

Without Service Mesh:
  • Implement retries, timeouts, circuit breakers in every microservice
  • No visibility into service-to-service traffic
  • Difficult to enforce mTLS between services
With Service Mesh (Istio):
  • Traffic Management: Canary deployments, A/B testing, retries
  • Security: Automatic mTLS between services
  • Observability: Distributed tracing, metrics, logs

Istio Architecture

Installing Istio on AKS

Traffic Management Example

Canary Deployment (90% v1, 10% v2):
[!NOTE] Deep Dive: When to Use Service Mesh?
  • YES: Microservices (10+ services), need mTLS, complex traffic routing
  • NO: Monolith, simple apps, small teams (adds complexity)


13. AKS Security Deep Dive

Pod Security Standards

Pod Security Standards replace deprecated Pod Security Policies (PSPs). Three Levels:
  1. Privileged: Unrestricted (no restrictions)
  2. Baseline: Minimally restrictive (prevents known privilege escalations)
  3. Restricted: Heavily restricted (hardened, follows pod hardening best practices)
Example: Restricted Pod:

Network Policies

Network Policies = Firewall rules for pods.
Default Deny All:

Secrets Management with Azure Key Vault

CSI Driver for Azure Key Vault:
SecretProviderClass:
Pod using Key Vault secret:

14. StatefulSets & Persistent Storage

StatefulSet = For stateful applications (databases, message queues) that need stable network identity and persistent storage.

StatefulSet vs Deployment

StatefulSet Example

Accessing pods:

Azure Disk vs Azure Files


15. KEDA: Event-Driven Autoscaling

KEDA Event-Driven Autoscaling
KEDA (Kubernetes Event-Driven Autoscaling) = Scale pods based on external metrics (queue length, HTTP requests, database queries).

Installing KEDA

Example: Scale Based on Azure Service Bus Queue

Deployment:
How it works:
  • Queue has 50 messages → KEDA scales to 5 pods (50/10)
  • Queue has 200 messages → KEDA scales to 20 pods (max)
  • Queue empty → KEDA scales to 1 pod (min)
  • Azure Service Bus: Queue/Topic message count
  • Azure Storage Queue: Queue length
  • HTTP: Incoming HTTP requests
  • Prometheus: Custom metrics
  • Kafka: Consumer lag
  • Redis: List length
  • Cron: Time-based scaling

16. Interview Questions

Beginner Level

Answer:Pod:
  • Smallest deployable unit in Kubernetes
  • One or more containers running together
  • Ephemeral (dies when node fails)
  • No self-healing
Deployment:
  • Manages a set of identical Pods (ReplicaSet)
  • Ensures desired number of Pods are running
  • Self-healing (recreates failed Pods)
  • Supports rolling updates and rollbacks
In production: Always use Deployments, never bare Pods.
Answer:Namespaces = Virtual clusters within a physical cluster.Use cases:
  • Environment separation: dev, staging, prod
  • Team isolation: team-a, team-b
  • Resource quotas: Limit CPU/memory per namespace
Default namespaces:
  • default: Default namespace for resources
  • kube-system: Kubernetes system components
  • kube-public: Public resources (readable by all)
Example:
Answer:Service = Stable network endpoint for a set of Pods.Problem: Pods have dynamic IPs (change on restart) Solution: Service provides a stable IP and DNS nameTypes:
  • ClusterIP (default): Internal only (10.0.1.5)
  • NodePort: Exposes on each node’s IP (30000-32767)
  • LoadBalancer: Creates Azure Load Balancer (public IP)
Example:

Intermediate Level

Answer:HPA = Automatically scales pods based on CPU/memory usage.How it works:
  1. Metrics Server collects pod metrics every 15 seconds
  2. HPA controller checks metrics every 30 seconds
  3. If avg CPU > target, scale up
  4. If avg CPU < target (for 5 min), scale down
Formula:
Example:
  • Current: 3 pods, avg CPU 80%
  • Target: 50%
  • Desired: ceil(3 * (80/50)) = ceil(4.8) = 5 pods
Gotcha: Requires resources.requests to be set!
Answer:Recommendation: Use Azure CNI for production (better integration, performance).
Answer:Strategy: Rolling Update with readiness probes
Process:
  1. Create 1 new pod (v2)
  2. Wait for readiness probe to pass
  3. Terminate 1 old pod (v1)
  4. Repeat until all pods are v2
Result: Always 4-6 pods running (never less than 4).

Advanced Level

Answer:Requirements:
  • Isolate tenants (security, resources)
  • Cost allocation per tenant
  • Prevent noisy neighbor
Architecture:Option 1: Namespace per Tenant (Soft Isolation)
Option 2: Node Pool per Tenant (Hard Isolation)
Deployment with node affinity:
Cost Allocation: Use tags/labels + Azure Cost Management.
Answer:CrashLoopBackOff = Pod starts, crashes, Kubernetes restarts it, crashes again (loop).Troubleshooting Steps:
  1. Check pod events:
  1. Check logs:
  1. Common causes:
  • OOMKilled: Increase resources.limits.memory
  • Application error: Fix code, check environment variables
  • Missing dependencies: Database not ready → Add init container
  • Liveness probe failing: Adjust probe settings
  1. Debug with ephemeral container (Kubernetes 1.23+):
  1. Disable probes temporarily:
Answer:Blue-Green = Run two identical environments (blue=current, green=new), switch traffic instantly.Implementation with Services:
Cutover Process:
Pros: Instant rollback, zero downtime Cons: 2x resources during deployment

Troubleshooting: The AKS Production Triage

When a pod fails in production, don’t panic. Follow this 3-step triage:

1. The “Pods won’t start” Phase

  • ImagePullBackOff: Kubernetes can’t download your container image.
    • The Pro Check: Does the AKS Cluster have the AcrPull permission on your Container Registry?
  • CrashLoopBackOff: The container starts but immediately crashes.
    • The Pro Check: Run kubectl logs <pod-name> --previous. You need to see the logs from the failed instance, not the new one that just restarted.
  • Pending: The pod isn’t even trying to start.
    • The Pro Check: Run kubectl describe pod <pod-name>. Usually, it’s because you requested 2 GB of RAM but your nodes only have 1 GB available.

2. The “Network Ghost” Phase

  • Service but no Response: The service is running, but you get a 504 timeout.
    • The Pro Check: Do the selectors in your Service YAML exactly match the labels in your Deployment YAML? If not, the Load Balancer is sending traffic into a black hole.

3. The “Node Pressure” Phase

  • Evicted Pods: Your pods are being killed randomly.
    • The Pro Check: Your Node is out of disk space or RAM. Check “Azure Monitor for Containers” to see which app is leaking memory.
[!TIP] Pro Tool: Lens & k9s While kubectl is the standard, Principal Engineers often use Lens (Desktop UI) or k9s (Terminal UI) to visualize cluster health in real-time. These tools make it instantly obvious when a deployment is failing across multiple zones.

17. Key Takeaways

Managed Control Plane

AKS manages the master nodes (API server, etcd) for free. You only pay for worker nodes.

Declarative Config

Use YAML manifests to define desired state. Avoid imperative commands (kubectl run) in production.

Autoscaling

Use HPA for pods (CPU/Memory) and Cluster Autoscaler for nodes to handle variable loads efficiently.

Networking Choice

Use Kubenet for simplicity/IP conservation. Use Azure CNI for distinct IPs per pod and direct VNet connectivity.

Security

Integrate Azure AD for authentication. Use Network Policies to restrict traffic between pods.

Namespace Isolation

Use namespaces to logically separate teams, environments (dev/prod), or applications within a cluster.

Interview Deep-Dive

Strong Candidate Answer:
  • The cascade mechanism: Service B calls A synchronously. When A returns 500s, B retries aggressively, amplifying load on A. B’s response time increases as it waits for A’s timeouts, exhausting B’s connection pool. Service C, calling B, experiences the same cascade. Within 60 seconds, all three services are down.
  • Prevention 1 — Circuit Breaker: After 5 consecutive failures from A, the circuit opens and B returns a fallback response (cached data, degraded response) instead of waiting. Use Istio destination rules or application-level libraries like Polly.
  • Prevention 2 — Aggressive timeouts: Set 2-3 second timeouts for internal calls (not 30-second defaults). Configure retries with exponential backoff and jitter, limited to 3 attempts. Prevents retry storms.
  • Prevention 3 — Bulkhead pattern: Separate connection pools per downstream dependency. If the pool for A is exhausted, calls to other services continue unaffected.
  • Prevention 4 — Async communication: If B does not need synchronous response from A, switch to Service Bus messaging. B publishes “process payment” and immediately returns. A processes when recovered.
Follow-up: How do you test that circuit breakers work before a real incident?Chaos engineering with Chaos Mesh or Azure Chaos Studio. Inject 5-second latency on A, return 500 errors for 50% of requests, or kill A’s pods. Verify B’s circuit breaker opens and C remains healthy. Run in staging first, then production during low-traffic windows. The first chaos experiment always reveals misconfigured circuit breakers — discovering that in a test is worth 100x more than during a real incident.
Strong Candidate Answer:
  • ACI: Single container, no orchestration, pay per second. Best for batch jobs, build agents, burst capacity. ~$35/month for 1 vCPU 24/7. Not suitable for production web services.
  • Container Apps (ACA): Serverless containers on managed Kubernetes (KEDA + Envoy + Dapr). Auto-scales to zero. Built-in Dapr for service-to-service calls. Best for 3-10 engineer teams wanting container benefits without Kubernetes overhead.
  • AKS: Full Kubernetes control — networking, node pools, admission controllers, service mesh, GPU scheduling. Best for 20+ engineer orgs, complex architectures, multi-cloud portability, or custom operators.
  • Decision: Start with Container Apps unless you need a specific Kubernetes feature. Migrate to AKS when you outgrow it (custom networking, Windows containers, GPU nodes).
Follow-up: Your team has 15 microservices on AKS with 3 engineers. On-call burden is heavy. Should you migrate to Container Apps?This is the Container Apps sweet spot. Each engineer manages 5 services plus the Kubernetes platform. Container Apps eliminates node management, cluster upgrades, and ingress controller configuration. Migration is 2-4 weeks: convert K8s manifests to Container Apps YAML (mostly 1:1 mapping), use Dapr for service communication.
Strong Candidate Answer:
  • kubenet for this scenario. With a /24 subnet and Azure CNI, each node reserves 30 IPs by default. You can fit only 8 nodes (8 x 30 = 240 IPs). With kubenet, only nodes consume VNet IPs, so 251 addresses support 200+ nodes with thousands of pods on an overlay network.
  • Azure CNI is better when: Pods need direct VNet addressability (hybrid/ExpressRoute scenarios), Windows containers are needed, or Azure Network Policy is required. But it demands a /20 or larger subnet for production clusters.
  • The newer option — Azure CNI Overlay: Pods get overlay IPs (saving VNet space) but can use Azure Network Policy. My default recommendation for new deployments as it combines kubenet’s IP efficiency with CNI’s policy features.
Follow-up: The security team wants pod-to-pod encryption and all egress through a corporate proxy. How do you implement this?Service mesh (Istio/Linkerd) with mutual TLS for pod-to-pod encryption — zero application code changes. For egress, deploy Azure Firewall and add a UDR on the AKS subnet pointing 0.0.0.0/0 to the firewall. Use Kubernetes NetworkPolicies to deny direct internet access from pods. This gives security a single inspection point with full URL logging.

Next Steps

Continue to Chapter 8

Master Azure Functions and serverless event-driven architecture