​

Traffic Management & Network Security

​

What You’ll Learn

• How load balancers work (and why Azure has 4 different ones)
• When to use each load balancer (Layer 4 vs Layer 7, Regional vs Global)
• Real costs and performance trade-offs between load balancing options
• How to prevent outages with health probes and connection draining
• Common mistakes that cause production failures

​

Introduction: What is Load Balancing?

​

Start Here if You’re Completely New

• 100 users visit → Server handles it fine ✅
• 10,000 users visit → Server slows down ⚠️
• 100,000 users visit → Server crashes ❌

Before (Single Server):
100,000 users → [Server 1] → CRASH! ❌

After (Load Balancer):
100,000 users → [Load Balancer] → [Server 1] 33,333 users ✅
                                 → [Server 2] 33,333 users ✅
                                 → [Server 3] 33,334 users ✅

​

Real-World Analogy: Restaurant Hostess

• Customers walk in, sit anywhere
• One table gets 10 people (overcrowded)
• Other tables are empty
• Bad customer experience

• Hostess greets customers
• Assigns them to available tables evenly
• All tables equally busy
• Great customer experience

​

Why This Matters: Real Cost of Getting It Wrong

• The Setup: Used wrong type of load balancer
• The Problem: Load balancer couldn’t handle HTTP traffic properly
• The Incident: Website down for 4 hours during peak shopping
• The Cost: $440M in lost sales (that day alone)
• The Fix: Migrated to proper Layer 7 load balancer
• The Lesson: Choosing wrong load balancer cost $440M

​

1. Load Balancing Decision Tree

​

Understanding Azure’s 4 Load Balancers (From Absolute Zero)

​

Layer 4 vs Layer 7 (Explained Simply)

• Sees: IP address and port number only
• Example: “Send packet to 10.0.0.5 port 80”
• Doesn’t know: What’s in the packet (HTTP? SQL? Video?)
• Speed: Extremely fast (<1ms latency)
• Analogy: Mail carrier who only reads the address on envelope

• Sees: HTTP headers, URL paths, cookies, everything
• Example: “Send /api requests to Server A, /images to Server B”
• Knows: Content type, can inspect and modify
• Speed: Slower (3-20ms latency, must parse HTTP)
• Analogy: Mail carrier who opens mail, reads it, decides where it should go

[!TIP] Jargon Alert: Layer 4 vs Layer 7 Layer 4 (Transport): Knows IP and Port. “Send packet to 10.0.0.5:80”. (Dumb, fast pipe). Layer 7 (Application): Knows URL, Cookies, Headers. “Send /api to Service A and /images to Service B”. (Smart, CPU intensive).

• ✅ Need maximum speed (latency < 1ms)
• ✅ Non-HTTP traffic (databases, game servers)
• ✅ Don’t need to inspect content

• ✅ Need smart routing (/api → different server than /images)
• ✅ Need SSL termination (decrypt HTTPS once, not on every server)
• ✅ Need Web Application Firewall (WAF) protection
• ✅ HTTP/HTTPS traffic only

​

Global vs Regional Load Balancers (Simplified)

• Example: 3 servers in East US datacenter

• Example: Servers in 3 continents

​

Azure’s 4 Load Balancers Explained

START: Which load balancer do I need?

├─ Is your app in multiple regions (global)?
│  ├─ YES: Is it HTTP/HTTPS traffic?
│  │  ├─ YES → Azure Front Door ($35/month)
│  │  └─ NO → Traffic Manager ($1.35/M queries)
│  │
│  └─ NO (single region): Is it HTTP/HTTPS traffic?
│     ├─ YES: Do you need WAF (Web Application Firewall)?
│     │  ├─ YES → Application Gateway ($125/month)
│     │  └─ NO → Azure Load Balancer ($18/month)
│     │
│     └─ NO (TCP/UDP, databases) → Azure Load Balancer ($18/month)

​

Global (Multi-Region) vs Regional (Detailed)

​

Deep Dive: Load Balancer Comparison

​

Understanding Key Features (Explained Simply)

• Why It Matters: Saves CPU on your servers (encryption is expensive)
• Example: 1,000 HTTPS requests → Load balancer decrypts once → Servers get plain HTTP
• Cost Savings: 20-30% less CPU usage on servers

• Example: /api → API servers, /images → Image servers, /admin → Admin servers
• Why It Matters: Optimize server resources for specific tasks
• Analogy: Restaurant with different stations (grill, salad bar, dessert)

• Blocks: SQL injection, XSS (cross-site scripting), DDoS attacks
• Example: Hacker sends https://yoursite.com/api?id=1' OR '1'='1 → WAF blocks it
• Real Cost: Equifax breach cost $4B, could have been prevented with WAF

• Problem: User logs in to Server A, next request goes to Server B (session lost!)
• Solution: “Pin” user to Server A for entire session
• Better Solution: Use Redis for shared sessions (no sticky sessions needed)

[!WARNING] Gotcha: Traffic Manager Isn’t a Load Balancer Traffic Manager is a DNS service. It returns an IP address to the client, then the client connects directly to that backend. If the backend goes down after DNS resolution, Traffic Manager won’t reroute traffic until the next DNS lookup (60+ seconds later). Use it for coarse-grained multi-region failover, not for real-time load balancing. Visual Example:

Copy

User requests: www.mysite.com

Traffic Manager Response:
"www.mysite.com = 20.50.100.5 (valid for 60 seconds)"

User connects directly to 20.50.100.5

If 20.50.100.5 goes down after 10 seconds:
- User still tries to connect to 20.50.100.5 ❌
- Traffic Manager can't help (DNS already resolved)
- User waits 50 more seconds until DNS TTL expires

• Team deploys global app
• Uses Traffic Manager for failover
• Region goes down
• Problem: Users stuck on dead region for 60+ seconds (DNS TTL)
• Impact: Bad user experience, lost revenue

• Use Azure Front Door ($35/month)
• Failover in <10 seconds (no DNS caching)
• Cost: $35/month vs$1.35/M queries (similar price for most apps)

​

Session Affinity (Sticky Sessions) Explained

1. Step 1: You visit website → Load balancer sends you to Server A
2. Step 2: You log in → Server A stores “You are logged in” in memory
3. Step 3: You add item to cart → Load balancer sends you to Server B
4. Result: Server B doesn’t know you’re logged in → “401 Unauthorized” error ❌

Request 1: User → Load Balancer → Server A (login saved)
Request 2: User → Load Balancer → Server B (who are you?) ❌

Request 1: User → Load Balancer → Server A (login saved)
Request 2: User → Load Balancer → Server A (logged in!) ✅
Request 3: User → Load Balancer → Server A (still logged in!) ✅

​

Method 1: Azure Load Balancer (5-Tuple Hash)

Hash(SourceIP, SourcePort, DestIP, DestPort, Protocol) → Backend Server

• Load balancer looks at your IP address + port
• Creates a “fingerprint” (hash)
• Always sends same fingerprint to same server

Your IP: 203.0.113.50
Your Port: 54321
Hash Result: abc123
abc123 always goes to → Server A

• ✅ Works for any protocol (TCP, UDP, HTTP)
• ✅ Very fast (no cookies to parse)

• ❌ If your IP changes (mobile switching cell towers), you get routed to different server
• ❌ If you’re behind NAT (corporate network), everyone shares same IP

​

Method 2: Application Gateway / Front Door (Cookie-Based)

1. First request → Load balancer picks Server A
2. Response includes cookie: Set-Cookie: ApplicationGatewayAffinity=abc123
3. Future requests → Browser sends cookie → Load balancer reads it → Routes to Server A

First Request:
GET /cart
→ Load Balancer picks Server A

Response:
HTTP/1.1 200 OK
Set-Cookie: ApplicationGatewayAffinity=ServerA-abc123; Path=/; HttpOnly

Second Request:
GET /checkout
Cookie: ApplicationGatewayAffinity=ServerA-abc123
→ Load Balancer reads cookie → Routes to Server A ✅

• ✅ Survives IP changes (mobile networks, VPN switches)
• ✅ More accurate than IP-based

• ❌ If user clears cookies, session is lost
• ❌ Slightly slower (must parse HTTP headers)

[!TIP] Best Practice: Use Redis or Azure App Service Distributed Cache for session state, so sticky sessions aren’t required. This allows horizontal scaling without session loss. Why Shared Session Storage is Better:

Copy

Without Shared Storage (needs sticky sessions):
Request 1: User → Server A (session in memory)
Request 2: User → Server B (session lost!) ❌

With Shared Storage (no sticky sessions needed):
Request 1: User → Server A → Save session in Redis
Request 2: User → Server B → Read session from Redis ✅

Benefits:
- Server crashes → Session survives in Redis
- True load balancing (any server can handle any request)
- Horizontal scaling without limits

• App stores sessions in server memory
• Uses sticky sessions
• Server crashes → All sessions on that server lost
• Impact: Users forced to log in again

• E-commerce site during Black Friday
• Server crash lost 10,000 active sessions
• Users had to re-add items to cart
• 70% abandoned their carts
• Cost: $2.1M in lost sales

• Migrate sessions to Redis ($20/month)
• No sticky sessions needed
• Server crashes don’t lose sessions

​

Health Probes: Keeping Dead Servers Out of Rotation

• Server A: Running fine ✅
• Server B: Running fine ✅
• Server C: Crashed (out of memory) ❌

Load balancer sends traffic:
→ 33% to Server A ✅
→ 33% to Server B ✅
→ 33% to Server C ❌ (fails, users get errors!)

Result: 33% of users see errors!

Load balancer checks all servers every 15 seconds:
- Server A responds → Healthy ✅
- Server B responds → Healthy ✅
- Server C doesn't respond → Unhealthy ❌

Load balancer sends traffic:
→ 50% to Server A ✅
→ 50% to Server B ✅
→ 0% to Server C (removed from rotation!)

Result: 0% of users see errors!

​

How Health Probes Work

• Every 15-30 seconds: “Are you healthy?”
• Server responds: “Yes, I’m fine!” → Stays in rotation
• Server doesn’t respond: (Marked unhealthy after 2-3 failures) → Removed from rotation

​

Azure Load Balancer Health Probes (Simple)

{
  "protocol": "TCP",
  "port": 80,
  "intervalInSeconds": 15,
  "numberOfProbes": 2
}

• Every 15 seconds, try to connect to port 80
• If 2 consecutive failures → Mark server unhealthy
• Marks unhealthy after: 2 × 15s = 30 seconds

• Question: “Is port 80 open?”
• Server: “Yes, port is open” ✅
• Problem: Port might be open, but app crashed!

• Question: “GET /health → Give me HTTP 200 OK”
• Server: “HTTP 200 OK” ✅
• Better: Confirms app is actually responding

Time 0:00 - Server C crashes
Time 0:15 - Health probe #1 fails
Time 0:30 - Health probe #2 fails → Marked unhealthy
Time 0:30 - Load balancer stops sending traffic to Server C

Timeline: 30 seconds of potential errors before removal

​

Application Gateway Health Probes (Advanced)

{
  "protocol": "Http",
  "path": "/health",
  "interval": 30,
  "timeout": 30,
  "unhealthyThreshold": 3,
  "statusCodes": ["200-399"]
}

• Every 30 seconds, send GET /health
• If server doesn’t respond in 30 seconds → Timeout
• If 3 consecutive failures → Mark unhealthy
• Marks unhealthy after: 3 × 30s = 90 seconds

// /health endpoint
app.get('/health', async (req, res) => {
  // Check if database is connected
  const dbOk = await checkDatabase();

  // Check if Redis is connected
  const redisOk = await checkRedis();

  // Check if external API is reachable
  const apiOk = await checkExternalAPI();

  if (dbOk && redisOk && apiOk) {
    return res.status(200).send('Healthy');
  } else {
    return res.status(503).send('Unhealthy');
  }
});

• Port might be open ✅
• App might be running ✅
• But if database is down → Health check fails → Server removed from rotation ✅

[!WARNING] Gotcha: Health Probe IPs Health probes come from Azure’s internal IP range 168.63.129.16. You MUST allow this IP in your NSG, or all backends will be marked unhealthy! Visual:

Copy

Your NSG Rules:
1. Allow traffic from Front Door IP ranges ✅
2. Deny all other traffic ❌

Result:
- User traffic → Allowed ✅
- Health probes from 168.63.129.16 → BLOCKED ❌
- All servers marked unhealthy
- 502 errors for all users!

NSG Rules (WRONG):
1. Allow: Front Door IPs → Port 80 ✅
2. Deny: All other traffic ❌

Health Probes:
Source: 168.63.129.16 (Azure internal)
Destination: Port 80
Result: BLOCKED by rule #2 ❌

Application Gateway:
- Server A: Health probe blocked → Marked unhealthy ❌
- Server B: Health probe blocked → Marked unhealthy ❌
- Server C: Health probe blocked → Marked unhealthy ❌
- All backends unhealthy → 502 errors for all users!

NSG Rules (CORRECT):
1. Allow: 168.63.129.16 → Port 80 (health probes) ✅
2. Allow: Front Door IPs → Port 80 (user traffic) ✅
3. Deny: All other traffic ❌

• Incident duration: 4 hours
• Users affected: 500,000
• Revenue lost: $2.8M
• Prevention: One extra NSG rule (free!)

​

Connection Draining: Gracefully Shutting Down Backends

​

Azure Load Balancer

az network lb rule update \
  --lb-name myLB \
  --name myRule \
  --floating-ip true \
  --idle-timeout 30

• Idle Timeout: After 30 minutes of inactivity, connection is closed.
• No Graceful Draining: Azure Load Balancer doesn’t support draining. Use a rolling update strategy.

​

Application Gateway

{
  "connectionDraining": {
    "enabled": true,
    "drainTimeoutInSec": 300
  }
}

• How it works: When you remove a backend, App Gateway stops sending new requests to it, but allows existing connections to finish for up to 300 seconds.

[!TIP] Best Practice: Set drain timeout to your P99 request latency. If 99% of requests finish in 10 seconds, set drain timeout to 15s.

​

Cross-Region Load Balancing: Front Door vs Traffic Manager

​

Decision Flowchart

​

2. Azure Front Door

• CDN: Caches static content at the edge.
• Anycast: Users connect to the nearest Microsoft Edge node (POPs).
• WAF: Web Application Firewall protects against SQL Injection, XSS.

[!WARNING] Gotcha: The 5-minute timeout Front Door has a hard 100-timeout for connections. If your backend takes 5 minutes to process a report, Front Door will cut the connection. Use Async patterns!

​

3. Azure Application Gateway

• WAF: Uses OWASP rules (same as Front Door).
• Autoscaling: Scales up based on traffic load.
• AGIC: Application Gateway Ingress Controller for AKS.

​

4. Azure NAT Gateway

• Dedicated resource for outbound traffic.
• Provides 64,000 SNAT ports per Public IP.
• You can attach up to 16 Public IPs (1 Million+ connections).

[!IMPORTANT] Best Practice: Always attach a NAT Gateway to your subnets if you have high outbound traffic (e.g., API scrapers, high-volume webhooks).

​

5. Azure DNS

​

Public Zones

​

Private Zones

• Resolve hostnames across VNets.
• Auto-registration: When you create a VM, it automatically gets a DNS record (vm1.app.internal).
• Used heavily by Private Link to map mypaas.privatelink.database.windows.net.

​

Split-Horizon DNS

​

6. Case Study: E-Commerce Architecture

1. User hits www.shop.com.
2. Azure Front Door intercepts, checks WAF, serves global cache.
3. Forwards dynamic request to Application Gateway in Region A.
4. App Gateway routes /cart to AKS Cluster (in a private subnet).
5. AKS Pod talks to Azure SQL via Private Link (traffic never leaves VNet).
6. AKS Pod sends email via SendGrid using NAT Gateway (to prevent SNAT failing).
7. DevOps Engineer connects via VPN Gateway to debug DB issues.