Module Overview
Estimated Time: 4-5 hours | Difficulty: Intermediate | Prerequisites: Core Concepts
- Lambda execution model and lifecycle
- Event sources and triggers
- Lambda Layers and custom runtimes
- Cold starts and performance optimization
- VPC configuration and networking
- Concurrency and scaling
- Best practices and production patterns
- Cost optimization strategies
Why Lambda?
No Servers
Zero infrastructure management—AWS handles all server provisioning and maintenance
Auto-Scaling
Scales automatically from zero to thousands of concurrent executions
Pay Per Use
Pay only for compute time—no charges when code isn’t running
Event-Driven
Integrates with 200+ AWS services as event sources
Lambda Execution Model
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Lambda Execution Lifecycle │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ COLD START │ │
│ │ ┌───────────┐ ┌────────────┐ ┌─────────────┐ ┌───────────┐ │ │
│ │ │ Download │ │ Start │ │ Initialize │ │ Run │ │ │
│ │ │ Code │─►│ Container │─►│ Runtime │─►│ Handler │ │ │
│ │ │ │ │ │ │ + Init Code │ │ │ │ │
│ │ └───────────┘ └────────────┘ └─────────────┘ └───────────┘ │ │
│ │ │ │ │ │ │ │
│ │ AWS AWS YOUR CODE YOUR CODE │ │
│ │ (100-500ms) (50-100ms) (varies) (billed) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ WARM START │ │
│ │ ┌───────────────────────────────────────────┐ ┌───────────┐ │ │
│ │ │ Container Already Running │ │ Run │ │ │
│ │ │ (execution environment reused) │─►│ Handler │ │ │
│ │ └───────────────────────────────────────────┘ └───────────┘ │ │
│ │ SKIPPED YOUR CODE │ │
│ │ (milliseconds) (billed) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
│ Execution Environment: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ /tmp (512 MB - 10 GB) │ Memory (128 MB - 10 GB) │ │
│ │ Persisted between │ CPU proportional to memory │ │
│ │ warm invocations │ 1769 MB = 1 vCPU │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────┘
Handler Function Structure
Copy
import json
import boto3
from typing import Any, Dict
# INIT CODE - Runs once per container (cold start)
# Put connections, clients, config here
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Orders')
def lambda_handler(event: Dict[str, Any], context) -> Dict[str, Any]:
"""
Lambda handler function.
Args:
event: Event data from trigger (S3, API Gateway, etc.)
context: Runtime information
- context.function_name
- context.memory_limit_in_mb
- context.invoked_function_arn
- context.aws_request_id
- context.get_remaining_time_in_millis()
Returns:
Response format depends on trigger type
"""
# Log the request ID for tracing
print(f"Request ID: {context.aws_request_id}")
print(f"Remaining time: {context.get_remaining_time_in_millis()}ms")
try:
# Process event
body = json.loads(event.get('body', '{}'))
order_id = body.get('order_id')
# Business logic
result = table.get_item(Key={'order_id': order_id})
return {
'statusCode': 200,
'headers': {'Content-Type': 'application/json'},
'body': json.dumps(result.get('Item', {}))
}
except Exception as e:
print(f"Error: {str(e)}")
return {
'statusCode': 500,
'body': json.dumps({'error': str(e)})
}
Event Sources and Triggers
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Lambda Event Source Types │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ SYNCHRONOUS (Request-Response) │
│ ───────────────────────────── │
│ • API Gateway (REST, HTTP, WebSocket) │
│ • Application Load Balancer │
│ • Cognito User Pools │
│ • Alexa Skills │
│ • CloudFront (Lambda@Edge) │
│ → Caller waits for response │
│ → Errors returned to caller │
│ │
│ ASYNCHRONOUS (Fire-and-Forget) │
│ ───────────────────────────── │
│ • S3 Events │
│ • SNS │
│ • EventBridge │
│ • SES │
│ • CloudWatch Logs │
│ → Caller gets 202 Accepted immediately │
│ → Lambda retries on failure (2 attempts by default) │
│ → Can configure DLQ for failed events │
│ │
│ POLL-BASED (Event Source Mapping) │
│ ──────────────────────────────── │
│ • SQS │
│ • DynamoDB Streams │
│ • Kinesis Data Streams │
│ • Amazon MQ │
│ • Kafka (MSK, self-managed) │
│ → Lambda polls the source │
│ → Batch processing supported │
│ → Lambda manages checkpointing │
│ │
└────────────────────────────────────────────────────────────────────────┘
Common Event Formats
Copy
# API Gateway Event (REST API)
api_gateway_event = {
"httpMethod": "POST",
"path": "/orders",
"pathParameters": {"id": "123"},
"queryStringParameters": {"status": "pending"},
"headers": {"Content-Type": "application/json"},
"body": "{\"item\": \"widget\", \"qty\": 2}",
"requestContext": {
"requestId": "abc-123",
"authorizer": {"claims": {"sub": "user-123"}}
}
}
# S3 Event
s3_event = {
"Records": [{
"eventName": "ObjectCreated:Put",
"s3": {
"bucket": {"name": "my-bucket"},
"object": {
"key": "uploads/image.jpg",
"size": 1024
}
}
}]
}
# SQS Event
sqs_event = {
"Records": [{
"messageId": "msg-123",
"body": "{\"order_id\": \"12345\"}",
"attributes": {
"ApproximateReceiveCount": "1"
}
}]
}
# DynamoDB Stream Event
dynamodb_stream_event = {
"Records": [{
"eventName": "INSERT",
"dynamodb": {
"NewImage": {
"PK": {"S": "ORDER#123"},
"status": {"S": "CREATED"}
},
"StreamViewType": "NEW_AND_OLD_IMAGES"
}
}]
}
# EventBridge Event
eventbridge_event = {
"source": "custom.myapp",
"detail-type": "OrderCreated",
"detail": {
"order_id": "12345",
"amount": 99.99
}
}
Lambda Layers
Layers allow you to share code and dependencies across multiple functions.Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Lambda Layers Architecture │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ Lambda Function │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Your Code (handler.py) │ │
│ ├─────────────────────────────────────────────────────────────────┤ │
│ │ Layer 1: Shared Libraries (boto3, requests) │ │
│ ├─────────────────────────────────────────────────────────────────┤ │
│ │ Layer 2: Custom Utilities (logging, auth) │ │
│ ├─────────────────────────────────────────────────────────────────┤ │
│ │ Layer 3: AWS SDK Extensions (X-Ray, Powertools) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
│ Benefits: │
│ • Reduce deployment package size │
│ • Share code across functions │
│ • Separate dependencies from business logic │
│ • Faster deployments (layers cached) │
│ │
│ Limits: │
│ • Maximum 5 layers per function │
│ • Total unzipped size: 250 MB (function + layers) │
│ • Layer path: /opt/ │
│ - Python: /opt/python/ │
│ - Node.js: /opt/nodejs/node_modules/ │
│ │
└────────────────────────────────────────────────────────────────────────┘
Creating a Lambda Layer
Copy
# Create layer structure for Python
mkdir -p python/lib/python3.11/site-packages
# Install dependencies
pip install requests boto3 aws-lambda-powertools \
-t python/lib/python3.11/site-packages
# Package the layer
zip -r layer.zip python/
# Deploy layer
aws lambda publish-layer-version \
--layer-name my-dependencies \
--zip-file fileb://layer.zip \
--compatible-runtimes python3.11 \
--description "Common Python dependencies"
AWS Lambda Powertools
Copy
# Using AWS Lambda Powertools for production-ready Lambda
from aws_lambda_powertools import Logger, Tracer, Metrics
from aws_lambda_powertools.utilities.typing import LambdaContext
from aws_lambda_powertools.utilities.validation import validate
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
logger = Logger(service="order-service")
tracer = Tracer(service="order-service")
metrics = Metrics(service="order-service", namespace="MyApp")
app = APIGatewayRestResolver()
@app.post("/orders")
@tracer.capture_method
def create_order():
"""Create a new order."""
body = app.current_event.json_body
logger.info("Creating order", extra={"order_data": body})
metrics.add_metric(name="OrdersCreated", unit="Count", value=1)
# Business logic...
order_id = process_order(body)
return {"order_id": order_id}
@logger.inject_lambda_context
@tracer.capture_lambda_handler
@metrics.log_metrics
def lambda_handler(event: dict, context: LambdaContext) -> dict:
return app.resolve(event, context)
Cold Starts and Optimization
Understanding Cold Starts
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Cold Start Factors │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ Factor │ Impact │ How to Optimize │
│ ────────────────────┼─────────────────┼───────────────────────────── │
│ Package Size │ High │ Minimize dependencies │
│ Runtime │ Medium │ Python/Node faster than Java │
│ Memory │ High │ More memory = faster init │
│ VPC │ Very High │ Use VPC only if needed │
│ Provisioned Conc. │ Eliminates │ Pre-warm for critical paths │
│ │
│ Typical Cold Start Times: │
│ ───────────────────────── │
│ Python (no VPC): 100-300ms │
│ Node.js (no VPC): 100-300ms │
│ Java (no VPC): 500-3000ms │
│ .NET (no VPC): 200-500ms │
│ │
│ With VPC (before improvements): +10-30 seconds │
│ With VPC (after ENI improvements): +200-500ms │
│ │
└────────────────────────────────────────────────────────────────────────┘
Optimization Strategies
Copy
import json
import boto3
# ✅ GOOD: Initialize outside handler (reused in warm starts)
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Orders')
# ✅ GOOD: Lazy loading for optional dependencies
_heavy_library = None
def get_heavy_library():
global _heavy_library
if _heavy_library is None:
import heavy_library
_heavy_library = heavy_library
return _heavy_library
def lambda_handler(event, context):
# ❌ BAD: Creating clients inside handler
# dynamodb = boto3.resource('dynamodb') # Don't do this!
# ✅ GOOD: Use connection pooling
# boto3 automatically reuses connections
# ✅ GOOD: Only import when needed
if event.get('needs_heavy_processing'):
lib = get_heavy_library()
result = lib.process(event)
return {'statusCode': 200}
Provisioned Concurrency
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Provisioned Concurrency │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Pre-warmed Execution Environments │ │
│ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ │
│ │ │ Warm │ │ Warm │ │ Warm │ │ Warm │ │ Warm │ = 5 PC │ │
│ │ │ #1 │ │ #2 │ │ #3 │ │ #4 │ │ #5 │ │ │
│ │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ │ │
│ │ │ │ │ │ │ │ │
│ │ └────────┴────────┴────────┴────────┘ │ │
│ │ │ │ │
│ │ Always ready to handle requests │ │
│ │ (no cold starts) │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
│ Pricing: │
│ • $0.000004463 per GB-second (provisioned) │
│ • + $0.0000097 per GB-second (invocation, same as on-demand) │
│ • Example: 1GB, 100 PC, 24/7 = ~$350/month │
│ │
│ Use Cases: │
│ • Latency-critical APIs │
│ • Predictable traffic patterns │
│ • Functions with heavy initialization │
│ │
└────────────────────────────────────────────────────────────────────────┘
VPC Configuration
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Lambda in VPC │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ When to Use VPC: │
│ ───────────────── │
│ ✓ Access RDS in private subnet │
│ ✓ Access ElastiCache │
│ ✓ Access EC2 instances │
│ ✓ Compliance requirements │
│ ✗ Don't use VPC if not needed (adds latency) │
│ │
│ Architecture: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ VPC │ │
│ │ ┌───────────────────────┐ ┌───────────────────────┐ │ │
│ │ │ Private Subnet (AZ1) │ │ Private Subnet (AZ2) │ │ │
│ │ │ ┌─────────────────┐ │ │ ┌─────────────────┐ │ │ │
│ │ │ │ Lambda ENI │ │ │ │ Lambda ENI │ │ │ │
│ │ │ └────────┬────────┘ │ │ └────────┬────────┘ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ ┌────────▼────────┐ │ │ ┌────────▼────────┐ │ │ │
│ │ │ │ RDS │ │ │ │ ElastiCache │ │ │ │
│ │ │ └─────────────────┘ │ │ └─────────────────┘ │ │ │
│ │ └───────────────────────┘ └───────────────────────┘ │ │
│ │ │ │
│ │ To access internet (S3, DynamoDB, external APIs): │ │
│ │ ┌───────────────────┐ │ │
│ │ │ NAT Gateway │ ──► Internet Gateway ──► Internet │ │
│ │ └───────────────────┘ │ │
│ │ OR │ │
│ │ ┌───────────────────┐ │ │
│ │ │ VPC Endpoints │ ──► S3, DynamoDB (private, no NAT) │ │
│ │ └───────────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────┘
VPC Configuration Best Practices
Copy
# Lambda VPC configuration (Terraform)
vpc_config = """
resource "aws_lambda_function" "vpc_lambda" {
function_name = "my-vpc-function"
runtime = "python3.11"
handler = "handler.lambda_handler"
vpc_config {
# Use at least 2 subnets for high availability
subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id]
security_group_ids = [aws_security_group.lambda_sg.id]
}
# Lambda needs these permissions to create ENIs
role = aws_iam_role.lambda_vpc_role.arn
}
# Security group for Lambda
resource "aws_security_group" "lambda_sg" {
name = "lambda-sg"
vpc_id = aws_vpc.main.id
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
# VPC Endpoint for DynamoDB (avoid NAT for AWS services)
resource "aws_vpc_endpoint" "dynamodb" {
vpc_id = aws_vpc.main.id
service_name = "com.amazonaws.us-east-1.dynamodb"
vpc_endpoint_type = "Gateway"
route_table_ids = [aws_route_table.private.id]
}
"""
Concurrency and Scaling
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Lambda Concurrency │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ Concurrent Executions = Invocations/sec × Duration (sec) │
│ │
│ Example: 100 requests/sec × 0.5 sec duration = 50 concurrent │
│ │
│ Account Limits: │
│ ─────────────── │
│ • Default: 1,000 concurrent executions (per region) │
│ • Can request increase to 10,000+ │
│ • Burst limit: 500-3,000 (varies by region) │
│ │
│ Concurrency Types: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Unreserved (default) Reserved Provisioned │ │
│ │ ───────────────────── ───────── ──────────── │ │
│ │ Shared pool for all Guaranteed Pre-initialized │ │
│ │ functions in account minimum for (no cold starts) │ │
│ │ this function │ │
│ │ │ │
│ │ Account Limit: 1000 │ │
│ │ ┌────────────────────────────────────────────────────────────┐ │ │
│ │ │████████████████│████████│██████████████│ │ │ │
│ │ │ Unreserved │ Func A │ Func B │ Not allocated │ │ │
│ │ │ (600) │ (100) │ (200) │ (100) │ │ │
│ │ └────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────┘
Reserved Concurrency
Copy
# Setting reserved concurrency prevents one function from
# consuming all account capacity
import boto3
lambda_client = boto3.client('lambda')
# Reserve 100 concurrent executions for critical function
lambda_client.put_function_concurrency(
FunctionName='critical-payment-processor',
ReservedConcurrentExecutions=100
)
# Setting to 0 = disable function (throttle all invocations)
lambda_client.put_function_concurrency(
FunctionName='temporarily-disabled-function',
ReservedConcurrentExecutions=0
)
Error Handling and Retries
Copy
┌────────────────────────────────────────────────────────────────────────┐
│ Lambda Error Handling │
├────────────────────────────────────────────────────────────────────────┤
│ │
│ SYNCHRONOUS INVOCATIONS (API Gateway, ALB): │
│ ───────────────────────────────────────── │
│ • No automatic retries │
│ • Errors returned to caller │
│ • Implement retry logic in client │
│ │
│ ASYNCHRONOUS INVOCATIONS (S3, SNS, EventBridge): │
│ ────────────────────────────────────────────── │
│ • 2 automatic retries (total 3 attempts) │
│ • Exponential backoff between retries │
│ • Configure DLQ or On-Failure destination │
│ │
│ EVENT SOURCE MAPPINGS (SQS, DynamoDB, Kinesis): │
│ ────────────────────────────────────────────── │
│ • Retry until record expires or succeeds │
│ • Can configure max age, retry attempts │
│ • Bisect batch on failure (find problematic record) │
│ │
│ Destinations (async only): │
│ ┌───────────────────┐ Success ┌───────────────────┐ │
│ │ Lambda Function │──────────────►│ SQS/SNS/Lambda │ │
│ │ │ │ EventBridge │ │
│ │ │ Failure ┌───────────────────┐ │
│ │ │──────────────►│ SQS/SNS/Lambda │ │
│ └───────────────────┘ │ EventBridge │ │
│ └───────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────┘
Copy
import json
from aws_lambda_powertools import Logger
from aws_lambda_powertools.utilities.idempotency import (
IdempotencyConfig, DynamoDBPersistenceLayer, idempotent
)
logger = Logger()
# Configure idempotency to prevent duplicate processing
persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
config = IdempotencyConfig(
event_key_jmespath="body",
expires_after_seconds=3600
)
@idempotent(config=config, persistence_store=persistence_layer)
def process_payment(event: dict):
"""
Idempotent payment processing.
Safe to retry - won't charge customer twice.
"""
body = json.loads(event['body'])
# Process payment...
return {"status": "success", "transaction_id": "tx-123"}
def lambda_handler(event, context):
try:
result = process_payment(event)
return {
'statusCode': 200,
'body': json.dumps(result)
}
except Exception as e:
logger.exception("Payment processing failed")
# Re-raise to trigger retry (for async) or return error (for sync)
raise
Best Practices
Keep Functions Focused
Single responsibility - one function per task
Minimize Package Size
Only include necessary dependencies
Use Environment Variables
Store configuration, not secrets (use Secrets Manager)
Implement Idempotency
Handle retries safely
Set Realistic Timeouts
Default 3s is often too short
Monitor Everything
CloudWatch metrics, X-Ray traces, custom dashboards
Production Checklist
Copy
production_checklist = {
"code": [
"✓ Handler is focused and small",
"✓ Dependencies minimized",
"✓ Connections initialized outside handler",
"✓ Proper error handling with logging",
"✓ Idempotent operations",
],
"configuration": [
"✓ Appropriate memory (test to find optimal)",
"✓ Realistic timeout (p99 latency + buffer)",
"✓ Environment variables for config",
"✓ Secrets Manager for secrets",
"✓ X-Ray tracing enabled",
],
"reliability": [
"✓ DLQ configured for async invocations",
"✓ Reserved concurrency for critical functions",
"✓ Provisioned concurrency for latency-sensitive",
"✓ Retry logic for downstream failures",
],
"security": [
"✓ Least-privilege IAM role",
"✓ VPC only if needed (private resources)",
"✓ No hardcoded credentials",
"✓ Input validation",
],
"cost": [
"✓ Right-sized memory",
"✓ Avoid unnecessary VPC",
"✓ Use ARM (Graviton2) for 20% savings",
"✓ Monitor and alert on invocation spikes",
]
}
🎯 Interview Questions
Q1: How do you minimize Lambda cold starts?
Q1: How do you minimize Lambda cold starts?
Optimization strategies:
- Code level:
- Minimize deployment package size
- Lazy load heavy dependencies
- Initialize SDK clients outside handler
- Configuration:
- Increase memory (faster CPU = faster init)
- Use compiled languages carefully
- Avoid VPC unless necessary
- Provisioned Concurrency:
- Pre-warm execution environments
- Eliminate cold starts for critical paths
- Use scheduled scaling for traffic patterns
- Architecture:
- Keep functions warm with scheduled pings (anti-pattern, prefer PC)
- Use Lambda SnapStart for Java
Q2: How does Lambda scale?
Q2: How does Lambda scale?
Scaling behavior:
- Lambda scales by creating more execution environments
- Burst: 500-3,000 concurrent executions immediately
- After burst: 500 additional per minute
- Account limit: 1,000 default (can increase)
- Function reserved concurrency: up to account limit
- Provisioned concurrency: pre-warmed instances
Q3: Lambda vs EC2 vs ECS - when to use each?
Q3: Lambda vs EC2 vs ECS - when to use each?
Lambda:
- Short-lived, event-driven workloads
- Unpredictable/spiky traffic
- < 15 minutes execution
- No server management needed
- Long-running containers
- Microservices architecture
- Consistent traffic patterns
- Need more control than Lambda
- Maximum control/customization
- Specialized hardware needs
- Persistent workloads
- Legacy applications
Q4: How do you handle secrets in Lambda?
Q4: How do you handle secrets in Lambda?
Best practices:Never:
Copy
# Use Secrets Manager (cached with Powertools)
from aws_lambda_powertools.utilities.parameters import get_secret
# Cached for 5 minutes by default
db_password = get_secret("prod/db/password")
# Or use Parameter Store for non-sensitive config
from aws_lambda_powertools.utilities.parameters import get_parameter
api_endpoint = get_parameter("/myapp/api/endpoint")
- Hardcode secrets in code
- Store secrets in environment variables
- Log secrets
Q5: Explain Lambda@Edge vs Lambda
Q5: Explain Lambda@Edge vs Lambda
Lambda@Edge:
- Runs at CloudFront edge locations
- Lower latency (closer to users)
- Limited: 128MB memory, 5s timeout (viewer), 30s (origin)
- Use cases: URL rewrite, A/B testing, auth, headers
- Runs in a single region
- Full capabilities: 10GB memory, 15min timeout
- More triggers and integrations
- Even faster, cheaper than Lambda@Edge
- 2MB code, 1ms timeout
- Simple header manipulation only
🧪 Hands-On Lab
1
Create Basic Lambda
Create a Python Lambda function with API Gateway trigger
2
Add Dependencies with Layers
Create a layer with requests and boto3, attach to function
3
Configure VPC Access
Set up Lambda in VPC to access RDS, configure VPC endpoints
4
Implement Error Handling
Add DLQ, structured logging, and X-Ray tracing
5
Optimize Performance
Configure provisioned concurrency and measure cold start impact
Next Module
AWS Step Functions
Orchestrate serverless workflows with Step Functions