AWS Step Functions lets you coordinate multiple AWS services into serverless workflows using visual state machines. Think of Step Functions as a recipe card for your cloud kitchen — each step says “do this, then check the result, then decide what to do next.” Without it, you would wire up Lambda-to-Lambda calls with SQS queues, build your own retry logic, and pray that error handling works. Step Functions gives you that orchestration layer with built-in retries, timeouts, and a visual execution history so you can see exactly where a workflow failed at 3 AM. This module covers workflow design patterns, error handling, and production best practices.What You’ll Learn:
State machine concepts and design
State types (Task, Choice, Parallel, Map)
Error handling and retries
Standard vs Express workflows
Service integrations
Workflow patterns for common use cases
Why Step Functions?
Visual Workflows
Design and visualize complex business processes as state machines
Built-in Error Handling
Automatic retries, catch blocks, and compensation logic
200+ Integrations
Native integration with Lambda, DynamoDB, SQS, SNS, and more
Audit Trail
Complete execution history for debugging and compliance
Performs work by invoking an AWS service or activity.
{ "ValidateOrder": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:validate-order", "InputPath": "$.order", "ResultPath": "$.validation", "OutputPath": "$", // Always set TimeoutSeconds. Without it, a stuck Lambda (e.g., waiting // on a downstream service that never responds) will hold the execution // open indefinitely. Standard workflows charge per state transition AND // keep the execution in "Running" state, which counts against your // concurrent execution quota (1M default, but still finite). "TimeoutSeconds": 30, "Next": "CheckValidation" }}
class PaymentDeclinedException(Exception): passclass InsufficientFundsException(Exception): passdef lambda_handler(event, context): try: result = process_payment(event) return result except CardDeclinedError: # Step Functions will catch this raise PaymentDeclinedException("Card was declined") except InsufficientFundsError: raise InsufficientFundsException("Insufficient funds in account")
Standard vs Express Workflows
┌────────────────────────────────────────────────────────────────────────┐│ Workflow Type Comparison │├────────────────────────────────────────────────────────────────────────┤│ ││ Feature │ Standard │ Express ││ ─────────────────────┼──────────────────────┼────────────────────── ││ Duration │ Up to 1 year │ Up to 5 minutes ││ Execution History │ Stored (90 days) │ CloudWatch Logs only ││ Start Rate │ 2,000/sec │ 100,000/sec ││ Pricing │ Per state transition │ Per execution + dur. ││ Idempotency │ Exactly-once │ At-least-once ││ Execution Type │ Async (default) │ Sync or Async ││ ││ When to Use Standard: ││ ───────────────────── ││ ✓ Long-running workflows (hours/days) ││ ✓ Need execution history for audit ││ ✓ Require exactly-once semantics ││ ✓ Human approval workflows ││ ││ When to Use Express: ││ ──────────────────── ││ ✓ High-volume, short-duration workflows ││ ✓ Event processing pipelines ││ ✓ API orchestration ││ ✓ Cost-sensitive scenarios ││ ││ Pricing Example (1M executions, 5 state transitions each): ││ Standard: 5M transitions x $0.000025 = $125 ││ Express: 1M x $1.00/M + duration charges = ~$10-20 ││ ││ Cost mistake: Using Standard workflows for high-volume, short-lived ││ tasks (like API orchestration). A team processing 10M API requests/ ││ month with 8 transitions each pays $2,000 on Standard vs ~$100 on ││ Express. Rule of thumb: if it finishes in under 5 minutes and you ││ can tolerate at-least-once semantics, use Express. ││ │└────────────────────────────────────────────────────────────────────────┘
Service Integrations
Direct Service Integrations
{ "Comment": "Direct integrations without Lambda", "States": { "PutItemInDynamoDB": { "Type": "Task", // Direct service integration -- no Lambda needed. This calls // DynamoDB directly from Step Functions, saving both the cost of // a Lambda invocation (~$0.20/M) and ~100ms of cold-start latency. // Use direct integrations whenever the operation is a simple // service call that doesn't need custom business logic. "Resource": "arn:aws:states:::dynamodb:putItem", "Parameters": { "TableName": "Orders", "Item": { "order_id": {"S.$": "$.orderId"}, "status": {"S": "CREATED"}, "created_at": {"S.$": "$$.State.EnteredTime"} } }, "Next": "SendNotification" }, "SendNotification": { "Type": "Task", "Resource": "arn:aws:states:::sns:publish", "Parameters": { "TopicArn": "arn:aws:sns:us-east-1:123456789012:order-notifications", "Message.$": "States.Format('Order {} created', $.orderId)" }, "Next": "AddToQueue" }, "AddToQueue": { "Type": "Task", "Resource": "arn:aws:states:::sqs:sendMessage", "Parameters": { "QueueUrl": "https://sqs.us-east-1.amazonaws.com/123456789012/orders", "MessageBody.$": "States.JsonToString($.order)" }, "Next": "Done" }, "InvokeLambda": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke", "Parameters": { "FunctionName": "process-order", "Payload.$": "$" }, "Next": "Done" }, "StartECSTask": { "Type": "Task", "Resource": "arn:aws:states:::ecs:runTask.sync", "Parameters": { "Cluster": "my-cluster", "TaskDefinition": "process-batch", "LaunchType": "FARGATE" }, "Next": "Done" } }}
Wait for Callback Pattern
This is one of the most powerful Step Functions patterns. The workflow pauses and waits for an external system (a human, a webhook, a third-party API) to call back with a result. The execution stays in “Waiting” state without consuming compute or costing money beyond the initial state transition. Real-world use cases include: manager approval for expenses, waiting for a payment processor webhook, or pausing until a manual QA review is complete.
{ "WaitForHumanApproval": { "Type": "Task", // waitForTaskToken pauses the workflow until an external system calls // SendTaskSuccess or SendTaskFailure with the token. The workflow // consumes no compute while waiting -- you only pay for the state // transition, not the wait time. This is the pattern for human // approvals, external webhooks, or any async callback. "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken", "Parameters": { "FunctionName": "send-approval-request", "Payload": { "orderId.$": "$.orderId", "approver": "manager@company.com", "taskToken.$": "$$.Task.Token" } }, // Common mistake: Not setting a timeout on callback tasks. Without // TimeoutSeconds, the execution waits forever if the callback never // arrives (e.g., the approver ignores the email). This counts against // your open executions quota and the execution eventually becomes // unrecoverable. Always set a reasonable timeout with a Catch block // for States.Timeout that sends a reminder or auto-rejects. "TimeoutSeconds": 86400, "Next": "ProcessApprovedOrder" }}
# Lambda that sends approval requestdef send_approval_request(event, context): task_token = event['taskToken'] order_id = event['orderId'] # Store token for later callback dynamodb.put_item( TableName='PendingApprovals', Item={ 'order_id': order_id, 'task_token': task_token, 'status': 'PENDING' } ) # Send email with approval link ses.send_email( To=event['approver'], Subject=f'Approval needed for order {order_id}', Body=f'Click to approve: https://api.../approve?order={order_id}' )# API endpoint that handles approvaldef handle_approval(event, context): order_id = event['queryStringParameters']['order_id'] action = event['queryStringParameters']['action'] # approve/reject # Get stored token item = dynamodb.get_item( TableName='PendingApprovals', Key={'order_id': order_id} ) task_token = item['Item']['task_token'] # Resume Step Function if action == 'approve': stepfunctions.send_task_success( taskToken=task_token, output=json.dumps({'approved': True}) ) else: stepfunctions.send_task_failure( taskToken=task_token, error='Rejected', cause='Manager rejected the order' )