Documentation Index
Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt
Use this file to discover all available pages before exploring further.
Chapter 8: Microservices
Microservices architecture enables you to build scalable, distributed systems. NestJS provides first-class support for microservices, message brokers, and event-driven design. This chapter covers TCP, Redis, RabbitMQ, message patterns, distributed tracing, and real-world deployment tips. We’ll walk through practical examples and explain how to design robust microservices.
8.1 What is a Microservice?
A microservice is a small, independent process that communicates with others over a network. Each service is responsible for a specific business capability and can be deployed, scaled, and updated independently.
Microservices vs Monolith
Monolithic Architecture:
- Single codebase
- Deploy as one unit
- Scale entire application
- Shared database
Microservices Architecture:
- Multiple independent services
- Deploy services independently
- Scale individual services
- Each service has its own database
Analogy:
Imagine a city’s postal system. Each microservice is a different office (tax office, passport office, vehicle registration). Each office has its own building, its own staff, and its own filing system (database). They communicate by sending letters (messages) through the postal service (message broker). If the tax office burns down, the passport office keeps working. You can hire more staff at the tax office during tax season without expanding the passport office. But managing a city full of independent offices is inherently more complex than managing one big building — you need reliable mail delivery, consistent address formats, and a way to track letters across offices. That is the fundamental trade-off of microservices: operational complexity in exchange for independent scalability and resilience.
Benefits
Scalability:
- Scale only what you need
- Independent scaling per service
- Optimize resources
Fault Isolation:
- One service can fail without crashing the whole system
- Isolated failures
- Better resilience
Technology Diversity:
- Use the best tool for each job
- Different languages/frameworks per service
- Technology flexibility
Independent Deployment:
- Deploy and update services separately
- Faster release cycles
- Reduced risk
Team Autonomy:
- Teams can work independently
- Clear ownership
- Faster development
Challenges
These are not hypothetical downsides — they are the daily reality of running microservices in production. Do not adopt microservices unless these challenges are worth the benefits for your specific situation.
- Complexity: More moving parts means more things to deploy, monitor, version, and debug. A monolith with 10 modules becomes 10 services with 10 deployment pipelines, 10 health checks, and 10 log streams.
- Network Latency: Every service-to-service call crosses the network. What was a 1ms in-process function call becomes a 5-50ms network round trip. Chained calls compound this — Service A calls B calls C means 3x latency.
- Data Consistency: Without distributed transactions (which are hard), you rely on eventual consistency and saga patterns. This means your system may temporarily show stale or inconsistent data.
- Testing: You cannot just run
npm test — you need the entire ecosystem running (or good contract tests). E2E testing across services requires orchestration tools like Docker Compose.
- Monitoring: A single request may touch 5 services. Without distributed tracing (OpenTelemetry, Jaeger), debugging production issues is like finding a needle in a haystack.
Honest Advice: Start with a well-structured monolith. Extract services only when you have a concrete scaling or team-autonomy reason. Most NestJS applications never need microservices — a modular monolith with clear module boundaries gives you 80% of the benefits with 20% of the complexity.
8.2 Microservices in NestJS
NestJS supports multiple transport layers for microservices, each with different characteristics.
Supported Transports
TCP:
- Default, simple and fast
- Direct service-to-service communication
- Low latency
- Good for internal services
Redis:
- Pub/sub messaging
- Caching support
- Simple setup
- Good for event-driven systems
NATS:
- Lightweight messaging
- High performance
- Simple protocol
- Good for cloud-native apps
RabbitMQ:
- Robust message broker
- Advanced routing
- Reliable delivery
- Good for complex workflows
MQTT:
- IoT messaging
- Lightweight protocol
- Good for IoT applications
gRPC:
- High-performance RPC
- Type-safe
- HTTP/2 based
- Good for inter-service communication
Kafka:
- Distributed streaming
- High throughput
- Event sourcing support
- Good for big data
Transport Layer Comparison
| Transport | Latency | Throughput | Persistence | Delivery Guarantee | Complexity | Best For |
|---|
| TCP | Very low (~1ms) | High | None | At-most-once | Low | Internal service-to-service calls |
| Redis | Low (~2-5ms) | High | Optional (Streams) | At-most-once (pub/sub) | Low | Event broadcasting, simple pub/sub |
| NATS | Very low (~1ms) | Very high | Optional (JetStream) | Configurable | Low | Cloud-native, high-frequency events |
| RabbitMQ | Low (~5-10ms) | High | Yes (durable queues) | At-least-once | Medium | Reliable task queues, complex routing |
| Kafka | Medium (~10-50ms) | Very high | Yes (log-based) | At-least-once | High | Event streaming, big data pipelines |
| gRPC | Very low (~1ms) | Very high | None | At-most-once | Medium | High-performance RPC, polyglot systems |
| MQTT | Low | Moderate | Configurable | Configurable (QoS) | Low | IoT devices, low-bandwidth clients |
Decision Framework — Which Transport?
Do you need guaranteed message delivery (no lost messages)?
YES --> RabbitMQ (durable queues) or Kafka (persistent log)
NO --> TCP (fastest), Redis pub/sub, or NATS
Do you need to process millions of events per second?
YES --> Kafka or NATS JetStream
NO --> RabbitMQ or Redis
Do consumers need to replay past messages?
YES --> Kafka (log-based, messages persist) or NATS JetStream
NO --> RabbitMQ, Redis, or TCP
Are your services in different languages?
YES --> gRPC (strongly typed contracts, code generation for all languages)
NO --> Any transport works, TCP is simplest for NestJS-to-NestJS
Is it IoT or constrained devices?
YES --> MQTT
NO --> Any of the above
Monolith vs Microservices Decision Framework:
| Factor | Stay Monolith | Consider Microservices |
|---|
| Team Size | 1-5 developers | 10+ developers in independent teams |
| Deployment Frequency | Weekly or less | Multiple deploys per day by different teams |
| Scaling Needs | Uniform load across features | One feature has 100x the load of others |
| Fault Isolation | Acceptable if whole app goes down briefly | Individual feature failure must not cascade |
| Data Complexity | Shared database is manageable | Features need independent data stores |
| Operational Maturity | No Kubernetes, no distributed tracing | Strong DevOps, observability in place |
8.3 Creating a TCP Microservice
TCP is the simplest transport for microservices. Let’s create a basic TCP microservice.
Installation
npm install @nestjs/microservices
Microservice Bootstrap
// main.ts (microservice)
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
AppModule,
{
transport: Transport.TCP,
options: {
host: 'localhost',
port: 3001,
},
},
);
await app.listen();
console.log('Microservice is listening on port 3001');
}
bootstrap();
Message Pattern Handler
Message patterns are the microservice equivalent of HTTP routes. Instead of @Get('/users/:id'), you use @MessagePattern({ cmd: 'get_user' }). The pattern object is matched exactly — think of it as a topic or address for the message.
// app.controller.ts
import { Controller } from '@nestjs/common';
import { MessagePattern, Payload } from '@nestjs/microservices';
@Controller()
export class AppController {
// @MessagePattern defines a request-response handler.
// When a client sends { cmd: 'sum' } with payload [1, 2, 3],
// this handler runs and returns the result (6) back to the caller.
@MessagePattern({ cmd: 'sum' })
accumulate(@Payload() data: number[]): number {
return data.reduce((a, b) => a + b, 0);
}
// @Payload() extracts the data from the incoming message, similar to
// @Body() in HTTP controllers. The message format depends on the
// transport layer (TCP, RabbitMQ, Redis, etc.).
@MessagePattern({ cmd: 'get_user' })
getUser(@Payload() data: { id: number }) {
return { id: data.id, name: 'John Doe' };
}
}
Key Distinction: @MessagePattern is for request-response (caller waits for a reply). @EventPattern is for fire-and-forget (caller does not wait). Choosing the wrong one is a common mistake — if you use @MessagePattern but the caller uses .emit(), the handler will never fire.
Client Service
// app.service.ts
import { Injectable, Inject, OnModuleInit } from '@nestjs/common';
import { ClientProxy, ClientProxyFactory, Transport } from '@nestjs/microservices';
@Injectable()
export class AppService implements OnModuleInit {
private client: ClientProxy;
constructor() {
this.client = ClientProxyFactory.create({
transport: Transport.TCP,
options: {
host: 'localhost',
port: 3001,
},
});
}
async onModuleInit() {
await this.client.connect();
}
async sum(data: number[]) {
return this.client.send({ cmd: 'sum' }, data).toPromise();
}
async getUser(id: number) {
return this.client.send({ cmd: 'get_user' }, { id }).toPromise();
}
}
Hybrid Application
Run both HTTP and microservice in the same app:
// main.ts
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';
async function bootstrap() {
// HTTP application
const app = await NestFactory.create(AppModule);
// Microservice
app.connectMicroservice<MicroserviceOptions>({
transport: Transport.TCP,
options: {
host: 'localhost',
port: 3001,
},
});
await app.startAllMicroservices();
await app.listen(3000);
console.log('HTTP server on port 3000, Microservice on port 3001');
}
bootstrap();
8.4 Message Patterns
Message patterns define how microservices communicate. Use consistent patterns for maintainability.
Request-Response Pattern
// Handler
@MessagePattern({ cmd: 'get_user' })
getUser(@Payload() data: { id: number }) {
return this.usersService.findOne(data.id);
}
// Client
const user = await this.client.send({ cmd: 'get_user' }, { id: 1 }).toPromise();
Event Pattern (Fire and Forget)
// Handler
@EventPattern('user_created')
handleUserCreated(@Payload() data: { id: number; email: string }) {
console.log('User created:', data);
// Send welcome email, etc.
}
// Client
this.client.emit('user_created', { id: 1, email: 'user@example.com' });
Multiple Patterns
@Controller()
export class UsersController {
@MessagePattern({ cmd: 'create_user' })
async createUser(@Payload() data: CreateUserDto) {
return this.usersService.create(data);
}
@MessagePattern({ cmd: 'get_user' })
async getUser(@Payload() data: { id: number }) {
return this.usersService.findOne(data.id);
}
@EventPattern('user_updated')
async handleUserUpdated(@Payload() data: { id: number }) {
// Handle event
}
}
Tip: Use clear, consistent message patterns for maintainability. Document your message contracts.
8.5 Using RabbitMQ
RabbitMQ is a popular message broker for event-driven systems. It enables publish/subscribe and queue-based communication.
Installation
npm install @nestjs/microservices amqplib amqp-connection-manager
npm install --save-dev @types/amqplib
RabbitMQ Microservice
// main.ts
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
AppModule,
{
transport: Transport.RMQ,
options: {
urls: [process.env.RABBITMQ_URL || 'amqp://localhost:5672'],
queue: 'orders_queue',
queueOptions: {
durable: true,
},
},
},
);
await app.listen();
console.log('RabbitMQ microservice is listening');
}
bootstrap();
RabbitMQ Client
// app.service.ts
import { Injectable } from '@nestjs/common';
import { ClientProxy, ClientProxyFactory, Transport } from '@nestjs/microservices';
@Injectable()
export class AppService {
private client: ClientProxy;
constructor() {
this.client = ClientProxyFactory.create({
transport: Transport.RMQ,
options: {
urls: [process.env.RABBITMQ_URL || 'amqp://localhost:5672'],
queue: 'orders_queue',
queueOptions: {
durable: true,
},
},
});
}
async createOrder(orderData: any) {
return this.client.send({ cmd: 'create_order' }, orderData).toPromise();
}
async publishOrderCreated(order: any) {
this.client.emit('order_created', order);
}
}
Diagram: Event-Driven Flow
Producer Service
↓
Publish Event → [RabbitMQ Queue]
↓
Consumer Service(s) ← Subscribe to Queue
↓
Process Event
8.6 Using Redis
Redis provides pub/sub messaging and can be used as a transport layer.
Redis Microservice
// main.ts
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
AppModule,
{
transport: Transport.REDIS,
options: {
host: 'localhost',
port: 6379,
},
},
);
await app.listen();
}
bootstrap();
Redis Client
this.client = ClientProxyFactory.create({
transport: Transport.REDIS,
options: {
host: 'localhost',
port: 6379,
},
});
8.7 Event-Driven Design
Event-driven architecture decouples services and enables scalability.
Principles
Decoupling:
- Services don’t need to know about each other
- Communicate via events
- Loose coupling
Scalability:
- Publish/subscribe patterns
- Multiple consumers
- Horizontal scaling
Reliability:
- Message persistence
- Retries and dead-letter queues
- Event replay
Event Sourcing Integration
@EventPattern('order_created')
async handleOrderCreated(@Payload() event: OrderCreatedEvent) {
// Store event
await this.eventStore.append('order_created', event);
// Update read model
await this.updateReadModel(event);
// Publish to other services
this.client.emit('order_created_notification', event);
}
Correlation IDs
Track requests across services:
@MessagePattern({ cmd: 'process_order' })
async processOrder(@Payload() data: any, @Ctx() context: RmqContext) {
const correlationId = context.getMessage().properties.correlationId;
// Use correlation ID for tracing
this.logger.log(`Processing order with correlation ID: ${correlationId}`);
return this.ordersService.process(data);
}
Saga Pattern for Distributed Transactions
When an operation spans multiple services (e.g., “create order, charge payment, reserve inventory”), you cannot use a database transaction because each service has its own database. The Saga pattern coordinates these steps with compensation logic for failures.
// Choreography-based Saga: each service publishes events, next service reacts
// Order Service publishes "OrderCreated"
// Payment Service listens, charges card, publishes "PaymentCompleted" or "PaymentFailed"
// Inventory Service listens to "PaymentCompleted", reserves stock
// If Payment fails, Order Service listens to "PaymentFailed" and cancels the order
// Orchestration-based Saga: a coordinator service manages the flow
@Injectable()
export class OrderSagaOrchestrator {
async execute(orderData: CreateOrderDto) {
let orderId: string;
let paymentId: string;
try {
// Step 1: Create order (pending)
orderId = await this.orderClient.send({ cmd: 'create_order' }, orderData).toPromise();
// Step 2: Charge payment
paymentId = await this.paymentClient.send({ cmd: 'charge' }, {
orderId, amount: orderData.total,
}).toPromise();
// Step 3: Reserve inventory
await this.inventoryClient.send({ cmd: 'reserve' }, {
orderId, items: orderData.items,
}).toPromise();
// Step 4: Confirm order
await this.orderClient.send({ cmd: 'confirm_order' }, { orderId }).toPromise();
} catch (error) {
// Compensation: undo completed steps in reverse order
if (paymentId) {
await this.paymentClient.send({ cmd: 'refund' }, { paymentId }).toPromise();
}
if (orderId) {
await this.orderClient.send({ cmd: 'cancel_order' }, { orderId }).toPromise();
}
throw error;
}
}
}
| Saga Style | Coordination | Complexity | Best For |
|---|
| Choreography | Decentralized (events) | Lower initially, harder to debug | Simple flows (2-3 steps) |
| Orchestration | Centralized (coordinator) | Higher initially, easier to debug | Complex flows (4+ steps) |
8.8 Distributed Tracing & Monitoring
Monitor and trace requests across microservices.
OpenTelemetry Integration
import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common';
import { Observable } from 'rxjs';
import { trace, context } from '@opentelemetry/api';
@Injectable()
export class TracingInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const tracer = trace.getTracer('nestjs');
const span = tracer.startSpan('http_request');
return next.handle().pipe(
tap(() => span.end()),
catchError((err) => {
span.recordException(err);
span.end();
throw err;
}),
);
}
}
Logging with Correlation IDs
@Injectable()
export class LoggerService {
log(message: string, correlationId?: string) {
console.log(JSON.stringify({
message,
correlationId,
timestamp: new Date().toISOString(),
}));
}
}
8.9 Error Handling
Handle errors gracefully in microservices.
Error Handling Pattern
@MessagePattern({ cmd: 'process_order' })
async processOrder(@Payload() data: any) {
try {
return await this.ordersService.process(data);
} catch (error) {
// Log error
this.logger.error('Failed to process order', error);
// Emit error event
this.client.emit('order_processing_failed', {
orderId: data.id,
error: error.message,
});
throw error;
}
}
Retry Logic
In distributed systems, transient failures (network blips, temporary service unavailability) are normal, not exceptional. Retrying with backoff is the standard response — but be careful about retrying non-idempotent operations. Retrying a “charge credit card” command without idempotency keys will double-charge your customer.
import { retry, catchError, delay } from 'rxjs/operators';
this.client.send({ cmd: 'process_order' }, data).pipe(
// retry(3) will retry up to 3 times on failure.
// For production, use retryWhen() with exponential backoff:
// retry 1 after 1s, retry 2 after 2s, retry 3 after 4s.
retry(3),
catchError((error) => {
// All retries exhausted. Log, send to dead-letter queue, or alert.
this.logger.error(`Failed after 3 retries: ${error.message}`);
return throwError(() => error);
}),
).toPromise();
Production Pattern: Combine retries with circuit breakers. After N consecutive failures, the circuit breaker “opens” and stops making requests for a cooldown period, preventing cascade failures across services. Libraries like opossum integrate well with NestJS.
Error Handling Decision Framework for Microservices
| Error Type | Retry? | Compensation? | Example |
|---|
| Transient (network timeout, temporary unavailability) | Yes, with exponential backoff | No | Connection refused, 503 from downstream |
| Permanent (invalid data, business rule violation) | No (retrying will never succeed) | No | Invalid order total, user not found |
| Partial failure (one step in a saga fails) | Depends on the failed step | Yes (undo completed steps) | Payment succeeded but inventory reservation failed |
| Poison message (message that always fails processing) | No (move to dead-letter queue) | Depends | Malformed JSON, missing required field |
Message arrives
|
+--> Process successfully? --> ACK and continue
|
+--> Transient error? --> Retry with backoff (max 3 retries)
| |
| +--> Still failing? --> Move to dead-letter queue + alert
|
+--> Permanent error? --> Move to dead-letter queue immediately
|
+--> Partial saga failure? --> Execute compensation logic
8.10 Best Practices
Following best practices ensures your microservices are robust and maintainable.
Service Design
- Keep services small and focused (single responsibility)
- Define clear service boundaries
- Use domain-driven design
- Document service contracts
Communication
- Use message brokers for async communication
- Use request-response for synchronous needs
- Implement circuit breakers
- Handle timeouts gracefully
Data Management
- Each service owns its data
- Avoid shared databases
- Use event sourcing for consistency
- Implement saga pattern for distributed transactions
Security
- Secure communication between services (TLS)
- Authenticate service-to-service calls
- Use API keys or mTLS
- Validate all inputs
Deployment
- Containerize services (Docker)
- Use orchestration (Kubernetes)
- Implement health checks
- Enable auto-scaling
Monitoring
- Log all important events
- Use distributed tracing
- Monitor latency and errors
- Set up alerts
Documentation
- Document message patterns
- Document service APIs
- Maintain service registry
- Keep architecture diagrams updated
8.11 Summary
You’ve learned how to build and scale microservices with NestJS:
Key Concepts:
- Microservices: Independent, scalable services
- Transport Layers: TCP, Redis, RabbitMQ, NATS, gRPC
- Message Patterns: Request-response and event patterns
- Event-Driven: Decoupled, scalable architecture
- Distributed Tracing: Monitor requests across services
Best Practices:
- Keep services small and focused
- Use message brokers for communication
- Handle errors gracefully
- Secure service communication
- Monitor and trace requests
- Document service contracts
Next Chapter: Learn about deployment, production optimization, Docker, Kubernetes, and monitoring strategies.