Skip to main content

Documentation Index

Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt

Use this file to discover all available pages before exploring further.

Chapter 8: Microservices

Microservices architecture enables you to build scalable, distributed systems. NestJS provides first-class support for microservices, message brokers, and event-driven design. This chapter covers TCP, Redis, RabbitMQ, message patterns, distributed tracing, and real-world deployment tips. We’ll walk through practical examples and explain how to design robust microservices.

8.1 What is a Microservice?

A microservice is a small, independent process that communicates with others over a network. Each service is responsible for a specific business capability and can be deployed, scaled, and updated independently.

Microservices vs Monolith

Monolithic Architecture:
  • Single codebase
  • Deploy as one unit
  • Scale entire application
  • Shared database
Microservices Architecture:
  • Multiple independent services
  • Deploy services independently
  • Scale individual services
  • Each service has its own database
Analogy:
Imagine a city’s postal system. Each microservice is a different office (tax office, passport office, vehicle registration). Each office has its own building, its own staff, and its own filing system (database). They communicate by sending letters (messages) through the postal service (message broker). If the tax office burns down, the passport office keeps working. You can hire more staff at the tax office during tax season without expanding the passport office. But managing a city full of independent offices is inherently more complex than managing one big building — you need reliable mail delivery, consistent address formats, and a way to track letters across offices. That is the fundamental trade-off of microservices: operational complexity in exchange for independent scalability and resilience.

Benefits

Scalability:
  • Scale only what you need
  • Independent scaling per service
  • Optimize resources
Fault Isolation:
  • One service can fail without crashing the whole system
  • Isolated failures
  • Better resilience
Technology Diversity:
  • Use the best tool for each job
  • Different languages/frameworks per service
  • Technology flexibility
Independent Deployment:
  • Deploy and update services separately
  • Faster release cycles
  • Reduced risk
Team Autonomy:
  • Teams can work independently
  • Clear ownership
  • Faster development

Challenges

These are not hypothetical downsides — they are the daily reality of running microservices in production. Do not adopt microservices unless these challenges are worth the benefits for your specific situation.
  • Complexity: More moving parts means more things to deploy, monitor, version, and debug. A monolith with 10 modules becomes 10 services with 10 deployment pipelines, 10 health checks, and 10 log streams.
  • Network Latency: Every service-to-service call crosses the network. What was a 1ms in-process function call becomes a 5-50ms network round trip. Chained calls compound this — Service A calls B calls C means 3x latency.
  • Data Consistency: Without distributed transactions (which are hard), you rely on eventual consistency and saga patterns. This means your system may temporarily show stale or inconsistent data.
  • Testing: You cannot just run npm test — you need the entire ecosystem running (or good contract tests). E2E testing across services requires orchestration tools like Docker Compose.
  • Monitoring: A single request may touch 5 services. Without distributed tracing (OpenTelemetry, Jaeger), debugging production issues is like finding a needle in a haystack.
Honest Advice: Start with a well-structured monolith. Extract services only when you have a concrete scaling or team-autonomy reason. Most NestJS applications never need microservices — a modular monolith with clear module boundaries gives you 80% of the benefits with 20% of the complexity.

8.2 Microservices in NestJS

NestJS supports multiple transport layers for microservices, each with different characteristics.

Supported Transports

TCP:
  • Default, simple and fast
  • Direct service-to-service communication
  • Low latency
  • Good for internal services
Redis:
  • Pub/sub messaging
  • Caching support
  • Simple setup
  • Good for event-driven systems
NATS:
  • Lightweight messaging
  • High performance
  • Simple protocol
  • Good for cloud-native apps
RabbitMQ:
  • Robust message broker
  • Advanced routing
  • Reliable delivery
  • Good for complex workflows
MQTT:
  • IoT messaging
  • Lightweight protocol
  • Good for IoT applications
gRPC:
  • High-performance RPC
  • Type-safe
  • HTTP/2 based
  • Good for inter-service communication
Kafka:
  • Distributed streaming
  • High throughput
  • Event sourcing support
  • Good for big data

Transport Layer Comparison

TransportLatencyThroughputPersistenceDelivery GuaranteeComplexityBest For
TCPVery low (~1ms)HighNoneAt-most-onceLowInternal service-to-service calls
RedisLow (~2-5ms)HighOptional (Streams)At-most-once (pub/sub)LowEvent broadcasting, simple pub/sub
NATSVery low (~1ms)Very highOptional (JetStream)ConfigurableLowCloud-native, high-frequency events
RabbitMQLow (~5-10ms)HighYes (durable queues)At-least-onceMediumReliable task queues, complex routing
KafkaMedium (~10-50ms)Very highYes (log-based)At-least-onceHighEvent streaming, big data pipelines
gRPCVery low (~1ms)Very highNoneAt-most-onceMediumHigh-performance RPC, polyglot systems
MQTTLowModerateConfigurableConfigurable (QoS)LowIoT devices, low-bandwidth clients
Decision Framework — Which Transport?
Do you need guaranteed message delivery (no lost messages)?
  YES --> RabbitMQ (durable queues) or Kafka (persistent log)
  NO  --> TCP (fastest), Redis pub/sub, or NATS

Do you need to process millions of events per second?
  YES --> Kafka or NATS JetStream
  NO  --> RabbitMQ or Redis

Do consumers need to replay past messages?
  YES --> Kafka (log-based, messages persist) or NATS JetStream
  NO  --> RabbitMQ, Redis, or TCP

Are your services in different languages?
  YES --> gRPC (strongly typed contracts, code generation for all languages)
  NO  --> Any transport works, TCP is simplest for NestJS-to-NestJS

Is it IoT or constrained devices?
  YES --> MQTT
  NO  --> Any of the above
Monolith vs Microservices Decision Framework:
FactorStay MonolithConsider Microservices
Team Size1-5 developers10+ developers in independent teams
Deployment FrequencyWeekly or lessMultiple deploys per day by different teams
Scaling NeedsUniform load across featuresOne feature has 100x the load of others
Fault IsolationAcceptable if whole app goes down brieflyIndividual feature failure must not cascade
Data ComplexityShared database is manageableFeatures need independent data stores
Operational MaturityNo Kubernetes, no distributed tracingStrong DevOps, observability in place

8.3 Creating a TCP Microservice

TCP is the simplest transport for microservices. Let’s create a basic TCP microservice.

Installation

npm install @nestjs/microservices

Microservice Bootstrap

// main.ts (microservice)
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.TCP,
      options: {
        host: 'localhost',
        port: 3001,
      },
    },
  );

  await app.listen();
  console.log('Microservice is listening on port 3001');
}
bootstrap();

Message Pattern Handler

Message patterns are the microservice equivalent of HTTP routes. Instead of @Get('/users/:id'), you use @MessagePattern({ cmd: 'get_user' }). The pattern object is matched exactly — think of it as a topic or address for the message.
// app.controller.ts
import { Controller } from '@nestjs/common';
import { MessagePattern, Payload } from '@nestjs/microservices';

@Controller()
export class AppController {
  // @MessagePattern defines a request-response handler.
  // When a client sends { cmd: 'sum' } with payload [1, 2, 3],
  // this handler runs and returns the result (6) back to the caller.
  @MessagePattern({ cmd: 'sum' })
  accumulate(@Payload() data: number[]): number {
    return data.reduce((a, b) => a + b, 0);
  }

  // @Payload() extracts the data from the incoming message, similar to
  // @Body() in HTTP controllers. The message format depends on the
  // transport layer (TCP, RabbitMQ, Redis, etc.).
  @MessagePattern({ cmd: 'get_user' })
  getUser(@Payload() data: { id: number }) {
    return { id: data.id, name: 'John Doe' };
  }
}
Key Distinction: @MessagePattern is for request-response (caller waits for a reply). @EventPattern is for fire-and-forget (caller does not wait). Choosing the wrong one is a common mistake — if you use @MessagePattern but the caller uses .emit(), the handler will never fire.

Client Service

// app.service.ts
import { Injectable, Inject, OnModuleInit } from '@nestjs/common';
import { ClientProxy, ClientProxyFactory, Transport } from '@nestjs/microservices';

@Injectable()
export class AppService implements OnModuleInit {
  private client: ClientProxy;

  constructor() {
    this.client = ClientProxyFactory.create({
      transport: Transport.TCP,
      options: {
        host: 'localhost',
        port: 3001,
      },
    });
  }

  async onModuleInit() {
    await this.client.connect();
  }

  async sum(data: number[]) {
    return this.client.send({ cmd: 'sum' }, data).toPromise();
  }

  async getUser(id: number) {
    return this.client.send({ cmd: 'get_user' }, { id }).toPromise();
  }
}

Hybrid Application

Run both HTTP and microservice in the same app:
// main.ts
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';

async function bootstrap() {
  // HTTP application
  const app = await NestFactory.create(AppModule);

  // Microservice
  app.connectMicroservice<MicroserviceOptions>({
    transport: Transport.TCP,
    options: {
      host: 'localhost',
      port: 3001,
    },
  });

  await app.startAllMicroservices();
  await app.listen(3000);
  console.log('HTTP server on port 3000, Microservice on port 3001');
}
bootstrap();

8.4 Message Patterns

Message patterns define how microservices communicate. Use consistent patterns for maintainability.

Request-Response Pattern

// Handler
@MessagePattern({ cmd: 'get_user' })
getUser(@Payload() data: { id: number }) {
  return this.usersService.findOne(data.id);
}

// Client
const user = await this.client.send({ cmd: 'get_user' }, { id: 1 }).toPromise();

Event Pattern (Fire and Forget)

// Handler
@EventPattern('user_created')
handleUserCreated(@Payload() data: { id: number; email: string }) {
  console.log('User created:', data);
  // Send welcome email, etc.
}

// Client
this.client.emit('user_created', { id: 1, email: 'user@example.com' });

Multiple Patterns

@Controller()
export class UsersController {
  @MessagePattern({ cmd: 'create_user' })
  async createUser(@Payload() data: CreateUserDto) {
    return this.usersService.create(data);
  }

  @MessagePattern({ cmd: 'get_user' })
  async getUser(@Payload() data: { id: number }) {
    return this.usersService.findOne(data.id);
  }

  @EventPattern('user_updated')
  async handleUserUpdated(@Payload() data: { id: number }) {
    // Handle event
  }
}
Tip: Use clear, consistent message patterns for maintainability. Document your message contracts.

8.5 Using RabbitMQ

RabbitMQ is a popular message broker for event-driven systems. It enables publish/subscribe and queue-based communication.

Installation

npm install @nestjs/microservices amqplib amqp-connection-manager
npm install --save-dev @types/amqplib

RabbitMQ Microservice

// main.ts
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.RMQ,
      options: {
        urls: [process.env.RABBITMQ_URL || 'amqp://localhost:5672'],
        queue: 'orders_queue',
        queueOptions: {
          durable: true,
        },
      },
    },
  );

  await app.listen();
  console.log('RabbitMQ microservice is listening');
}
bootstrap();

RabbitMQ Client

// app.service.ts
import { Injectable } from '@nestjs/common';
import { ClientProxy, ClientProxyFactory, Transport } from '@nestjs/microservices';

@Injectable()
export class AppService {
  private client: ClientProxy;

  constructor() {
    this.client = ClientProxyFactory.create({
      transport: Transport.RMQ,
      options: {
        urls: [process.env.RABBITMQ_URL || 'amqp://localhost:5672'],
        queue: 'orders_queue',
        queueOptions: {
          durable: true,
        },
      },
    });
  }

  async createOrder(orderData: any) {
    return this.client.send({ cmd: 'create_order' }, orderData).toPromise();
  }

  async publishOrderCreated(order: any) {
    this.client.emit('order_created', order);
  }
}
Diagram: Event-Driven Flow
Producer Service

Publish Event → [RabbitMQ Queue]

Consumer Service(s) ← Subscribe to Queue

Process Event

8.6 Using Redis

Redis provides pub/sub messaging and can be used as a transport layer.

Redis Microservice

// main.ts
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.REDIS,
      options: {
        host: 'localhost',
        port: 6379,
      },
    },
  );

  await app.listen();
}
bootstrap();

Redis Client

this.client = ClientProxyFactory.create({
  transport: Transport.REDIS,
  options: {
    host: 'localhost',
    port: 6379,
  },
});

8.7 Event-Driven Design

Event-driven architecture decouples services and enables scalability.

Principles

Decoupling:
  • Services don’t need to know about each other
  • Communicate via events
  • Loose coupling
Scalability:
  • Publish/subscribe patterns
  • Multiple consumers
  • Horizontal scaling
Reliability:
  • Message persistence
  • Retries and dead-letter queues
  • Event replay

Event Sourcing Integration

@EventPattern('order_created')
async handleOrderCreated(@Payload() event: OrderCreatedEvent) {
  // Store event
  await this.eventStore.append('order_created', event);
  
  // Update read model
  await this.updateReadModel(event);
  
  // Publish to other services
  this.client.emit('order_created_notification', event);
}

Correlation IDs

Track requests across services:
@MessagePattern({ cmd: 'process_order' })
async processOrder(@Payload() data: any, @Ctx() context: RmqContext) {
  const correlationId = context.getMessage().properties.correlationId;
  
  // Use correlation ID for tracing
  this.logger.log(`Processing order with correlation ID: ${correlationId}`);
  
  return this.ordersService.process(data);
}

Saga Pattern for Distributed Transactions

When an operation spans multiple services (e.g., “create order, charge payment, reserve inventory”), you cannot use a database transaction because each service has its own database. The Saga pattern coordinates these steps with compensation logic for failures.
// Choreography-based Saga: each service publishes events, next service reacts
// Order Service publishes "OrderCreated"
// Payment Service listens, charges card, publishes "PaymentCompleted" or "PaymentFailed"
// Inventory Service listens to "PaymentCompleted", reserves stock
// If Payment fails, Order Service listens to "PaymentFailed" and cancels the order

// Orchestration-based Saga: a coordinator service manages the flow
@Injectable()
export class OrderSagaOrchestrator {
  async execute(orderData: CreateOrderDto) {
    let orderId: string;
    let paymentId: string;

    try {
      // Step 1: Create order (pending)
      orderId = await this.orderClient.send({ cmd: 'create_order' }, orderData).toPromise();

      // Step 2: Charge payment
      paymentId = await this.paymentClient.send({ cmd: 'charge' }, {
        orderId, amount: orderData.total,
      }).toPromise();

      // Step 3: Reserve inventory
      await this.inventoryClient.send({ cmd: 'reserve' }, {
        orderId, items: orderData.items,
      }).toPromise();

      // Step 4: Confirm order
      await this.orderClient.send({ cmd: 'confirm_order' }, { orderId }).toPromise();

    } catch (error) {
      // Compensation: undo completed steps in reverse order
      if (paymentId) {
        await this.paymentClient.send({ cmd: 'refund' }, { paymentId }).toPromise();
      }
      if (orderId) {
        await this.orderClient.send({ cmd: 'cancel_order' }, { orderId }).toPromise();
      }
      throw error;
    }
  }
}
Saga StyleCoordinationComplexityBest For
ChoreographyDecentralized (events)Lower initially, harder to debugSimple flows (2-3 steps)
OrchestrationCentralized (coordinator)Higher initially, easier to debugComplex flows (4+ steps)

8.8 Distributed Tracing & Monitoring

Monitor and trace requests across microservices.

OpenTelemetry Integration

import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common';
import { Observable } from 'rxjs';
import { trace, context } from '@opentelemetry/api';

@Injectable()
export class TracingInterceptor implements NestInterceptor {
  intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
    const tracer = trace.getTracer('nestjs');
    const span = tracer.startSpan('http_request');
    
    return next.handle().pipe(
      tap(() => span.end()),
      catchError((err) => {
        span.recordException(err);
        span.end();
        throw err;
      }),
    );
  }
}

Logging with Correlation IDs

@Injectable()
export class LoggerService {
  log(message: string, correlationId?: string) {
    console.log(JSON.stringify({
      message,
      correlationId,
      timestamp: new Date().toISOString(),
    }));
  }
}

8.9 Error Handling

Handle errors gracefully in microservices.

Error Handling Pattern

@MessagePattern({ cmd: 'process_order' })
async processOrder(@Payload() data: any) {
  try {
    return await this.ordersService.process(data);
  } catch (error) {
    // Log error
    this.logger.error('Failed to process order', error);
    
    // Emit error event
    this.client.emit('order_processing_failed', {
      orderId: data.id,
      error: error.message,
    });
    
    throw error;
  }
}

Retry Logic

In distributed systems, transient failures (network blips, temporary service unavailability) are normal, not exceptional. Retrying with backoff is the standard response — but be careful about retrying non-idempotent operations. Retrying a “charge credit card” command without idempotency keys will double-charge your customer.
import { retry, catchError, delay } from 'rxjs/operators';

this.client.send({ cmd: 'process_order' }, data).pipe(
  // retry(3) will retry up to 3 times on failure.
  // For production, use retryWhen() with exponential backoff:
  // retry 1 after 1s, retry 2 after 2s, retry 3 after 4s.
  retry(3),
  catchError((error) => {
    // All retries exhausted. Log, send to dead-letter queue, or alert.
    this.logger.error(`Failed after 3 retries: ${error.message}`);
    return throwError(() => error);
  }),
).toPromise();
Production Pattern: Combine retries with circuit breakers. After N consecutive failures, the circuit breaker “opens” and stops making requests for a cooldown period, preventing cascade failures across services. Libraries like opossum integrate well with NestJS.

Error Handling Decision Framework for Microservices

Error TypeRetry?Compensation?Example
Transient (network timeout, temporary unavailability)Yes, with exponential backoffNoConnection refused, 503 from downstream
Permanent (invalid data, business rule violation)No (retrying will never succeed)NoInvalid order total, user not found
Partial failure (one step in a saga fails)Depends on the failed stepYes (undo completed steps)Payment succeeded but inventory reservation failed
Poison message (message that always fails processing)No (move to dead-letter queue)DependsMalformed JSON, missing required field
Message arrives
    |
    +--> Process successfully? --> ACK and continue
    |
    +--> Transient error? --> Retry with backoff (max 3 retries)
    |       |
    |       +--> Still failing? --> Move to dead-letter queue + alert
    |
    +--> Permanent error? --> Move to dead-letter queue immediately
    |
    +--> Partial saga failure? --> Execute compensation logic

8.10 Best Practices

Following best practices ensures your microservices are robust and maintainable.

Service Design

  • Keep services small and focused (single responsibility)
  • Define clear service boundaries
  • Use domain-driven design
  • Document service contracts

Communication

  • Use message brokers for async communication
  • Use request-response for synchronous needs
  • Implement circuit breakers
  • Handle timeouts gracefully

Data Management

  • Each service owns its data
  • Avoid shared databases
  • Use event sourcing for consistency
  • Implement saga pattern for distributed transactions

Security

  • Secure communication between services (TLS)
  • Authenticate service-to-service calls
  • Use API keys or mTLS
  • Validate all inputs

Deployment

  • Containerize services (Docker)
  • Use orchestration (Kubernetes)
  • Implement health checks
  • Enable auto-scaling

Monitoring

  • Log all important events
  • Use distributed tracing
  • Monitor latency and errors
  • Set up alerts

Documentation

  • Document message patterns
  • Document service APIs
  • Maintain service registry
  • Keep architecture diagrams updated

8.11 Summary

You’ve learned how to build and scale microservices with NestJS: Key Concepts:
  • Microservices: Independent, scalable services
  • Transport Layers: TCP, Redis, RabbitMQ, NATS, gRPC
  • Message Patterns: Request-response and event patterns
  • Event-Driven: Decoupled, scalable architecture
  • Distributed Tracing: Monitor requests across services
Best Practices:
  • Keep services small and focused
  • Use message brokers for communication
  • Handle errors gracefully
  • Secure service communication
  • Monitor and trace requests
  • Document service contracts
Next Chapter: Learn about deployment, production optimization, Docker, Kubernetes, and monitoring strategies.