Skip to main content

Documentation Index

Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt

Use this file to discover all available pages before exploring further.

​
Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. Think of it as the “recipe card” for your entire application stack: instead of running five separate docker run commands with various flags you will inevitably forget, you declare everything in one YAML file and bring the whole system up with a single command.

​
The docker-compose.yml File

Compose uses a YAML file to configure your application’s services.

​
Complete Example

version: '3.8'

services:
  # 1. Frontend Service -- built from local Dockerfile
  web:
    build: 
      context: ./frontend       # Build context is the frontend directory
      dockerfile: Dockerfile    # Explicit Dockerfile path (useful if you have Dockerfile.dev too)
    ports:
      - "3000:3000"             # host:container -- access the frontend at localhost:3000
    environment:
      - NODE_ENV=development
      - API_URL=http://api:8080 # "api" resolves via Docker DNS to the backend container
    depends_on:
      - api                     # Starts api before web (but does NOT wait until api is healthy)
    networks:
      - app-net

  # 2. Backend Service -- uses a pre-built image from a registry
  api:
    image: my-api:latest
    ports:
      - "8080:8080"
    environment:
      - DB_HOST=db              # Service name "db" resolves to the database container's IP
      - DB_PASS=${DB_PASSWORD}  # Pulled from .env file -- never hardcode passwords in YAML
    depends_on:
      db:
        condition: service_healthy  # Waits until db's healthcheck passes before starting
    networks:
      - app-net

  # 3. Database Service
  db:
    image: postgres:14-alpine   # Alpine variant: smaller image, same PostgreSQL
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD}
    volumes:
      - db-data:/var/lib/postgresql/data  # Named volume so data survives container restarts
    networks:
      - app-net
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]  # pg_isready is a built-in Postgres utility
      interval: 10s             # Check every 10 seconds
      timeout: 5s               # Fail if no response in 5 seconds
      retries: 5                # Mark unhealthy after 5 consecutive failures

# 4. Volumes -- declared here so Docker manages lifecycle and cleanup
volumes:
  db-data:

# 5. Networks -- explicit network gives you DNS resolution by service name
networks:
  app-net:
    driver: bridge
Common mistake: depends_on without condition: service_healthy only controls start order, not readiness. Your API might start before Postgres is accepting connections, causing connection refused errors on first boot. Always pair depends_on with a healthcheck for databases and caches.

​
Real-World Architecture Example

A typical production-like setup with Nginx Reverse Proxy, Backend API, Redis Cache, and PostgreSQL Database. Notice the use of two separate networks — public (internet-facing) and private (internal only). This is a security pattern: the database and cache are unreachable from outside the Docker network, even if someone compromises the reverse proxy. 
version: '3.8'

services:
  # Reverse Proxy -- the ONLY service exposed to the internet.
  # All external traffic enters through nginx and is routed internally.
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"       # HTTP
      - "443:443"     # HTTPS
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro   # :ro = read-only mount (security)
      - ./certs:/etc/nginx/certs:ro             # TLS certificates
    depends_on:
      - api
      - web
    networks:
      - public        # Can receive external traffic
      - private       # Can route to internal services

  # Frontend (React/Next.js) -- only accessible via nginx, not directly from outside
  web:
    build: ./frontend
    environment:
      - API_URL=http://api:3000  # Internal service discovery via Docker DNS
    networks:
      - private       # No public network = no direct external access

  # Backend API -- talks to database and cache but is not directly exposed
  api:
    build: ./backend
    environment:
      - DB_HOST=db            # Resolved by Docker DNS to the db container
      - REDIS_HOST=redis      # Resolved by Docker DNS to the redis container
    depends_on:
      - db
      - redis
    networks:
      - private

  # Cache -- in-memory store for sessions, rate limiting, etc.
  redis:
    image: redis:alpine
    networks:
      - private

  # Database -- uses Docker secrets instead of environment variables for the password
  db:
    image: postgres:14-alpine
    volumes:
      - db_data:/var/lib/postgresql/data        # Named volume for persistence
    environment:
      - POSTGRES_PASSWORD_FILE=/run/secrets/db_password  # Read password from file, not env var
    secrets:
      - db_password   # Mounted at /run/secrets/db_password inside the container
    networks:
      - private

networks:
  public:               # Connected to the host network -- reachable from outside
  private:
    internal: true      # internal: true means NO external connectivity at all

volumes:
  db_data:

# Docker secrets: more secure than environment variables because they are
# stored encrypted at rest and mounted as tmpfs (RAM-only) in the container.
secrets:
  db_password:
    file: ./secrets/db_password.txt

​
Key Concepts

​
Services

Services are the computing units of your application. Each service becomes one or more running containers.
  • build: Build an image from a local Dockerfile. Use this for your own application code.
  • image: Pull and use a pre-built image from a registry. Use this for third-party services (databases, caches, proxies).
  • depends_on: Controls startup order. With condition: service_healthy, it waits for the dependency’s healthcheck to pass.

​
Environment Variables

  • Inline: Defined directly in YAML. Fine for non-sensitive values like NODE_ENV=development.
  • .env file: Compose automatically reads variables from a .env file in the same directory. Use this for secrets and values that change between environments.
# .env -- this file should be in .gitignore!
DB_PASSWORD=secret123
REDIS_URL=redis://cache:6379
Never commit .env files to version control. Instead, commit a .env.example with placeholder values so teammates know which variables are needed. In CI/CD, inject secrets from your pipeline’s secret store (GitHub Secrets, Vault, AWS Secrets Manager).

​
Networking

By default, Compose creates a single bridge network for your entire stack. Services can reach each other by service name as the hostname (e.g., web can connect to http://api:8080). This works because Docker runs an embedded DNS server that maps service names to container IPs. No /etc/hosts hacking required.

​
Essential Commands

# Start all services in background (detached mode)
docker-compose up -d

# Rebuild images before starting -- use this after changing Dockerfiles or source code
docker-compose up -d --build

# View logs of all services (follow mode -- streams new logs in real time)
docker-compose logs -f

# View logs of specific service -- invaluable when debugging one misbehaving container
docker-compose logs -f api

# Stop and remove containers and networks (volumes are preserved)
docker-compose down

# Stop and remove EVERYTHING including volumes (WARNING: destroys database data!)
docker-compose down -v

# List running services with their status and port mappings
docker-compose ps

# Open an interactive shell in the database container
docker-compose exec db psql -U postgres

# Scale a service to multiple instances (useful for load testing)
docker-compose up -d --scale api=3
Practical tip: On modern Docker installations (Docker Compose V2), the command is docker compose (no hyphen) instead of docker-compose. Both work, but the V2 plugin is faster and actively maintained. Check yours with docker compose version.

​
Production vs Development

​
Overriding Configuration

You can use multiple Compose files to handle different environments.
  1. docker-compose.yml: Base config.
  2. docker-compose.prod.yml: Production overrides (restart policies, extra volumes).
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

​
Key Takeaways

  • Use Docker Compose for local development and testing — it replaces a page of docker run commands with a single declarative file.
  • Use .env files to manage secrets and environment-specific configuration. Never commit them to git.
  • Use depends_on with healthchecks to ensure services start in the correct order. Without healthchecks, depends_on only guarantees start order, not readiness.
  • Use Named Volumes for database data. Without them, a docker-compose down destroys everything.
  • Use multiple Compose files (docker-compose.yml + docker-compose.prod.yml) to handle environment differences without duplicating configuration.

​
Interview Deep-Dive

Strong Answer:
  • The depends_on directive without a condition only controls container start order — it ensures the database container starts before the API container. But “started” does not mean “ready to accept connections.” PostgreSQL takes several seconds to initialize its data directory, run recovery, and begin listening on port 5432. The API container starts a fraction of a second after the database container and immediately tries to connect, hitting a refused connection.
  • The fix is to use depends_on with condition: service_healthy and add a healthcheck to the database service. For PostgreSQL, the built-in pg_isready utility is the standard check: test: ["CMD-SHELL", "pg_isready -U postgres"] with an interval of 5-10 seconds and 5 retries.
  • An alternative application-level fix is retry logic with exponential backoff in the API’s database connection code. In production, this is actually the better pattern because orchestrators like Kubernetes do not have a depends_on equivalent — services must be resilient to dependencies being temporarily unavailable.
  • The deeper lesson: startup ordering is a development convenience, not a reliability strategy. Production systems should handle dependency failures gracefully at the application level.
Follow-up: In Kubernetes, there is no depends_on equivalent. How do you handle service startup ordering there?You combine readiness probes with application-level retry logic. The API’s database client should retry connections with exponential backoff (e.g., 1s, 2s, 4s, 8s). The API’s readiness probe should fail until the database connection is established, so Kubernetes does not send traffic to the API pod until it is truly ready. Init containers can also be used to block pod startup until a dependency is reachable, but retry logic in the application is the more robust long-term pattern.
Strong Answer:
  • internal: true on a Docker network means containers on that network have zero route to the host network or the internet. They can only communicate with other containers on the same internal network. There is no NAT, no gateway, no outbound connectivity at all.
  • The classic use case is a three-tier architecture: a reverse proxy (nginx) on a public network, an API on both public and private networks, and a database on the private internal network only. The database is completely unreachable from outside Docker — even if an attacker compromises the nginx container, they cannot connect directly to the database because nginx is not on the private network.
  • This is defense in depth. The API acts as the only bridge between the tiers. If an attacker compromises nginx, they can only reach the API. If they compromise the API, they can reach the database — but they had to chain two exploits to get there, not one.
  • In practice, I combine this with read-only volume mounts (:ro) for configuration files and Docker secrets (mounted as tmpfs) for passwords. The goal is that even in a compromise scenario, the attacker’s lateral movement is maximally constrained.
Follow-up: A developer pushes back saying this is over-engineering for a small team. How do you respond?The Compose YAML to implement this is about 5 extra lines. The security benefit is that a misconfigured port mapping or a vulnerability in a public-facing service cannot directly expose the database. For a small team, this is especially important because small teams often lack dedicated security reviews. The “over-engineering” cost is trivial; the risk reduction is significant. I would also point out that this pattern becomes mandatory in any SOC 2 or PCI-DSS compliant environment, so building the habit early saves painful retrofitting later.
Strong Answer:
  • The -v flag removes named volumes along with containers and networks. Since the PostgreSQL data was stored in a named volume (db-data:/var/lib/postgresql/data), the entire database is now gone. The data is not recoverable from Docker — once a named volume is removed, the underlying directory in /var/lib/docker/volumes/ is deleted.
  • Immediate recovery depends on your backup strategy. If you have automated pg_dump backups stored externally (S3, a backup server), you restore from the most recent backup. If you used Kafka CDC or logical replication to mirror data elsewhere, you can rebuild from that source.
  • If there are no backups (worst case), the data is gone. This becomes a lesson in backup hygiene.
  • Prevention going forward: First, use a wrapper script or alias that always prompts for confirmation before running down -v. Second, implement automated database backups on a schedule (daily pg_dump to S3 at minimum). Third, for critical environments, use an external volume driver (NFS, EBS) where the volume lifecycle is managed outside of Docker Compose. Fourth, consider adding external: true to the volume definition — this tells Compose that the volume is managed externally and refuses to delete it during down -v.
Follow-up: How does the ‘external: true’ volume option work, and when would you use it versus a Compose-managed volume?When you declare external: true on a volume, Compose expects the volume to already exist (created with docker volume create manually). Compose will not create it or destroy it during up and down. This is ideal for databases in shared environments where the data lifecycle should not be tied to the application lifecycle. The trade-off is that your Compose setup is no longer fully self-contained — a new developer running docker-compose up for the first time gets an error unless they create the volume first. For development environments, Compose-managed volumes are more convenient. For staging and production, external volumes are safer.
Strong Answer:
  • When Compose creates a custom bridge network (e.g., app-net), Docker sets up an embedded DNS server at 127.0.0.11 inside each container. Every container on that network has its /etc/resolv.conf pointing to this embedded DNS.
  • When the API container does a DNS lookup for “db,” the request goes to the embedded DNS server. Docker’s DNS server maintains a mapping of container names (and service names in Compose) to their current IP addresses on that bridge network. It resolves “db” to something like 172.18.0.3.
  • Under the hood, each container gets a virtual ethernet pair (veth). One end is inside the container’s network namespace (visible as eth0), and the other end connects to the bridge interface on the host. Traffic from api to db flows through the bridge — the packet goes out of api’s veth, through the bridge, and into db’s veth.
  • If the database container restarts and gets a new IP, the DNS server updates automatically. This is why custom bridge networks are essential — the default bridge network does not support DNS resolution by container name, so you would have to hard-code IP addresses that change on every restart.
  • For cross-host communication (Docker Swarm overlay networks), VXLAN encapsulation wraps the packet in a UDP datagram that travels over the physical network to the other host, where it is de-encapsulated and delivered to the target container.
Follow-up: The API can resolve “db” but connections are timing out. The database container is running and healthy. How do you debug this?I would start by exec’ing into the API container and running nslookup db to confirm DNS is resolving. If it resolves, I would ping the IP to check basic connectivity. If ping works, I would nc -zv db 5432 to check TCP connectivity on the database port. If that fails, the database might be listening on a different interface (localhost only instead of 0.0.0.0) — check with docker exec db netstat -tlnp. I would also verify both containers are on the same network with docker network inspect app-net. If they are on different networks, they cannot communicate. Finally, I would check if any iptables rules or the host firewall are interfering.
Next: Docker Best Practices →