> ## Documentation Index
> Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Docker Networking & Volumes
> Mastering container communication and data persistence
# Docker Networking & Volumes
Learn how containers communicate with each other and the outside world, and how to persist data.
***
## Docker Networking
Docker provides several network drivers to control how containers communicate.
### 1. Bridge Network (Default)
The default network driver. Think of a bridge network like an office floor with its own private ethernet switch -- containers plugged into the same switch can talk to each other, but they are isolated from containers on a different switch.
* **Isolation**: Containers on different bridge networks cannot communicate directly.
* **Use Case**: Standalone containers communicating on the same host.
```bash theme={null}
# Create a custom bridge network (always prefer this over the default bridge)
# The default bridge network does NOT support DNS resolution by container name.
docker network create my-net
# Run containers on this network -- they discover each other by service name
docker run -d --name db --network my-net postgres
docker run -d --name app --network my-net my-app
# 'app' can now reach 'db' using the hostname "db" -- no hardcoded IPs needed
```
**Why custom networks over the default bridge?** The default `bridge` network only supports communication via IP address, which changes every time a container restarts. Custom bridge networks provide automatic DNS resolution by container name, making your setup resilient to restarts and redeployments.
### 2. Host Network
Removes network isolation between the container and the Docker host. The container shares the host's network stack directly -- no virtual bridge, no NAT translation, no port mapping. It is like removing the office wall entirely and sitting your app at the receptionist's desk.
* **Performance**: Best throughput because there is zero network overhead from NAT or bridge forwarding.
* **Port Conflicts**: Container ports bind directly to host ports. Two containers cannot both use port 80.
* **Linux only**: Host networking does not work on Docker Desktop for macOS or Windows (Docker runs inside a VM there, so "host" means the VM, not your laptop).
```bash theme={null}
docker run --network host nginx
# Nginx is now accessible on the host's port 80 directly -- no -p flag needed
```
### 3. None Network
Disables all networking.
* **Use Case**: Security-sensitive batch jobs that don't need network access.
```bash theme={null}
docker run --network none alpine
```
### DNS Resolution
Docker has an embedded DNS server.
* On default `bridge` network: DNS by container name is **NOT** supported (only IP).
* On **custom** networks: DNS by container name **IS** supported.
***
## Docker Volumes
Containers are ephemeral -- their filesystem is a read-write layer that vanishes when the container is removed. This is great for stateless services but a disaster for databases. **Volumes** are Docker's mechanism for persistent storage that outlives any individual container.
Think of it this way: the container is a whiteboard (wiped clean when erased), but a volume is a filing cabinet in the room (stays even if you demolish the room).
### 1. Named Volumes (Recommended)
Managed entirely by Docker. Stored in `/var/lib/docker/volumes/` (on Linux). Docker handles permissions, cleanup, and backup tooling.
```bash theme={null}
# Create a named volume
docker volume create db-data
# Mount the volume to the Postgres data directory.
# Even if you "docker rm" this container, the data survives in the volume.
docker run -d \
-v db-data:/var/lib/postgresql/data \
postgres
```
### 2. Bind Mounts
Maps a specific file or directory on the **host** filesystem directly into the container. The container sees exactly what is on your disk, in real time.
* **Use Case**: Development workflows where you want live code reloading, or sharing configuration files.
* **Caution**: Bind mounts tie your container to a specific host path, making it non-portable.
```bash theme={null}
# Mount current directory to /app -- file changes on your laptop
# are instantly visible inside the container (great for hot reload)
docker run -d \
-v $(pwd):/app \
node:18-alpine
```
### 3. Tmpfs Mounts
Stored in the host's memory only. Never written to disk. Data is lost when the container stops.
* **Use Case**: Sensitive secrets that should not persist on disk, or high-performance scratch space for temporary computations.
```bash theme={null}
# Secrets are stored in RAM only -- never touch the filesystem
docker run --tmpfs /run/secrets my-app
```
**Production rule of thumb**: Use named volumes for databases and persistent state, bind mounts for local development, and tmpfs for secrets or throwaway temp files. If you find yourself using bind mounts in production, reconsider -- named volumes or external storage (NFS, cloud block storage) are almost always a better choice.
***
## Managing Networks & Volumes
```bash theme={null}
# List networks
docker network ls
# Inspect network (see connected containers)
docker network inspect my-net
# Remove unused networks
docker network prune
# List volumes
docker volume ls
# Remove unused volumes
docker volume prune
```
***
## Key Takeaways
* Always use **Custom Bridge Networks** for inter-container communication -- the default bridge does not support DNS resolution by container name.
* Use **Named Volumes** for database persistence -- without them, `docker rm` destroys your data permanently.
* Use **Bind Mounts** for development workflows (live code reloading), but avoid them in production.
* Use **Host Networking** only when you need maximum throughput and can accept the trade-off of no network isolation. Remember it only works natively on Linux.
* Use **multiple networks** to enforce security boundaries -- a database should not be on the same network as a public-facing proxy.
***
## Docker Networking Deep Dive
### How Bridge Networking Works
```
┌─────────────────────────────────────────────────────────┐
│ Host │
│ ┌─────────────────────────────────────────────────┐ │
│ │ docker0 (bridge) │ │
│ │ 172.17.0.1 │ │
│ └──────┬─────────────────┬─────────────────┬───────┘ │
│ │ │ │ │
│ ┌────┴────┐ ┌────┴────┐ ┌────┴────┐ │
│ │Container│ │Container│ │Container│ │
│ │172.17.0.2│ │172.17.0.3│ │172.17.0.4│ │
│ └─────────┘ └─────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────┘
```
### iptables Rules
Docker manipulates iptables for networking:
```bash theme={null}
# View Docker's iptables rules
sudo iptables -t nat -L -n
# DNAT rule for port mapping (-p 8080:80)
-A DOCKER -p tcp --dport 8080 -j DNAT --to-destination 172.17.0.2:80
```
### DNS Resolution
Docker's embedded DNS server (127.0.0.11):
```bash theme={null}
# Inside container
cat /etc/resolv.conf
# nameserver 127.0.0.11
# Resolve other containers by name
ping db # Works on custom networks
```
***
## Overlay Networks (Multi-Host)
For containers across multiple Docker hosts (Swarm mode or Kubernetes). Overlay networks solve a hard problem: how do you make containers on different physical machines communicate as if they were on the same LAN? The answer is VXLAN tunneling -- packets are encapsulated inside UDP datagrams and sent over the physical network, then de-encapsulated at the destination host.
```bash theme={null}
# Create overlay network -- requires Swarm mode (docker swarm init)
docker network create --driver overlay my-overlay
# Services on different hosts can communicate by name, as if they were local.
# Docker handles the VXLAN tunneling transparently.
docker service create --network my-overlay --name web nginx
docker service create --network my-overlay --name api myapi
```
### How Overlay Works
```
┌────────────────────────┐ ┌────────────────────────┐
│ Host 1 │ │ Host 2 │
│ ┌──────────────────┐ │ │ ┌──────────────────┐ │
│ │ Container A │ │ │ │ Container B │ │
│ │ 10.0.0.2 │ │ │ │ 10.0.0.3 │ │
│ └────────┬─────────┘ │ │ └────────┬─────────┘ │
│ │ │ │ │ │
│ ┌────────┴─────────┐ │ │ ┌────────┴─────────┐ │
│ │ VXLAN Tunnel │◄─┼─────┼─►│ VXLAN Tunnel │ │
│ └──────────────────┘ │ │ └──────────────────┘ │
└────────────────────────┘ └────────────────────────┘
```
***
## Container Network Interface (CNI)
CNI is the standard for container networking plugins (used by Kubernetes).
| Plugin | Features |
| ----------- | ----------------------------- |
| **Calico** | Network policies, BGP routing |
| **Cilium** | eBPF-based, advanced security |
| **Flannel** | Simple overlay, easy setup |
| **Weave** | Encryption, multicast |
***
## Volume Drivers
Beyond local storage, Docker supports pluggable volume drivers.
| Driver | Description |
| ------------- | ----------------------- |
| **local** | Default, stores on host |
| **nfs** | Network File System |
| **aws** | AWS EBS/EFS |
| **azure** | Azure File/Disk |
| **glusterfs** | Distributed storage |
```bash theme={null}
# Create volume with specific driver
docker volume create --driver=nfs \
--opt share=192.168.1.1:/data \
nfs-data
```
***
## Docker Compose Networking
This example demonstrates network segmentation -- a security best practice where different tiers of your application live on different networks. The API service bridges both networks because it needs to serve web requests and query the database.
```yaml theme={null}
version: '3.8'
services:
web:
image: nginx
networks:
- frontend # Can talk to api but NOT directly to db
ports:
- "80:80"
api:
image: myapi
networks:
- frontend # Receives requests from nginx
- backend # Connects to database -- api is the bridge between tiers
db:
image: postgres
networks:
- backend # Only reachable from the backend network
volumes:
- db-data:/var/lib/postgresql/data
networks:
frontend:
driver: bridge
backend:
driver: bridge
internal: true # internal: true = no route to the host or internet!
# The database is completely unreachable from outside Docker.
volumes:
db-data:
```
**Why this matters**: If an attacker compromises the nginx container, they can only reach services on the `frontend` network (the API). They cannot connect to the database directly because it is on the `backend` network, which nginx is not part of. This is defense in depth -- even a compromised front-end does not give direct database access.
### Network Aliases
```yaml theme={null}
services:
db:
image: postgres
networks:
backend:
aliases:
- database
- postgres-primary
```
***
## Interview Questions & Answers
| Driver | Scope | Use Case |
| ----------- | ----------- | ------------------------------------- |
| **bridge** | Single host | Default, container-to-container |
| **host** | Single host | No network isolation, max performance |
| **none** | Single host | No networking |
| **overlay** | Multi-host | Docker Swarm/K8s |
| **macvlan** | Single host | Container gets MAC address on LAN |
**Same bridge network:**
1. Container A sends packet to Container B's name
2. Docker DNS resolves name to IP
3. Packet goes through bridge interface
4. iptables routes to Container B
**Different networks:**
* Cannot communicate directly
* Need a container connected to both networks, or use host network
| Aspect | Named Volume | Bind Mount |
| --------------- | -------------------------- | -------------------- |
| **Location** | Docker manages | You specify path |
| **Portability** | Portable | Tied to host |
| **Backup** | `docker volume` commands | Standard file tools |
| **Performance** | Better on non-Linux | Native |
| **Use Case** | Databases, persistent data | Development, configs |
Use **named volumes**:
```bash theme={null}
docker volume create db-data
docker run -v db-data:/var/lib/postgresql/data postgres
```
**Important:**
* Volume survives container deletion
* Back up with `docker run --volumes-from backup-container`
* For production, consider external volume drivers (NFS, cloud storage)
macvlan assigns a MAC address to the container, making it appear as a physical device on the network.
**Use Cases:**
* Legacy apps requiring specific MAC addresses
* Apps needing to appear as physical hosts
* Direct LAN access without port mapping
```bash theme={null}
docker network create -d macvlan \
--subnet=192.168.1.0/24 \
--gateway=192.168.1.1 \
-o parent=eth0 \
my-macvlan
```
```bash theme={null}
# Inspect network
docker network inspect bridge
# Check container's network
docker inspect --format='{{.NetworkSettings.Networks}}' container
# Test connectivity from inside container
docker exec -it container ping other-container
# Check DNS resolution
docker exec -it container nslookup db
# Check iptables rules
sudo iptables -t nat -L -n | grep DOCKER
# Use network debug container
docker run --net container:target nicolaka/netshoot
```
***
## Common Pitfalls
**1. Default Bridge DNS**: The default bridge network doesn't support DNS resolution by container name. Use custom networks.
**2. Not Using Custom Networks**: Containers on default bridge can only communicate by IP, which changes.
**3. Data Loss Without Volumes**: Container filesystem is ephemeral. Always use volumes for persistent data.
**4. Bind Mounts in Production**: Bind mounts tie containers to specific host paths. Use named volumes.
**5. Port Conflicts**: Multiple containers can't bind to the same host port. Plan port mappings.
**6. Forgetting Network Isolation**: All containers on a network can communicate. Use multiple networks for isolation.
***
## Interview Deep-Dive
**Strong Answer:**
* Bridge networking adds overhead at two layers: NAT translation (iptables DNAT for port mapping) and virtual ethernet bridge traversal. Every packet entering the container goes through the host's network stack, hits the iptables DNAT rule, crosses the docker0 bridge, traverses the veth pair, and enters the container's network namespace. At 50K requests/second, this overhead becomes measurable -- typically 5-15% throughput reduction and 50-200 microseconds of added latency per packet.
* Host networking eliminates all of this. The container shares the host's network stack directly -- no NAT, no bridge, no veth pair. The application binds directly to a host interface. This gives you native network performance.
* However, the trade-offs are significant. First, port conflicts: if two containers both want port 80, only one can have it. You lose the ability to run multiple instances of the same service on the same host without changing the application's listen port. Second, no network isolation: the container can see and access every network interface on the host, including management networks, metadata endpoints (like the cloud provider's instance metadata at 169.254.169.254), and other containers' traffic. Third, host networking only works natively on Linux. On macOS and Windows, Docker Desktop runs inside a VM, so "host" means the VM's network, not your laptop's -- which defeats the purpose.
* My recommendation: for a service handling 50K req/s, I would first benchmark the actual overhead of bridge networking with `iperf3` or `wrk`. If the latency overhead is under your SLA threshold, keep bridge networking for its isolation benefits. If you genuinely need every microsecond, use host networking but compensate with other security measures (seccomp, read-only filesystem, dropped capabilities). In Kubernetes, the equivalent is `hostNetwork: true` on the pod spec, but you sacrifice pod-to-pod isolation entirely.
**Follow-up: In Kubernetes, how does networking differ from Docker's bridge model, and what role does a CNI plugin play?**
Kubernetes does not use Docker's bridge networking. Instead, every pod gets its own IP address in a flat network -- any pod can reach any other pod by IP without NAT. This is enforced by the CNI (Container Network Interface) plugin. The CNI plugin (Calico, Cilium, Flannel, etc.) is responsible for assigning IPs, setting up routes, and optionally enforcing network policies. Calico uses BGP routing for high performance. Cilium uses eBPF to bypass iptables entirely, providing both networking and security enforcement in the kernel with lower overhead. The key architectural difference is that Kubernetes mandates a flat pod network, while Docker gives you isolated bridge networks by default.
**Strong Answer:**
* Docker runs an embedded DNS server at 127.0.0.11 inside every container on a user-defined network. When a container does a DNS lookup (e.g., `getaddrinfo("db")`), the resolver in `/etc/resolv.conf` sends the query to 127.0.0.11. Docker's DNS server checks its internal registry of container names and service names, and returns the corresponding container IP address.
* A scenario where this fails despite both containers being on the same network: the target container was started with `--hostname custom-hostname` but the source container is resolving by container name. Docker DNS registers both the container name and the hostname, but they might not match what the application expects. Another common failure: the application uses a hardcoded DNS cache (like JVM's default behavior of caching DNS forever). The target container restarts, gets a new IP, DNS updates correctly, but the source container's JVM still has the old IP cached and connections fail.
* Another subtle case: if you create a container, attach it to a network, then detach and reattach it, there can be a brief window where the DNS entry is stale. I have also seen failures when a container's internal resolver is overridden -- for example, a base image that sets a custom `/etc/resolv.conf` pointing to an external DNS server instead of Docker's 127.0.0.11.
* The debugging path is: exec into the container, check `/etc/resolv.conf` (should show `nameserver 127.0.0.11`), run `nslookup db` to test resolution, and if that fails, check `docker network inspect network-name` to verify both containers are listed as attached.
**Follow-up: How does DNS resolution work differently on the default bridge network versus a user-defined bridge?**
On the default bridge network (`docker0`), Docker does not provide DNS resolution by container name at all. Containers can only communicate by IP address, and those IPs change on every container restart. This is a legacy behavior preserved for backward compatibility. On user-defined bridges, Docker enables its embedded DNS server, which maps container names to IPs and updates dynamically. This is the primary reason the documentation (and every best-practice guide) says to always use user-defined networks. The `--link` flag on the default bridge was a workaround that is now deprecated.
**Strong Answer:**
* The container's writable layer is stored inside Docker's storage directory (typically `/var/lib/docker/overlay2/` on Linux). When the application writes 10GB/day to the container filesystem, it fills up the host's disk -- specifically, the partition where `/var/lib/docker` lives. Once the disk is full, Docker cannot create new containers, pull images, or even write logs. All containers on the host are affected, not just the offending one.
* The root cause is architectural: application data should never live in the container layer. The container layer is ephemeral (lost on `docker rm`), slow for heavy writes (copy-on-write overhead), and contributes to the Docker storage pool that all containers share.
* The correct architecture depends on the data type. For databases and persistent application state, use named volumes. Named volumes live in `/var/lib/docker/volumes/`, which can be on a separate disk or partition, and they persist independently of the container lifecycle. For temporary data (caches, scratch computation), use tmpfs mounts that store data in memory and never touch disk. For data that needs to be shared with the host or archived, use bind mounts to a dedicated data partition.
* As an operational safeguard, I would configure Docker's storage driver with a maximum container size: `--storage-opt dm.basesize=20G` (for devicemapper) or use disk quotas. I would also monitor `/var/lib/docker` disk usage with Prometheus and set alerts at 70% and 90% capacity.
* In Kubernetes, this is handled by ephemeral storage limits (`resources.limits.ephemeral-storage`), which kill the pod if it writes too much to the container layer -- preventing the node from filling up.
**Follow-up: The application team says they cannot use volumes because the data path is determined at runtime and varies per deployment. What do you suggest?**
I would suggest mounting a volume at a predictable base path (e.g., `/data`) and configuring the application to write under that path regardless of the specific subdirectory structure. If the application truly requires dynamic paths, you can mount the volume at a high-level directory and let the application create subdirectories within it. If the application writes to paths outside your control (e.g., a third-party binary that insists on writing to `/var/log/app`), use a bind mount or a symbolic link in the Dockerfile that redirects that path to the volume mount point.
***
Next: [Docker Compose →](/courses/devops-tools/docker-compose)