Skip to main content

Documentation Index

Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt

Use this file to discover all available pages before exploring further.

​
Kubernetes on Windows & Linux

Kubernetes was originally Linux-only, but now supports Windows worker nodes, enabling hybrid clusters. This matters because many enterprises run critical workloads on .NET Framework, IIS, and SQL Server — applications that cannot run on Linux without a costly rewrite. Hybrid clusters let you manage everything from one control plane while running each workload on its native operating system.

​
Why Windows Containers?

Think of a hybrid Kubernetes cluster like a bilingual office: everyone follows the same company policies (Kubernetes API, scheduling, networking), but different teams work in different languages (Linux or Windows). The control plane is the management layer that speaks both.
  • Legacy Apps: Lift-and-shift .NET Framework applications that are tightly coupled to Windows APIs (Win32, COM, registry). Rewriting them for Linux is often a multi-year project. Containerizing them on Windows buys you orchestration benefits immediately.
  • Unified Management: Manage Windows and Linux apps with the same tool (Kubernetes), same CI/CD pipelines, same monitoring stack, same RBAC model. Without this, teams maintain two completely separate infrastructure stacks.
  • Modernization: Gradually refactor monoliths into microservices. New services run on Linux, legacy services stay on Windows, and they communicate through Kubernetes Services and DNS. This is the “strangler fig” pattern applied to infrastructure.

​
Architecture: Hybrid Cluster

A Kubernetes cluster can contain both Linux and Windows nodes.
  • Control Plane: MUST run on Linux.
  • Worker Nodes: Can be Linux or Windows.

​
Scheduling Workloads

You must ensure Windows Pods land on Windows Nodes and Linux Pods land on Linux Nodes.

​
Using nodeSelector

The kubernetes.io/os label is automatically applied to every node by the kubelet. You do not need to add it manually — it is a built-in label that Kubernetes populates based on the node’s operating system. 
# Windows Pod -- must explicitly target Windows nodes.
# Without the nodeSelector, the scheduler might place this on a Linux node,
# where the Windows container image simply cannot run.
apiVersion: v1
kind: Pod
metadata:
  name: iis-web
spec:
  containers:
  - name: iis
    # Windows images are large (3-5 GB). First pull can take several minutes.
    # Use a local registry or image cache to avoid slow deployments.
    image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019
  nodeSelector:
    kubernetes.io/os: windows   # Built-in label, no manual setup required

# Linux Pod -- in a hybrid cluster, you should ALSO add an explicit
# nodeSelector for Linux pods. Without it, they could theoretically
# land on a Windows node (unless you use taints to prevent it).
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:latest
  nodeSelector:
    kubernetes.io/os: linux

​
Using Taints and Tolerations

nodeSelector tells Kubernetes where a pod wants to go. Taints and tolerations tell Kubernetes where a pod cannot go. In a hybrid cluster, you need both — nodeSelector is the “pull” (attract to the right node) and taints are the “push” (repel from the wrong node). This belt-and-suspenders approach prevents scheduling accidents, especially from DaemonSets and system pods that do not have OS-specific selectors.
  1. Taint the Windows Node (prevents any pod without a matching toleration from landing here): 
    # NoSchedule means "do not schedule new pods here unless they tolerate this taint."
# Existing pods are NOT evicted -- use NoExecute for that.
kubectl taint nodes win-node1 os=windows:NoSchedule
  2. Add Toleration to Windows Pod (allows this pod to be scheduled on tainted nodes): 
    spec:
  tolerations:
  - key: "os"
    operator: "Equal"
    value: "windows"
    effect: "NoSchedule"
  nodeSelector:
    kubernetes.io/os: windows   # Always pair with nodeSelector
Production gotcha: Many Helm charts and operators deploy DaemonSets (e.g., Fluentd, Prometheus node-exporter) without OS-specific node selectors. In a hybrid cluster, these DaemonSets will attempt to schedule on Windows nodes and fail repeatedly, polluting your logs with CrashLoopBackOff errors. Taint your Windows nodes from day one to prevent this noise.

​
Key Differences & Limitations

Understanding these differences is critical for capacity planning and debugging. The most common surprise for teams new to Windows containers is image size and its cascading effects on pull times, disk usage, and node scaling speed.
FeatureLinuxWindows
Container Base ImageSmall (Alpine ~5MB, Debian ~120MB)Large (Server Core ~3GB, Nano Server ~250MB)
Startup TimeSecondsSeconds to Minutes (image pull dominates)
NetworkingBridge, Overlay, HostHost Networking not supported
Privileged ContainersSupportedNot Supported (HostProcess containers are the closest alternative, K8s 1.26+)
FilesystemCase-sensitiveCase-insensitive (can break apps that rely on case-sensitive paths)
Active DirectoryVia LDAP/KerberosGMSA (Group Managed Service Accounts)
Container IsolationKernel namespacesProcess isolation (same kernel) or Hyper-V isolation (separate kernel)
OS Version MatchingN/A (shared kernel)Container OS version must match host OS version for process isolation

​
Best Practices

Always taint Windows nodes to prevent Linux pods (like DaemonSets) from failing to start on them. This is not optional in a hybrid cluster — it is the first thing you configure when adding Windows nodes.
Windows images are large (3-5 GB). Use a local registry mirror or pre-pull images onto nodes to avoid slow first deployments. Match the container OS version with the host OS version when using process isolation — a mismatch causes cryptic “The operating system of the container does not match the host” errors. Use nerdctl or ctr to inspect the OS version of an image before deploying.
Use Group Managed Service Accounts (GMSA) for Windows pods that need to authenticate with Active Directory or access Windows file shares. This requires a CredentialSpec resource and a webhook admission controller. Plan for this early — retrofitting GMSA onto existing deployments is painful.
Windows Server Core (~3 GB): Full Win32 API surface. Use for .NET Framework, IIS, and apps that need the full Windows API. Nano Server (~250 MB): Minimal footprint. Use for .NET 6+/8+ apps, Go binaries, and anything that does not need the full Win32 API. Always prefer Nano Server when possible — the smaller image means faster pulls, less disk, and a smaller attack surface.

​
Common Pitfalls

1. OS Version Mismatch: Windows process isolation requires the container OS build number to match the host OS build number. LTSC 2019 containers will not run on LTSC 2022 hosts (and vice versa) in process isolation mode. Hyper-V isolation removes this restriction but adds overhead. Check build numbers with systeminfo on the host and docker inspect on the image.2. DaemonSet Failures on Windows Nodes: Most monitoring and logging DaemonSets (Fluentd, Datadog, Prometheus node-exporter) are Linux-only. Without taints on your Windows nodes, these DaemonSets will schedule there and enter CrashLoopBackOff, creating noise in your monitoring and wasting scheduler cycles.3. Windows Node Scaling Latency: Because Windows base images are 3-5 GB, the first pod on a freshly scaled Windows node takes minutes to start (image pull time). In autoscaling scenarios, this can violate SLAs. Mitigate with pre-pulled images, image caching (e.g., Spegel), or warm node pools.4. Assuming Linux Tooling Works: Common debugging tools (kubectl exec -- /bin/bash, ps aux, curl) do not exist in Windows containers by default. You need to use cmd.exe or powershell.exe instead, and many slim Windows images do not include PowerShell. Plan your debugging strategy before production.5. Network Policy Gaps: Not all CNI plugins support NetworkPolicy enforcement on Windows nodes. Check your CNI documentation. Calico has Windows support, but Flannel (Windows) does not enforce NetworkPolicies, leaving your Windows pods without network-level isolation.

​
Interview Questions & Answers

No. The control plane (API server, etcd, scheduler, controller manager) must run on Linux. Only worker nodes can be Windows. This is a fundamental architectural constraint — etcd and the control plane components are not available as Windows binaries. In a hybrid cluster, you need at least one Linux node for the control plane and system DaemonSets.
Use a two-layer approach: 1) nodeSelector with kubernetes.io/os: windows (or linux) to attract pods to the right nodes, and 2) Taints and tolerations to repel pods from the wrong nodes. The nodeSelector alone is not enough because DaemonSets and third-party charts may not include selectors. Taints provide the safety net.
Process isolation: Containers share the host kernel. Fast, lightweight, but requires the container OS version to match the host OS version exactly. This is the default in Kubernetes.Hyper-V isolation: Each container runs in a lightweight VM with its own kernel. Slower and heavier, but allows running different Windows versions on the same host. Not natively supported in Kubernetes without special configuration.
Congratulations! You have completed the Kubernetes Crash Course and the entire DevOps Tools Mastery course!

​
Interview Deep-Dive

Strong Answer:
  • The control plane must run on Linux — there is no Windows control plane. Windows nodes are workers only.
  • Every pod spec must declare which OS it targets using nodeSelector: kubernetes.io/os: windows (or linux). Without this, Kubernetes might schedule a Linux container on a Windows node, which fails immediately.
  • DaemonSets are the biggest pain point. Fluentd, node-exporter, and CNI plugins are Linux containers and fail on Windows nodes. You need Windows-specific DaemonSets for monitoring, and those tools have less mature Windows support.
  • Windows images are huge (Server Core is 3+ GB vs 5MB Alpine). Pull times affect deployment speed and autoscaling responsiveness. Use image caching or pre-pulling.
  • Windows containers do not support privileged mode, so security tools relying on privileged access will not work.
Follow-up: What is the difference between Process Isolation and Hyper-V Isolation for Windows containers?Process Isolation runs the container on the host kernel directly, requiring the container OS build to match the host OS build exactly. Hyper-V Isolation runs each container in a lightweight VM, allowing version mismatches but adding overhead. In Kubernetes, Process Isolation is the default. This means you must pin your container base image version to match your node OS version. In managed Kubernetes (AKS), the node OS version is managed for you.
Strong Answer:
  • Windows containers cannot join a domain directly, so Kubernetes supports Group Managed Service Accounts (GMSA). A GMSA is a domain account that multiple machines share without knowing the password — the domain controller manages credentials.
  • Setup: create a GMSA in Active Directory, configure a GMSACredentialSpec resource in Kubernetes, and reference it in the pod’s securityContext.windowsOptions.gmsaCredentialSpecName.
  • The kubelet on the Windows node retrieves credentials from AD and makes them available to the container. The application authenticates to SQL Server, file shares, or other AD services using Kerberos.
  • The operational challenge: Windows nodes must be domain-joined. This complicates autoscaling because new nodes need domain joining before they can run GMSA-enabled pods.
Follow-up: What happens if the domain controller is unreachable when a GMSA-enabled pod starts?The pod fails to start because the kubelet cannot retrieve the GMSA credential spec. If the DC becomes unreachable after startup, existing Kerberos tickets continue working until they expire (typically 10 hours). After expiry, authentication fails. This is why redundant domain controllers and reliable network connectivity from Windows nodes to AD are critical.
Strong Answer:
  • Use taints and tolerations (repel wrong workloads) combined with nodeSelectors (attract right workloads).
  • GPU nodes: taint with nvidia.com/gpu=true:NoSchedule. Only ML pods with the toleration and a GPU resource request land there. Without the taint, non-GPU workloads consume GPU node resources.
  • Windows nodes: taint with os=windows:NoSchedule. Only Windows pods with the toleration and nodeSelector: kubernetes.io/os: windows land there. Prevents Linux DaemonSets from failing on Windows nodes.
  • Standard Linux nodes: no special taints, but add nodeSelector: kubernetes.io/os: linux to Linux workloads as a safety net.
  • For cost optimization, GPU and Windows nodes should be in separate autoscaling node pools that can scale to zero when idle.
Follow-up: What is the difference between a taint with NoSchedule versus NoExecute?NoSchedule prevents new pods from scheduling on the node but leaves existing pods alone. NoExecute also evicts existing pods that do not tolerate the taint. You can add tolerationSeconds to NoExecute tolerations for graceful eviction — “tolerate for 300 seconds, then get evicted” gives in-flight requests time to complete before the pod is removed.
Next: Back to Overview →