> ## Documentation Index
> Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Kubernetes on Windows & Linux

> Running hybrid clusters with Windows and Linux nodes

# Kubernetes on Windows & Linux

Kubernetes was originally Linux-only, but now supports Windows worker nodes, enabling hybrid clusters. This matters because many enterprises run critical workloads on .NET Framework, IIS, and SQL Server -- applications that cannot run on Linux without a costly rewrite. Hybrid clusters let you manage everything from one control plane while running each workload on its native operating system.

***

## Why Windows Containers?

Think of a hybrid Kubernetes cluster like a bilingual office: everyone follows the same company policies (Kubernetes API, scheduling, networking), but different teams work in different languages (Linux or Windows). The control plane is the management layer that speaks both.

* **Legacy Apps**: Lift-and-shift .NET Framework applications that are tightly coupled to Windows APIs (Win32, COM, registry). Rewriting them for Linux is often a multi-year project. Containerizing them on Windows buys you orchestration benefits immediately.
* **Unified Management**: Manage Windows and Linux apps with the same tool (Kubernetes), same CI/CD pipelines, same monitoring stack, same RBAC model. Without this, teams maintain two completely separate infrastructure stacks.
* **Modernization**: Gradually refactor monoliths into microservices. New services run on Linux, legacy services stay on Windows, and they communicate through Kubernetes Services and DNS. This is the "strangler fig" pattern applied to infrastructure.

***

## Architecture: Hybrid Cluster

A Kubernetes cluster can contain both Linux and Windows nodes.

* **Control Plane**: MUST run on Linux.
* **Worker Nodes**: Can be Linux or Windows.

```mermaid theme={null}
graph TD
    subgraph Control Plane (Linux)
        API[API Server]
        Sched[Scheduler]
    end
    
    subgraph Linux Worker
        KubeletL[Kubelet]
        PodL[Linux Pod (Nginx)]
    end
    
    subgraph Windows Worker
        KubeletW[Kubelet.exe]
        PodW[Windows Pod (IIS)]
    end
    
    API --- KubeletL
    API --- KubeletW
```

***

## Scheduling Workloads

You must ensure Windows Pods land on Windows Nodes and Linux Pods land on Linux Nodes.

### Using `nodeSelector`

The `kubernetes.io/os` label is automatically applied to every node by the kubelet. You do not need to add it manually -- it is a built-in label that Kubernetes populates based on the node's operating system.

```yaml theme={null}
# Windows Pod -- must explicitly target Windows nodes.
# Without the nodeSelector, the scheduler might place this on a Linux node,
# where the Windows container image simply cannot run.
apiVersion: v1
kind: Pod
metadata:
  name: iis-web
spec:
  containers:
  - name: iis
    # Windows images are large (3-5 GB). First pull can take several minutes.
    # Use a local registry or image cache to avoid slow deployments.
    image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019
  nodeSelector:
    kubernetes.io/os: windows   # Built-in label, no manual setup required
```

```yaml theme={null}
# Linux Pod -- in a hybrid cluster, you should ALSO add an explicit
# nodeSelector for Linux pods. Without it, they could theoretically
# land on a Windows node (unless you use taints to prevent it).
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:latest
  nodeSelector:
    kubernetes.io/os: linux
```

### Using Taints and Tolerations

`nodeSelector` tells Kubernetes where a pod *wants* to go. Taints and tolerations tell Kubernetes where a pod *cannot* go. In a hybrid cluster, you need both -- `nodeSelector` is the "pull" (attract to the right node) and taints are the "push" (repel from the wrong node). This belt-and-suspenders approach prevents scheduling accidents, especially from DaemonSets and system pods that do not have OS-specific selectors.

1. **Taint the Windows Node** (prevents any pod without a matching toleration from landing here):
   ```bash theme={null}
   # NoSchedule means "do not schedule new pods here unless they tolerate this taint."
   # Existing pods are NOT evicted -- use NoExecute for that.
   kubectl taint nodes win-node1 os=windows:NoSchedule
   ```

2. **Add Toleration to Windows Pod** (allows this pod to be scheduled on tainted nodes):
   ```yaml theme={null}
   spec:
     tolerations:
     - key: "os"
       operator: "Equal"
       value: "windows"
       effect: "NoSchedule"
     nodeSelector:
       kubernetes.io/os: windows   # Always pair with nodeSelector
   ```

<Tip>
  **Production gotcha**: Many Helm charts and operators deploy DaemonSets (e.g., Fluentd, Prometheus node-exporter) without OS-specific node selectors. In a hybrid cluster, these DaemonSets will attempt to schedule on Windows nodes and fail repeatedly, polluting your logs with CrashLoopBackOff errors. Taint your Windows nodes from day one to prevent this noise.
</Tip>

***

## Key Differences & Limitations

Understanding these differences is critical for capacity planning and debugging. The most common surprise for teams new to Windows containers is image size and its cascading effects on pull times, disk usage, and node scaling speed.

| Feature                   | Linux                                | Windows                                                                       |
| ------------------------- | ------------------------------------ | ----------------------------------------------------------------------------- |
| **Container Base Image**  | Small (Alpine \~5MB, Debian \~120MB) | Large (Server Core \~3GB, Nano Server \~250MB)                                |
| **Startup Time**          | Seconds                              | Seconds to Minutes (image pull dominates)                                     |
| **Networking**            | Bridge, Overlay, Host                | Host Networking not supported                                                 |
| **Privileged Containers** | Supported                            | Not Supported (HostProcess containers are the closest alternative, K8s 1.26+) |
| **Filesystem**            | Case-sensitive                       | Case-insensitive (can break apps that rely on case-sensitive paths)           |
| **Active Directory**      | Via LDAP/Kerberos                    | GMSA (Group Managed Service Accounts)                                         |
| **Container Isolation**   | Kernel namespaces                    | Process isolation (same kernel) or Hyper-V isolation (separate kernel)        |
| **OS Version Matching**   | N/A (shared kernel)                  | Container OS version must match host OS version for process isolation         |

***

## Best Practices

<AccordionGroup>
  <Accordion title="Use Taints and Tolerations" icon="shield-halved">
    Always taint Windows nodes to prevent Linux pods (like DaemonSets) from failing to start on them. This is not optional in a hybrid cluster -- it is the first thing you configure when adding Windows nodes.
  </Accordion>

  <Accordion title="Image Management" icon="image">
    Windows images are large (3-5 GB). Use a local registry mirror or pre-pull images onto nodes to avoid slow first deployments. Match the container OS version with the host OS version when using process isolation -- a mismatch causes cryptic "The operating system of the container does not match the host" errors. Use `nerdctl` or `ctr` to inspect the OS version of an image before deploying.
  </Accordion>

  <Accordion title="GMSA for Auth" icon="id-card">
    Use Group Managed Service Accounts (GMSA) for Windows pods that need to authenticate with Active Directory or access Windows file shares. This requires a CredentialSpec resource and a webhook admission controller. Plan for this early -- retrofitting GMSA onto existing deployments is painful.
  </Accordion>

  <Accordion title="Choose the Right Base Image" icon="box">
    **Windows Server Core** (\~3 GB): Full Win32 API surface. Use for .NET Framework, IIS, and apps that need the full Windows API.
    **Nano Server** (\~250 MB): Minimal footprint. Use for .NET 6+/8+ apps, Go binaries, and anything that does not need the full Win32 API.
    Always prefer Nano Server when possible -- the smaller image means faster pulls, less disk, and a smaller attack surface.
  </Accordion>
</AccordionGroup>

***

## Common Pitfalls

<Warning>
  **1. OS Version Mismatch**: Windows process isolation requires the container OS build number to match the host OS build number. LTSC 2019 containers will not run on LTSC 2022 hosts (and vice versa) in process isolation mode. Hyper-V isolation removes this restriction but adds overhead. Check build numbers with `systeminfo` on the host and `docker inspect` on the image.

  **2. DaemonSet Failures on Windows Nodes**: Most monitoring and logging DaemonSets (Fluentd, Datadog, Prometheus node-exporter) are Linux-only. Without taints on your Windows nodes, these DaemonSets will schedule there and enter CrashLoopBackOff, creating noise in your monitoring and wasting scheduler cycles.

  **3. Windows Node Scaling Latency**: Because Windows base images are 3-5 GB, the first pod on a freshly scaled Windows node takes minutes to start (image pull time). In autoscaling scenarios, this can violate SLAs. Mitigate with pre-pulled images, image caching (e.g., Spegel), or warm node pools.

  **4. Assuming Linux Tooling Works**: Common debugging tools (`kubectl exec -- /bin/bash`, `ps aux`, `curl`) do not exist in Windows containers by default. You need to use `cmd.exe` or `powershell.exe` instead, and many slim Windows images do not include PowerShell. Plan your debugging strategy before production.

  **5. Network Policy Gaps**: Not all CNI plugins support NetworkPolicy enforcement on Windows nodes. Check your CNI documentation. Calico has Windows support, but Flannel (Windows) does not enforce NetworkPolicies, leaving your Windows pods without network-level isolation.
</Warning>

***

## Interview Questions & Answers

<AccordionGroup>
  <Accordion title="Can the Kubernetes control plane run on Windows?" icon="circle-question">
    **No.** The control plane (API server, etcd, scheduler, controller manager) must run on Linux. Only worker nodes can be Windows. This is a fundamental architectural constraint -- etcd and the control plane components are not available as Windows binaries. In a hybrid cluster, you need at least one Linux node for the control plane and system DaemonSets.
  </Accordion>

  <Accordion title="How do you ensure pods land on the correct OS node?" icon="circle-question">
    Use a two-layer approach: 1) **nodeSelector** with `kubernetes.io/os: windows` (or `linux`) to attract pods to the right nodes, and 2) **Taints and tolerations** to repel pods from the wrong nodes. The nodeSelector alone is not enough because DaemonSets and third-party charts may not include selectors. Taints provide the safety net.
  </Accordion>

  <Accordion title="What is process isolation vs Hyper-V isolation?" icon="circle-question">
    **Process isolation**: Containers share the host kernel. Fast, lightweight, but requires the container OS version to match the host OS version exactly. This is the default in Kubernetes.

    **Hyper-V isolation**: Each container runs in a lightweight VM with its own kernel. Slower and heavier, but allows running different Windows versions on the same host. Not natively supported in Kubernetes without special configuration.
  </Accordion>
</AccordionGroup>

***

**Congratulations!** You have completed the Kubernetes Crash Course and the entire DevOps Tools Mastery course!

## Interview Deep-Dive

<AccordionGroup>
  <Accordion title="Your company wants to run a legacy .NET Framework 4.8 application alongside Linux microservices in the same Kubernetes cluster. What are the architectural constraints and operational challenges?" icon="circle-question">
    **Strong Answer:**

    * The control plane must run on Linux -- there is no Windows control plane. Windows nodes are workers only.
    * Every pod spec must declare which OS it targets using `nodeSelector: kubernetes.io/os: windows` (or linux). Without this, Kubernetes might schedule a Linux container on a Windows node, which fails immediately.
    * DaemonSets are the biggest pain point. Fluentd, node-exporter, and CNI plugins are Linux containers and fail on Windows nodes. You need Windows-specific DaemonSets for monitoring, and those tools have less mature Windows support.
    * Windows images are huge (Server Core is 3+ GB vs 5MB Alpine). Pull times affect deployment speed and autoscaling responsiveness. Use image caching or pre-pulling.
    * Windows containers do not support privileged mode, so security tools relying on privileged access will not work.

    **Follow-up: What is the difference between Process Isolation and Hyper-V Isolation for Windows containers?**

    Process Isolation runs the container on the host kernel directly, requiring the container OS build to match the host OS build exactly. Hyper-V Isolation runs each container in a lightweight VM, allowing version mismatches but adding overhead. In Kubernetes, Process Isolation is the default. This means you must pin your container base image version to match your node OS version. In managed Kubernetes (AKS), the node OS version is managed for you.
  </Accordion>

  <Accordion title="How do you handle Active Directory authentication for Windows containers in Kubernetes?" icon="circle-question">
    **Strong Answer:**

    * Windows containers cannot join a domain directly, so Kubernetes supports Group Managed Service Accounts (GMSA). A GMSA is a domain account that multiple machines share without knowing the password -- the domain controller manages credentials.
    * Setup: create a GMSA in Active Directory, configure a `GMSACredentialSpec` resource in Kubernetes, and reference it in the pod's `securityContext.windowsOptions.gmsaCredentialSpecName`.
    * The kubelet on the Windows node retrieves credentials from AD and makes them available to the container. The application authenticates to SQL Server, file shares, or other AD services using Kerberos.
    * The operational challenge: Windows nodes must be domain-joined. This complicates autoscaling because new nodes need domain joining before they can run GMSA-enabled pods.

    **Follow-up: What happens if the domain controller is unreachable when a GMSA-enabled pod starts?**

    The pod fails to start because the kubelet cannot retrieve the GMSA credential spec. If the DC becomes unreachable after startup, existing Kerberos tickets continue working until they expire (typically 10 hours). After expiry, authentication fails. This is why redundant domain controllers and reliable network connectivity from Windows nodes to AD are critical.
  </Accordion>

  <Accordion title="You need to run GPU nodes for ML, Windows nodes for .NET, and standard Linux nodes in the same cluster. How do you ensure workloads land on the right nodes?" icon="circle-question">
    **Strong Answer:**

    * Use taints and tolerations (repel wrong workloads) combined with nodeSelectors (attract right workloads).
    * GPU nodes: taint with `nvidia.com/gpu=true:NoSchedule`. Only ML pods with the toleration and a GPU resource request land there. Without the taint, non-GPU workloads consume GPU node resources.
    * Windows nodes: taint with `os=windows:NoSchedule`. Only Windows pods with the toleration and `nodeSelector: kubernetes.io/os: windows` land there. Prevents Linux DaemonSets from failing on Windows nodes.
    * Standard Linux nodes: no special taints, but add `nodeSelector: kubernetes.io/os: linux` to Linux workloads as a safety net.
    * For cost optimization, GPU and Windows nodes should be in separate autoscaling node pools that can scale to zero when idle.

    **Follow-up: What is the difference between a taint with NoSchedule versus NoExecute?**

    `NoSchedule` prevents new pods from scheduling on the node but leaves existing pods alone. `NoExecute` also evicts existing pods that do not tolerate the taint. You can add `tolerationSeconds` to NoExecute tolerations for graceful eviction -- "tolerate for 300 seconds, then get evicted" gives in-flight requests time to complete before the pod is removed.
  </Accordion>
</AccordionGroup>

***

Next: [Back to Overview →](/courses/devops-tools/overview)