The process subsystem is the heart of Linux. Understanding task_struct, process creation, and the scheduler is essential for infrastructure engineers who need to debug performance issues and understand container behavior.
Interview Frequency: Very High Key Topics: task_struct, clone/fork, CFS scheduler, CPU affinity Time to Master: 14-16 hours
Why is task_struct so large? Because it’s the kernel’s complete representation of a process. Every subsystem needs to track its own data about each process:
The scheduler needs priority and runtime statistics
Memory management needs page tables and memory limits
The filesystem needs current directory and open files
Security needs credentials and capabilities
Signals need pending signals and handlers
Rather than scattering this data across multiple global structures (which would require expensive lookups), Linux packs everything into one structure. This makes context switches faster - the kernel just needs to switch the current pointer to a different task_struct.Think of task_struct as the DNA of a process. It contains absolutely everything the kernel needs to know to manage that process. If it’s not in task_struct (or structures linked from it), the kernel doesn’t know about it.It acts as a “process control block” (PCB) and tracks:
State: Is it running? Waiting? Zombie?
Resources: What files are open? How much memory is used?
Identity: Who owns it? What group is it in?
Relationships: Who is the parent? Who are the children?
Scheduling: How much CPU time does it deserve?
Copy
// include/linux/sched.h (simplified)struct task_struct { // Scheduler state volatile long state; // TASK_RUNNING, TASK_INTERRUPTIBLE, etc. unsigned int flags; // PF_EXITING, PF_KTHREAD, etc. int prio, static_prio, normal_prio; struct sched_entity se; // CFS scheduling entity struct sched_rt_entity rt; // Real-time scheduling entity unsigned int policy; // SCHED_NORMAL, SCHED_FIFO, etc. // CPU and preemption int on_cpu; // Currently running on CPU int on_rq; // On runqueue cpumask_t cpus_mask; // Allowed CPUs // Identity pid_t pid; // Process ID pid_t tgid; // Thread group ID (= pid of main thread) // Process relationships struct task_struct *real_parent; // Real parent struct task_struct *parent; // Current parent (may differ due to ptrace) struct list_head children; // List of children struct list_head sibling; // Linkage in parent's children list struct task_struct *group_leader; // Thread group leader // Memory management struct mm_struct *mm; // Memory descriptor (NULL for kernel threads) struct mm_struct *active_mm; // Active memory descriptor // Filesystem struct fs_struct *fs; // Filesystem info (cwd, root) struct files_struct *files; // Open file descriptors // Namespaces struct nsproxy *nsproxy; // Namespace proxy // Credentials const struct cred *cred; // Credentials (uid, gid, capabilities) // Signals struct signal_struct *signal; struct sighand_struct *sighand; sigset_t blocked; // Blocked signals // Timing u64 utime, stime; // User and system time u64 start_time; // Process start time // And much more...};
The clone() system call is the Swiss Army knife of process creation. Unlike fork() which copies everything, clone() allows you to selectively choose exactly what to share and what to copy.All process/thread creation goes through clone():
Copy
long clone(unsigned long flags, void *child_stack, int *ptid, int *ctid, unsigned long tls);
The Completely Fair Scheduler (CFS) uses the concept of virtual runtime (vruntime) to ensure fairness. The goal is simple: model an “ideal multi-tasking CPU” where every process gets an equal share of processor power.How it works:
Tracking Runtime: As a task runs, its vruntime increases.
Weighting: Tasks with higher priority (lower nice value) accumulate vruntime more slowly, allowing them to run longer for the same “virtual” cost.
Selection: The scheduler always picks the task with the lowest vruntime (the one that has been treated most unfairly so far).
Copy
┌─────────────────────────────────────────────────────────────────────────────┐│ CFS VIRTUAL RUNTIME │├─────────────────────────────────────────────────────────────────────────────┤│ ││ The idea: Track how much CPU time each task has had, always run ││ the task with the LEAST virtual runtime. ││ ││ Virtual runtime accumulation: ││ vruntime += delta_exec * (NICE_0_WEIGHT / task_weight) ││ ││ - Nice 0 task: vruntime = actual runtime ││ - Nice -20 task: vruntime accumulates slower (runs more) ││ - Nice +19 task: vruntime accumulates faster (runs less) ││ ││ Example: 3 tasks with same vruntime at start ││ ││ Time ─────────────────────────────────────────────────────────→ ││ ││ Task A (nice 0): |====| |====| |====| vruntime = 12ms ││ Task B (nice -10): |========| |========| vruntime = 12ms ││ Task C (nice +10): |==| |==| |==| |==| vruntime = 12ms ││ ││ All have same vruntime, but B ran more actual time! ││ │└─────────────────────────────────────────────────────────────────────────────┘
# Scheduler tuning via sysctlsysctl -a | grep sched# Key parameters:# sched_latency_ns = 6000000 (6ms)# Target latency - how long before all tasks run once## sched_min_granularity_ns = 750000 (0.75ms) # Minimum time slice## sched_wakeup_granularity_ns = 1000000 (1ms)# How much better vruntime needed to preempt
#define _GNU_SOURCE#include <sched.h>// Set CPU affinitycpu_set_t mask;CPU_ZERO(&mask);CPU_SET(0, &mask); // Only CPU 0CPU_SET(2, &mask); // And CPU 2sched_setaffinity(pid, sizeof(mask), &mask);// Get CPU affinitysched_getaffinity(pid, sizeof(mask), &mask);
Copy
# Command linetaskset -c 0,2 ./my_programtaskset -p 1234 # Show affinity of PID 1234
# Kernel boot parameter: isolate CPUs from scheduler# In /etc/default/grub:GRUB_CMDLINE_LINUX="isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3"# isolcpus - No normal tasks scheduled on these CPUs# nohz_full - No timer ticks on these CPUs (when single task)# rcu_nocbs - RCU callbacks offloaded from these CPUs
# View NUMA topologynumactl --hardware# Run on specific NUMA nodenumactl --cpunodebind=0 --membind=0 ./my_program# View task's NUMA statscat /proc/<pid>/numa_maps
Context switches are one of the most expensive operations in an operating system. Here’s why:
Direct costs (~2-5 μs):
Saving/restoring CPU registers
Switching page tables (CR3 register)
TLB flush (thousands of cached address translations lost)
Indirect costs (~10-100 μs):
Cache pollution: New process brings different data into CPU caches, evicting the previous process’s data
Cache misses: After switch, almost every memory access misses cache initially
Branch predictor reset: CPU’s prediction tables are now wrong
Real-world impact: A process doing 10,000 context switches/second spends 2-5% of CPU time just on switching overhead, plus 10-50% on cache misses.This is why:
Threads are cheaper than processes (no TLB flush if same address space)
CPU affinity matters (keeps cache warm)
Reducing context switches improves performance
Understanding context switch overhead:
Copy
### Measuring Context Switch Overhead```c// Measure using perf// perf stat -e context-switches,cpu-migrations ./my_program// Or programmatically#include <unistd.h>#include <sys/resource.h>struct rusage usage;getrusage(RUSAGE_SELF, &usage);printf("Voluntary context switches: %ld\n", usage.ru_nvcsw);printf("Involuntary context switches: %ld\n", usage.ru_nivcsw);
# Python script to visualize schedulingimport osimport timedef get_sched_info(pid): with open(f'/proc/{pid}/sched') as f: return f.read()pid = os.getpid()for i in range(5): print(f"\n--- Iteration {i} ---") # Do some work sum(range(10000000)) info = get_sched_info(pid) for line in info.split('\n')[:10]: print(line) time.sleep(0.1)
Lab 4: CPU Affinity and Isolation
Objective: Control process placement
Copy
#define _GNU_SOURCE#include <stdio.h>#include <sched.h>#include <unistd.h>void print_affinity() { cpu_set_t mask; CPU_ZERO(&mask); sched_getaffinity(0, sizeof(mask), &mask); printf("CPU affinity: "); for (int i = 0; i < CPU_SETSIZE; i++) { if (CPU_ISSET(i, &mask)) printf("%d ", i); } printf("\n");}int main() { printf("Before: "); print_affinity(); // Set affinity to CPU 0 only cpu_set_t mask; CPU_ZERO(&mask); CPU_SET(0, &mask); sched_setaffinity(0, sizeof(mask), &mask); printf("After: "); print_affinity(); // Do some work on CPU 0 volatile long sum = 0; for (long i = 0; i < 100000000; i++) sum += i; printf("Work done on CPU %d\n", sched_getcpu()); return 0;}
