Skip to main content

Documentation Index

Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt

Use this file to discover all available pages before exploring further.

Go Concurrency Patterns

​
Concurrency in Go

Go is famous for its concurrency model. Unlike many other languages that treat concurrency as an afterthought or a library add-on, Go builds concurrency directly into the language core. The core philosophy comes from Tony Hoare’s 1978 paper on Communicating Sequential Processes (CSP): instead of sharing memory and coordinating with locks, you share data by sending it through channels. As the Go proverb says: “Do not communicate by sharing memory; instead, share memory by communicating.”

​
Goroutines

A goroutine is a lightweight thread managed by the Go runtime. Think of goroutines as lightweight workers in a factory: you can have thousands of them running simultaneously, each doing a small job. Unlike operating system threads (which are like hiring full-time employees with benefits, office space, and a 1-2MB stack), goroutines start with just 2KB of stack and are managed by Go’s own scheduler rather than the OS kernel.

​
The go Keyword

To start a goroutine, simply use the go keyword before a function call. 
package main

import (
	"fmt"
	"time"
)

func say(s string) {
	for i := 0; i < 3; i++ {
		time.Sleep(100 * time.Millisecond)
		fmt.Println(s)
	}
}

func main() {
	go say("world") // Starts a new goroutine
	say("hello")    // Runs in the main goroutine
}

​
How Goroutines Work

Goroutines are multiplexed onto a smaller number of OS threads using the GMP model (Goroutines, Machine threads, Processors). This M:N scheduling allows Go to efficiently run thousands of goroutines on just a handful of OS threads. Key Characteristics:
  • Lightweight: Start with a tiny 2KB stack that grows/shrinks dynamically.
  • Fast Context Switching: Switching between goroutines is much faster than OS thread context switches.
  • Work Stealing: Idle processors can steal work from busy ones for load balancing.
Go Runtime Architecture Memory Allocation: Local variables in goroutines typically live on the goroutine’s stack. However, if a variable “escapes” (e.g., returned as a pointer, captured by a closure that outlives the function), it’s allocated on the heap. Go Memory Allocation 
func createPerson() *Person {
    p := Person{Name: "Alice"} // Allocated on heap (escapes via return)
    return &p
}

func localOnly() {
    x := 42 // Allocated on stack (doesn't escape)
    fmt.Println(x)
}

​
Channels

Channels are the pipes that connect concurrent goroutines. Think of a channel as a physical pipe between two rooms: one goroutine puts a message in one end, and another goroutine pulls it out the other end. The pipe can be either unbuffered (a direct handoff — the sender waits until the receiver is ready, like a relay race baton pass) or buffered (the pipe has room to hold a few messages, like a mailbox that can queue up letters).

​
Creating Channels

ch := make(chan int) // Unbuffered channel of integers

​
Sending and Receiving

ch <- v    // Send v to channel ch.
v := <-ch  // Receive from ch, and assign value to v.

​
Unbuffered vs. Buffered Channels

  • Unbuffered Channels: Sending blocks until the receiver is ready. Receiving blocks until the sender is ready. This provides synchronization.
  • Buffered Channels: Sending only blocks if the buffer is full. Receiving only blocks if the buffer is empty.
ch := make(chan int, 100) // Buffered channel with capacity 100

​
Channel Internals

Under the hood, a channel is a struct hchan that contains a circular buffer, send/receive indices, and wait queues for blocked goroutines. Go Channel Internals

​
The select Statement

The select statement lets a goroutine wait on multiple communication operations. A select blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready. 
select {
case msg1 := <-c1:
    fmt.Println("received", msg1)
case msg2 := <-c2:
    fmt.Println("received", msg2)
case <-time.After(1 * time.Second):
    fmt.Println("timeout")
default:
    fmt.Println("no activity")
}

​
Synchronization Primitives (sync package)

While channels are great for passing data, sometimes you just need to coordinate state.

​
WaitGroup

sync.WaitGroup waits for a collection of goroutines to finish. 
var wg sync.WaitGroup

for i := 0; i < 5; i++ {
    wg.Add(1)
    go func(i int) {
        defer wg.Done()
        fmt.Printf("Worker %d starting\n", i)
        time.Sleep(time.Second)
        fmt.Printf("Worker %d done\n", i)
    }(i)
}

wg.Wait() // Blocks until the WaitGroup counter is zero

​
Mutex

sync.Mutex provides a mutual exclusion lock to prevent data races. 
type SafeCounter struct {
	mu sync.Mutex
	v  map[string]int
}

func (c *SafeCounter) Inc(key string) {
	c.mu.Lock()
	// Lock so only one goroutine at a time can access the map c.v.
	c.v[key]++
	c.mu.Unlock()
}

​
Common Pitfalls and Practical Tips

​
Goroutine Leaks

A goroutine leak is when a goroutine is blocked forever — waiting on a channel that nobody will ever send to or receive from. These are the memory leaks of Go: they accumulate silently and eventually exhaust memory. 
// LEAK: This goroutine blocks forever if nobody reads from ch
func leakyFunction() {
    ch := make(chan int)
    go func() {
        result := expensiveComputation()
        ch <- result // Blocks forever if the caller gives up
    }()
    // If we return early (timeout, error), the goroutine is stuck
}

// FIX: Use a buffered channel so the goroutine can send and exit
func safeFunction() {
    ch := make(chan int, 1) // Buffer of 1: goroutine can send even if nobody reads
    go func() {
        result := expensiveComputation()
        ch <- result // Never blocks: buffer absorbs the value
    }()
}

// BETTER FIX: Use context for cancellation
func bestFunction(ctx context.Context) (int, error) {
    ch := make(chan int, 1)
    go func() {
        ch <- expensiveComputation()
    }()

    select {
    case result := <-ch:
        return result, nil
    case <-ctx.Done():
        return 0, ctx.Err() // The goroutine will complete and its result
    }                       // is absorbed by the buffer
}
Detecting goroutine leaks in production: Monitor runtime.NumGoroutine() over time. If the count climbs steadily, you have a leak. In tests, use the go.uber.org/goleak package to catch leaks automatically.

​
Race Conditions

Race conditions occur when multiple goroutines access shared data concurrently with at least one write. Go’s race detector (go test -race or go run -race) is one of the most valuable tools in the language — use it always during development and in CI. 
// RACE CONDITION: Multiple goroutines write to the same variable
counter := 0
for i := 0; i < 1000; i++ {
    go func() {
        counter++ // Data race: unsynchronized read-modify-write
    }()
}

// FIX with atomic (best for simple counters):
var counter atomic.Int64
for i := 0; i < 1000; i++ {
    go func() {
        counter.Add(1) // Thread-safe
    }()
}

// FIX with mutex (for complex shared state):
var mu sync.Mutex
counter := 0
for i := 0; i < 1000; i++ {
    go func() {
        mu.Lock()
        counter++
        mu.Unlock()
    }()
}
Concurrent map access causes runtime panics, not just data races. Go’s runtime actively detects concurrent reads and writes to a map and crashes with “fatal error: concurrent map read and map write.” This is distinct from a normal data race — it is an intentional crash to prevent data corruption. Always protect maps with a sync.RWMutex or use sync.Map.

​
Channel Direction Types

Using directional channel types in function signatures communicates intent and prevents bugs at compile time: 
// producer can only send to the channel
func producer(out chan<- int) {
    for i := 0; i < 10; i++ {
        out <- i
    }
    close(out) // Sender closes the channel when done
}

// consumer can only receive from the channel
func consumer(in <-chan int) {
    for val := range in { // Iterates until channel is closed
        fmt.Println(val)
    }
}

​
Best Practices

  1. Share Memory by Communicating: Don’t communicate by sharing memory. Use channels to pass ownership of data.
  2. Detect Race Conditions: Run your tests with go test -race to detect data races. Make this part of your CI pipeline.
  3. Avoid Leaking Goroutines: Ensure every goroutine you start has a way to exit. Use context.Context for cancellation.
  4. Close Channels from the Sender: Only the sender should close a channel, never the receiver. Closing a channel signals “no more values.” Sending to a closed channel panics.
  5. Use WaitGroup for Fan-Out: When spawning multiple goroutines, use sync.WaitGroup to wait for all of them to complete before proceeding.

​
Summary

  • Goroutines are lightweight threads managed by the Go runtime.
  • Channels allow safe communication and synchronization between goroutines.
  • Select allows waiting on multiple channel operations.
  • Sync Package provides low-level primitives like Mutex and WaitGroup for finer control.

​
Interview Deep-Dive

Strong Answer:
  • At the scheduler level, 10,000 goroutines are created with roughly 2KB of stack each (about 20MB total stack memory). They are distributed across P run queues (one per GOMAXPROCS, typically equal to CPU cores). As each goroutine issues an HTTP request, it eventually hits a network I/O operation. The runtime’s network poller (using epoll on Linux) puts the goroutine to sleep without consuming an OS thread — the M is freed to run other goroutines from the P’s queue.
  • What could go wrong: First, 10,000 simultaneous outbound HTTP connections will likely exhaust file descriptors (default ulimit is often 1024), causing “too many open files” errors. Second, the target server may reject or rate-limit this many connections. Third, if the HTTP client uses the default transport, MaxIdleConnsPerHost defaults to 2, so connection reuse is minimal and you create far more TCP connections than necessary. Fourth, if the responses are large and not consumed quickly, you can exhaust memory.
  • The production-grade approach: use a worker pool or semaphore to limit concurrency to a reasonable number (say, 50-100 concurrent requests). Configure the HTTP transport with appropriate MaxIdleConns and MaxIdleConnsPerHost. Use context.WithTimeout on each request to prevent hanging connections. Use errgroup with SetLimit for clean goroutine management and error propagation.
  • A key insight: just because goroutines are cheap to create does not mean the resources they consume (network connections, file descriptors, target server capacity) are cheap. The goroutine is lightweight, but the work it does may not be.
Follow-up: How would you detect and debug a goroutine leak in a long-running production service?Monitor runtime.NumGoroutine() over time — export it as a Prometheus gauge. In a healthy service, the goroutine count should be relatively stable with some fluctuation during load. A steadily climbing count indicates a leak. To diagnose, hit the pprof endpoint at /debug/pprof/goroutine?debug=2 to get a full dump of all goroutine stacks. Look for goroutines blocked on channel operations or select statements with no timeout or cancellation. The stack trace tells you exactly where the goroutine is stuck. For automated detection in tests, use go.uber.org/goleak which checks that no unexpected goroutines remain after a test completes. In my experience, the most common causes are: forgetting to close a channel that a goroutine is reading from, missing context cancellation in long-running workers, and sending to an unbuffered channel with no receiver.
Strong Answer:
  • Channels and mutexes solve different problems, even though both can protect shared state. Channels are for communication — transferring ownership of data between goroutines. Mutexes are for synchronization — protecting data that multiple goroutines access in place.
  • Use channels when: you are transferring data from a producer to a consumer, implementing a pipeline where data flows through stages, coordinating a fan-out/fan-in pattern, or signaling events (like shutdown). The mental model is “passing a baton” — once you send data through a channel, the sender no longer uses it.
  • Use mutexes when: you have shared state that multiple goroutines read and write (like a cache, a counter, or a configuration map), the protected operation is a quick read-modify-write, or you need RWMutex semantics where multiple readers can proceed concurrently but writers are exclusive.
  • The Go proverb “share memory by communicating” does not mean “never use mutexes.” It means prefer channel-based designs where data ownership transfers cleanly. But a sync.Mutex protecting a map is perfectly idiomatic when multiple goroutines need to access the same data structure.
  • In practice, I use sync.RWMutex for caches and shared state, channels for work distribution and signaling, and atomic operations for simple counters. If I find myself building a complex state machine with channels, I reconsider whether a mutex-protected struct would be simpler.
Follow-up: What happens if you send to a closed channel? What about receiving from a closed channel?Sending to a closed channel causes an immediate, unrecoverable panic. There is no safe way to “check if a channel is closed” before sending because that check would be a race condition. The rule is: only the sender should close a channel, and only when it is done sending. Receiving from a closed channel returns immediately with the zero value of the channel’s type. The two-value receive val, ok := <-ch returns ok == false when the channel is closed and drained. The for val := range ch loop consumes all values and exits when the channel is closed. A common mistake is closing a channel from the receiver side “to signal done” — this panics if the sender is still active. Use a separate “done” channel or context cancellation for receiver-to-sender signaling instead.
Strong Answer:
  • The race detector output tells you exactly which two goroutines are conflicting and which memory location they are accessing. It shows the stack traces for both the “previous write” and the “concurrent read” (or write). My first step is reading both stack traces to identify the shared variable and understand the access pattern.
  • Common patterns: a struct field being read by one goroutine while another writes to it, a map being read and written concurrently (which actually causes a fatal crash, not just a race), or a slice being appended to from multiple goroutines.
  • Fixes depend on the pattern. For a simple counter: use atomic.Int64. For a shared map: protect with sync.RWMutex (use RLock for reads, Lock for writes). For shared state that is set once and read many times: use sync.Once. For data that flows between goroutines: restructure to use channels so only one goroutine owns the data at a time.
  • Why data races are particularly dangerous in Go: Go’s memory model provides very few guarantees about what one goroutine sees when another writes without synchronization. A data race is not just “you might read a stale value” — it is undefined behavior. The compiler and CPU may reorder memory operations, and without a synchronization point (channel send/receive, mutex lock/unlock, atomic operation), there is no happens-before relationship between the goroutines. This means the racy read might see a partially written value, an arbitrarily old value, or even a value that was never written. In practice, races cause intermittent, non-reproducible bugs that are nearly impossible to debug without the race detector.
Follow-up: The race detector slows down your program by 5-10x. How do you use it effectively in CI without making tests too slow?Run -race on unit tests in CI always — the overhead is acceptable for test workloads. For integration tests or end-to-end tests that are already slow, you can run -race on a subset or in a separate CI stage that runs less frequently (nightly instead of per-commit). Some teams tag their most concurrency-sensitive tests and run only those with -race on every commit. The race detector only catches races on code paths that are actually exercised, so you also need good test coverage of concurrent paths. A practical pattern is to have a “race” CI stage that runs go test -race -count=5 ./... — the -count=5 runs tests multiple times to increase the chance of exposing timing-dependent races. Importantly, never ship a binary compiled with -race to production — the performance overhead is too high for any production workload.