Skip to main content

Documentation Index

Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt

Use this file to discover all available pages before exploring further.

Python interviews span an unusually wide range — from basic syntax and data structure manipulation to CPython internals, concurrency models, and framework-specific architecture decisions. What makes Python interviews distinctive is that the language’s simplicity on the surface hides significant depth underneath. Interviewers use this gap to distinguish candidates who have genuine understanding from those who have only written scripts. The GIL, descriptor protocol, metaclasses, and memory management model are the topics where strong candidates separate themselves. The questions below are organized from fundamentals through advanced topics. For each question, the provided answer goes beyond the textbook definition to include the “why,” the trade-offs, and the production context that interviewers want to hear. Practice by explaining each concept out loud before reading — your ability to articulate clearly matters as much as knowing the right answer.

1. Python Basics

Python is a high-level, interpreted, general-purpose programming language created by Guido van Rossum (first released 1991). But the real answer an interviewer wants goes beyond the textbook definition.Key features that matter in production:
  • Interpreted with bytecode compilation: Python is not purely interpreted. CPython compiles source to .pyc bytecode first, then the VM interprets that bytecode. This is why you see __pycache__ directories. Understanding this distinction shows you know what actually happens when you run python script.py.
  • Dynamic typing with strong typing: Python is dynamically typed (variables do not declare types) but strongly typed (you cannot add a string to an integer without explicit conversion). Many candidates confuse “dynamic” with “weak” typing — JavaScript is weakly typed, Python is not.
  • First-class functions: Functions are objects. You can assign them to variables, pass them as arguments, return them from other functions. This enables decorators, closures, and functional patterns that are central to idiomatic Python.
  • Extensive standard library (“batteries included”): collections, itertools, functools, pathlib, dataclasses, typing — knowing these separates a Python developer from someone who just writes Python syntax.
  • Memory management via reference counting + cyclic GC: Automatic memory management, but understanding it matters for long-running services where memory leaks from circular references can crash your app at 3 AM.
  • GIL (Global Interpreter Lock): The elephant in the room. One thread executes Python bytecode at a time in CPython. This has massive implications for concurrency design.
What interviewers are really testing: Whether you understand Python beyond “it is easy to learn.” A senior candidate connects features to engineering trade-offs (e.g., dynamic typing speeds development but requires more testing discipline).Red flag answer: “Python is easy to learn and has lots of libraries.” This is a junior answer that shows no depth. Another red flag: not knowing the difference between dynamic and weak typing.War Story: At Instagram (one of the largest Python deployments in the world), the team hit GIL-related bottleneck serving 2 billion+ daily active users. They disabled Python’s cyclic garbage collector, saving ~10% of CPU, and used Cython for hot paths. Understanding these production-level trade-offs is what separates a Python user from a Python engineer.Follow-up:
  • “You mentioned Python compiles to bytecode. What is the difference between CPython, PyPy, and Cython, and when would you choose each?”
  • “How does Python’s strong dynamic typing affect how you design a large codebase with 50+ engineers? What tooling fills the gap?”
  • “What is one feature of Python you think is a design mistake, and how do you work around it?”
  • “Python 3.13 introduced a free-threaded build. What does this mean for the GIL and existing C extensions?”
Python’s built-in data types, organized by category:
  • Numeric: int (arbitrary precision — no overflow), float (64-bit IEEE 754 double), complex (real + imaginary)
  • Sequence: list (mutable, ordered), tuple (immutable, ordered), range (lazy integer sequence), str (immutable Unicode text)
  • Mapping: dict (insertion-ordered since 3.7, hash map under the hood)
  • Set: set (mutable, hash-based unique collection), frozenset (immutable, can be dict key)
  • Boolean: bool (subclass of intTrue == 1, False == 0)
  • Binary: bytes (immutable), bytearray (mutable), memoryview (zero-copy buffer access)
  • None Type: NoneType (singleton — only one None object exists)
What most people miss:
  • bool being a subclass of int means True + True == 2. This is by design (PEP 285) and occasionally causes bugs: isinstance(True, int) returns True.
  • dict ordering is an implementation detail in CPython 3.6 and a language guarantee since 3.7. If you are writing code that must run on older Python, do not rely on it.
  • memoryview is critical for high-performance work — it lets you slice bytes without copying data. NumPy uses this concept extensively.
  • Python integers have arbitrary precision: 2**10000 works fine. There is no int overflow. Under the hood, CPython uses a C struct with a variable-length array of digits.
What interviewers are really testing: Whether you know the behavior of these types, not just their names. Especially: mutability, hashability, and when to choose one over another.Red flag answer: Just listing the types without understanding mutability or when to use each.Follow-up:
  • “Why can you use a tuple as a dictionary key but not a list? What is the relationship between hashability and immutability?”
  • “What happens when you put a float('nan') into a set? Can you have duplicate NaN values? Why does float('nan') == float('nan') return False?”
  • “When would you use bytearray over bytes? Give a network protocol example.”
  • sys.getsizeof(1) returns 28 bytes for a single integer. Why is a Python int so large compared to a C int?”
The surface answer is mutability, but the real differences go much deeper:Mutability: Lists are mutable (append, pop, sort in place). Tuples are immutable (cannot change after creation). But a tuple can contain mutable objects: t = ([1, 2], [3, 4]) — the tuple itself is immutable, but the lists inside it can change. This is a common gotcha.Performance (real numbers):
  • Tuple creation is ~5-10x faster than list creation for small sizes (CPython optimizes tuple allocation with a free list cache)
  • Tuples use less memory: a 3-element tuple is ~64 bytes vs ~88 bytes for a 3-element list (CPython 3.11), because lists over-allocate space for future append() calls
  • Tuple element access is marginally faster (simpler C struct, no indirection for resizing metadata)
  • timeit benchmark: creating (1,2,3) takes ~15ns vs [1,2,3] at ~50ns on typical hardware
Hashability: Tuples are hashable (if all elements are hashable), so they can be dict keys and set members. Lists cannot. This is the reason you would use tuple as a dict key for multi-dimensional lookups: cache[(x, y, z)].Semantic meaning: In idiomatic Python, tuples represent heterogeneous fixed-structure records (like a row from a database: (name, age, email)), while lists represent homogeneous variable-length collections (like a list of users). This is why namedtuple exists.What interviewers are really testing: Whether you understand the why behind choosing one vs the other, not just the syntax difference.Red flag answer: “Tuples use parentheses and lists use brackets.” This shows zero understanding of the engineering implications.Follow-up:
  • “If tuples are immutable, why can t = ([1,2],) have its inner list modified? What does immutability actually mean here?”
  • “When would you use a namedtuple vs a dataclass? What are the trade-offs in terms of memory, mutability, and inheritance?”
  • “You have a function returning multiple values. Should you return a tuple or a list? Why? What about a dataclass?”
  • “Can you hash (1, [2, 3])? Why or why not? What exception is raised?”
Mutable types (can be modified in place): list, dict, set, bytearray, and most user-defined objects.Immutable types (create new objects on modification): int, float, str, tuple, frozenset, bytes, bool, NoneType.Why this matters in real code:
  1. The mutable default argument trap — the single most common Python bug:
def add_item(item, items=[]):  # BUG: shared across all calls
    items.append(item)
    return items
# add_item(1) returns [1]
# add_item(2) returns [1, 2]  -- surprise!
Fix: use None as default, create new list inside function. This bug has caused production incidents at companies processing millions of requests where a shared default list accumulated data across HTTP requests.
  1. Dictionary keys must be immutable (and hashable). If you mutate an object used as a dict key, the hash changes and the dict breaks silently — you can no longer find the key.
  2. Pass-by-object-reference implications: When you pass a list to a function, the function gets a reference to the same list object. Modifications inside the function affect the caller. With immutable types, operations create new objects, so the caller is unaffected.
  3. Thread safety: Immutable objects are inherently thread-safe because they cannot change. Mutable shared state requires locks. This is why str concatenation in a loop creates new strings each time (use ''.join() instead for O(n) vs O(n^2)).
What interviewers are really testing: Whether you can explain how mutability causes real bugs, not just recite the definition.Red flag answer: Explaining the concept without mentioning the mutable default argument trap or pass-by-reference implications.Follow-up:
  • “Show me a bug caused by mutability that would be hard to catch in code review. How would you write a test for it?”
  • “How does string immutability affect performance when building a large string in a loop? What is the fix, and what is the Big-O difference?”
  • “Is a tuple always immutable? What about tuple([mutable_list])? Can you hash() a tuple that contains a list?”
  • “You are designing a configuration object for a Flask app. Should its attributes be mutable or immutable? What pattern would you use?”
is and is not check whether two variables point to the same object in memory (identity), not whether they have the same value (equality).The critical distinction:
a = [1, 2, 3]
b = [1, 2, 3]
c = a

a == b   # True (same value)
a is b   # False (different objects in memory)
a is c   # True (c points to same object as a)
Where is should actually be used:
  • None checks: Always use if x is None, never if x == None. Why? Because == calls __eq__, which a class could override to return True for None comparison. is checks identity directly and cannot be overridden. PEP 8 explicitly mandates this.
  • Sentinel values: When you create a unique sentinel like _MISSING = object(), use is to check against it.
The integer interning gotcha:
a = 256
b = 256
a is b  # True -- CPython caches integers -5 to 256

a = 257
b = 257
a is b  # False in the REPL, but may be True in a .py file!
CPython interns small integers (-5 to 256) and short strings as an optimization. The boundary at 257 has tripped up many developers. In a .py file, the compiler may constant-fold and reuse the same object, giving different results than the REPL.What interviewers are really testing: Whether you know when to use is vs == and can explain the interning behavior.Red flag answer: Using is for general value comparison, or not knowing about None check conventions.Follow-up:
  • “Why does PEP 8 require is None instead of == None? Give me a case where == None would give a wrong result.”
  • “What is integer interning, and why does 257 is 257 give different results in the REPL vs a script?”
  • “How would you create your own sentinel object, and why would you use is to compare against it?”
The real answer here is Python’s LEGB scope resolution rule, which determines how variable names are looked up:
  1. Local — inside the current function
  2. Enclosing — in the enclosing function (for nested functions/closures)
  3. Global — at the module level
  4. Built-in — in Python’s built-in namespace (len, print, range, etc.)
Python searches in this exact order. The first match wins.The global and nonlocal keywords:
x = 10  # Global

def outer():
    y = 20  # Enclosing

    def inner():
        nonlocal y   # Modifies enclosing scope's y
        global x     # Modifies module-level x
        x = 99
        y = 99

    inner()
    print(y)  # 99 (modified by nonlocal)

outer()
print(x)  # 99 (modified by global)
The UnboundLocalError trap:
x = 10
def broken():
    print(x)  # UnboundLocalError!
    x = 20    # This assignment makes x local to entire function
Python determines scope at compile time, not runtime. If there is any assignment to x in the function, x is treated as local for the entire function, even before the assignment. This catches many developers off guard.Production advice: Global mutable state is a code smell. In production systems, you should use dependency injection, configuration objects, or module-level constants (uppercase by convention). Flask’s g object, Django’s settings module — frameworks provide patterns to avoid raw globals.What interviewers are really testing: Whether you know LEGB, understand UnboundLocalError, and have opinions about global state in real systems.Red flag answer: Only explaining global keyword without mentioning LEGB or the UnboundLocalError trap.Follow-up:
  • “What is an UnboundLocalError and when does it occur? Show me a 3-line function that triggers it. Why does Python decide scope at compile time, not runtime?”
  • “When would you use nonlocal vs global? Can you give a real example where nonlocal is genuinely useful?”
  • “In a production Flask app, how would you manage configuration instead of using global variables? Compare app.config, environment variables, and dependency injection approaches.”
  • “A developer added logging = custom_logger() inside a function and now logging.info() fails earlier in the same function. What happened?”
Python provides explicit type conversion functions and also performs implicit type coercion in specific situations.Explicit conversion (type constructors):
  • int('123') returns 123. int('0xff', 16) returns 255 (base conversion). int(3.9) returns 3 (truncates, does not round).
  • float('3.14') returns 3.14. float('inf') and float('nan') are valid.
  • str(42) returns '42'. But repr(42) also returns '42' — the difference matters for strings: str('hello') returns hello, repr('hello') returns 'hello' (with quotes). Use repr() for debugging, str() for user display.
  • bool() follows truthiness rules: bool(0), bool(''), bool([]), bool(None) are all False. Everything else is True.
  • list('abc') returns ['a', 'b', 'c'] — iterates the string.
Implicit coercion (where Python does it for you):
  • 3 + 4.0 returns 7.0 (int promoted to float)
  • True + 2 returns 3 (bool is subclass of int)
  • if my_list: implicitly calls bool(my_list) — this is the Pythonic way to check for empty collections
Gotchas that bite in production:
  • int('3.14') raises ValueError — you must do int(float('3.14')) for string-to-int via float
  • float('nan') == float('nan') is False (IEEE 754 spec). Use math.isnan() instead
  • bool([False]) is True — the list is not empty, even though its only element is falsy
  • json.loads() returns Python dict/list automatically but all numbers become float, losing integer precision for large numbers
What interviewers are really testing: Edge cases and gotchas, not basic syntax.Red flag answer: Just listing conversion functions without knowing any edge cases.Follow-up:
  • “What is the difference between str() and repr()? When does it matter?”
  • “Why does int('3.14') fail? How would you safely convert arbitrary user input to an integer?”
  • “Explain Python’s truthiness rules. What objects are falsy?”
PEP 8 is Python’s official style guide (authored by Guido van Rossum, Barry Warsaw, and Nick Coghlan). But the real interview question is about how you enforce consistency in a team.Key conventions:
  • Naming: snake_case for functions/variables/modules, PascalCase for classes, UPPER_SNAKE_CASE for constants, _single_leading_underscore for internal use, __double_leading for name mangling
  • Indentation: 4 spaces (never tabs). This is non-negotiable in the Python community.
  • Line length: 79 characters (72 for docstrings). Many teams relax this to 88 (Black’s default) or 100-120 for modern wide monitors.
  • Imports: Standard library first, then third-party, then local. One import per line. Tools like isort automate this.
How real teams enforce PEP 8 (the actual answer interviewers want):
  • black: Opinionated auto-formatter. Zero configuration. “Any color you want, as long as it is black.” Eliminates style debates in code review.
  • ruff: Blazing-fast linter (written in Rust), replaces flake8, isort, pyupgrade, and dozens of plugins. Adopted by major projects like FastAPI, pandas, and Airflow.
  • mypy/pyright: Not PEP 8 per se, but type checking is now standard in serious Python projects. PEP 484 type hints are checked by these tools.
  • Pre-commit hooks: Run black + ruff + mypy before every commit. This is the industry-standard setup for Python projects in 2024+.
  • CI/CD enforcement: Fail the build if linting does not pass. No exceptions.
What interviewers are really testing: Not whether you know what PEP 8 says, but whether you have experience with tooling that enforces code quality in a team environment.Red flag answer: “I follow PEP 8 by reading it and being careful.” This shows no tooling awareness.Follow-up:
  • “Walk me through the linting/formatting setup on your last project. What tools did you use?”
  • “Your team disagrees on line length (79 vs 120). How do you resolve this?”
  • “What is the difference between a linter (ruff/flake8) and a formatter (black)? Why do you need both?”
The Zen of Python (PEP 20) is a collection of 19 aphorisms by Tim Peters that guide Python’s design philosophy. Access with import this.The principles that actually matter in interviews (and why):
  • “Explicit is better than implicit” — This is why Python does not have implicit type coercion like JavaScript ("1" + 1 throws TypeError in Python, returns "11" in JS). This principle also explains why import * is discouraged.
  • “Simple is better than complex” — Prefer for item in items over for i in range(len(items)). Prefer list comprehensions over manual loops when they are readable. But: “Complex is better than complicated” — sometimes a class is clearer than a nested dict of functions.
  • “Readability counts” — This is why Python uses significant whitespace, why and/or/not exist instead of &&/||/!, and why the community obsesses over naming.
  • “Errors should never pass silently” — Never use bare except: pass. Log it, re-raise it, handle it specifically. This principle directly informs exception handling best practices.
  • “There should be one — and preferably only one — obvious way to do it” — Contrast with Perl’s TMTOWTDI (“There is more than one way to do it”). Python values consistency.
  • “If the implementation is hard to explain, it is a bad idea” — If your clever metaprogramming requires a 500-word comment, refactor it.
Real-world application: When reviewing a PR, the Zen of Python gives you vocabulary for feedback. “This lambda chain is clever, but ‘readability counts’ — can we refactor to a named function?” is much better feedback than “I don’t like this.”What interviewers are really testing: Whether you have internalized these principles and can apply them to code decisions, not just recite them.Red flag answer: Reciting principles without being able to give a concrete code example where one applies.Follow-up:
  • “Give me an example of real code where ‘explicit is better than implicit’ changed your design decision.”
  • “When does ‘simple is better than complex’ conflict with ‘practicality beats purity’? How do you decide?”
  • “How would you use the Zen of Python to argue for or against using a metaclass in a code review?”
Deep equality (==) recursively compares values. Python’s == operator calls __eq__ on objects, and built-in types implement deep comparison by default:
[1, [2, 3]] == [1, [2, 3]]  # True -- deep comparison
Structural pattern matching (Python 3.10+, PEP 634) uses match/case statements to destructure and match against patterns. It is not just a switch statement — it is a fundamentally different tool:
def handle_command(command):
    match command:
        case ["quit"]:
            return "Exiting"
        case ["go", direction]:
            return f"Going {direction}"
        case ["get", item] if item != "sword":
            return f"Picking up {item}"
        case _:
            return "Unknown command"
Why this matters: Pattern matching replaces chains of isinstance checks and nested if/elif blocks. Before 3.10, parsing a JSON API response with variable shapes required ugly conditional logic. With match/case, you can destructure the shape directly.Guards (if clauses in case) add conditional logic to patterns. Capture variables bind matched values to names. OR patterns (case "yes" | "y") match multiple options.Gotcha: case name: where name is a variable does NOT match against the variable’s value — it is a capture pattern that always matches and binds the value to name. Use case MyEnum.VALUE: for value matching, or use a guard: case x if x == name:.What interviewers are really testing: Whether you know modern Python features and can recognize when pattern matching simplifies code vs when it is overkill.Red flag answer: “It is just Python’s version of a switch statement.” Pattern matching is far more powerful.Follow-up:
  • “When would pattern matching be clearer than if/elif chains? When would it be worse?”
  • “What is the capture variable gotcha in match/case, and how do you avoid it?”
  • “How would you use pattern matching to parse different shapes of JSON API responses?”
The walrus operator (:=, PEP 572, Python 3.8+) is an assignment expression — it assigns a value to a variable as part of an expression, rather than as a standalone statement.Where it genuinely helps:
# Without walrus -- duplicate call or temp variable
data = get_data()
if data:
    process(data)

# With walrus -- single expression
if (data := get_data()):
    process(data)

# Especially useful in while loops
while (chunk := file.read(8192)):
    process(chunk)

# And in list comprehensions with expensive filtering
results = [clean for item in items if (clean := expensive_transform(item)) is not None]
When NOT to use it:
  • When it hurts readability. x := expr inside a complex expression makes code harder to parse visually.
  • In simple assignments. x := 5 is worse than x = 5 — it adds noise.
  • PEP 572 itself warns against overuse. Guido van Rossum stepped down as BDFL partly because of the contentious debate around this feature.
The key trade-off: The walrus operator reduces lines of code and avoids duplicate computation, but it increases the cognitive load per line. Senior engineers use it sparingly and only when the intent is clearer than the alternative.What interviewers are really testing: Judgment about when a feature helps vs when it hurts readability.Red flag answer: Either not knowing it exists (knowledge gap) or using it everywhere to show off (poor judgment).Follow-up:
  • “Show me a case where the walrus operator makes code genuinely clearer, and one where it makes it worse.”
  • “Why was PEP 572 so controversial in the Python community?”
  • “How does := differ from = in terms of scope inside comprehensions?“

2. Data Structures

Lists are Python’s workhorse data structure — a dynamic array that can hold mixed types. But understanding the performance characteristics is what separates a strong answer.Creation:
my_list = [1, 2, 3]          # Literal (fastest)
my_list = list(range(10))    # From iterable
my_list = [0] * 100          # Pre-allocated (useful for performance)
my_list = [x**2 for x in range(10)]  # Comprehension (Pythonic)
Common operations and their Big-O complexity:
  • append(item) — O(1) amortized. Uses over-allocation strategy: when the internal array is full, CPython allocates ~12.5% more space. This is why append is fast on average but occasionally triggers a resize.
  • insert(0, item) — O(n). Every element must shift. If you are doing frequent left-inserts, use collections.deque instead (O(1) on both ends).
  • pop() — O(1) from end, O(n) from beginning (pop(0)). Again, deque is better for FIFO patterns.
  • remove(item) — O(n). Linear search + shift.
  • in operator — O(n). If you need fast membership testing, use a set instead.
  • sort() — O(n log n). Uses Timsort (hybrid merge sort + insertion sort). Stable sort. sorted() returns a new list; sort() modifies in place.
  • Slice lst[a:b] — O(b-a). Creates a shallow copy of the slice.
What interviewers are really testing: Whether you know the time complexity of list operations and can choose the right data structure.Red flag answer: Listing methods without knowing their complexity or when lists are the wrong choice.Follow-up:
  • “Your code does list.insert(0, item) in a loop processing 1M items. What is the total complexity, and how do you fix it?”
  • “What is the difference between sort() and sorted()? When would you use each?”
  • “How does CPython’s list over-allocation strategy work, and why does it matter?”
List comprehensions are a concise, Pythonic way to create lists from iterables. But the real depth is in understanding when they help and when they hurt.Syntax: [expression for item in iterable if condition]Why they are faster than equivalent loops:
# Loop version (~30% slower in CPython benchmarks)
squares = []
for x in range(1000):
    squares.append(x**2)

# Comprehension (faster -- no repeated attribute lookup for .append)
squares = [x**2 for x in range(1000)]
The speed difference comes from CPython’s internal optimization: comprehensions use a dedicated LIST_APPEND bytecode instruction, avoiding the overhead of looking up and calling the append method on each iteration.Advanced patterns:
# Nested comprehension (flattening)
flat = [x for row in matrix for x in row]

# Dict comprehension
word_lengths = {word: len(word) for word in words}

# Set comprehension
unique_lengths = {len(word) for word in words}

# Conditional expression (ternary in comprehension)
labels = ["even" if x % 2 == 0 else "odd" for x in range(10)]
When NOT to use comprehensions:
  • When the logic requires multiple statements or side effects
  • When the comprehension exceeds ~80 characters or nests more than 2 levels deep — readability drops fast
  • When you do not need the full list in memory — use a generator expression (x**2 for x in range(10_000_000)) instead to avoid allocating a massive list
The memory trap: [process(x) for x in range(10_000_000)] creates a 10M-element list in memory. If you are only iterating once, use a generator expression or map().What interviewers are really testing: Whether you know the performance implications and readability boundaries, not just the syntax.Red flag answer: Writing deeply nested comprehensions as a flex. One-line does not mean readable.Follow-up:
  • “When should you use a generator expression (...) instead of a list comprehension [...]?”
  • “This 3-level nested comprehension is hard to read. Refactor it to be clearer.”
  • “What is the bytecode difference between a list comprehension and an equivalent for loop? Why is the comprehension faster?”
Dictionaries are Python’s hash map implementation. Since Python 3.7, they maintain insertion order as a language guarantee (not just a CPython implementation detail).Under the hood (CPython 3.6+): Python dicts use a compact hash table with two arrays: a sparse hash index array and a dense key-value entries array. This design reduced memory usage by 20-25% compared to Python 3.5’s implementation. Lookups are O(1) average, O(n) worst case (hash collisions).Essential methods and when to use them:
  • get(key, default) — Returns default (or None) if key missing. Avoids KeyError. Use this over dict[key] when the key might not exist.
  • setdefault(key, default) — Returns value if key exists, otherwise sets key to default and returns it. Atomic operation useful for building groupings:
groups = {}
for item in items:
    groups.setdefault(item.category, []).append(item)
But collections.defaultdict is usually cleaner for this pattern.
  • update(other_dict) — Merges another dict. Since Python 3.9, you can use | operator: merged = dict1 | dict2 (dict2 wins on conflicts).
  • pop(key, default) — Removes and returns value. Raises KeyError if no default.
  • items() / keys() / values() — Return view objects (not copies). Views reflect changes to the dict. Do not modify a dict while iterating its views.
Performance characteristics:
  • Lookup, insert, delete: O(1) average
  • Memory: ~50-70 bytes per key-value pair (CPython 3.11)
  • Hash function: CPython uses SipHash for strings (resistant to hash-flooding DoS attacks, changed after CVE-2012-1150)
  • Keys must be hashable (implement __hash__ and __eq__). Mutable objects like lists cannot be keys.
Python 3.9+ dict merge operators:
merged = dict1 | dict2      # Creates new dict (dict2 wins on conflicts)
dict1 |= dict2              # In-place merge
What interviewers are really testing: Whether you understand hash maps, know about ordering guarantees, and can choose between dict, defaultdict, OrderedDict, and Counter.Red flag answer: “Dictionaries store key-value pairs.” This is the level of a beginner tutorial.Follow-up:
  • “What happens if two keys have the same hash? How does Python resolve collisions?”
  • “When would you use collections.defaultdict vs dict.setdefault()?”
  • “Your dict lookup is O(1) in theory but slow in practice. What could cause this?”
Sets are mutable, unordered collections of unique, hashable elements. Frozensets are their immutable counterpart.Why frozensets exist — the real answer: Since sets are mutable, they are not hashable, which means you cannot put a set inside another set or use it as a dict key. Frozensets solve this:
# This fails -- sets are not hashable
# my_set = {  {1, 2}, {3, 4}  }  # TypeError

# This works -- frozensets are hashable
my_set = { frozenset({1, 2}), frozenset({3, 4}) }
cache_key = frozenset(user_permissions)  # Use as dict key
Performance (both types):
  • Membership testing (in): O(1) average. This is the primary reason to use sets — if item in my_set is O(1) vs O(n) for lists.
  • Add/remove: O(1) average (sets only).
  • Union/intersection/difference: O(min(len(s1), len(s2))) for intersection, O(len(s1) + len(s2)) for union.
Real-world use cases for frozensets:
  • Cache keys representing a set of features or permissions
  • Dict keys for memoization where the input is a collection of unique items
  • Elements of other sets (e.g., a set of sets for graph algorithms)
  • Immutable configuration that should not be modified after initialization
What interviewers are really testing: Whether you understand hashability and can articulate why immutability matters for hash-based containers.Red flag answer: “Frozensets are just sets you cannot change.” This misses the why — hashability and use as dict keys.Follow-up:
  • “Why cannot you use a regular set as a dictionary key?”
  • “You need to check if a user has any of 10,000 banned words in their input. What data structure do you use?”
  • “What happens to set performance when every element has the same hash?”
Sets support mathematical set operations with both operator and method syntax. The key interview insight is knowing when each is useful in production code.Operations and their complexity:
  • Union (s1 | s2 or s1.union(s2)) — O(len(s1) + len(s2)). All elements from both sets.
  • Intersection (s1 & s2 or s1.intersection(s2)) — O(min(len(s1), len(s2))). Only elements in both.
  • Difference (s1 - s2 or s1.difference(s2)) — O(len(s1)). Elements in s1 but not s2.
  • Symmetric difference (s1 ^ s2) — O(len(s1) + len(s2)). Elements in either but not both.
  • Subset/Superset (s1 <= s2, s1 >= s2) — O(len(s1)). Check containment.
The method syntax advantage: Methods accept any iterable, operators require both to be sets:
my_set = {1, 2, 3}
my_set.union([4, 5, 6])    # Works -- accepts list
my_set | [4, 5, 6]         # TypeError -- operator needs set on both sides
Real-world examples:
# Find users with both permissions (intersection)
admin_users = set(admins) & set(active_users)

# Find new users who signed up but have not verified (difference)
unverified = set(signups) - set(verified_emails)

# Find features unique to either A/B test group (symmetric diff)
unique_features = set(group_a_features) ^ set(group_b_features)

# Check if required permissions are all present (subset)
if required_perms <= user_perms:
    grant_access()
What interviewers are really testing: Whether you can apply set operations to solve real problems efficiently, not just define them.Red flag answer: Reciting operations without real-world examples or complexity awareness.Follow-up:
  • “You have two lists of 1M user IDs each. How do you find users in both lists? What is the complexity?”
  • “What is the difference between s1 - s2 and s2 - s1? When does order matter?”
  • “How would you use set operations to implement a simple permissions system?”
This is the “choose your data structure” question. The strong answer is a decision matrix, not a feature comparison.
Featurelisttupleset
OrderedYesYesNo
MutableYesNoYes (elements cannot be mutable)
DuplicatesAllowedAllowedNot allowed
IndexableYes, O(1)Yes, O(1)No
Membership testO(n)O(n)O(1)
HashableNoYes (if elements are hashable)No (use frozenset)
MemoryHighest (over-allocates)Lowest (fixed)Medium (hash table)
When to use each (the real interview answer):
  • List: When you need ordered, mutable, indexed access. Default choice for collections. Example: a list of user records you will sort, filter, paginate.
  • Tuple: When data is fixed and should not change. Function return values, dict keys, named records. Example: (latitude, longitude) coordinate pairs. Also used for heterogeneous data (name, age, email) vs lists for homogeneous data (list of names).
  • Set: When you need fast membership testing or deduplication. Example: checking if a URL has been visited, deduplicating a list of emails, computing permission intersections.
The collections module alternatives:
  • collections.deque — When you need O(1) append/pop from both ends (queue/stack)
  • collections.OrderedDict — When you need dict with explicit ordering control (e.g., LRU cache before 3.7)
  • collections.Counter — When you need to count occurrences
  • collections.defaultdict — When you want automatic default values
What interviewers are really testing: Whether you can justify your data structure choice based on access patterns and performance requirements.Red flag answer: Describing features without explaining when to use each one.Follow-up:
  • “You are processing 10M log entries and need to find unique IPs. Which data structure and why?”
  • “Your function returns (success, data, error). Should this be a tuple, list, or dict?”
  • “When would you reach for collections.deque instead of a list?”
This question tests whether you understand Python’s reference model. Objects are not values in Python — variables hold references to objects.Shallow copy (copy.copy(), list slicing lst[:], list(lst), dict.copy()): Creates a new container object but fills it with references to the same nested objects. Only the top-level container is new.Deep copy (copy.deepcopy()): Recursively creates new objects for everything, including all nested containers. A completely independent clone.
import copy

original = [[1, 2], [3, 4]]
shallow = copy.copy(original)
deep = copy.deepcopy(original)

# Shallow copy shares nested objects
shallow[0][0] = 99
print(original[0][0])  # 99 -- original affected!

# Deep copy is fully independent
deep[0][0] = 88
print(original[0][0])  # Still 99 -- original unaffected
The subtle cases that matter:
  1. Circular references: deepcopy handles them correctly by tracking already-copied objects with a memo dict. A naive recursive copy would infinite-loop.
  2. Custom objects: You can customize copy behavior by implementing __copy__() and __deepcopy__() methods. SQLAlchemy models, for example, need special handling because they carry database session state.
  3. Performance: deepcopy is slow — it must traverse the entire object graph. On a deeply nested dict with 100K entries, it can take hundreds of milliseconds. In a hot path, consider whether you actually need a full deep copy or can restructure to avoid it.
  4. Hidden shallow copies: list[:] slicing, dict.copy(), and the list() constructor all create shallow copies. Many developers assume these are deep copies.
Real-world bug example:
# A config template shared across requests
config_template = {"db": {"host": "localhost", "port": 5432}}

def get_config(user):
    config = config_template.copy()  # SHALLOW -- shares nested dict
    config["db"]["host"] = user.db_host  # Mutates template!
    return config
# After first call, config_template is corrupted for all future calls
Fix: config = copy.deepcopy(config_template) or restructure to use immutable data.What interviewers are really testing: Whether you have been burned by shallow copy bugs and can explain the reference model.Red flag answer: Defining the difference without mentioning a real bug scenario or knowing about circular reference handling.Follow-up:
  • “Show me a production bug caused by shallow copy. How would you find it?”
  • deepcopy is too slow for your hot path. What alternatives exist?”
  • “How does deepcopy handle circular references internally?”
defaultdict (from collections) automatically creates a default value when you access a missing key, eliminating the need for existence checks.How it works internally:
from collections import defaultdict

# Groups items by category -- no KeyError, no setdefault boilerplate
word_counts = defaultdict(int)      # Missing keys get 0
grouped = defaultdict(list)         # Missing keys get []
nested = defaultdict(lambda: defaultdict(int))  # Nested defaults

for word in words:
    word_counts[word] += 1          # No need to check if key exists

for item in items:
    grouped[item.category].append(item)  # No need for setdefault
defaultdict overrides __missing__(). When __getitem__ fails to find a key, it calls the factory function, stores the result, and returns it. This means accessing a missing key creates that key — which is a subtle difference from dict.get().defaultdict vs alternatives:
  • dict.setdefault(key, []) — similar but more verbose. Creates a new default object on every call even if the key exists (though CPython optimizes common cases).
  • dict.get(key, default) — does NOT insert the key into the dict. Read-only fallback.
  • collections.Counter — specialized defaultdict(int) with extra methods like most_common().
Gotcha: Because defaultdict creates entries on access, iterating with a for key in d loop and accessing keys that do not exist will pollute the dict:
d = defaultdict(int)
d["a"] = 1
print(d["b"])  # Returns 0 but ALSO creates key "b" in the dict
print(len(d))  # 2, not 1!
If you want read-only default behavior, use dict.get() instead.What interviewers are really testing: Whether you know the right tool for grouping/counting patterns and understand the side effects.Red flag answer: Not knowing defaultdict exists, or using manual if key in dict checks everywhere.Follow-up:
  • “What is the difference between defaultdict(list) and using dict.setdefault(key, [])?”
  • “When would defaultdict create unexpected entries? How do you prevent it?”
  • “How would you implement a defaultdict from scratch using __missing__?“

3. Object-Oriented Programming

The four pillars, but explained the way Python actually implements them (not Java-style textbook definitions):1. Encapsulation — Python takes a radically different approach than Java/C++. There are no private or protected keywords. Instead:
  • Single underscore _name: Convention for “internal use.” Not enforced. Other code can access it but should not.
  • Double underscore __name: Triggers name mangling — Python rewrites it to _ClassName__name. This prevents accidental override in subclasses, but it is NOT true privacy (you can still access it if you know the mangled name).
  • The philosophy: “We are all consenting adults here.” Python trusts developers to respect conventions.
2. Abstraction — Implemented via Abstract Base Classes (abc.ABC or abc.ABCMeta). You cannot instantiate a class with unimplemented abstract methods. Also achieved through Python 3.8+ Protocol classes (structural subtyping — no inheritance required).3. Inheritance — Python supports multiple inheritance with C3 linearization MRO. But the community strongly prefers composition over inheritance. Deep inheritance hierarchies are a code smell in Python. Use mixins sparingly.4. Polymorphism — In Python, this is primarily duck typing: “If it quacks like a duck…” You do not need inheritance for polymorphism. Any object with a read() method works wherever a file-like object is expected. typing.Protocol formalizes this.What makes Python’s OOP unique: It is opt-in and flexible. You can write purely procedural Python, purely functional Python, or full OOP. Most production Python is a pragmatic mix.What interviewers are really testing: Whether you understand how Python’s OOP differs from Java/C++ and can articulate the philosophy, not just the definitions.Red flag answer: Describing the four pillars exactly as a Java textbook would. Python’s implementation is fundamentally different.Follow-up:
  • “Python has no private keyword. How do you prevent other developers from accessing internal state?”
  • “When would you use abc.ABC vs typing.Protocol?”
  • “Give me an example where duck typing is more Pythonic than inheritance-based polymorphism.”
The basic syntax is the class keyword, but a strong answer covers the full lifecycle of Python object creation and modern alternatives.Basic class:
class Person:
    species = "Human"  # Class attribute (shared by all instances)

    def __init__(self, name, age):
        self.name = name   # Instance attribute
        self.age = age

    def greet(self):
        return f"Hello, I am {self.name}"

p = Person("Alice", 30)
What actually happens when you write Person("Alice", 30):
  1. Person.__new__(Person) is called to create the instance (allocates memory)
  2. Person.__init__(instance, "Alice", 30) initializes the instance
  3. The instance is returned
Most developers never override __new__, but it is essential for immutable types (you cannot modify self in __init__ for immutable objects) and Singleton patterns.Modern alternatives (what you should actually use in 2024+):
# dataclasses (Python 3.7+) -- eliminates boilerplate
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int
    email: str = ""  # Default value

# Gets __init__, __repr__, __eq__ for free
# Also supports frozen=True for immutability, slots=True for memory

# NamedTuple -- lightweight immutable records
from typing import NamedTuple

class Point(NamedTuple):
    x: float
    y: float
When to use what:
  • @dataclass — Most data-holding classes. Default choice for DTOs, configs, records.
  • NamedTuple — When you want immutability and tuple compatibility (hashable, can be dict key).
  • Regular class — When you need complex behavior, custom __init__ logic, or non-trivial methods.
  • attrs (third-party) — More features than dataclass (validators, converters). Used by major projects like pytest.
What interviewers are really testing: Whether you reach for @dataclass instead of writing boilerplate __init__/__repr__/__eq__ by hand.Red flag answer: Writing a class with manual __init__, __repr__, and __eq__ when @dataclass would do it in 3 lines.Follow-up:
  • “What is the difference between __new__ and __init__? When would you override __new__?”
  • “Compare @dataclass vs NamedTuple vs attrs. When would you choose each?”
  • “How does @dataclass(frozen=True, slots=True) work, and when would you use those options?”
Instance methods — The default. Take self as first parameter. Operate on instance data. Can access both instance and class attributes.Class methods — Decorated with @classmethod. Take cls as first parameter. Operate on the class itself, not instances. The killer use case is alternative constructors:
class Date:
    def __init__(self, year, month, day):
        self.year = year
        self.month = month
        self.day = day

    @classmethod
    def from_string(cls, date_string):
        year, month, day = map(int, date_string.split('-'))
        return cls(year, month, day)  # cls, not Date -- works with subclasses!

    @classmethod
    def today(cls):
        import datetime
        t = datetime.date.today()
        return cls(t.year, t.month, t.day)
The cls parameter is crucial: if SpecialDate(Date) calls Date.from_string(), it returns a SpecialDate instance, not a Date. This is why @classmethod exists instead of just using regular functions.Static methods — Decorated with @staticmethod. Take neither self nor cls. Pure utility functions that logically belong to the class namespace:
class MathUtils:
    @staticmethod
    def is_even(n):
        return n % 2 == 0
Controversial opinion: most @staticmethod uses should just be module-level functions. @staticmethod only makes sense when the function is conceptually bound to the class.What interviewers are really testing: Whether you know when to use @classmethod (alternative constructors, factory methods) vs @staticmethod (rare) vs instance methods (default).Red flag answer: Defining all three without giving the alternative constructor use case for @classmethod.Follow-up:
  • “Why does @classmethod use cls instead of the class name directly? What breaks if you hardcode the class name?”
  • “When would a @staticmethod be better as a module-level function?”
  • “How would you implement dict.fromkeys() using @classmethod?”
Inheritance allows a class to reuse code from a parent class. Python supports both single and multiple inheritance, which makes it more powerful — and more dangerous — than languages like Java.Single inheritance:
class Animal:
    def speak(self):
        raise NotImplementedError

class Dog(Animal):
    def speak(self):
        return "Woof"
Multiple inheritance and the Diamond Problem:
class A:
    def method(self): return "A"

class B(A):
    def method(self): return "B"

class C(A):
    def method(self): return "C"

class D(B, C):
    pass

D().method()  # Returns "B" -- MRO is D -> B -> C -> A -> object
Python resolves this using C3 linearization (MRO). The order is deterministic and predictable.super() is NOT “call the parent” — it is “call the next class in the MRO chain”:
class B(A):
    def method(self):
        return super().method()  # Calls C.method(), NOT A.method()!
In the diamond above, super() in B calls C’s method, not A’s. This is cooperative multiple inheritance, and misunderstanding it causes bugs.When to use inheritance vs composition:
  • Inheritance: True “is-a” relationships. Dog IS an Animal. Interface implementation via ABCs.
  • Composition: “has-a” relationships. Car HAS an Engine. Prefer this in most cases.
  • Mixins: Small, focused classes that add specific behavior. LoggingMixin, SerializableMixin. Keep them single-purpose.
What interviewers are really testing: Whether you understand MRO, super() in multiple inheritance, and the preference for composition.Red flag answer: “Inheritance lets you reuse code.” Without understanding MRO or the composition alternative.Follow-up:
  • “In the diamond problem, what does super() in class B actually call? Walk me through the MRO.”
  • “When would multiple inheritance cause problems that composition would not?”
  • “How do mixins work, and what rules should you follow when designing them?”
Magic methods (double underscore, aka “dunder” methods) are Python’s protocol system. They let your objects integrate with Python’s built-in operations, syntax, and standard library. This is arguably the most powerful feature of Python’s object model.The essential categories:Object lifecycle: __init__ (initializer), __new__ (constructor), __del__ (finalizer — avoid using this, use context managers instead)String representations:
  • __str__ — called by str() and print(). Human-readable. Example: "Alice (age 30)".
  • __repr__ — called by repr() and in the REPL. Developer-readable, ideally eval-able. Example: "Person('Alice', 30)". Rule: always implement __repr__. If you only implement one, implement __repr____str__ falls back to it.
Comparison: __eq__, __lt__, __le__, __gt__, __ge__, __ne__. Use @functools.total_ordering to implement only __eq__ and __lt__, and get the rest for free.Container protocol: __len__, __getitem__, __setitem__, __delitem__, __contains__, __iter__. Implementing these makes your object work with len(), [] indexing, in operator, and for loops.Arithmetic: __add__, __sub__, __mul__, etc. Also reverse versions (__radd__) for when your object is on the right side of the operator.Context manager: __enter__, __exit__ — enable with statement support.Callable: __call__ — makes instances callable like functions. Useful for stateful callables, decorators implemented as classes.Hashing: __hash__ — required for dict keys and set membership. Important rule: if you define __eq__, Python sets __hash__ to None (making objects unhashable). You must explicitly define __hash__ if you want hashability with custom equality.What interviewers are really testing: Whether you can design classes that integrate naturally with Python’s ecosystem, not just classes with methods.Red flag answer: Listing dunder methods without explaining the protocols they implement or the __repr__ vs __str__ distinction.Follow-up:
  • “You define __eq__ on your class. What happens to __hash__? What breaks?”
  • “How would you make a custom class work with for loops? What methods do you need?”
  • “What is the difference between __str__ and __repr__? Which should you always implement?”
__slots__ is a class-level declaration that tells Python: “These are the only attributes instances of this class will ever have.” It replaces the per-instance __dict__ with a fixed-size struct.How it works under the hood:
class Point:
    __slots__ = ('x', 'y')

    def __init__(self, x, y):
        self.x = x
        self.y = y

p = Point(1, 2)
p.z = 3  # AttributeError! Cannot add attributes not in __slots__
Normally, every Python instance has a __dict__ (a dict storing its attributes). That dict itself costs ~100 bytes per instance (empty dict overhead) plus ~50-70 bytes per attribute. With __slots__, Python stores attributes in a fixed C struct, eliminating the dict entirely.Real memory savings (CPython 3.11 benchmarks):
  • Regular class with 3 attributes: ~152 bytes per instance
  • __slots__ class with 3 attributes: ~72 bytes per instance
  • At 1 million instances: ~152 MB vs ~72 MB. That is 80 MB saved.
  • At 10 million instances (e.g., a graph with 10M nodes): ~800 MB saved. This can be the difference between OOM and running smoothly.
Trade-offs:
  • Cannot add attributes dynamically — no monkey patching instance state
  • Cannot use __dict__ — some serialization libraries expect it
  • Inheritance is tricky — if parent has __slots__, child must also declare __slots__ (can be empty __slots__ = ()) or it gets a __dict__ anyway
  • Cannot use __weakref__ unless you include it in __slots__
  • @dataclass(slots=True) (Python 3.10+) — the modern way to get slots without manual declaration
When to use:
  • Classes with millions of instances (nodes in a graph, points in a point cloud, rows in a dataset)
  • Internal library classes where the API is fixed
  • Performance-critical hot paths
What interviewers are really testing: Whether you have optimized memory in production and understand the trade-offs.Red flag answer: “It saves memory.” Without knowing how much, or the trade-offs.Follow-up:
  • “You add __slots__ to a class but it still has a __dict__. What went wrong?”
  • “How does __slots__ interact with inheritance? What happens in a child class?”
  • “When would __slots__ actually hurt you? Give me a scenario.”
@property turns method calls into attribute access, giving you controlled getters/setters without changing the API. This is Python’s answer to Java’s getName()/setName() pattern.The core pattern:
class Temperature:
    def __init__(self, celsius):
        self._celsius = celsius  # Store in private attribute

    @property
    def celsius(self):
        return self._celsius

    @celsius.setter
    def celsius(self, value):
        if value < -273.15:
            raise ValueError("Temperature below absolute zero")
        self._celsius = value

    @property
    def fahrenheit(self):
        return self._celsius * 9/5 + 32

    @fahrenheit.setter
    def fahrenheit(self, value):
        self.celsius = (value - 32) * 5/9  # Uses celsius setter for validation

t = Temperature(100)
print(t.fahrenheit)   # 212.0 -- looks like attribute access, calls method
t.celsius = -300      # ValueError -- validation fires
Why properties matter (the real answer):
  1. API evolution without breaking changes: Start with a simple attribute self.name = name. Later, when you need validation, replace it with @property — all calling code stays the same. No need to change obj.name to obj.get_name().
  2. Computed attributes: fahrenheit above is derived from celsius. The caller does not need to know it is calculated.
  3. Lazy loading: Compute expensive values only when accessed, cache the result.
  4. Validation at the boundary: Enforce invariants when attributes are set.
Under the hood: @property is syntactic sugar for the descriptor protocol (__get__, __set__, __delete__). Understanding descriptors explains how properties, methods, and classmethod/staticmethod all work.Gotcha: @property on a method that does I/O or expensive computation can surprise callers who expect attribute access to be cheap. If obj.data triggers a database query, it should probably be obj.get_data() or obj.load_data() to signal the cost.What interviewers are really testing: Whether you understand the Pythonic way to evolve APIs and enforce invariants.Red flag answer: Describing the syntax without explaining why you would use properties over plain attributes.Follow-up:
  • “You start with self.name = name and later need validation. How do you add it without breaking existing code?”
  • “What is the relationship between @property and the descriptor protocol?”
  • “When is @property a bad idea? Give me a case where a method is better.”
MRO determines the order Python searches for methods in a class hierarchy. This is critical for multiple inheritance and understanding super().Python uses C3 linearization (since Python 2.3). The algorithm guarantees:
  1. Children come before parents
  2. If a class inherits from multiple parents, they are searched in the order listed
  3. A valid linearization exists (otherwise Python raises TypeError at class definition time)
Concrete example:
class A: pass
class B(A): pass
class C(A): pass
class D(B, C): pass

print(D.__mro__)
# (D, B, C, A, object)
Why this matters for super(): super() does NOT mean “call the parent.” It means “call the next class in the MRO.” This is cooperative multiple inheritance:
class A:
    def method(self):
        print("A")

class B(A):
    def method(self):
        print("B")
        super().method()  # Calls C.method(), NOT A.method()!

class C(A):
    def method(self):
        print("C")
        super().method()  # Calls A.method()

class D(B, C):
    def method(self):
        print("D")
        super().method()  # Calls B.method()

D().method()  # Prints: D, B, C, A
When C3 linearization fails:
class X(A, B): pass
class Y(B, A): pass
# class Z(X, Y): pass  # TypeError! Cannot create consistent MRO
This happens when the ordering constraints are contradictory. Python refuses to guess and raises an error at class definition time.Production implications:
  • Always use super() consistently in a class hierarchy — mixing super() with direct parent calls breaks cooperative inheritance
  • Mixins should always call super() in their methods
  • Use ClassName.__mro__ or ClassName.mro() to debug method resolution issues
  • Consider whether you actually need multiple inheritance or if composition is simpler
What interviewers are really testing: Whether you understand that super() follows the MRO, not just the parent, and can trace through a diamond hierarchy.Red flag answer: “MRO is the order Python searches for methods.” Without being able to trace through a concrete diamond example.Follow-up:
  • “Walk me through the MRO for class D(B, C) where both B and C inherit from A. What does super() in B call?”
  • “When does C3 linearization fail? What causes a TypeError?”
  • “How would you debug a method resolution issue in a codebase with deep inheritance?“

4. Functions and Decorators

*args collects extra positional arguments into a tuple. **kwargs collects extra keyword arguments into a dict. But the real depth is in understanding how they interact with Python’s full argument resolution system.The complete argument order (must be in this order):
def func(pos_only, /, normal, *, kw_only, **kwargs):
    pass
# pos_only: positional-only (Python 3.8+, before /)
# normal: positional or keyword
# kw_only: keyword-only (after *)
# **kwargs: catches remaining keyword args
Real-world use cases:
# 1. Wrapper/decorator functions -- must pass through all arguments
def timing_decorator(func):
    def wrapper(*args, **kwargs):  # Accept anything
        start = time.time()
        result = func(*args, **kwargs)  # Pass through everything
        print(f"Took {time.time() - start:.3f}s")
        return result
    return wrapper

# 2. Extending parent class methods
class Child(Parent):
    def __init__(self, extra_param, *args, **kwargs):
        self.extra = extra_param
        super().__init__(*args, **kwargs)  # Forward to parent

# 3. Unpacking into function calls
args_tuple = (1, 2, 3)
kwargs_dict = {"name": "Alice", "age": 30}
func(*args_tuple, **kwargs_dict)  # Equivalent to func(1, 2, 3, name="Alice", age=30)
The / and * separators (Python 3.8+):
def example(a, b, /, c, d, *, e, f):
    pass
# a, b: positional only (cannot pass as keywords)
# c, d: either positional or keyword
# e, f: keyword only (cannot pass positionally)
This matters for API design: len takes obj as positional-only so you can write len(obj) but not len(obj=mylist).What interviewers are really testing: Whether you understand argument resolution order, unpacking, and real-world patterns like decorator forwarding.Red flag answer: Only knowing the basic definition without unpacking (*/**) or positional-only/keyword-only parameters.Follow-up:
  • “What is the difference between * and ** in a function call vs a function definition?”
  • “Why does Python 3.8 add positional-only parameters with /? Give a real API where this matters.”
  • “How would you use *args and **kwargs to implement a decorator that works with any function?”
Lambda functions are anonymous, single-expression functions. But the real interview question is about when they are appropriate and when they are an anti-pattern.Syntax: lambda arguments: expression (no statements, no assignments, no multi-line)Where lambdas shine:
# Sorting with custom key (most common legitimate use)
users.sort(key=lambda u: u.last_login)
sorted(items, key=lambda x: (x.priority, -x.created_at))

# Quick callbacks
button.on_click(lambda event: print(f"Clicked at {event.x}"))

# Simple transformations with map/filter
names = list(map(lambda u: u.name, users))
Where lambdas are an anti-pattern:
# BAD: Assigning lambda to a variable (use def instead)
square = lambda x: x**2  # flake8 E731 violation
# GOOD:
def square(x):
    return x**2

# BAD: Complex logic crammed into a lambda
process = lambda x: x.strip().lower().replace(' ', '_') if x else 'unknown'
# GOOD: Named function with clear intent
def normalize_name(x):
    if not x:
        return 'unknown'
    return x.strip().lower().replace(' ', '_')
Why named functions are usually better:
  • They show up with their name in stack traces (lambdas show as <lambda>)
  • They can have docstrings
  • They can contain multiple statements and early returns
  • ruff/flake8 flag lambda assignments (E731) for a reason
Alternative: operator module for common operations:
from operator import itemgetter, attrgetter
# Instead of: sorted(items, key=lambda x: x['name'])
sorted(items, key=itemgetter('name'))        # Faster, clearer
sorted(users, key=attrgetter('last_login'))   # Same for attributes
What interviewers are really testing: Judgment about when to use lambdas vs named functions.Red flag answer: Using lambdas everywhere to show off, or assigning them to variables.Follow-up:
  • “When would you use operator.itemgetter instead of a lambda?”
  • “A lambda appears in a stack trace as <lambda>. Why is that a problem?”
  • “Can a lambda contain an assignment? A conditional? Multiple expressions?”
Decorators are functions that take a function and return a modified function. But the real depth is understanding they are just syntactic sugar for higher-order functions, and knowing the production patterns.What @decorator actually does:
@my_decorator
def func():
    pass
# Is exactly equivalent to:
func = my_decorator(func)
The proper decorator template (production-grade):
import functools

def my_decorator(func):
    @functools.wraps(func)  # CRITICAL: preserves __name__, __doc__, __module__
    def wrapper(*args, **kwargs):
        # Before logic
        result = func(*args, **kwargs)
        # After logic
        return result
    return wrapper
Without @functools.wraps, the decorated function loses its name, docstring, and module — which breaks introspection, help(), Sphinx docs, and debugging. This is the #1 mistake in decorator implementations.Decorator with arguments (the two-level pattern):
def retry(max_attempts=3, delay=1):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception:
                    if attempt == max_attempts - 1:
                        raise
                    time.sleep(delay)
        return wrapper
    return decorator

@retry(max_attempts=5, delay=2)
def fetch_data(url):
    return requests.get(url)
Real-world decorator patterns:
  • @functools.lru_cache(maxsize=128) — memoization with LRU eviction. Use for expensive pure functions.
  • @functools.cache (Python 3.9+) — unbounded memoization (simpler than lru_cache).
  • @app.route('/path') (Flask) — URL routing.
  • @login_required (Django) — authentication enforcement.
  • @pytest.fixture — test dependency injection.
  • @contextmanager — turn a generator into a context manager.
Class-based decorators (for stateful decoration):
class CountCalls:
    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func
        self.count = 0

    def __call__(self, *args, **kwargs):
        self.count += 1
        return self.func(*args, **kwargs)
What interviewers are really testing: Whether you know @functools.wraps, can write decorators with arguments, and have used decorators in production.Red flag answer: Writing a decorator without @functools.wraps — this is the litmus test.Follow-up:
  • “What happens if you forget @functools.wraps? What breaks in production?”
  • “Write a decorator that takes arguments (e.g., @retry(max_attempts=3)).”
  • “How would you implement @lru_cache from scratch?”
A closure is a function that captures and remembers variables from its enclosing scope, even after the enclosing function has finished executing. The closure “closes over” those variables.How it works internally:
def make_multiplier(factor):
    def multiply(x):
        return x * factor  # 'factor' is a free variable, captured by closure
    return multiply

double = make_multiplier(2)
triple = make_multiplier(3)
print(double(5))   # 10
print(triple(5))   # 15
print(double.__closure__[0].cell_contents)  # 2 -- the captured value
Python stores captured variables in __closure__ as cell objects. The closure holds a reference to the variable, not a copy of the value. This leads to a famous gotcha:The late-binding closure bug:
# BUG: All functions return 4 (the final value of i)
functions = []
for i in range(5):
    functions.append(lambda: i)
print([f() for f in functions])  # [4, 4, 4, 4, 4]

# FIX 1: Default argument captures current value
functions = []
for i in range(5):
    functions.append(lambda i=i: i)
print([f() for f in functions])  # [0, 1, 2, 3, 4]

# FIX 2: Use functools.partial
from functools import partial
functions = [partial(lambda i: i, i) for i in range(5)]
This bug happens because the closure captures the variable i, not its value at closure creation time. When the lambda is called, it reads the current value of i, which is 4 after the loop finishes.Real-world closure use cases:
  • Factory functions (as shown above)
  • Callback registration with context
  • Implementing decorators (decorators are closures)
  • Data encapsulation (private state without classes)
  • Partial application of functions
What interviewers are really testing: Whether you understand the late-binding gotcha and can explain what __closure__ contains.Red flag answer: Defining closures correctly but not knowing about the loop variable capture bug.Follow-up:
  • “Why does [lambda: i for i in range(5)] produce five functions that all return 4? How do you fix it?”
  • “What is stored in func.__closure__? How can you inspect it?”
  • “How are closures related to decorators?”
Generator functions use yield to produce values lazily, one at a time, pausing execution between yields. They are the foundation of Python’s memory-efficient iteration.How generators work under the hood:
def fibonacci():
    a, b = 0, 1
    while True:
        yield a        # Pauses here, returns value
        a, b = b, a + b  # Resumes here on next() call

gen = fibonacci()     # No code executes yet -- returns generator object
next(gen)  # 0 -- executes until first yield
next(gen)  # 1 -- resumes after yield, executes until next yield
next(gen)  # 1
The generator’s execution frame is frozen on yield and thawed on next(). Local variables, instruction pointer, and stack are all preserved.Why generators matter (real numbers):
# List: allocates ALL 10M items in memory at once
data = [process(x) for x in range(10_000_000)]  # ~400MB for ints

# Generator: processes ONE item at a time, ~negligible memory
data = (process(x) for x in range(10_000_000))  # ~120 bytes
This difference is the reason range() in Python 3 returns a lazy object, and why tools like csv.reader() yield rows one at a time.yield vs return:
  • return terminates the function and sends one value
  • yield suspends the function and sends one value, resuming on next call
  • A generator can yield many times. After the function body is exhausted, StopIteration is raised.
send() and bidirectional communication:
def accumulator():
    total = 0
    while True:
        value = yield total   # yield sends total, then pauses; send() resumes with value
        total += value

acc = accumulator()
next(acc)          # Initialize (runs to first yield) -- returns 0
acc.send(10)       # Returns 10
acc.send(20)       # Returns 30
Generator pipelines (Unix-pipe style):
def read_lines(path):
    with open(path) as f:
        for line in f:
            yield line.strip()

def filter_errors(lines):
    for line in lines:
        if "ERROR" in line:
            yield line

def parse_timestamp(lines):
    for line in lines:
        yield line.split()[0]

# Pipeline processes one line at a time -- handles files larger than RAM
pipeline = parse_timestamp(filter_errors(read_lines("huge.log")))
What interviewers are really testing: Memory implications, send() for advanced use, and pipeline patterns.Red flag answer: Explaining yield without mentioning memory benefits or knowing about send().Follow-up:
  • “How would you process a 50GB log file without running out of memory?”
  • “What does generator.send(value) do? When would you use it?”
  • “What is the difference between a generator function and a generator expression?”
Function annotations (PEP 3107) provide metadata about parameters and return types. Type hints (PEP 484+) are the modern evolution, forming Python’s gradual typing system. This is one of the biggest shifts in Python culture in the last decade.Basic syntax:
def process_user(
    name: str,
    age: int,
    emails: list[str],              # Python 3.9+ (lowercase generics)
    metadata: dict[str, Any] | None = None  # Python 3.10+ (union with |)
) -> User:
    ...
Key point: Type hints are NOT enforced at runtime. Python ignores them entirely during execution. They are for:
  1. Static analysis: mypy, pyright, pytype catch type errors before running code
  2. IDE support: Autocomplete, refactoring, inline errors
  3. Documentation: Self-documenting function signatures
  4. Runtime validation: Libraries like pydantic use hints for runtime type checking and serialization
The typing module essentials:
from typing import (
    Optional,    # Optional[X] == X | None
    Union,       # Union[X, Y] == X | Y (use | in 3.10+)
    Any,         # Disables type checking for this value
    TypeVar,     # Generic type variable
    Protocol,    # Structural subtyping (duck typing for type checkers)
    TypeAlias,   # Explicit type alias
    Literal,     # Literal["read", "write"] -- exact values
    TypedDict,   # Dict with specific key types
)
typing.Protocol — the game changer (Python 3.8+):
from typing import Protocol

class Readable(Protocol):
    def read(self) -> str: ...

def process(source: Readable) -> str:
    return source.read()
# ANY object with a read() -> str method satisfies this -- no inheritance needed
This formalizes duck typing for static analysis.Production adoption:
  • Major projects like FastAPI, SQLAlchemy 2.0, and Django 4.1+ are fully typed
  • mypy --strict catches entire categories of bugs (null reference, wrong type, missing attributes)
  • Some teams report 15-30% fewer production bugs after adopting strict type checking
  • pyright (Microsoft, used in VS Code/Pylance) is faster and stricter than mypy
What interviewers are really testing: Whether you use type hints in practice and understand the tooling ecosystem.Red flag answer: “Annotations are just comments for documentation.” This misses the entire static analysis ecosystem.Follow-up:
  • “What is the difference between mypy and pyright? Which have you used?”
  • “How does typing.Protocol differ from abc.ABC? When would you use each?”
  • “How does Pydantic use type hints differently than mypy?”
Duck typing: “If it walks like a duck and quacks like a duck, it is a duck.” Python determines an object’s suitability by its behavior (methods and attributes), not its type (class hierarchy).How this differs from Java/C++: In Java, you must declare that a class implements an interface. In Python, any object that has the required methods just works:
class File:
    def read(self): return "file data"

class HttpResponse:
    def read(self): return "http data"

class MockData:
    def read(self): return "test data"

def process(source):
    return source.read()  # Works with ALL three -- no shared base class needed

process(File())         # Works
process(HttpResponse()) # Works
process(MockData())     # Works -- great for testing!
The Python standard library is built on duck typing protocols:
  • Iterable protocol: Has __iter__() — works with for loops
  • Sequence protocol: Has __getitem__() and __len__() — works with indexing and len()
  • Context manager protocol: Has __enter__() and __exit__() — works with with
  • Callable protocol: Has __call__() — can be called like a function
  • Hashable protocol: Has __hash__() — can be dict key or set member
The evolution: duck typing to structural typing:
  • Python 3.8+ typing.Protocol formalizes duck typing for static analysis
  • You define the expected interface, and mypy/pyright verify it at check time — without requiring inheritance
  • This gives you the flexibility of duck typing with the safety of static type checking
Trade-offs:
  • Pro: Extremely flexible, easy testing (mock anything), no rigid hierarchies
  • Con: Runtime AttributeError if object does not have required methods, harder to discover expected interfaces without type hints
What interviewers are really testing: Whether you understand that Python’s polymorphism model is fundamentally different from interface-based languages, and how Protocol bridges the gap.Red flag answer: Explaining duck typing without connecting it to Python’s protocol system or typing.Protocol.Follow-up:
  • “How would you make duck typing safer in a large codebase? What tools help?”
  • “What is the difference between typing.Protocol and abc.ABC? When do you want each?”
  • “Give me an example where duck typing makes testing dramatically easier than inheritance-based polymorphism.”

5. File Handling and I/O

File I/O is fundamental, but production code requires understanding encoding, buffering, and resource management.The correct pattern (always use with):
# Reading -- always specify encoding explicitly
with open('file.txt', 'r', encoding='utf-8') as f:
    content = f.read()         # Entire file into string (dangerous for large files)

# Line-by-line (memory-efficient for large files)
with open('huge.log', 'r', encoding='utf-8') as f:
    for line in f:             # File object is an iterator -- reads one line at a time
        process(line)

# Writing
with open('output.txt', 'w', encoding='utf-8') as f:
    f.write('Hello World\n')

# Binary mode (images, protobuf, pickle)
with open('image.png', 'rb') as f:
    data = f.read()            # Returns bytes, not str
Why with is non-negotiable: Without with, if an exception occurs between open() and f.close(), the file handle leaks. In a long-running server processing thousands of requests, leaked file handles exhaust the OS limit (typically 1024 on Linux by default) and crash the process. The with statement guarantees close() via __exit__.The encoding trap: open() without encoding uses the system default (often cp1252 on Windows, utf-8 on Linux). This causes cross-platform bugs. Always specify encoding='utf-8'. Python 3.15 will warn about missing encoding (PEP 686).Reading strategies by file size:
  • Small files (<10MB): f.read() into memory
  • Medium files (10MB-1GB): Iterate line-by-line with for line in f
  • Large files (1GB+): Use f.read(chunk_size) in a loop, or mmap for random access
  • Structured data: Use csv, json, pandas.read_csv() with chunksize parameter
What interviewers are really testing: Whether you handle encoding, large files, and resource cleanup correctly.Red flag answer: Using f = open(...) without with, or calling f.read() on a potentially large file.Follow-up:
  • “Your script works on Linux but produces garbled text on Windows. What is likely wrong?”
  • “How would you read a 50GB log file without loading it into memory?”
  • “What happens if you forget to close a file in a web server handling 10,000 requests/second?”
'w' (write): Creates file if it does not exist. Truncates to zero length if it does exist — all previous content is destroyed immediately on open(), not on write(). This is the #1 way developers accidentally delete data.'a' (append): Creates file if it does not exist. Writes are always appended to end. Atomic on most POSIX systems — multiple processes can append without corruption (this is why log files use append mode).The complete mode table (what most people miss):
  • 'r' — Read only. File must exist.
  • 'w' — Write only. Creates or truncates.
  • 'a' — Append only. Creates or appends.
  • 'x' — Exclusive creation. Fails if file exists. Use this for safe file creation — prevents overwriting.
  • 'r+' — Read and write. File must exist. Does not truncate.
  • 'w+' — Read and write. Creates or truncates.
  • 'b' suffix — Binary mode ('rb', 'wb'). No encoding/decoding, returns bytes.
  • 't' suffix — Text mode (default). Applies encoding, returns str.
Production safety patterns:
# DANGEROUS: truncates on open, data lost if crash before write completes
with open('config.json', 'w') as f:
    json.dump(config, f)

# SAFE: atomic write using temp file
import tempfile, os
with tempfile.NamedTemporaryFile('w', dir='.', delete=False, suffix='.tmp') as tmp:
    json.dump(config, tmp)
    tmp_path = tmp.name
os.replace(tmp_path, 'config.json')  # Atomic rename on POSIX
What interviewers are really testing: Whether you know about data loss risks with 'w' mode and safer alternatives.Red flag answer: Only knowing 'w' and 'a' without mentioning 'x' mode or atomic write patterns.Follow-up:
  • “Your app writes a config file with 'w' mode. It crashes mid-write. What state is the file in?”
  • “How do you safely overwrite a file without risking data loss?”
  • “What is 'x' mode and when would you use it?”
The csv module handles CSV parsing and writing, but production use involves edge cases that trip up most developers.Basic usage:
import csv

# Reading
with open('data.csv', 'r', encoding='utf-8') as f:
    reader = csv.DictReader(f)  # Prefer DictReader -- access by column name
    for row in reader:
        print(row['name'], row['email'])

# Writing
with open('output.csv', 'w', encoding='utf-8', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=['name', 'age'])
    writer.writeheader()
    writer.writerow({'name': 'Alice', 'age': 30})
The newline='' trap: On Windows, without newline='', you get double line endings in CSV output because the csv module writes \r\n and Python’s text mode adds another \r. Always pass newline='' when opening files for csv writing.When to use csv vs pandas:
  • csv module: Simple parsing, streaming large files (memory efficient), no dependencies
  • pandas.read_csv(): Data analysis, type inference, handling missing values, chunksize for large files, much faster for large datasets (C engine)
  • polars.read_csv(): Even faster than pandas for pure CSV loading, no GIL limitations
Edge cases that break naive parsers: Fields containing commas, quotes within quoted fields, newlines within fields, different delimiters (TSV, pipe-separated). The csv module handles all of these correctly.What interviewers are really testing: Whether you handle encodings, the newline parameter, and know when to graduate to pandas/polars.Red flag answer: Parsing CSV by splitting on commas (line.split(',')) instead of using the csv module.Follow-up:
  • “Why does splitting on commas not work for real CSV files?”
  • “How would you process a 10GB CSV file that does not fit in memory?”
  • “When would you use csv.DictReader vs pandas.read_csv()?”
The json module serializes Python objects to JSON strings and back. But production use requires understanding its limitations and alternatives.Core API:
import json

# Serialization (Python -> JSON)
data = {'name': 'Alice', 'age': 30, 'active': True}
json_str = json.dumps(data, indent=2)        # To string
json.dump(data, open('data.json', 'w'))       # To file

# Deserialization (JSON -> Python)
parsed = json.loads(json_str)                 # From string
parsed = json.load(open('data.json', 'r'))    # From file
Type mapping gotchas:
  • JSON has no integer type — json.loads('{"n": 12345678901234567890}') converts to Python int, but JavaScript loses precision for integers > 2^53.
  • JSON has no datetime, bytes, set, or tuple type. You need custom serialization.
  • json.dumps fails on non-serializable types (datetime, Decimal, numpy arrays). Fix with default parameter:
import datetime
from decimal import Decimal

def json_serializer(obj):
    if isinstance(obj, datetime.datetime):
        return obj.isoformat()
    if isinstance(obj, Decimal):
        return str(obj)
    raise TypeError(f"Not serializable: {type(obj)}")

json.dumps(data, default=json_serializer)
Performance and alternatives:
  • json (stdlib): ~50MB/s. Fine for most use cases.
  • orjson (third-party): ~1GB/s. Auto-serializes datetime, UUID, numpy. Used by FastAPI internally.
  • ujson: ~300MB/s. Drop-in replacement.
  • msgpack: Binary format, faster and smaller than JSON. Good for internal service communication.
Security concern: Never use eval() to parse JSON. Always use json.loads(). eval() executes arbitrary code.What interviewers are really testing: Whether you know the type limitations, custom serialization, and when to use faster alternatives.Red flag answer: Not knowing that json.dumps fails on datetime or Decimal objects.Follow-up:
  • “Your API returns a Decimal and datetime. How do you serialize them to JSON?”
  • “When would you use orjson instead of the stdlib json module?”
  • json.loads is slow for your 500MB response. What are your options?”
Context managers implement the with statement protocol, guaranteeing resource cleanup even when exceptions occur. They are Python’s answer to C++‘s RAII and Java’s try-with-resources.The protocol — two magic methods:
class DatabaseConnection:
    def __enter__(self):
        self.conn = connect_to_db()
        return self.conn       # Value bound by 'as' in 'with ... as'

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.conn.close()      # Always called, even if exception occurs
        return False           # False = re-raise exception, True = suppress it
The simpler way — @contextmanager decorator:
from contextlib import contextmanager

@contextmanager
def timer(label):
    start = time.time()
    try:
        yield            # Code inside 'with' block runs here
    finally:
        elapsed = time.time() - start
        print(f"{label}: {elapsed:.3f}s")

with timer("data processing"):
    process_data()       # Automatically timed
Real-world context managers you should know:
  • open() — file cleanup
  • threading.Lock() — lock acquire/release
  • contextlib.suppress(ExceptionType) — silently ignore specific exceptions
  • tempfile.TemporaryDirectory() — auto-cleanup temp directories
  • unittest.mock.patch() — mock scoping in tests
  • decimal.localcontext() — temporary decimal precision changes
  • contextlib.ExitStack() — manage dynamic number of context managers
The __exit__ return value matters: If __exit__ returns True, the exception is suppressed (swallowed). If False (default), the exception propagates. Almost always return False — suppressing exceptions silently is dangerous.What interviewers are really testing: Whether you can write context managers and understand when to use class-based vs @contextmanager.Red flag answer: Knowing with works with files but not being able to create your own context manager.Follow-up:
  • “Write a context manager that temporarily changes the working directory and restores it.”
  • “What does the return value of __exit__ control? When would you return True?”
  • “How would you manage a dynamic number of resources with contextlib.ExitStack?“

6. Exception Handling

Exception handling provides structured error recovery. But the real interview topic is the philosophy of how and when to use exceptions in Python.Python’s EAFP vs LBYL philosophy:
  • EAFP (Easier to Ask Forgiveness than Permission) — the Pythonic way. Try the operation, catch the exception if it fails:
# EAFP (Pythonic)
try:
    value = my_dict[key]
except KeyError:
    value = default

# LBYL (non-Pythonic in most cases)
if key in my_dict:
    value = my_dict[key]
else:
    value = default
EAFP is preferred because: (1) it avoids race conditions (the key could be deleted between the check and the access), (2) it is faster in the happy path (no extra lookup), and (3) it is more readable for complex conditions.Exception handling is NOT error hiding:
# TERRIBLE -- hides all bugs, makes debugging impossible
try:
    complex_operation()
except:
    pass

# CORRECT -- catch specific exceptions, handle or log them
try:
    user = fetch_user(user_id)
except ConnectionError:
    logger.warning(f"DB connection failed for user {user_id}")
    user = get_cached_user(user_id)
except UserNotFoundError:
    raise HTTPException(status_code=404, detail="User not found")
The Zen of Python says: “Errors should never pass silently. Unless explicitly silenced.” This means every except block should either handle the error meaningfully, log it, or re-raise it.What interviewers are really testing: Whether you understand EAFP, catch specific exceptions, and never use bare except: pass.Red flag answer: Using try/except as a replacement for input validation, or catching Exception everywhere.Follow-up:
  • “What is EAFP and why does Python prefer it over LBYL?”
  • “When is LBYL actually better than EAFP? Give me a concrete case.”
  • “What is wrong with except Exception as e: pass?”
Python’s exception hierarchy is a tree rooted at BaseException. Understanding the hierarchy matters more than memorizing names.The hierarchy (what most people miss):
BaseException
+-- SystemExit          # sys.exit() -- do NOT catch this
+-- KeyboardInterrupt   # Ctrl+C -- do NOT catch this
+-- GeneratorExit       # Generator cleanup
+-- Exception           # Base for all "normal" exceptions
    +-- StopIteration   # Signals iterator exhaustion
    +-- ArithmeticError
    |   +-- ZeroDivisionError
    +-- LookupError
    |   +-- IndexError
    |   +-- KeyError
    +-- OSError
    |   +-- FileNotFoundError
    |   +-- PermissionError
    |   +-- ConnectionError
    +-- ValueError
    +-- TypeError
    +-- AttributeError
    +-- RuntimeError
        +-- RecursionError
Critical rule: Catch Exception, NEVER BaseException. Catching BaseException traps KeyboardInterrupt and SystemExit, making your program impossible to kill with Ctrl+C.Exception groups (Python 3.11+): ExceptionGroup wraps multiple exceptions raised concurrently (e.g., from asyncio.TaskGroup). Use except* to catch specific types from the group. This is a fundamental change for async error handling.What interviewers are really testing: Whether you know the hierarchy, especially the BaseException vs Exception distinction.Red flag answer: Listing exceptions without understanding the hierarchy or catching BaseException.Follow-up:
  • “Why should you never catch BaseException? What goes wrong?”
  • “What are ExceptionGroup and except* in Python 3.11+?”
  • “When would you catch OSError vs FileNotFoundError specifically?”
The full try statement has four blocks, each with a specific purpose. The else clause is the one most developers misunderstand.
try:
    result = fetch_data(url)       # Code that might fail
except ConnectionError as e:
    logger.error(f"Connection failed: {e}")
    result = get_cached_data()     # Recovery logic
except ValueError as e:
    raise InvalidDataError(f"Bad response: {e}") from e  # Re-raise with chain
else:
    save_to_database(result)       # Only runs if NO exception in try block
    logger.info("Data saved successfully")
finally:
    cleanup_temp_files()           # ALWAYS runs (even if return/break/continue in try)
Why else exists (the misunderstood clause): Code in else only runs if try succeeds with no exceptions. Why not just put it in try? Because if save_to_database raises an exception, you do not want it caught by the except ConnectionError handler — that would hide a database bug as a connection error.finally guarantees (and gotchas):
  • finally runs even if try has a return statement
  • finally runs even if an exception is raised and not caught
  • Gotcha: If finally has a return, it silently overrides the try block’s return value and suppresses any active exception. Never put return in finally.
Exception chaining (from keyword):
try:
    data = json.loads(response)
except json.JSONDecodeError as e:
    raise ValidationError("Invalid JSON response") from e
# The original exception is preserved in __cause__ for debugging
This preserves the full traceback chain. Without from e, the original context is still available in __context__ but the intent is less clear.What interviewers are really testing: Whether you know what else is for and understand exception chaining.Red flag answer: Not knowing what else does, or putting all logic in try instead of using else.Follow-up:
  • “What happens if finally contains a return statement?”
  • “Why would you put code in else instead of at the end of try?”
  • “What is exception chaining with raise ... from ...? When do you use it?”
Custom exceptions create a domain-specific error hierarchy for your application. This is essential for APIs and libraries.Production-grade custom exception pattern:
class AppError(Exception):
    """Base exception for our application."""
    def __init__(self, message: str, code: str = "UNKNOWN", details: dict = None):
        self.message = message
        self.code = code
        self.details = details or {}
        super().__init__(self.message)

class ValidationError(AppError):
    """Raised when input validation fails."""
    def __init__(self, field: str, message: str):
        super().__init__(
            message=f"Validation failed for '{field}': {message}",
            code="VALIDATION_ERROR",
            details={"field": field}
        )

class NotFoundError(AppError):
    """Raised when a requested resource does not exist."""
    def __init__(self, resource: str, resource_id: str):
        super().__init__(
            message=f"{resource} '{resource_id}' not found",
            code="NOT_FOUND",
            details={"resource": resource, "id": resource_id}
        )
Design principles:
  1. Create a base exception for your library/app. Callers can catch AppError to handle all your exceptions.
  2. Always inherit from Exception, never BaseException.
  3. Add structured data (error codes, details dict) for machine-readable error handling, especially in APIs.
  4. Map to HTTP status codes in web apps: ValidationError -> 400, NotFoundError -> 404, AuthError -> 401.
  5. Keep the hierarchy shallow — 2-3 levels max. Deep exception hierarchies are as bad as deep inheritance.
Anti-pattern — generic exceptions:
# BAD: meaningless exception
raise Exception("Something went wrong")

# GOOD: specific, actionable
raise InsufficientFundsError(account_id=acct, requested=100, available=50)
What interviewers are really testing: Whether you design exception hierarchies for real applications, not just toy examples.Red flag answer: Creating a custom exception that just inherits Exception with no additional context or structure.Follow-up:
  • “How would you design an exception hierarchy for a payment processing API?”
  • “How do your custom exceptions map to HTTP status codes in a REST API?”
  • “When is it better to return an error value (like Result type) vs raising an exception?”
assert tests conditions that should ALWAYS be true if the code is correct. They are a debugging tool, not an error handling mechanism.Syntax and behavior:
assert condition, "Error message"
# Equivalent to:
if __debug__:
    if not condition:
        raise AssertionError("Error message")
The critical rule: assertions are REMOVED by python -O (optimize flag). Running python -O script.py sets __debug__ = False and strips all assert statements. This means:
# DANGEROUS: Using assert for input validation
def withdraw(amount):
    assert amount > 0, "Amount must be positive"  # Removed by -O flag!
    self.balance -= amount  # Oops, negative withdrawal goes through

# CORRECT: Use explicit validation
def withdraw(amount):
    if amount <= 0:
        raise ValueError("Amount must be positive")
    self.balance -= amount
When to use assert:
  • Internal invariants: assert len(self._items) == self._count
  • Preconditions in internal functions (not public APIs)
  • Sanity checks during development
  • Test assertions: assert result == expected (pytest uses this)
When NOT to use assert:
  • User input validation (use ValueError, TypeError)
  • Security checks (use explicit conditionals)
  • Anything that must run in production (assertions are stripped by -O)
What interviewers are really testing: Whether you know assertions are removed in optimized mode and should not be used for validation.Red flag answer: Using assertions for input validation or not knowing about the -O flag.Follow-up:
  • “What happens to assertions when you run python -O? What breaks if you rely on them?”
  • “When should you use assert vs raise ValueError?”
  • “How does pytest use assertions differently from standard Python?“

7. Modules and Packages

Module: A single .py file. When imported, Python executes the file top-to-bottom and creates a module object. The module’s functions, classes, and variables become attributes of that object.Package: A directory containing modules and (typically) an __init__.py file. Packages create namespaces for organizing related modules hierarchically.What actually happens on import (the import machinery):
  1. Python checks sys.modules (import cache). If already imported, returns cached module.
  2. Python searches sys.path (list of directories) to find the module.
  3. Python creates a module object and executes the module’s code.
  4. The module is cached in sys.modules.
This means:
  • Module code runs exactly once on first import, regardless of how many files import it. Subsequent imports return the cached module.
  • Modules are effectively singletons — this is why module-level state is shared across the entire application.
  • sys.path determines where Python looks. It includes the script’s directory, PYTHONPATH env var, and installed packages.
Namespace packages (Python 3.3+): Directories without __init__.py can be namespace packages, allowing a single logical package to be split across multiple directories on disk. Used by large frameworks with plugin systems.What interviewers are really testing: Whether you understand the import machinery (caching, sys.modules, execution order).Red flag answer: “A module is a file, a package is a directory.” Without understanding caching or sys.path.Follow-up:
  • “What happens if two different files import the same module? Does the module code run twice?”
  • “How do you add a custom directory to Python’s import search path?”
  • “What are namespace packages and when would you use them?”
Python offers several import styles, each with different implications for readability, performance, and namespace management.Import styles and when to use each:
import os                          # Full module -- use os.path.join()
from os.path import join, exists   # Specific names -- use join() directly
from os.path import join as pjoin  # Alias -- when name conflicts exist
import numpy as np                 # Community-standard aliases (np, pd, plt)
Why from module import * is dangerous:
  • Pollutes the namespace with unknown names
  • Can silently override existing variables
  • Makes it impossible to know where a name came from during code review
  • Only acceptable in __init__.py to re-export a public API, controlled by __all__
Circular import resolution:
# a.py imports b.py, b.py imports a.py -- ImportError or AttributeError

# Fix 1: Import inside function (lazy import)
def process():
    from other_module import helper  # Only imported when function is called
    return helper()

# Fix 2: Restructure into a third module
# Fix 3: Use TYPE_CHECKING for type hints only
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from other_module import MyType  # Only imported by type checkers, not at runtime
Import performance:
  • First import executes the module (can be slow for heavy modules like pandas — ~200ms)
  • Subsequent imports are O(1) dict lookups in sys.modules
  • Lazy imports (importlib.import_module() or in-function imports) can improve startup time
What interviewers are really testing: Whether you know about circular imports, import * dangers, and performance implications.Red flag answer: Using from module import * in production code.Follow-up:
  • “How do you resolve circular imports? Give me three strategies.”
  • “What is TYPE_CHECKING and when do you use it?”
  • “Your application takes 5 seconds to start. You suspect heavy imports. How do you diagnose it?”
__init__.py is the initializer for a Python package. It runs when the package is first imported.Key responsibilities:
  1. Marks directory as a package (required before Python 3.3, recommended after)
  2. Controls the public API via __all__:
# mypackage/__init__.py
from .core import Engine
from .config import Config
from .exceptions import AppError

__all__ = ['Engine', 'Config', 'AppError']  # Controls 'from mypackage import *'
  1. Re-exports for convenience: Users write from mypackage import Engine instead of from mypackage.core import Engine
  2. Package-level initialization: Setup logging, register plugins, validate environment
__all__ does two things:
  • Controls what from package import * exports
  • Signals to type checkers and IDEs what the public API is
Common patterns in production:
  • Empty __init__.py: Most common. Just marks as package.
  • Re-export pattern: Flatten nested imports for cleaner public API.
  • Lazy loading: Import submodules on first access to improve startup time (used by large libraries like numpy).
What interviewers are really testing: Whether you understand __all__, re-exporting, and API design via __init__.py.Red flag answer: “It makes a directory a package.” Without knowing about __all__ or re-exporting.Follow-up:
  • “How does __all__ affect from package import *?”
  • “How would you design __init__.py for a library with a clean public API?”
  • “What are the pros and cons of an empty __init__.py vs one that re-exports?”
__name__ is a special variable that Python sets depending on how the file is executed.
  • Executed directly (python script.py): __name__ == '__main__'
  • Imported as module (import script): __name__ == 'script'
The standard guard pattern:
def main():
    # Application logic here
    print("Running as main program")

if __name__ == '__main__':
    main()
Why this pattern matters:
  1. Testability: import mymodule does not trigger side effects. Tests can import and test individual functions.
  2. Reusability: The module can be both a script and a library.
  3. Multiprocessing on Windows: multiprocessing spawns new Python processes that import the module. Without the guard, the spawned process re-runs the main code, causing infinite process spawning.
The __main__.py convention: For packages, python -m mypackage runs mypackage/__main__.py. This is how python -m pytest, python -m http.server, and python -m venv work.What interviewers are really testing: Whether you know why the guard exists, especially the multiprocessing and testability reasons.Red flag answer: Knowing the pattern without understanding why it prevents side effects on import.Follow-up:
  • “What happens on Windows with multiprocessing if you forget the if __name__ == '__main__' guard?”
  • “What is __main__.py and how does python -m package_name use it?”
  • “How does the guard pattern improve testability?”
Virtual environments isolate project dependencies, preventing conflicts between projects. This is non-negotiable for professional Python development.The modern tool landscape (2024+):uv (by Astral, the ruff creators) — the new standard:
uv venv                    # Create venv (10-100x faster than python -m venv)
source .venv/bin/activate  # Activate (Linux/Mac)
uv pip install flask       # Install (10-100x faster than pip)
uv pip compile requirements.in -o requirements.txt  # Lock dependencies
uv is a drop-in replacement for pip, pip-tools, virtualenv, and venv. Written in Rust, it is dramatically faster.poetry — dependency management + packaging:
poetry init                # Create pyproject.toml
poetry add flask           # Add dependency (auto-resolves versions)
poetry lock                # Lock exact versions
poetry install             # Install from lock file
pip + requirements.txt — the classic approach:
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip freeze > requirements.txt  # Pin current versions
Lock files matter (the real production lesson): Without pinned versions, pip install flask might install Flask 2.0 today and Flask 3.0 tomorrow, breaking your app. Always use lock files (poetry.lock, requirements.txt with pinned versions, uv.lock).pyproject.toml (PEP 621) is the modern standard for project metadata, replacing setup.py, setup.cfg, and requirements.txt in many workflows.What interviewers are really testing: Whether you have opinions about dependency management tooling and understand lock files.Red flag answer: “I use pip install without a virtual environment.” This is a career-ending answer for a senior role.Follow-up:
  • “What is the difference between poetry and uv? When would you choose each?”
  • “Why are lock files important? What breaks without them?”
  • “How would you set up a reproducible development environment for a new team member?“

8. Advanced Python Concepts

This is one of the most fundamental concepts in Python. Understanding the iterator protocol explains how for loops, comprehensions, map, filter, generators, and zip all work.The protocol:
  • Iterable: Any object with __iter__() method that returns an iterator. Lists, strings, dicts, files, generators are all iterables.
  • Iterator: An object with both __iter__() (returns self) and __next__() (returns next value or raises StopIteration).
What for loop actually does:
# This:
for item in [1, 2, 3]:
    print(item)

# Is equivalent to:
it = iter([1, 2, 3])   # Calls list.__iter__(), returns list_iterator
while True:
    try:
        item = next(it)  # Calls iterator.__next__()
        print(item)
    except StopIteration:
        break
Building a custom iterator:
class CountDown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self       # Iterator returns itself

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        self.current -= 1
        return self.current + 1
Key distinction: iterable vs iterator. A list is iterable (you can get fresh iterators from it repeatedly). An iterator is a one-pass object — once exhausted, it is done:
nums = [1, 2, 3]         # Iterable -- can iterate multiple times
it = iter(nums)           # Iterator -- single pass
list(it)                  # [1, 2, 3]
list(it)                  # [] -- exhausted!
Generators are iterators (single-pass). Files are iterators. This is why you can only loop through a generator once.What interviewers are really testing: Whether you understand the protocol mechanics, not just the vocabulary.Red flag answer: Saying “iterables are things you can iterate over” without explaining __iter__ and __next__.Follow-up:
  • “What is the difference between an iterable and an iterator? Can something be both?”
  • “Why can you loop over a list multiple times but only over a generator once?”
  • “Build me an iterator class that yields Fibonacci numbers.”
Multithreading: Multiple threads within a single process, sharing the same memory space. The GIL allows only one thread to execute Python bytecode at a time.Multiprocessing: Multiple separate processes, each with its own Python interpreter and memory space. True parallel execution on multiple CPU cores.Decision matrix:
ThreadingMultiprocessingasyncio
Best forI/O-bound (network, disk)CPU-bound (computation)High-concurrency I/O
MemoryShared (lightweight)Separate (heavy ~30-50MB per process)Shared (very lightweight)
GIL impactBlocked for CPU workNo GIL issue (separate interpreters)Single thread, no GIL issue
CommunicationShared objects + locksQueues, Pipes (serialization overhead)Coroutine chaining
Overhead per unit~100KB per thread~30-50MB per process~1KB per coroutine
DebuggingHard (race conditions)Medium (process isolation)Medium (async stack traces)
Real-world examples with numbers:
# I/O-bound: Scraping 100 URLs
# Sequential: ~100 seconds (1s per URL)
# Threading (10 threads): ~10 seconds
# asyncio (100 concurrent): ~1-2 seconds

# CPU-bound: Processing 1M images
# Sequential: ~1000 seconds
# Threading: ~1000 seconds (GIL prevents parallelism)
# Multiprocessing (8 cores): ~125 seconds
The modern API (concurrent.futures):
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

# Switch between threading and multiprocessing by changing one word
with ThreadPoolExecutor(max_workers=10) as executor:
    results = list(executor.map(fetch_url, urls))

with ProcessPoolExecutor(max_workers=8) as executor:
    results = list(executor.map(process_image, images))
What interviewers are really testing: Whether you can choose the right concurrency model based on the task type.Red flag answer: “Use multithreading for parallelism.” This shows fundamental GIL misunderstanding.Follow-up:
  • “Your image processing is slow. You add threads but it does not speed up. Why?”
  • “What are the risks of shared memory in multithreading?”
  • “How does concurrent.futures simplify switching between threads and processes?”
The GIL is a mutex in CPython that ensures only one thread executes Python bytecode at a time. It is the most misunderstood and debated feature of Python.Why the GIL exists: CPython’s memory management uses reference counting. Without the GIL, every reference count update would need its own lock, which would be far slower for single-threaded code (the common case). The GIL was a pragmatic trade-off: simplify memory management at the cost of limiting multithreaded CPU parallelism.What the GIL does and does not prevent:
  • Prevents: True parallel execution of Python bytecode across threads
  • Does NOT prevent: I/O parallelism (the GIL is released during I/O operations — network, disk, sleep)
  • Does NOT prevent: Parallel execution of C extensions (NumPy releases the GIL during array operations)
  • Does NOT prevent: Multiprocessing (separate processes, separate GILs)
The free-threaded Python initiative (PEP 703, Python 3.13+): Python 3.13 introduced an experimental build with the GIL disabled (python --disable-gil). This is the biggest change to CPython in decades. The t suffix in version tags (e.g., python3.13t) indicates the free-threaded build. As of 3.13, it is experimental; expected to become default in 3-5 years.Workarounds used in production today:
  1. multiprocessing / ProcessPoolExecutor for CPU-bound work
  2. asyncio for I/O-bound concurrency
  3. C extensions (NumPy, Cython) that release the GIL
  4. Use a different Python implementation (PyPy has the GIL; Jython and GraalPython do not)
What interviewers are really testing: Whether you understand the GIL’s actual impact (limited to CPU-bound threading) and know the workarounds.Red flag answer: “The GIL makes Python slow.” The GIL only limits CPU-bound threading. Single-threaded Python is not affected.Follow-up:
  • “If the GIL limits threading, why does Python even have a threading module?”
  • “What is PEP 703 (free-threaded Python) and what is its status?”
  • “How does NumPy achieve parallelism despite the GIL?”
Metaclasses are “classes of classes.” Just as an object is an instance of a class, a class is an instance of a metaclass. The default metaclass is type.The mind-bending truth:
class Foo:
    pass

type(Foo)        # <class 'type'> -- Foo is an instance of type
type(type)       # <class 'type'> -- type is an instance of itself
isinstance(Foo, type)  # True -- classes are objects too
How class creation works: When Python encounters class Foo(Base, metaclass=MyMeta):, it calls MyMeta('Foo', (Base,), namespace_dict). The metaclass controls:
  • __new__: Creates the class object
  • __init__: Initializes the class object
  • __call__: Controls what happens when the class is instantiated
When metaclasses are actually used (rare but powerful):
  1. ORMs: Django’s Model metaclass automatically creates database fields from class attributes
  2. API frameworks: Abstract base classes use ABCMeta to enforce abstract method implementation
  3. Singleton pattern: Metaclass that caches instances
  4. Automatic registration: Metaclass that registers all subclasses in a registry
  5. Validation: Ensure class definitions follow specific rules at definition time
Tim Peters’ quote (creator of Timsort and the Zen): “Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t.”Modern alternatives that replaced most metaclass use cases:
  • __init_subclass__ (Python 3.6+) — hook called when a class is subclassed. Handles most registration/validation patterns without a metaclass.
  • __set_name__ (Python 3.6+) — descriptor hook for attribute naming.
  • @dataclass and typing.Protocol — handle many patterns that previously required metaclasses.
What interviewers are really testing: Whether you understand the object model deeply enough to explain metaclasses, AND whether you have the judgment to know when simpler alternatives exist.Red flag answer: Either “I do not know what metaclasses are” (knowledge gap) or immediately reaching for metaclasses when __init_subclass__ would suffice (poor judgment).Follow-up:
  • “When would you use __init_subclass__ instead of a metaclass?”
  • “How does Django’s Model class use metaclasses to create database tables from class definitions?”
  • “Write a metaclass that enforces all subclass method names are lowercase.”
Coroutines enable cooperative multitasking — a single thread handles many concurrent I/O operations by switching between them when one is waiting. This is Python’s answer to the C10K problem.The mental model:
import asyncio
import aiohttp

async def fetch_url(session, url):
    async with session.get(url) as response:   # I/O wait -- yield control
        return await response.text()             # I/O wait -- yield control

async def main():
    async with aiohttp.ClientSession() as session:
        # All 100 requests run concurrently on a SINGLE thread
        tasks = [fetch_url(session, url) for url in urls]
        results = await asyncio.gather(*tasks)

asyncio.run(main())
How it works:
  1. async def creates a coroutine function (calling it returns a coroutine object, does not execute it)
  2. await suspends the coroutine and yields control back to the event loop
  3. The event loop runs other ready coroutines while this one waits for I/O
  4. When I/O completes, the event loop resumes the coroutine
Key asyncio patterns:
  • asyncio.gather(*coros) — Run multiple coroutines concurrently, return all results
  • asyncio.TaskGroup() (Python 3.11+) — Structured concurrency with proper error handling
  • asyncio.Queue — Producer-consumer patterns
  • async for — Iterate over async iterators
  • async with — Async context managers
When async is worth the complexity:
  • Web servers handling thousands of concurrent connections (FastAPI, Starlette)
  • API clients making many concurrent requests
  • Chat applications, WebSocket servers
  • NOT for CPU-bound work (async is single-threaded)
The “function color” problem: Once you use async, it infects your entire call stack. You cannot call an async function from a regular function without asyncio.run(). This leads to “async all the way up” or painful refactoring.What interviewers are really testing: Whether you understand the event loop model and when async is the right choice.Red flag answer: “Async makes things run in parallel.” It does not — it is concurrent, not parallel. One thread, cooperative scheduling.Follow-up:
  • “What is the difference between concurrency and parallelism? Where does asyncio fit?”
  • “What is the ‘function color’ problem with async/await?”
  • “When would you use asyncio.TaskGroup vs asyncio.gather?”
All four are syntactic sugar for creating collections from iterables, but they differ in output type and memory behavior.Side-by-side comparison:
data = range(10)

# List comprehension -- creates full list in memory
squares_list = [x**2 for x in data]              # [0, 1, 4, 9, ...]

# Dict comprehension -- creates full dict in memory
squares_dict = {x: x**2 for x in data}           # {0: 0, 1: 1, 2: 4, ...}

# Set comprehension -- creates full set in memory (deduplicates)
squares_set = {x % 3 for x in data}              # {0, 1, 2}

# Generator expression -- lazy, produces one value at a time
squares_gen = (x**2 for x in data)                # <generator object>
The critical memory difference:
# This allocates ~800MB for 100M integers
big_list = [x for x in range(100_000_000)]

# This uses ~120 bytes regardless of size
big_gen = (x for x in range(100_000_000))
When to use each:
  • List comprehension: When you need the full collection (indexing, slicing, multiple iterations, len())
  • Dict comprehension: Transforming/filtering dicts, inverting key-value pairs
  • Set comprehension: Extracting unique values from a collection
  • Generator expression: When you only iterate once, especially for large/infinite sequences. Also when feeding directly into sum(), min(), max(), any(), all() — no intermediate list needed.
Pro tip: Generator expressions can be passed directly to functions without extra parentheses:
total = sum(x**2 for x in range(1000))  # No need for sum([x**2 for ...])
Scope gotcha (Python 3 only): Comprehensions have their own scope. The loop variable does not leak:
x = "before"
[x for x in range(5)]
print(x)  # "before" in Python 3, 4 in Python 2
What interviewers are really testing: Whether you choose the right comprehension type based on memory and access patterns.Red flag answer: Using list comprehensions when a generator expression would be more memory-efficient.Follow-up:
  • “Why does sum(x**2 for x in range(10)) not need square brackets?”
  • “You have a 100GB dataset. How do you process it with comprehensions?”
  • “What is the scope of the loop variable in a comprehension? Does it differ between Python 2 and 3?”
Python uses a two-part memory management system: reference counting (primary) and cyclic garbage collector (secondary, for circular references).1. Reference counting (immediate cleanup): Every object has a reference count. When it reaches zero, memory is freed immediately:
a = [1, 2, 3]    # refcount = 1
b = a             # refcount = 2
del a             # refcount = 1
del b             # refcount = 0 -> immediately freed
You can inspect reference counts with sys.getrefcount(obj) (returns count + 1 because the function call itself creates a temporary reference).2. Cyclic garbage collector (for circular references): Reference counting cannot handle cycles:
a = []
b = []
a.append(b)  # a references b
b.append(a)  # b references a -- circular reference!
del a, b     # refcounts are 1 (not 0) -- memory leaked without cyclic GC
The cyclic GC uses a generational algorithm with three generations:
  • Gen 0: Newly created objects. Collected most frequently (threshold: ~700 objects).
  • Gen 1: Survived one collection. Collected less frequently.
  • Gen 2: Long-lived objects. Collected rarely. Objects that survive collection are promoted to the next generation. This is based on the “generational hypothesis”: most objects die young.
Production implications:
  • GC pauses can cause latency spikes in real-time systems. Instagram famously disabled the cyclic GC in their Django servers because it caused 10% of CPU time to be spent on GC.
  • gc.disable() is safe IF you avoid circular references (use weak references or restructure data).
  • gc.collect() forces a collection — useful before memory-critical sections.
  • gc.get_threshold() returns (700, 10, 10) — the generation thresholds.
  • __del__ finalizers complicate GC (objects with __del__ in a cycle were uncollectable before Python 3.4).
What interviewers are really testing: Whether you understand both mechanisms and the generational approach.Red flag answer: Only knowing reference counting, or thinking Python uses Java-style stop-the-world GC.Follow-up:
  • “Why did Instagram disable Python’s garbage collector? What did they do instead?”
  • “How does the generational GC decide when to run?”
  • “What happens if two objects in a reference cycle both have __del__ methods?“

9. Web Development with Python

A production-grade Flask app looks very different from the “Hello World” tutorial:Tutorial version (what everyone shows):
from flask import Flask
app = Flask(__name__)

@app.route('/')
def home():
    return 'Hello, World!'
Production version (what you should actually discuss):
# Application factory pattern (testable, configurable)
from flask import Flask

def create_app(config_name='default'):
    app = Flask(__name__)
    app.config.from_object(config[config_name])

    # Register blueprints (modular route organization)
    from .api import api_bp
    app.register_blueprint(api_bp, url_prefix='/api/v1')

    # Error handlers
    @app.errorhandler(404)
    def not_found(error):
        return {"error": "Not found"}, 404

    return app
Key production concerns:
  • Application factory pattern: Do not create app at module level. Use a factory function for testability (create fresh app per test).
  • Blueprints: Organize routes into modules. Without them, a 50-endpoint API becomes unmaintainable.
  • WSGI server: Never use app.run() in production. Use Gunicorn: gunicorn -w 4 "myapp:create_app()".
  • Request validation: Use marshmallow or flask-pydantic for input validation.
  • Configuration: Environment-based config (dev/staging/prod) via environment variables.
What interviewers are really testing: Whether you know the application factory pattern and production deployment.Red flag answer: Showing only the “Hello World” example. In an interview, always discuss the factory pattern.Follow-up:
  • “What is the application factory pattern and why is it important for testing?”
  • “How do you deploy a Flask app in production? What WSGI server do you use?”
  • “How would you structure a Flask app with 50+ endpoints?”
Django ORM maps Python classes to database tables and translates Python operations into SQL. It is both powerful and a common source of performance problems.The model-to-SQL mapping:
class User(models.Model):
    name = models.CharField(max_length=100)
    email = models.EmailField(unique=True)
    created_at = models.DateTimeField(auto_now_add=True)

# QuerySet API (lazy -- SQL not executed until evaluated)
active_users = User.objects.filter(is_active=True).order_by('-created_at')[:10]
# Generates: SELECT * FROM user WHERE is_active = true ORDER BY created_at DESC LIMIT 10
The N+1 query problem (most common ORM performance issue):
# BAD: N+1 queries (1 query for posts + N queries for authors)
for post in Post.objects.all():
    print(post.author.name)  # Each access triggers a separate SQL query

# GOOD: 2 queries total using select_related (JOIN)
for post in Post.objects.select_related('author').all():
    print(post.author.name)

# GOOD: 2 queries for many-to-many using prefetch_related
for post in Post.objects.prefetch_related('tags').all():
    print(post.tags.all())
At Disqus, fixing N+1 queries reduced page load time from 3 seconds to 200ms on some endpoints.When the ORM is not enough:
  • Complex aggregations: Use .annotate() and F() expressions
  • Raw SQL: User.objects.raw('SELECT ...') for complex queries the ORM cannot express
  • Database-specific features: Use django.db.connection for raw cursor access
  • High-volume writes: Use bulk_create() and bulk_update() instead of saving one object at a time
What interviewers are really testing: Whether you know about N+1 queries, select_related/prefetch_related, and when to bypass the ORM.Red flag answer: Describing the ORM without mentioning N+1 queries or performance implications.Follow-up:
  • “What is the N+1 query problem? How do you detect and fix it in Django?”
  • “What is the difference between select_related and prefetch_related?”
  • “When would you use raw SQL instead of the ORM?”
The requests library is the standard for synchronous HTTP calls, but production code requires handling timeouts, retries, connection pooling, and async alternatives.Basic usage with production-grade error handling:
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

# Session with connection pooling and retries
session = requests.Session()
retry = Retry(total=3, backoff_factor=0.5, status_forcelist=[500, 502, 503])
session.mount('https://', HTTPAdapter(max_retries=retry))

try:
    response = session.get(
        'https://api.example.com/data',
        timeout=(3.05, 27),  # (connect_timeout, read_timeout) -- ALWAYS set timeouts
        headers={"Authorization": f"Bearer {token}"}
    )
    response.raise_for_status()  # Raises HTTPError for 4xx/5xx
    data = response.json()
except requests.ConnectionError:
    logger.error("Failed to connect to API")
except requests.Timeout:
    logger.error("API request timed out")
except requests.HTTPError as e:
    logger.error(f"API returned error: {e.response.status_code}")
The #1 production mistake: forgetting timeouts. Without timeout, requests.get() will hang indefinitely if the server never responds. This has caused production outages at many companies.Async alternative (httpx or aiohttp):
import httpx

async with httpx.AsyncClient() as client:
    response = await client.get('https://api.example.com/data', timeout=10)
httpx is a modern alternative that supports both sync and async, with HTTP/2 support.What interviewers are really testing: Timeouts, retries, error handling, and connection pooling.Red flag answer: Using requests.get() without timeout or error handling.Follow-up:
  • “What happens if you do not set a timeout on requests.get()?”
  • “How would you implement exponential backoff retry logic?”
  • “When would you use httpx or aiohttp instead of requests?”
REST (Representational State Transfer) is an architectural style for networked applications. But the real interview depth is in understanding REST constraints and knowing what “RESTful” actually means vs how APIs work in practice.The six REST constraints:
  1. Client-Server: Separation of concerns
  2. Stateless: Each request contains all needed information (no server-side session)
  3. Cacheable: Responses must declare if cacheable
  4. Uniform Interface: Consistent URL structure, standard HTTP methods
  5. Layered System: Client cannot tell if connected directly to server
  6. Code on Demand (optional): Server can extend client with executable code
HTTP methods and idempotency (what most people get wrong):
  • GET — Read. Safe (no side effects). Idempotent. Cacheable.
  • POST — Create. NOT idempotent (calling twice creates two resources).
  • PUT — Full replace. Idempotent (calling twice produces same result).
  • PATCH — Partial update. NOT necessarily idempotent.
  • DELETE — Remove. Idempotent (deleting twice, second is a no-op).
Good URL design:
GET    /api/v1/users          -- List users
POST   /api/v1/users          -- Create user
GET    /api/v1/users/123      -- Get specific user
PUT    /api/v1/users/123      -- Replace user
PATCH  /api/v1/users/123      -- Update user fields
DELETE /api/v1/users/123      -- Delete user
GET    /api/v1/users/123/orders -- List user's orders (nested resource)
Status codes that matter:
  • 200 OK, 201 Created, 204 No Content (successful DELETE)
  • 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 409 Conflict
  • 422 Unprocessable Entity (validation error — common with FastAPI)
  • 429 Too Many Requests (rate limiting)
  • 500 Internal Server Error, 503 Service Unavailable
What interviewers are really testing: Idempotency, proper status codes, and URL design — not just CRUD definitions.Red flag answer: “REST uses GET, POST, PUT, DELETE.” Without understanding idempotency or status code semantics.Follow-up:
  • “What does idempotent mean? Which HTTP methods are idempotent?”
  • “Your API returns 200 for everything, even errors. What is wrong with this?”
  • “How would you design a REST API for a file upload that supports resumable uploads?“

10. Data Science and Libraries

NumPy provides the multi-dimensional array (ndarray) that underlies virtually all numerical computing in Python. It is fast because array operations run in compiled C code, not interpreted Python.Why NumPy is 10-100x faster than Python lists:
import numpy as np

# Python list: ~500ms for 10M elements
python_list = list(range(10_000_000))
result = [x * 2 for x in python_list]

# NumPy array: ~15ms for 10M elements (30x faster)
np_array = np.arange(10_000_000)
result = np_array * 2  # Vectorized operation -- no Python loop
The speedup comes from: (1) contiguous memory layout (cache-friendly), (2) C-level loops (no Python interpreter overhead per element), (3) SIMD vectorization, (4) no type checking per element (homogeneous dtype).Essential concepts:
  • Broadcasting: Operations between arrays of different shapes without copying:
a = np.array([[1, 2, 3],
              [4, 5, 6]])     # Shape (2, 3)
b = np.array([10, 20, 30])    # Shape (3,)
result = a + b                 # Broadcasting: b is "stretched" to (2, 3)
# [[11, 22, 33], [14, 25, 36]]
  • Vectorization: Replace Python loops with array operations
  • Views vs copies: Slicing creates a view (shared memory), not a copy. Modifying a slice modifies the original. Use .copy() for independent data.
  • dtype: All elements share one type (float64, int32, etc.). This enables the performance gains.
What interviewers are really testing: Whether you understand why NumPy is fast (memory layout, vectorization) and can explain broadcasting.Red flag answer: “NumPy is fast because it is written in C.” This is one reason but misses memory layout and vectorization.Follow-up:
  • “Explain broadcasting. What happens when you add a (3,4) array and a (4,) array?”
  • “What is the difference between a NumPy view and a copy? When does it matter?”
  • “Why is np.sum(arr) faster than sum(arr) for a NumPy array?”
pandas provides high-level data manipulation built on NumPy. The two core structures are Series (1D) and DataFrame (2D).DataFrame is the workhorse:
import pandas as pd

df = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'],
    'age': [30, 25, 35],
    'city': ['NYC', 'LA', 'NYC']
})

# SQL-like operations
nyc_users = df[df['city'] == 'NYC']                    # WHERE
avg_age_by_city = df.groupby('city')['age'].mean()     # GROUP BY
merged = pd.merge(df, orders, on='name', how='left')   # JOIN
Performance gotchas that matter in production:
  • Never iterate with for row in df.iterrows() — it is 100-1000x slower than vectorized operations. Use .apply() (better) or vectorized NumPy operations (best).
  • chunksize for large files: pd.read_csv('huge.csv', chunksize=10_000) returns an iterator of DataFrames, enabling processing files larger than RAM.
  • category dtype for strings with few unique values: Reduces memory by 90%+ for columns like “status” or “country”.
  • inplace=True does NOT save memory — pandas internally creates a copy anyway. It is deprecated in newer pandas.
When pandas is not enough:
  • polars: 5-10x faster than pandas for most operations. Written in Rust. No GIL limitations. Lazy evaluation. The future of Python dataframes.
  • Dask: Parallel pandas for datasets larger than RAM. Distributed computing.
  • PySpark: For truly massive datasets (TB+) on clusters.
What interviewers are really testing: Whether you know performance best practices and alternatives to pandas.Red flag answer: Using iterrows() for DataFrame operations, or not knowing about polars.Follow-up:
  • “Why is df.iterrows() slow? What should you use instead?”
  • “How would you process a 50GB CSV file with pandas?”
  • “When would you choose polars over pandas?”
Missing data handling is where data engineering meets data science. The strategy depends entirely on context — there is no universal correct approach.Detection:
df.isnull().sum()          # Count NaN per column
df.isnull().sum() / len(df) * 100  # Percentage missing per column
Strategies and when to use each:
  1. Drop rows (df.dropna()): When missing data is <5% and random (MCAR). Risky if missing data is systematic.
  2. Fill with constant (df.fillna(0) or df.fillna('Unknown')): When a default value makes domain sense.
  3. Forward/backward fill (df.fillna(method='ffill')): For time series data where the last known value is a reasonable proxy.
  4. Fill with mean/median (df.fillna(df.mean())): When the distribution is roughly normal (mean) or skewed (median). Warning: reduces variance and can bias models.
  5. Interpolation (df.interpolate()): For continuous time series.
  6. Model-based imputation (scikit-learn’s SimpleImputer, KNNImputer): When relationships between features can predict missing values.
The critical gotcha — missing data types:
  • NaN (NumPy float('nan')): The default for missing numeric data. NaN != NaN is True (IEEE 754).
  • None: Python’s null. Used in object-dtype columns.
  • pd.NA (pandas 1.0+): The new nullable type. Works with integer columns without converting to float.
  • Older pandas: Missing integers become floats because NaN is a float. pd.Int64Dtype() fixes this.
What interviewers are really testing: Whether you choose strategies based on domain context, not just what is easy.Red flag answer: “I use fillna(0) for everything.” This corrupts data when 0 has semantic meaning.Follow-up:
  • “Your temperature column has 20% missing values. fillna(0) would be wrong. What do you do?”
  • “What is the difference between NaN, None, and pd.NA in pandas?”
  • “How would you detect if missing data is random (MCAR) vs systematic (MNAR)?”
matplotlib is Python’s foundational plotting library. It is powerful but verbose. The real interview depth is knowing its place in the visualization ecosystem and when to use alternatives.The two interfaces:
import matplotlib.pyplot as plt

# 1. pyplot (MATLAB-like, quick and dirty)
plt.plot([1, 2, 3], [1, 4, 9])
plt.title("Quick Plot")
plt.show()

# 2. Object-oriented (recommended for production/reusable code)
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
axes[0].plot(x, y1, label='Revenue')
axes[0].set_title('Revenue Over Time')
axes[0].legend()
axes[1].bar(categories, values)
fig.tight_layout()
fig.savefig('report.png', dpi=150, bbox_inches='tight')
When to use alternatives:
  • seaborn: Statistical visualizations (distributions, regressions, heatmaps). Built on matplotlib, much less code for complex plots.
  • plotly: Interactive web-based charts. Zoom, hover, click. Great for dashboards and Jupyter notebooks.
  • Altair: Declarative visualization based on Vega-Lite. Concise syntax, excellent for exploratory analysis.
  • bokeh: Interactive web plots with Python backend. Similar to plotly.
What interviewers are really testing: Whether you know the OO interface (not just pyplot) and can choose the right visualization library.Red flag answer: Only knowing the pyplot interface without the object-oriented API.Follow-up:
  • “What is the difference between the pyplot and object-oriented matplotlib interfaces?”
  • “When would you use plotly vs matplotlib? What are the trade-offs?”
  • “How do you create publication-quality figures with matplotlib?”
scikit-learn is Python’s standard machine learning library. Its consistent API (fit/predict/transform) and comprehensive algorithm collection make it the go-to for classical ML.The universal API pattern:
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
model = RandomForestClassifier(n_estimators=100)
model.fit(X_train, y_train)           # Train
predictions = model.predict(X_test)   # Predict
print(classification_report(y_test, predictions))
Every algorithm follows this pattern. Switch from RandomForestClassifier to LogisticRegression by changing one line.The Pipeline pattern (production-critical):
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

pipeline = Pipeline([
    ('scaler', StandardScaler()),      # Preprocessing
    ('model', RandomForestClassifier()) # Model
])
pipeline.fit(X_train, y_train)          # Applies all steps in order
pipeline.predict(X_test)                # Applies all steps in order
Pipelines prevent data leakage (fitting the scaler on test data) — a common and serious mistake in ML workflows.When scikit-learn is NOT enough:
  • Deep learning: Use PyTorch or TensorFlow
  • Gradient boosting at scale: Use XGBoost, LightGBM, or CatBoost (faster than sklearn’s GradientBoosting)
  • Very large datasets (millions of rows): Some sklearn algorithms do not scale. Use partial_fit for incremental learning, or switch to Spark MLlib.
What interviewers are really testing: Whether you know the Pipeline pattern and can discuss trade-offs between algorithms.Red flag answer: Knowing the API without understanding Pipelines or data leakage prevention.Follow-up:
  • “What is data leakage and how do Pipelines prevent it?”
  • “When would you use XGBoost instead of sklearn’s RandomForest?”
  • “How would you handle a dataset with 100M rows that does not fit in memory?“

11. Additional Important Topics

Slicing extracts a portion of a sequence using sequence[start:end:step]. But the real depth is in understanding how slicing interacts with the object model.The basics:
  • start is inclusive, end is exclusive
  • Negative indices count from end: lst[-3:] gets last 3 elements
  • step controls stride: lst[::2] gets every other element
  • Omitted values: lst[:] copies entire list, lst[::-1] reverses
What slicing actually does under the hood: Slicing calls __getitem__ with a slice object: lst[1:4] is equivalent to lst.__getitem__(slice(1, 4)). You can implement slicing in custom classes by accepting slice objects in __getitem__.Critical nuance — shallow copy:
original = [[1, 2], [3, 4]]
sliced = original[:]      # Shallow copy -- new list, same nested objects
sliced[0][0] = 99
print(original[0][0])     # 99 -- original modified!
Slice assignment (powerful but underused):
lst = [1, 2, 3, 4, 5]
lst[1:3] = [20, 30, 40]  # Replace slice with different-length iterable
# lst is now [1, 20, 30, 40, 4, 5]

lst[::2] = [0, 0, 0]     # Replace every other element (must match length)
What interviewers are really testing: Whether you understand shallow copy behavior and slice objects.Red flag answer: Only knowing basic slicing without mentioning shallow copy or __getitem__.Follow-up:
  • “What does lst[:] actually do? Is it a deep copy or shallow copy?”
  • “How would you implement slicing in a custom class?”
  • “What happens when you assign to a slice: lst[1:3] = [10, 20, 30, 40]?”
== calls __eq__ to compare values. is compares memory addresses (object identity). This is covered in the identity operators question above — the key addition here is when each goes wrong.The dangerous cases:
# WRONG: Never use 'is' for value comparison
if x is 1:       # Unreliable! Works for small ints due to interning, fails for large
if x is "hello":  # Unreliable! Works for interned strings only

# CORRECT
if x == 1:        # Always reliable for values
if x is None:     # Always correct for None (singleton)
if x is True:     # Technically correct but usually prefer 'if x:'
Why is with strings seems to work (but should not be relied on): CPython interns string literals and identifier-like strings. So "hello" is "hello" is True in practice, but "hello world" is "hello world" may or may not be True depending on context. Never rely on this behavior.What interviewers are really testing: Whether you use is correctly (only for None and sentinels).Red flag answer: Using is for any value comparison besides None.Follow-up:
  • “Can == return True while is returns False? Give an example.”
  • “Why does a = 256; b = 256; a is b return True but a = 257; b = 257; a is b might return False?”
and, or, and not are logical operators, but what most people miss is that and/or return values, not booleans.Short-circuit evaluation with value return:
# 'and' returns first falsy value, or last value if all truthy
0 and "hello"        # Returns 0 (first falsy)
"hello" and "world"  # Returns "world" (last value, all truthy)

# 'or' returns first truthy value, or last value if all falsy
0 or "hello"         # Returns "hello" (first truthy)
"" or 0 or None      # Returns None (all falsy, returns last)
Real-world patterns using this behavior:
# Default value pattern (before walrus operator existed)
name = user_input or "Anonymous"

# Guard clause pattern
result = data and data.get('key')  # Returns None if data is falsy

# Conditional assignment
debug = os.environ.get('DEBUG') or False
Gotcha: and/or do NOT always return booleans. bool(x and y) returns a boolean, but x and y returns the actual value. This matters when you store the result.What interviewers are really testing: Whether you know about value return behavior, not just True/False logic.Red flag answer: “They return True or False.” They return values.Follow-up:
  • “What does [] or 'default' return? What about [1,2] or 'default'?”
  • “How does short-circuit evaluation work with and?”
enumerate() adds a counter to an iterable, yielding (index, value) tuples. It is essential for Pythonic code — any time you need an index in a loop.Why it exists:
# Non-Pythonic (C-style indexing)
for i in range(len(fruits)):
    print(i, fruits[i])

# Pythonic
for i, fruit in enumerate(fruits):
    print(i, fruit)

# With custom start index
for line_num, line in enumerate(file, start=1):
    print(f"Line {line_num}: {line}")
Under the hood: enumerate is a lazy iterator. It does not create a list of tuples in memory. It yields one (count, value) tuple at a time.Advanced pattern — enumerate with unpacking in comprehensions:
indexed_data = {i: val for i, val in enumerate(data)}
What interviewers are really testing: Whether you use enumerate instead of range(len(...)).Red flag answer: Using for i in range(len(list)): when enumerate would be cleaner.Follow-up:
  • “How does enumerate interact with generators? Does it consume the entire generator?”
  • “What does the start parameter do, and when would you use it?”
zip() pairs up elements from multiple iterables, creating tuples of corresponding elements. It is fundamental for parallel iteration.Core behavior:
names = ['Alice', 'Bob', 'Charlie']
ages = [30, 25, 35]
cities = ['NYC', 'LA', 'SF']

for name, age, city in zip(names, ages, cities):
    print(f"{name}, {age}, {city}")

# Creating dicts from parallel lists
user_dict = dict(zip(names, ages))  # Pythonic dict construction
Handling unequal lengths:
# zip() silently truncates to shortest -- DATA LOSS risk
list(zip([1, 2, 3], ['a', 'b']))  # [(1, 'a'), (2, 'b')] -- 3 is lost!

# itertools.zip_longest fills missing values
from itertools import zip_longest
list(zip_longest([1, 2, 3], ['a', 'b'], fillvalue='?'))
# [(1, 'a'), (2, 'b'), (3, '?')]

# Python 3.10+ strict mode raises on length mismatch
list(zip([1, 2, 3], ['a', 'b'], strict=True))  # ValueError!
The unzip pattern:
pairs = [('Alice', 30), ('Bob', 25), ('Charlie', 35)]
names, ages = zip(*pairs)  # Unpack with * -- transpose operation
What interviewers are really testing: Whether you know about the truncation behavior and strict=True.Red flag answer: Not knowing that zip silently drops elements from longer iterables.Follow-up:
  • “You zip two lists of different lengths. What happens to the extra elements?”
  • “How do you unzip a list of tuples? Explain the zip(*) pattern.”
  • “When would you use zip_longest vs zip(..., strict=True)?“

12. Core Programming Concepts

The key distinction: append treats its argument as a single element, extend treats its argument as an iterable of elements.
a = [1, 2, 3]
a.append([4, 5])     # a = [1, 2, 3, [4, 5]]  -- list added as one element
a = [1, 2, 3]
a.extend([4, 5])     # a = [1, 2, 3, 4, 5]     -- elements added individually
Performance:
  • append(x): O(1) amortized. Single element.
  • extend(iterable): O(k) where k is length of iterable. But faster than calling append k times because it avoids k separate method lookups and resizes once.
  • += [items] is equivalent to extend, not append.
The gotcha with extend and strings:
lst = [1, 2]
lst.extend("abc")    # lst = [1, 2, 'a', 'b', 'c'] -- strings are iterable!
lst.append("abc")    # lst = [1, 2, 'a', 'b', 'c', 'abc']
What interviewers are really testing: Whether you understand the iterable vs element distinction.Red flag answer: “append adds one thing, extend adds many” without the string gotcha.Follow-up:
  • “What happens when you extend a list with a string?”
  • “Is lst += [1, 2] the same as lst.extend([1, 2]) or lst.append([1, 2])?”
Default arguments are evaluated once at function definition time, not at each call. This leads to Python’s most infamous bug.The mutable default argument trap:
# BUG: Default list is shared across ALL calls
def add_item(item, items=[]):
    items.append(item)
    return items

add_item(1)  # [1]
add_item(2)  # [1, 2] -- surprise! The default list persists.
add_item(3)  # [1, 2, 3]
The fix — use None sentinel:
def add_item(item, items=None):
    if items is None:
        items = []
    items.append(item)
    return items
Why this happens: The default value [] is evaluated once when def is executed (at import time). The list object is stored in add_item.__defaults__. Every call that uses the default shares the same list object.When mutable defaults are intentional (rare but valid): Caching/memoization patterns use mutable defaults deliberately:
def expensive_function(n, _cache={}):
    if n not in _cache:
        _cache[n] = compute(n)
    return _cache[n]
This is a hack — prefer @functools.lru_cache instead.What interviewers are really testing: Whether you know about the mutable default bug and the None sentinel pattern.Red flag answer: Not knowing about this trap, or not using None as sentinel.Follow-up:
  • “Why are default arguments evaluated at definition time, not call time?”
  • “How can you inspect a function’s default values?”
  • “When would a mutable default argument be intentionally useful?”
Three ways to remove elements, each with different behavior and use cases:
remove(value)pop(index)del lst[index]
Removes byValueIndexIndex
Returns removed itemNoYesNo
On missingValueErrorIndexErrorIndexError
ComplexityO(n) — linear searchO(n) from front, O(1) from endO(n) from front, O(1) from end
When to use each:
  • remove(x): When you know the value but not the index. Only removes first occurrence.
  • pop(i): When you need the removed value (e.g., stack operations: stack.pop())
  • del lst[i]: When you just want deletion by index without needing the value
  • del lst[1:3]: Slice deletion — removes a range
Gotcha with remove in a loop:
# BUG: Skips elements because list shrinks during iteration
lst = [1, 2, 2, 3]
for item in lst:
    if item == 2:
        lst.remove(item)
# lst = [1, 2, 3] -- one 2 remains!

# FIX: Use list comprehension
lst = [item for item in lst if item != 2]
What interviewers are really testing: Whether you know the loop-removal bug and complexity differences.Red flag answer: Not mentioning the danger of remove() during iteration.Follow-up:
  • “Why does removing items from a list during iteration skip elements?”
  • “What is the time complexity of remove() vs pop()?”
Python has three string formatting approaches. f-strings are the standard for new code (Python 3.6+).f-strings (formatted string literals) — the modern way:
name = "Alice"
age = 30
# Basic
f"{name} is {age} years old"

# Expressions
f"{name.upper()} is {age * 2} in dog years"

# Format specifiers
pi = 3.14159
f"{pi:.2f}"           # "3.14" -- 2 decimal places
f"{1000000:,}"        # "1,000,000" -- thousands separator
f"{0.75:.1%}"         # "75.0%" -- percentage
f"{'hello':>20}"      # "               hello" -- right-aligned, padded

# Debugging (Python 3.8+)
x = 42
f"{x = }"             # "x = 42" -- prints variable name and value!
The f"{x = }" trick (Python 3.8+) is a game-changer for debugging. It automatically shows the variable name alongside its value.When to use each approach:
  • f-strings: Default choice. Fast (compiled at parse time), readable, flexible.
  • str.format(): When the format string is determined at runtime (e.g., loaded from a config file). f-strings cannot be reused as templates.
  • % formatting: Legacy code only. Still common in logging module for performance reasons (string not formatted if log level is filtered out).
  • string.Template: For user-provided format strings (safe — no arbitrary code execution).
Security note: f-strings can execute arbitrary code: f"{__import__('os').system('rm -rf /')}". Never build f-strings from user input. Use string.Template for user-provided format strings.What interviewers are really testing: Whether you use f-strings, know the = debugging feature, and understand security implications.Red flag answer: Using % formatting or str.format() when f-strings would be simpler.Follow-up:
  • “What is the f'{x = }' syntax in Python 3.8+? When would you use it?”
  • “When would str.format() be better than an f-string?”
  • “Why is f-string formatting a security risk with user input?”
any() and all() are built-in functions for checking conditions across iterables. They short-circuit for efficiency.Behavior:
any([False, False, True])   # True -- at least one True
all([True, True, True])     # True -- all True
any([False, False, False])  # False
all([True, True, False])    # False

# Edge cases
any([])  # False (no element is True -- vacuous truth edge)
all([])  # True  (no element is False -- vacuous truth)
Real-world patterns:
# Check if any user is admin
if any(user.is_admin for user in users):
    grant_admin_access()

# Validate all required fields are present
required = ['name', 'email', 'password']
if all(field in request.data for field in required):
    process_registration()

# Check if any file is too large (with early termination)
if any(f.size > MAX_SIZE for f in uploaded_files):
    raise ValidationError("File too large")
The short-circuit advantage: With generator expressions, any() and all() stop as soon as the result is determined. any stops at first True, all stops at first False. This means you can use them with expensive checks and they will not evaluate everything unnecessarily.De Morgan’s equivalence:
  • not any(x) is equivalent to all(not x) — “none are true” == “all are false”
  • not all(x) is equivalent to any(not x) — “not all are true” == “some are false”
What interviewers are really testing: Whether you use these with generator expressions for elegant, efficient condition checking.Red flag answer: Not knowing the empty iterable behavior or not using generator expressions with them.Follow-up:
  • “Why does all([]) return True? Is this intuitive?”
  • “How does short-circuit evaluation work with any(expensive_check(x) for x in items)?”
  • “Rewrite this nested if-chain using any() or all().”

13. Python Internals and Memory Management

Python’s memory management is a multi-layered system. This was covered in depth in Section 8 (garbage collector), but the internals go deeper.CPython’s memory allocator hierarchy:
  1. OS allocator (malloc/free): Large allocations (>512 bytes)
  2. Python object allocator (pymalloc): Small object allocations. Uses memory pools of fixed-size blocks (8, 16, 24, … 512 bytes). This avoids calling malloc for every small object, which would be slow.
  3. Object-specific allocators: int, float, list, dict each have optimized allocation strategies. For example, small ints (-5 to 256) are pre-allocated and reused.
Memory pools and arenas: pymalloc manages memory in 256KB arenas, divided into 4KB pools, divided into fixed-size blocks. Objects of similar sizes share a pool. When all blocks in a pool are freed, the pool is returned to the arena for reuse.Practical memory management:
  • Use sys.getsizeof(obj) for shallow size of one object
  • Use pympler.asizeof for deep/recursive size (includes referenced objects)
  • tracemalloc module traces memory allocations to their source line
  • memory_profiler shows line-by-line memory usage
  • gc.get_objects() lists all tracked objects (useful for leak debugging)
What interviewers are really testing: Whether you know about pymalloc, memory profiling tools, and can debug memory issues.Red flag answer: Only knowing about reference counting without understanding the allocator layers.Follow-up:
  • “How would you find a memory leak in a long-running Python service?”
  • “What tools do you use to profile memory usage?”
  • “Why does CPython have its own memory allocator instead of just using malloc?”
Python uses neither pass-by-value nor pass-by-reference. It uses pass-by-object-reference (also called “pass-by-assignment”). Understanding this eliminates an entire class of confusion.The model: Variables are names that point to objects. Assignment binds a name to an object. Function arguments are new names bound to the same objects.
def modify(lst, num, text):
    lst.append(4)      # Mutates the object -- caller sees this
    num += 1            # Rebinds local name to new int object -- caller does NOT see this
    text += " world"    # Rebinds local name to new string -- caller does NOT see this

my_list = [1, 2, 3]
my_num = 5
my_text = "hello"

modify(my_list, my_num, my_text)
print(my_list)   # [1, 2, 3, 4] -- modified (mutable, mutated in place)
print(my_num)    # 5             -- unchanged (immutable, rebinding created new object)
print(my_text)   # "hello"       -- unchanged (immutable, rebinding created new object)
The mental model: Think of variables as sticky notes on objects, not as boxes containing values. = moves the sticky note. Mutation changes the object the sticky note is on. Rebinding moves the sticky note to a different object.The augmented assignment gotcha:
def tricky(lst):
    lst += [4, 5]  # For lists, += calls __iadd__ which MUTATES in place
    # This DOES affect the caller because list.__iadd__ modifies self

def also_tricky(tup):
    tup += (4, 5)  # For tuples, += creates a new tuple (immutable)
    # This does NOT affect the caller
+= behaves differently for mutable vs immutable types. For lists, it mutates. For tuples and strings, it creates new objects.What interviewers are really testing: Whether you can predict what happens when you pass mutable vs immutable objects to functions.Red flag answer: Saying Python is “pass by reference” or “pass by value.”Follow-up:
  • “Is Python pass-by-value or pass-by-reference? Neither — explain what it actually is.”
  • “Why does lst += [4] inside a function affect the caller but num += 1 does not?”
  • “How would you write a function that cannot accidentally modify its arguments?”
Object interning is CPython’s optimization where frequently-used immutable objects are cached and reused instead of creating new objects.What gets interned:
  • Integers -5 to 256: Pre-allocated at interpreter startup. a = 100; b = 100; a is b is always True.
  • Short strings that look like identifiers (alphanumeric + underscore): Automatically interned. "hello" is "hello" is True.
  • Strings used as variable names, function names, etc.: Interned by the compiler.
  • True, False, None: Singletons (always interned).
What does NOT get interned (surprising cases):
a = 257; b = 257
a is b   # Often False in REPL, may be True in .py file (compiler optimization)

a = "hello world"  # Contains space -- NOT interned
b = "hello world"
a is b  # May be True or False -- implementation-dependent

a = "hello"; b = "hello"
a is b  # True -- looks like an identifier, so it is interned
Force interning with sys.intern():
import sys
a = sys.intern("some long string that repeats a lot")
b = sys.intern("some long string that repeats a lot")
a is b  # True -- forced interning
Use sys.intern() when you have many duplicate strings in memory (e.g., column names in a data processing pipeline with millions of rows).Why this matters: Interning saves memory and makes is comparisons O(1). But never rely on interning for correctness — always use == for value comparison. Interning is a CPython optimization detail that may change.What interviewers are really testing: Whether you know the interning boundaries and understand it is implementation-specific.Red flag answer: “Integers are interned” without knowing the -5 to 256 range or the REPL vs script difference.Follow-up:
  • “Why does 257 is 257 give different results in the REPL vs a .py file?”
  • “When would you use sys.intern() in production code?”
  • “Why is interning an implementation detail you should never rely on?”
Weak references let you reference an object without preventing its garbage collection. The reference becomes None when the object is collected.The problem weak references solve:
# Without weak references -- cache prevents garbage collection
cache = {}
def get_data(key):
    obj = expensive_computation(key)
    cache[key] = obj   # Strong reference -- obj can NEVER be garbage collected
    return obj
# Memory grows forever if keys are unique
With weakref.WeakValueDictionary:
import weakref

cache = weakref.WeakValueDictionary()
def get_data(key):
    if key in cache:
        return cache[key]     # May be garbage collected -- check first
    obj = expensive_computation(key)
    cache[key] = obj          # Weak reference -- GC can collect obj
    return obj
When no strong references to obj remain, the GC collects it and the cache entry automatically disappears.Use cases:
  • Caches: Allow cached objects to be collected when memory is tight
  • Observer pattern: Observers hold weak references to subjects to avoid preventing collection
  • Circular reference avoidance: Break cycles by making one reference weak
  • Object tracking/debugging: Monitor objects without affecting their lifecycle
Limitations: Not all objects support weak references. int, str, tuple, and other built-in immutable types cannot have weak references. Custom classes support them by default (unless __slots__ is used without __weakref__).What interviewers are really testing: Whether you understand the GC implications and can identify real use cases.Red flag answer: Knowing the syntax without understanding why — the garbage collection interaction.Follow-up:
  • “How does a WeakValueDictionary differ from a regular dict in terms of garbage collection?”
  • “What types of objects cannot have weak references? Why?”
  • “How would you implement an LRU cache that respects memory pressure using weak references?”
Python is a language specification. CPython, PyPy, Jython, etc. are implementations of that specification.CPython (the default):
  • Written in C. Reference implementation — defines what “Python” means.
  • Compiles to bytecode, interpreted by CPython VM.
  • Has the GIL. Uses reference counting + cyclic GC.
  • Best ecosystem compatibility (C extensions, pip packages).
  • Performance: baseline. ~10-100x slower than C for CPU-bound work.
PyPy:
  • Written in Python (RPython). Uses JIT (Just-In-Time) compilation.
  • Can be 4-10x faster than CPython for long-running programs (the JIT “warms up”).
  • Has a GIL (same as CPython).
  • Compatibility: Most pure-Python code works. C extensions may not (no ctypes API compatibility guarantee). NumPy works via cpyext but is slower than on CPython.
  • Best for: Long-running servers, computation-heavy pure Python code.
GraalPython:
  • Runs on GraalVM (polyglot runtime by Oracle).
  • No GIL. Can interop with Java, JavaScript, Ruby, R.
  • Relatively new, limited ecosystem support.
Cython (not a Python implementation, but relevant):
  • Compiles Python-like code to C extensions. Gives C-level performance for annotated code.
  • Used by many high-performance libraries (NumPy, pandas, scikit-learn).
Practical decision matrix:
  • Need C extensions (NumPy, pandas, TensorFlow)? CPython
  • Need faster pure-Python execution? PyPy
  • Need Java interop? GraalPython or Jython
  • Need to write C-speed extensions? Cython
What interviewers are really testing: Whether you know alternatives exist and can make informed choices.Red flag answer: Not knowing that CPython is just one implementation, or thinking Python == CPython.Follow-up:
  • “When would you deploy PyPy instead of CPython? What would you lose?”
  • “What is JIT compilation and why does it make PyPy faster?”
  • “How does Cython differ from CPython?”
The dis module disassembles Python functions into CPython bytecode instructions. This is invaluable for understanding performance and debugging mysterious behavior.Example:
import dis

def example(a, b):
    return a + b

dis.dis(example)
#   0 LOAD_FAST    0 (a)
#   2 LOAD_FAST    1 (b)
#   4 BINARY_ADD
#   6 RETURN_VALUE
Why this matters for performance:
# List comprehension vs loop -- bytecode explains the speed difference
# Comprehension uses LIST_APPEND (fast, no attribute lookup)
# Loop calls list.append (method lookup each iteration)

dis.dis(compile("[x**2 for x in range(10)]", "", "eval"))
dis.dis(compile("""
result = []
for x in range(10):
    result.append(x**2)
""", "", "exec"))
Common bytecodes to know:
  • LOAD_FAST / STORE_FAST: Local variable access (fastest)
  • LOAD_GLOBAL / STORE_GLOBAL: Global variable access (slower — dict lookup)
  • LOAD_ATTR: Attribute access (dict lookup on object)
  • CALL_FUNCTION: Function call overhead
  • BINARY_ADD, BINARY_MULTIPLY: Arithmetic operations
  • FOR_ITER: Loop iteration
Practical use: If two apparently equivalent code paths have different performance, dis.dis() shows you exactly what the interpreter does differently.What interviewers are really testing: Whether you can use low-level tools to understand and debug Python performance.Red flag answer: Never having used dis or not understanding why local variables are faster than global ones.Follow-up:
  • “Why is accessing a local variable faster than a global variable in CPython?”
  • “How would you use dis to explain why list comprehensions are faster than equivalent for loops?”
  • “What bytecode instruction does a for loop compile to?“

14. Descriptors and Magic Methods

The descriptor protocol is one of Python’s most powerful and least understood features. Properties, methods, classmethod, staticmethod, and super() all use it.The protocol — three optional methods:
  • __get__(self, obj, objtype=None) — Called when attribute is accessed
  • __set__(self, obj, value) — Called when attribute is assigned
  • __delete__(self, obj) — Called when attribute is deleted
Data descriptors vs non-data descriptors:
  • Data descriptor: Defines __set__ and/or __delete__. Takes priority over instance __dict__.
  • Non-data descriptor: Defines only __get__. Instance __dict__ takes priority.
  • @property creates a data descriptor. Functions are non-data descriptors (which is why you can override methods on instances).
How @property works internally (simplified):
class Property:
    def __init__(self, fget=None, fset=None, fdel=None):
        self.fget = fget
        self.fset = fset
        self.fdel = fdel

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self      # Accessed on class, return descriptor itself
        return self.fget(obj)

    def __set__(self, obj, value):
        if self.fset is None:
            raise AttributeError("can't set attribute")
        self.fset(obj, value)

    def setter(self, fset):
        return type(self)(self.fget, fset, self.fdel)
How functions become methods (descriptor magic):
class MyClass:
    def method(self):
        pass

obj = MyClass()
# MyClass.__dict__['method'] is a function (non-data descriptor)
# obj.method triggers function.__get__(obj, MyClass) which returns a bound method
This is why obj.method and MyClass.method behave differently.Attribute lookup order (the full picture):
  1. Data descriptors on the class (e.g., properties)
  2. Instance __dict__
  3. Non-data descriptors on the class (e.g., methods)
  4. __getattr__ (fallback)
What interviewers are really testing: Whether you understand the lookup order and can explain how @property and methods work.Red flag answer: Knowing @property syntax without understanding descriptors.Follow-up:
  • “What is the difference between a data descriptor and a non-data descriptor?”
  • “How does a function become a bound method when accessed on an instance?”
  • “Build a descriptor that validates attribute types (e.g., must be int).”
These two methods control attribute access at different levels:__getattr__(self, name) — The fallback. Only called when normal attribute lookup fails (not found in __dict__, class hierarchy, or data descriptors):
class FlexibleObject:
    def __init__(self):
        self.data = {"name": "Alice"}

    def __getattr__(self, name):
        # Only called for attributes not found normally
        if name in self.data:
            return self.data[name]
        raise AttributeError(f"No attribute '{name}'")

obj = FlexibleObject()
obj.data     # Normal lookup -- __getattr__ NOT called
obj.name     # Not in __dict__ -- __getattr__ IS called, returns "Alice"
__getattribute__(self, name) — Total control. Called for EVERY attribute access:
class LoggedAccess:
    def __getattribute__(self, name):
        print(f"Accessing: {name}")
        return object.__getattribute__(self, name)  # MUST call super to avoid infinite recursion
The infinite recursion trap: Inside __getattribute__, any self.x access calls __getattribute__ again. You MUST use object.__getattribute__(self, name) to break the recursion.Use cases:
  • __getattr__: Proxy objects, lazy loading, default values, API wrappers
  • __getattribute__: Logging, access control, transparent proxies (rare — use with extreme caution)
What interviewers are really testing: Whether you know when each is called and can avoid the recursion trap.Red flag answer: Confusing the two, or not knowing about the infinite recursion danger in __getattribute__.Follow-up:
  • “How do you avoid infinite recursion in __getattribute__?”
  • “Build a proxy object using __getattr__ that delegates all attribute access to a wrapped object.”
  • “When would you use __getattribute__ instead of __getattr__?”
Using type() as a metaclass demonstrates that classes are objects created by calling a metaclass.
# These are equivalent:

# Using class keyword
class Person:
    def __init__(self, name):
        self.name = name
    def greet(self):
        return f"Hello, I'm {self.name}"

# Using type() directly
def __init__(self, name):
    self.name = name
def greet(self):
    return f"Hello, I'm {self.name}"

Person = type('Person', (object,), {
    '__init__': __init__,
    'greet': greet,
})
What this reveals about Python’s object model:
  • The class statement is syntactic sugar for a type() call
  • type is both a function (returning an object’s type) and a metaclass (creating new classes)
  • Classes are first-class objects that can be created, modified, passed as arguments, and returned from functions
Dynamic class creation use cases:
  • ORMs that create model classes from database schemas
  • Plugin systems that register classes at runtime
  • Testing frameworks that generate test classes dynamically
  • Serialization libraries that create classes from schemas (Pydantic, marshmallow)
What interviewers are really testing: Whether you understand that classes are objects and metaclasses are their factories.Red flag answer: Being surprised that type() can create classes.Follow-up:
  • “If type creates classes, what creates type? (Answer: type creates itself)”
  • “How would you dynamically create a class with attributes determined at runtime?”
  • “How does this relate to metaclasses?”
Monkey patching is modifying code at runtime by replacing methods, functions, or attributes of existing classes/modules. It is powerful but dangerous.The pattern:
# Original
import requests

# Monkey patch -- replace requests.get with a mock
original_get = requests.get  # Save original

def mock_get(url, **kwargs):
    return MockResponse(status_code=200, json_data={"mock": True})

requests.get = mock_get      # Patch

# All code that calls requests.get() now gets our mock
data = requests.get("https://api.example.com/data")
# data is MockResponse, no network call made

requests.get = original_get  # Restore (important!)
Legitimate use cases:
  1. Testing: unittest.mock.patch is structured monkey patching with automatic cleanup
  2. Hotfixing third-party bugs: Patch a broken method in a library you cannot update immediately
  3. Feature flags: Replace behavior at runtime based on configuration
  4. gevent/eventlet: Monkey-patches the standard library to make blocking I/O non-blocking
Why it is dangerous:
  • Code behavior depends on import order and patch order
  • Breaks IDE navigation and static analysis (the patched function is not the one in the source)
  • Library updates may change the patched interface, breaking your patch silently
  • Makes debugging extremely difficult (“why is requests.get returning a mock in production?”)
The proper way — unittest.mock.patch:
from unittest.mock import patch

with patch('module.requests.get') as mock_get:
    mock_get.return_value.json.return_value = {"data": "test"}
    result = function_under_test()
# Original automatically restored when exiting 'with' block
What interviewers are really testing: Whether you know when monkey patching is appropriate (testing, hotfixes) vs when it is reckless (production behavior changes).Red flag answer: Enthusiastically monkey patching in production without discussing the risks.Follow-up:
  • “How does unittest.mock.patch improve on raw monkey patching?”
  • “What is gevent’s monkey patching and why is it necessary?”
  • “You need to fix a bug in a third-party library. Monkey patch or fork the repo?“

15. Advanced Generators and Iterators

yield from delegates to a sub-generator, but it is far more than syntactic sugar for a for loop. It creates a bidirectional channel between the caller and the sub-generator.Basic delegation:
def sub_generator():
    yield 1
    yield 2
    return "sub done"  # Return value is captured by yield from

def main_generator():
    result = yield from sub_generator()  # Delegates completely
    print(f"Sub returned: {result}")     # "sub done"
    yield 3
What yield from handles that a manual loop does not:
  1. send() forwarding: Values sent to the outer generator are forwarded to the sub-generator
  2. throw() forwarding: Exceptions thrown into the outer generator are thrown into the sub-generator
  3. close() forwarding: Closing the outer generator closes the sub-generator
  4. Return value capture: The sub-generator’s return value becomes the value of the yield from expression
Without yield from, you would need ~40 lines of boilerplate to handle all edge cases correctly. PEP 380 formalized this.Real-world use case — flattening nested structures:
def flatten(items):
    for item in items:
        if isinstance(item, (list, tuple)):
            yield from flatten(item)  # Recursive delegation
        else:
            yield item

list(flatten([1, [2, [3, 4], 5], 6]))  # [1, 2, 3, 4, 5, 6]
Connection to asyncio: yield from was the original mechanism for coroutine composition before async/await was introduced in Python 3.5. await is essentially yield from for coroutines.What interviewers are really testing: Whether you understand the bidirectional channel, not just the iteration delegation.Red flag answer: “It is the same as for x in gen: yield x.” It is not — send(), throw(), and close() forwarding are the whole point.Follow-up:
  • “What is the difference between yield from gen and for x in gen: yield x?”
  • “How does yield from relate to await in asyncio?”
  • “How do you capture the return value of a sub-generator?”
Generators are unique in Python because they maintain a suspended execution frame — the function’s local variables, instruction pointer, and stack are frozen on yield and thawed on next().The execution lifecycle:
def counter():
    print("Starting")      # Runs on first next()
    i = 0
    while True:
        i += 1
        print(f"Before yield {i}")
        yield i             # SUSPEND: frame frozen, value sent to caller
        print(f"After yield {i}")  # RESUME: runs on next next()

gen = counter()             # NO code executes -- generator object created
print(next(gen))            # "Starting", "Before yield 1", prints 1
print(next(gen))            # "After yield 1", "Before yield 2", prints 2
Frame persistence internals (CPython):
  • Generator objects hold a reference to their frame object (gen.gi_frame)
  • The frame stores: local variables, instruction pointer (f_lasti), operand stack, block stack
  • gen.gi_frame is None after the generator is exhausted
  • You can inspect the current state: gen.gi_frame.f_locals, gen.gi_frame.f_lineno
Generator states:
import inspect
gen = counter()
inspect.getgeneratorstate(gen)  # 'GEN_CREATED' -- never started
next(gen)
inspect.getgeneratorstate(gen)  # 'GEN_SUSPENDED' -- paused at yield
# After exhaustion: 'GEN_CLOSED'
Why this matters for memory: The frame is a lightweight object (~200-400 bytes). Compare this to a thread (~8MB default stack on Linux) or a process (~30MB+). This is why you can have millions of generators but only thousands of threads.What interviewers are really testing: Whether you understand that generators are coroutines (suspended functions) not just iterators.Red flag answer: Describing generators as “functions that return multiple values” without understanding frame suspension.Follow-up:
  • “How can you inspect the current state of a suspended generator?”
  • “Why can you have millions of generators but only thousands of threads?”
  • “What happens to the generator frame when it is exhausted?”
The itertools module provides composable, lazy building blocks for creating efficient data processing pipelines. It is one of the most underused modules in Python.Essential itertools functions:
import itertools

# chain -- concatenate iterables without creating a combined list
all_data = itertools.chain(file1_lines, file2_lines, file3_lines)

# islice -- slice an iterator (cannot use [] on iterators)
first_100 = itertools.islice(huge_generator, 100)

# groupby -- group consecutive elements (MUST be sorted first)
sorted_data = sorted(records, key=lambda r: r.category)
for category, group in itertools.groupby(sorted_data, key=lambda r: r.category):
    print(category, list(group))

# product -- cartesian product (replaces nested for loops)
for x, y, z in itertools.product(range(10), range(10), range(10)):
    pass  # 1000 combinations, O(1) memory

# combinations, permutations
pairs = itertools.combinations(users, 2)  # All unique pairs

# accumulate -- running totals
running_sum = itertools.accumulate([1, 2, 3, 4])  # 1, 3, 6, 10

# starmap -- map with argument unpacking
results = itertools.starmap(pow, [(2, 3), (3, 2), (10, 3)])  # 8, 9, 1000
Building Unix-pipe-style processing:
def read_logs(path):
    with open(path) as f:
        yield from f

def filter_errors(lines):
    return (line for line in lines if "ERROR" in line)

def extract_timestamps(lines):
    return (line.split()[0] for line in lines)

# Process 100GB log file with constant memory
pipeline = extract_timestamps(filter_errors(read_logs("huge.log")))
for timestamp in itertools.islice(pipeline, 1000):  # First 1000 errors
    print(timestamp)
What interviewers are really testing: Whether you can build memory-efficient pipelines using lazy iterators.Red flag answer: Loading everything into lists instead of using lazy iterators for large data.Follow-up:
  • “How would you process a 100GB log file using itertools and generators?”
  • “What is the gotcha with itertools.groupby? (Must be sorted first)”
  • “When would itertools.product be better than nested for loops?“

16. Advanced Concurrency and Parallelism

This question trips up many candidates. The GIL prevents CPU-bound parallelism, but threading is essential for I/O-bound concurrency.The key insight: The GIL is released during I/O operations (network calls, disk reads, database queries, time.sleep()). While one thread waits for I/O, other threads can execute Python code.Real-world performance impact:
import concurrent.futures
import requests

urls = ["https://example.com"] * 100

# Sequential: ~100 seconds (1s per request)
for url in urls:
    requests.get(url)

# Threaded: ~10 seconds (10 threads, each handles 10 requests)
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
    list(executor.map(requests.get, urls))
The threaded version is 10x faster despite the GIL because each thread spends 99% of its time waiting for the network, not executing Python bytecode.Threading is also needed for:
  • GUI applications (separate thread for UI responsiveness)
  • Background tasks (periodic cleanup, monitoring)
  • Real-time systems (audio playback while processing)
  • Libraries that release the GIL in C extensions (NumPy, PIL, cryptography)
Thread safety concerns (the real production issue): Even though the GIL prevents data races at the bytecode instruction level, operations like counter += 1 are NOT atomic (they compile to multiple bytecodes: LOAD, ADD, STORE). You still need threading.Lock for shared mutable state.What interviewers are really testing: Whether you understand that the GIL is released during I/O and that threading is valuable for I/O-bound work.Red flag answer: “Threading is useless in Python because of the GIL.”Follow-up:
  • “Is counter += 1 thread-safe in Python? Why or why not?”
  • “When is the GIL released? Give me specific examples.”
  • “How many threads should you use for a web scraper hitting 1000 URLs?”
The decision depends on two factors: (1) Is the work I/O-bound or CPU-bound? (2) How many concurrent tasks do you need?Decision matrix:
CPU-bound work?
├── Yes → ProcessPoolExecutor (num_workers = num_cores)
└── No (I/O-bound) →
    How many concurrent tasks?
    ├── < 100 → ThreadPoolExecutor (simpler, sufficient)
    └── > 100-1000+ → asyncio (lower overhead per task)
ThreadPoolExecutor — the workhorse for I/O-bound work:
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=20) as pool:
    results = list(pool.map(fetch_url, urls))
~100KB per thread. Good for 10-100 concurrent tasks. Shared memory (easy data sharing). Simple mental model.ProcessPoolExecutor — for CPU-bound work:
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers=os.cpu_count()) as pool:
    results = list(pool.map(process_image, images))
~30-50MB per process. Arguments must be picklable (serialized across process boundaries). Each process has its own GIL.asyncio — for high-concurrency I/O:
async def main():
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
~1KB per coroutine. Can handle 10,000+ concurrent connections on a single thread. Requires async-compatible libraries.Hybrid approach (common in production):
async def main():
    # I/O-bound: use asyncio
    data = await fetch_all_data()

    # CPU-bound: offload to ProcessPool from within asyncio
    loop = asyncio.get_event_loop()
    with ProcessPoolExecutor() as pool:
        results = await loop.run_in_executor(pool, cpu_heavy_work, data)
What interviewers are really testing: Whether you can match the concurrency model to the task type with clear reasoning.Red flag answer: Always choosing the same model regardless of task type.Follow-up:
  • “Your task involves both API calls (I/O) and data processing (CPU). What architecture do you use?”
  • “What happens if you use ThreadPoolExecutor for CPU-bound work? Show me the math.”
  • “Why is asyncio more efficient than threading for 10,000 concurrent connections?”
Understanding resource costs is critical for production capacity planning.Per-unit cost comparison:
ApproachMemory per unitCreation timeMax practical count
Thread~100KB (stack)~1ms~1,000-5,000
Process~30-50MB (full Python interpreter)~50-100ms~CPU count (4-64)
Coroutine~1KB~0.01ms~100,000+
Thread-specific costs:
  • Default stack size: 8MB on Linux (most unused). Set threading.stack_size(65536) for low-memory threads.
  • Context switching: ~1-10 microseconds (OS-managed, preemptive)
  • Shared memory: Pro for data sharing, con for needing locks
  • GIL contention: Under heavy CPU load, threads fight for the GIL, adding ~5% overhead per thread
Process-specific costs:
  • Each process gets a full Python interpreter copy
  • IPC (Inter-Process Communication) requires serialization: pickle for multiprocessing, which is slow for large objects
  • Copy-on-write after fork(): Memory is shared until modified. Linux is efficient here; macOS less so.
  • initializer functions in ProcessPoolExecutor run once per worker — use them to load large models/data per worker instead of passing through pickle
asyncio-specific costs:
  • Coroutines are lightweight Python objects (~1KB)
  • Event loop overhead: ~10 microseconds per coroutine switch
  • No parallelism: All coroutines share one CPU core
  • Blocking call in any coroutine blocks ALL coroutines (the “foot gun”)
What interviewers are really testing: Whether you can do back-of-envelope capacity calculations.Red flag answer: Not knowing the approximate memory cost of threads vs processes vs coroutines.Follow-up:
  • “You need to handle 50,000 concurrent WebSocket connections. Calculate the memory needed for threads vs asyncio.”
  • “Your process pool workers each load a 500MB ML model. How much total memory?”
  • “How does copy-on-write affect memory usage after forking?”
The fundamental difference is preemptive (threading) vs cooperative (async) multitasking.Threading (preemptive):
  • The OS decides when to switch threads (can happen at any bytecode instruction)
  • You need locks to protect shared data (race conditions are possible)
  • Blocking I/O is fine — the OS blocks the thread and switches to another
  • Simple mental model: write sequential code, wrap in threads
async/await (cooperative):
  • Coroutines explicitly yield control at await points (you control when switching happens)
  • No locks needed for coroutine-local data (no preemptive switching between awaits)
  • Blocking I/O is FORBIDDEN — one blocking call freezes the entire event loop
  • Requires async-compatible libraries for everything (aiohttp, asyncpg, aiofiles)
Side-by-side comparison:
# Threading -- simple, works with any library
def fetch_all_threaded(urls):
    with ThreadPoolExecutor(10) as pool:
        return list(pool.map(requests.get, urls))

# Async -- more efficient, requires async libraries
async def fetch_all_async(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [session.get(url) for url in urls]
        return await asyncio.gather(*tasks)
The “function color” problem: Once you have one async def, the entire call stack above it must also be async. You cannot await from a regular function. This creates a viral effect that can require rewriting large portions of code.When threading wins: Quick scripts, code that uses sync-only libraries, teams without async experience.When asyncio wins: High-concurrency servers (>1000 connections), WebSocket servers, API gateways.What interviewers are really testing: Whether you understand the trade-offs, especially the cooperative nature of async.Red flag answer: “Async is always faster than threading.” It is more efficient for many connections but adds complexity.Follow-up:
  • “What happens if you accidentally call time.sleep(10) inside an async handler?”
  • “How do you run blocking code from within an async function?”
  • “What is the ‘function color’ problem and how does it affect your architecture decisions?”
The golden rule: measure first, optimize second. Performance intuition is wrong more often than it is right.The profiling toolkit:
# 1. cProfile -- function-level CPU profiling
python -m cProfile -s cumulative your_script.py

# 2. py-spy -- sampling profiler (attach to running process, no code changes)
py-spy top --pid 12345
py-spy record -o profile.svg -- python your_script.py  # Flame graph

# 3. line_profiler -- line-by-line profiling
@profile  # Add decorator, run with kernprof
def slow_function():
    pass

# 4. timeit -- micro-benchmarks
python -m timeit -s "lst = list(range(1000))" "sum(lst)"

# 5. memory_profiler -- line-by-line memory usage
@profile
def memory_hungry():
    pass

# 6. tracemalloc -- memory allocation tracking
tracemalloc.start()
# ... code ...
snapshot = tracemalloc.take_snapshot()
The profiling workflow:
  1. Reproduce the slow behavior with realistic data
  2. Profile to find the hot path (usually 5% of code is 95% of runtime)
  3. Optimize the hot path using appropriate techniques:
    • Algorithm improvement (O(n^2) to O(n log n))
    • Data structure change (list to set for lookups)
    • Caching (functools.lru_cache)
    • Vectorization (NumPy instead of Python loops)
    • Concurrency (appropriate model for task type)
    • C extension (Cython, pybind11) as last resort
  4. Measure again to verify improvement
  5. Add regression benchmarks to prevent future regressions
py-spy is the most practical tool for production profiling — it attaches to a running process with zero overhead, requires no code changes, and produces flame graphs. It has saved hours of debugging in my experience.What interviewers are really testing: Whether you follow a systematic profiling workflow or guess at optimizations.Red flag answer: Optimizing without profiling first, or only knowing print(time.time()).Follow-up:
  • “How would you profile a production service without restarting it?”
  • “Your function is slow. cProfile shows it spends 80% of time in one line. What is your next step?”
  • “What is a flame graph and how do you read one?”
Matching the right concurrency model to the workload is a critical production skill:Multithreading use cases (I/O-bound, moderate concurrency):
  • Web scraping: Fetch 100 URLs concurrently. Each thread blocks on network I/O while others proceed. ThreadPoolExecutor with 10-50 workers.
  • File processing: Read/write multiple files simultaneously. Disk I/O is the bottleneck, not CPU.
  • Database operations: Execute multiple queries in parallel. Database drivers release the GIL during network waits.
  • GUI applications: Keep the UI responsive while doing background work.
Multiprocessing use cases (CPU-bound):
  • Image/video processing: Resize 10,000 images using all 8 CPU cores. Each worker process handles a batch.
  • Data transformation: ETL pipelines with heavy computation (parsing, transforming, aggregating).
  • Scientific computing: Monte Carlo simulations, numerical integration, matrix operations (when NumPy is insufficient).
  • ML model training: Parallel hyperparameter search with ProcessPoolExecutor.
asyncio use cases (I/O-bound, high concurrency):
  • Web servers: FastAPI/Starlette handling 10,000+ concurrent HTTP connections.
  • WebSocket servers: Real-time chat, live dashboards, gaming servers.
  • API gateways: Fan-out requests to multiple backend services concurrently.
  • Message queue consumers: Process Kafka/RabbitMQ messages with async handlers.
  • Web crawlers: Crawl thousands of pages with aiohttp, rate-limited.
Hybrid architectures (production reality): Most production systems combine approaches:
# FastAPI (asyncio) + ProcessPool for CPU work
@app.post("/process-image")
async def process_image(file: UploadFile):
    data = await file.read()                          # asyncio for I/O
    loop = asyncio.get_event_loop()
    result = await loop.run_in_executor(              # Process pool for CPU
        process_pool, heavy_processing, data
    )
    await save_to_s3(result)                          # asyncio for I/O
    return {"status": "done"}
What interviewers are really testing: Whether you can design a real system using the right concurrency model for each component.Red flag answer: Only knowing one approach, or using threading for CPU-bound work.Follow-up:
  • “Design the concurrency architecture for an image-processing API that receives uploads (I/O), processes them (CPU), and stores results (I/O).”
  • “How would you handle a mixed workload that is both I/O-bound and CPU-bound?”
  • “What is the maximum number of concurrent WebSocket connections a single asyncio server can handle? What is the bottleneck?“

17. Design Patterns and Architecture

SOLID principles apply to Python, but Python’s dynamic nature means the implementation looks different than in Java/C#.S — Single Responsibility: Each class/module has one reason to change. In Python, this often means splitting a “god class” into smaller classes or using standalone functions:
# BAD: UserService does everything
class UserService:
    def create_user(self): ...
    def send_email(self): ...
    def generate_report(self): ...

# GOOD: Separate concerns
class UserRepository:   # Data access only
class EmailService:     # Communication only
class ReportGenerator:  # Reporting only
O — Open/Closed: Open for extension, closed for modification. In Python, this is achieved through composition, protocols, and plugin systems — not necessarily inheritance:
# Instead of modifying existing code, extend through Protocol
from typing import Protocol

class Serializer(Protocol):
    def serialize(self, data: dict) -> str: ...

class JSONSerializer:
    def serialize(self, data: dict) -> str: return json.dumps(data)

class XMLSerializer:
    def serialize(self, data: dict) -> str: ...  # New format without changing existing code
L — Liskov Substitution: Subclasses must be usable wherever their parent is expected. The classic violation in Python: overriding a method to raise NotImplementedError.I — Interface Segregation: Use typing.Protocol (Python 3.8+) for narrow, focused interfaces. Do not force classes to implement methods they do not need.D — Dependency Inversion: Depend on abstractions (Protocols, ABCs), not concrete implementations. In Python, this is often as simple as passing a function or Protocol-typed parameter.What interviewers are really testing: Whether you can translate SOLID from Java-speak to Pythonic patterns.Red flag answer: Describing SOLID exactly as a Java textbook would without Python-specific adaptations.Follow-up:
  • “How does typing.Protocol implement the Interface Segregation principle in Python?”
  • “Give me a real example of Liskov Substitution violation in Python.”
  • “How do you implement Dependency Inversion in Python without a DI framework?”
Composition means building objects by combining simpler objects, rather than inheriting behavior from parent classes. The Python community strongly favors composition.Why inheritance causes problems:
# Inheritance: tight coupling, diamond problem, rigid hierarchy
class Animal: ...
class FlyingAnimal(Animal): ...
class SwimmingAnimal(Animal): ...
class Duck(FlyingAnimal, SwimmingAnimal): ...  # Diamond problem!
Composition: flexible, testable, explicit:
# Behaviors as composable objects
@dataclass
class Animal:
    name: str
    movement: MovementStrategy     # Inject behavior
    sound: SoundStrategy

# Behaviors can be mixed freely
duck = Animal("Duck", CanFlyAndSwim(), Quack())
dog = Animal("Dog", CanRun(), Bark())
penguin = Animal("Penguin", CanSwim(), Squawk())
Real-world refactoring — Django views:
# Before: Deep inheritance chain (common Django anti-pattern)
class MyView(LoginRequiredMixin, PermissionRequiredMixin,
             FormView, DetailView):  # What methods are available? Who knows.
    pass

# After: Composition with explicit delegation
class MyView:
    def __init__(self):
        self.auth = AuthService()
        self.form_handler = FormHandler()
        self.data_loader = DataLoader()

    def handle_request(self, request):
        self.auth.require_login(request)
        self.auth.check_permission(request, "edit")
        data = self.data_loader.get_detail(request.id)
        return self.form_handler.process(request, data)
The rule of thumb: Use inheritance for “is-a” relationships with shared interfaces (ABCs). Use composition for “has-a” relationships and behavior assembly. When in doubt, compose.What interviewers are really testing: Whether you can articulate why composition is better with a concrete example.Red flag answer: “Composition is better” without explaining the specific problems inheritance causes.Follow-up:
  • “When IS inheritance the right choice in Python?”
  • “How do mixins fit into the composition vs inheritance debate?”
  • “Refactor this 5-level inheritance chain into composition.”
The Factory pattern creates objects without hardcoding their exact class. Python’s dynamic nature makes factories simpler than in Java, but the pattern is still valuable for complex creation logic.Simple factory (dict-based):
class JSONParser:
    def parse(self, data): return json.loads(data)

class XMLParser:
    def parse(self, data): return ET.fromstring(data)

class YAMLParser:
    def parse(self, data): return yaml.safe_load(data)

def create_parser(format_type: str):
    parsers = {
        'json': JSONParser,
        'xml': XMLParser,
        'yaml': YAMLParser,
    }
    parser_cls = parsers.get(format_type)
    if parser_cls is None:
        raise ValueError(f"Unknown format: {format_type}")
    return parser_cls()
Registry-based factory (scalable, plugin-friendly):
parser_registry = {}

def register_parser(format_type):
    def decorator(cls):
        parser_registry[format_type] = cls
        return cls
    return decorator

@register_parser('json')
class JSONParser:
    def parse(self, data): return json.loads(data)

@register_parser('csv')
class CSVParser:
    def parse(self, data): ...

def create_parser(format_type: str):
    return parser_registry[format_type]()
This pattern allows adding new parsers without modifying the factory — just decorate a new class.When to use factories vs direct instantiation:
  • Direct instantiation: When the class to create is known at the call site
  • Factory function: When the class depends on runtime configuration (format type, environment)
  • Abstract factory: When you need families of related objects (rare in Python)
What interviewers are really testing: Whether you can implement a factory using Python’s first-class classes and decorators, not Java-style abstract factory hierarchies.Red flag answer: Implementing a Java-style AbstractFactory with 5 classes when a dict would do.Follow-up:
  • “How does the registry decorator pattern work?”
  • “When is a factory function overkill and you should just use @classmethod?”
  • “How would you make this factory extensible for plugins?”
Singletons ensure only one instance of a class exists. Python offers several approaches, each with different trade-offs:1. Module-level variable (the Pythonic way):
# config.py
_settings = None

def get_settings():
    global _settings
    if _settings is None:
        _settings = Settings()
    return _settings
Simplest. Modules are cached in sys.modules and only executed once. Most Pythonic approach. Used by the standard library.2. __new__ override:
class Singleton:
    _instance = None

    def __new__(cls, *args, **kwargs):
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance
Straightforward. But __init__ runs on every call, which can re-initialize state.3. Decorator:
def singleton(cls):
    instances = {}
    @functools.wraps(cls)
    def get_instance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]
    return get_instance

@singleton
class Database:
    def __init__(self, url):
        self.connection = connect(url)
Clean syntax. But changes the class to a function, which can break isinstance checks.4. Metaclass:
class SingletonMeta(type):
    _instances = {}
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class Database(metaclass=SingletonMeta):
    pass
Most robust. Preserves class semantics, isinstance works. But complex.The honest answer: In Python, just use a module-level instance or a function that lazily creates one. The other approaches are mostly academic exercises. “The Borg pattern” (sharing state across instances) is another alternative.What interviewers are really testing: Whether you know multiple approaches AND can articulate that the simple one is usually best.Red flag answer: Reaching for the metaclass approach first.Follow-up:
  • “Why is a module-level singleton considered the most Pythonic approach?”
  • “Are any of these approaches thread-safe? How would you make them safe?”
  • “What is the Borg pattern and when is it better than a traditional Singleton?”
In languages like Java, many design patterns exist to work around the lack of first-class functions. In Python, functions are objects: they can be assigned to variables, passed as arguments, returned from functions, and stored in data structures.Strategy pattern (no classes needed):
# Java-style: Strategy interface, ConcreteStrategyA, ConcreteStrategyB, Context class...
# Python-style:
def process_data(data, strategy):
    return strategy(data)

process_data(data, sorted)
process_data(data, reversed)
process_data(data, lambda x: sorted(x, key=len))
Command pattern (closures replace command objects):
def make_command(action, *args):
    def command():
        return action(*args)
    return command

undo_stack = []
undo_stack.append(make_command(delete_file, "temp.txt"))
undo_stack.append(make_command(rename_file, "a.txt", "b.txt"))
Observer pattern (callables as observers):
class EventBus:
    def __init__(self):
        self.listeners = defaultdict(list)

    def on(self, event, callback):  # callback is any callable
        self.listeners[event].append(callback)

    def emit(self, event, data):
        for callback in self.listeners[event]:
            callback(data)
Template Method pattern (just pass the varying step as a function):
def etl_pipeline(extract_fn, transform_fn, load_fn):
    data = extract_fn()
    transformed = transform_fn(data)
    load_fn(transformed)
What interviewers are really testing: Whether you recognize that Python’s design patterns are simpler than GoF patterns and can explain why.Red flag answer: Implementing Java-style Strategy with interfaces and concrete classes in Python.Follow-up:
  • “Which GoF patterns are unnecessary in Python? Why?”
  • “When IS a class-based strategy better than a function in Python?”
  • “How does functools.partial relate to the Command pattern?”
This was covered in Section 5, but here is the deeper dive on advanced patterns:Class-based (full control):
class Timer:
    def __init__(self, label):
        self.label = label

    def __enter__(self):
        self.start = time.perf_counter()
        return self  # Available as the 'as' variable

    def __exit__(self, exc_type, exc_val, exc_tb):
        elapsed = time.perf_counter() - self.start
        print(f"{self.label}: {elapsed:.3f}s")
        return False  # Do not suppress exceptions

with Timer("database query") as t:
    result = db.query("SELECT * FROM users")
@contextmanager (concise):
from contextlib import contextmanager

@contextmanager
def temporary_directory():
    path = tempfile.mkdtemp()
    try:
        yield path          # Everything before yield is __enter__
    finally:
        shutil.rmtree(path) # Everything after yield is __exit__
Advanced: contextlib.ExitStack for dynamic resource management:
from contextlib import ExitStack

def process_files(file_paths):
    with ExitStack() as stack:
        files = [stack.enter_context(open(p)) for p in file_paths]
        # All files are open. All will be closed when exiting 'with'.
        for f in files:
            process(f)
Async context managers (async with):
class AsyncDBConnection:
    async def __aenter__(self):
        self.conn = await connect()
        return self.conn
    async def __aexit__(self, *exc):
        await self.conn.close()
What interviewers are really testing: Whether you can write both forms and know about ExitStack for dynamic cases.Red flag answer: Only knowing with open() without being able to create custom context managers.Follow-up:
  • “When would you use ExitStack instead of nested with statements?”
  • “How do async context managers differ from sync ones?”
  • “What happens if code inside a @contextmanager’s try block raises an exception?”
Circular imports are one of the most common Python frustrations. Understanding why they happen reveals the fix.Why circular imports fail: When module A imports module B, and B imports A, Python’s import machinery partially initializes A before B can access its attributes. B gets a half-built module object, causing AttributeError or ImportError.Solution 1 — Restructure (best): Extract shared dependencies into a third module. If A needs something from B and B needs something from A, there is probably a shared concept that deserves its own module:
# Before: a.py imports b.py, b.py imports a.py
# After: both import from common.py
Solution 2 — Lazy imports (pragmatic):
def process():
    from other_module import helper  # Import when needed, not at module level
    return helper()
Solution 3 — TYPE_CHECKING guard (for type hints only):
from __future__ import annotations  # Makes all annotations strings (lazy)
from typing import TYPE_CHECKING

if TYPE_CHECKING:  # Only True when mypy/pyright runs, False at runtime
    from other_module import OtherClass

def process(obj: "OtherClass") -> None:  # String annotation, no runtime import
    ...
This is the standard pattern for type hints that would cause circular imports.Solution 4 — Dependency injection:
class Service:
    def __init__(self, repository):  # Accept dependency as parameter
        self.repository = repository
# Caller assembles dependencies, breaking the circular chain
What interviewers are really testing: Whether you can diagnose circular imports and know the TYPE_CHECKING pattern.Red flag answer: “I just move the import into the function” without understanding why the cycle exists or knowing about TYPE_CHECKING.Follow-up:
  • “How does from __future__ import annotations help with circular imports?”
  • “What is the TYPE_CHECKING constant and how does it work?”
  • “When is restructuring better than lazy imports?“

18. Testing and Debugging

The rule is simple: mock at the boundary. Mock I/O and external services. Do NOT mock the code under test or its internal collaborators.What to mock (external boundaries):
  • Network calls (HTTP APIs, database queries, message queues)
  • File system operations
  • System clock (datetime.now(), time.time())
  • Environment variables
  • Third-party services (payment gateways, email providers)
What NOT to mock (internal logic):
  • Your own business logic classes
  • Data transformations
  • Validation functions
  • Anything that does not cross a process/network boundary
The patch target rule (most common mistake):
# mymodule.py
from requests import get

def fetch_data():
    return get("https://api.example.com")

# test_mymodule.py
# WRONG: patch where it is defined
@patch('requests.get')

# CORRECT: patch where it is USED
@patch('mymodule.get')
def test_fetch_data(mock_get):
    mock_get.return_value.json.return_value = {"data": "test"}
    result = fetch_data()
    assert result.json() == {"data": "test"}
    mock_get.assert_called_once_with("https://api.example.com")
patch replaces the name at the import location, not the definition location. Getting this wrong is the #1 source of “my mock is not working” bugs.What interviewers are really testing: Whether you know the boundary rule and the patch target rule.Red flag answer: Mocking internal methods or not knowing where to target patch.Follow-up:
  • “Why do you patch('mymodule.get') instead of patch('requests.get')?”
  • “When would you use a real database in tests instead of mocking?”
  • “How do you test code that depends on datetime.now()?”
Stubs provide canned responses. Mocks also verify interactions. In Python’s unittest.mock, Mock() does both, but the distinction matters for test design.Stubbing — provides data, does not verify behavior:
# Stub: "When called, return this data"
mock_db = Mock()
mock_db.get_user.return_value = User(name="Alice", age=30)

# The test verifies the output, not how the mock was called
result = process_user(mock_db, user_id=1)
assert result.name == "Alice"
Mocking — verifies interactions:
# Mock: "Verify it was called correctly"
mock_email = Mock()
register_user(user_data, email_service=mock_email)

mock_email.send.assert_called_once_with(
    to="user@example.com",
    subject="Welcome!",
    body=ANY  # We do not care about exact body
)
When to use each:
  • Stub when you care about the output of the function under test
  • Mock when you care about side effects (was the email sent? was the cache invalidated?)
Over-asserting on mock calls (anti-pattern):
# BAD: Testing implementation details
mock_db.query.assert_called_with("SELECT * FROM users WHERE id = %s", (1,))
# This breaks if you refactor the SQL, even if the behavior is correct

# GOOD: Test behavior, not implementation
result = get_user(1)
assert result.name == "Alice"
What interviewers are really testing: Whether you distinguish testing behavior from testing implementation.Red flag answer: Asserting on every mock call, testing implementation details instead of behavior.Follow-up:
  • “When does verifying mock calls become an anti-pattern?”
  • “How does unittest.mock.ANY help you write less brittle mock assertions?”
  • “What is spec=True on a Mock and why should you use it?”
Over-mocking is one of the most common testing anti-patterns. It produces tests that pass but give false confidence — they test your mocks, not your code.Signs of over-mocking:
  1. Tests break on refactoring but code still works — you changed internal structure, not behavior, but 50 tests fail because mocks expected specific method calls
  2. Mocks return mocks return mocksmock.get.return_value.json.return_value.data is a code smell
  3. Tests are harder to read than the code — 40 lines of mock setup for 5 lines of actual test
  4. Bugs slip through — the real database driver returns Decimal, your mock returns float, causing a production bug that tests never caught
Real-world horror story: A team had 2,000 unit tests, all passing, all heavily mocked. They upgraded their database driver. Zero tests failed. But production crashed on the first query because the new driver returned different types. Their mocks were testing a fantasy world.The testing pyramid approach:
  • Unit tests (70%): Test pure logic, no I/O. No mocks needed because there is nothing to mock.
  • Integration tests (20%): Test with real database, real file system. Use pytest-docker or testcontainers for reproducible environments.
  • End-to-end tests (10%): Test the full system. Slow but catches integration issues.
Better alternatives to mocking:
  • Fakes: In-memory implementations of interfaces (e.g., dict-based repository instead of database repository)
  • Test databases: SQLite for unit tests, Docker-based PostgreSQL for integration tests
  • Dependency injection: Design code so dependencies are passed in, not imported
What interviewers are really testing: Whether you have experienced the pain of over-mocking and can design a balanced test strategy.Red flag answer: “I mock everything for fast tests.” Speed without accuracy is worthless.Follow-up:
  • “How do you decide between a mock and a fake?”
  • “How do you test database interactions without mocking?”
  • “Your team has 5,000 unit tests that all pass, but production bugs keep slipping through. What is likely wrong?”
Effective debugging follows a systematic approach. The best debuggers use the right tool for the situation.The debugging decision tree:
  1. Obvious error with traceback: Read the traceback bottom-up. The last frame shows where the error occurred. The cause is usually 1-3 frames above.
  2. Logic error (wrong result, no crash): Use breakpoint() (Python 3.7+) or import pdb; pdb.set_trace() to inspect state at the problem point.
  3. Intermittent/timing issue: Add structured logging with context. Use logging.debug() with correlation IDs.
  4. Performance issue: Profile with py-spy or cProfile before optimizing.
  5. Memory issue: Use tracemalloc to track allocations.
Modern Python debugging (breakpoint()):
def process_order(order):
    total = calculate_total(order.items)
    breakpoint()  # Drops into pdb (or any configured debugger)
    # Set PYTHONBREAKPOINT=0 to disable all breakpoints in production
    # Set PYTHONBREAKPOINT=ipdb.set_trace for enhanced debugger
    apply_discount(total, order.coupon)
Essential pdb commands:
  • n (next) — execute current line, step over function calls
  • s (step) — step INTO function calls
  • c (continue) — run until next breakpoint
  • l (list) — show source code around current line
  • p expr — print expression value
  • pp expr — pretty-print (great for dicts/lists)
  • w (where) — show full stack trace
  • u/d (up/down) — navigate stack frames
  • !statement — execute arbitrary Python code
VS Code / PyCharm debugging is generally more productive than pdb for complex issues — conditional breakpoints, watch expressions, and variable inspection are far easier in a GUI.Post-mortem debugging (after a crash):
# Automatically drop into debugger on unhandled exception
python -m pdb script.py

# Or in code:
import pdb
try:
    failing_function()
except Exception:
    pdb.post_mortem()  # Debug the exception state
What interviewers are really testing: Whether you use breakpoint() and have a systematic debugging approach.Red flag answer: “I add print statements everywhere.” This works but is the least efficient approach.Follow-up:
  • “What is breakpoint() and how does PYTHONBREAKPOINT environment variable work?”
  • “How do you debug an issue that only happens in production?”
  • “Walk me through how you would debug a function that returns an incorrect result.”
Logging is the primary debugging tool for production systems. But most developers configure it incorrectly.Production-grade logging setup:
import logging
import json

# Structured JSON logging (parseable by ELK, Datadog, CloudWatch)
class JSONFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
            "module": record.module,
            "function": record.funcName,
            "line": record.lineno,
        })

handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())

logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
Log levels and when to use each:
  • DEBUG: Detailed diagnostic info. Disabled in production.
  • INFO: Normal operation milestones. “User created”, “Payment processed”.
  • WARNING: Something unexpected but not an error. “Retry attempt 2/3”, “Disk 80% full”.
  • ERROR: Something failed but the service continues. “Failed to send email, will retry”.
  • CRITICAL: Service is about to crash. “Database connection pool exhausted”.
The %s vs f-string performance trick:
# GOOD: String not formatted if level is filtered out
logger.debug("Processing user %s with %d items", user_id, len(items))

# BAD: f-string is always formatted, even if debug logging is disabled
logger.debug(f"Processing user {user_id} with {len(items)} items")
With %s formatting, the string is only built if the log level is active. With f-strings, it is always built. This matters in hot loops.Correlation IDs for distributed tracing:
import uuid

def handle_request(request):
    request_id = str(uuid.uuid4())
    logger = logging.getLogger(__name__)
    logger = logging.LoggerAdapter(logger, {"request_id": request_id})
    logger.info("Request received", extra={"path": request.path})
    # All logs from this request share the same request_id
What interviewers are really testing: Structured logging, log levels, and the %s performance pattern.Red flag answer: Using print() in production code, or only using logging.basicConfig().Follow-up:
  • “Why use %s formatting instead of f-strings in logging calls?”
  • “How do you correlate logs across microservices?”
  • “How would you set up logging that outputs JSON for a production ELK stack?”
Performance debugging follows a strict methodology: measure, identify, optimize, measure again.Step 1 — Identify the type of bottleneck:
  • CPU-bound: High CPU usage, slow computation. Profile with cProfile or py-spy.
  • I/O-bound: Low CPU, waiting for network/disk. Profile with asyncio debug mode or logging timestamps.
  • Memory-bound: Growing RSS, GC pauses. Profile with tracemalloc.
Step 2 — Profile with the right tool:
# cProfile -- function-level CPU profiling (built-in)
python -m cProfile -s cumulative script.py
# Output: function call counts and cumulative time

# py-spy -- sampling profiler (zero-overhead, attach to running process)
py-spy record -o profile.svg -- python script.py
# Output: flame graph SVG (visual, intuitive)

# line_profiler -- line-by-line timing (surgical)
@profile  # Add this decorator
def slow_function():
    data = load_data()        # 0.1s
    processed = transform(data)  # 45.2s  <-- found it!
    save(processed)           # 0.3s

# timeit -- micro-benchmarks (compare approaches)
python -m timeit -s "lst = list(range(10000))" "sum(lst)"
python -m timeit -s "import numpy as np; arr = np.arange(10000)" "np.sum(arr)"
Step 3 — Common optimizations (in order of impact):
  1. Algorithm change: O(n^2) to O(n log n) is the biggest win possible
  2. Data structure change: list in check to set in check (O(n) to O(1))
  3. Caching: @functools.lru_cache for pure functions called repeatedly
  4. Vectorization: Replace Python loops with NumPy/pandas operations
  5. Concurrency: Right model for the task (threading/async for I/O, multiprocessing for CPU)
  6. C extension: Cython, pybind11, or Rust via PyO3 (last resort)
What interviewers are really testing: Whether you profile first and optimize based on data.Red flag answer: “I would use Cython” without having profiled first.Follow-up:
  • “How do you read a flame graph? What pattern indicates a performance issue?”
  • “Your function is slow but cProfile shows no single slow function. What could cause this?”
  • “How does py-spy attach to a running process without restarting it?”
Memory leaks in Python usually fall into three categories: unbounded caches, circular references, and C extension leaks.Step 1 — Detect the leak:
import tracemalloc
tracemalloc.start(25)  # Track 25 frames deep

# Take baseline
snapshot1 = tracemalloc.take_snapshot()

# ... run code that you suspect leaks ...

# Compare
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
for stat in top_stats[:10]:
    print(stat)
# Shows: which lines allocated the most NEW memory between snapshots
Step 2 — Identify what objects are accumulating:
import objgraph
objgraph.show_most_common_types(limit=20)
# Output: dict: 50000, list: 30000, str: 25000...
# If a type count keeps growing, you have a leak

objgraph.show_growth(limit=10)  # Show types that increased since last call
Step 3 — Find what is holding references:
# Visualize reference chain for suspicious objects
objgraph.show_backrefs(
    objgraph.by_type('MyLeakyClass')[:3],
    max_depth=5,
    filename='refs.png'
)
Common leak patterns and fixes:
  1. Unbounded cache: @lru_cache(maxsize=None) or plain dict used as cache. Fix: set maxsize or use cachetools.TTLCache.
  2. Closures capturing large objects: Lambda in a loop captures a DataFrame. Fix: extract the value, do not close over the large object.
  3. Global collections: Module-level lists/dicts that append but never pop. Fix: use bounded collections or weak references.
  4. C extension leak: tracemalloc cannot see it. Monitor RSS via /proc/self/status and compare with tracemalloc totals. The difference is C-level allocations.
  5. Circular references with __del__: Before Python 3.4, these were uncollectable. After 3.4, the GC handles them, but __del__ can still cause issues by resurrecting objects.
What interviewers are really testing: Whether you have a systematic approach to memory debugging using specific tools.Red flag answer: “Call gc.collect() in a loop” (treats the symptom, not the cause).Follow-up:
  • “How do you distinguish a Python-level memory leak from a C extension leak?”
  • “Your service RSS grows by 1GB/day. Walk me through your debugging approach.”
  • “Why might gc.collect() not free memory, and why might freed Python memory not reduce RSS?”

Advanced Scenario-Based Questions

Scenario: Your team deployed a data-processing microservice that scores ML models in real time. It spawns 8 worker threads via ThreadPoolExecutor, yet htop shows a single core pegged at 100% while the other 7 sit idle. P99 latency is climbing toward your SLA. Your manager asks why the 8-core box is basically a single-core machine.What weak candidates say:
  • “Just add more threads” or “Increase the thread pool size to 64.”
  • Vaguely mention the GIL without explaining why it applies here or what concrete alternatives exist.
  • Suggest rewriting the whole service in Go or Rust as the only option.
What strong candidates say:
  • “This is the GIL in action. Because the workload is CPU-bound (NumPy scoring that drops back into Python between calls, or pure-Python feature preprocessing), the GIL serializes bytecode execution across all 8 threads. You get concurrency but not parallelism.”
  • “First, I would profile with py-spy or cProfile to confirm the hot path is actually CPU-bound Python, not I/O. If it is, the fix depends on context:”
    • Quick win: Switch from ThreadPoolExecutor to ProcessPoolExecutor. Each worker process gets its own GIL. Serialization cost for IPC is the trade-off — measure it.
    • Medium effort: Offload the hot loop to a C extension, Cython, or ensure the NumPy/SciPy calls release the GIL (most do via Py_BEGIN_ALLOW_THREADS). Verify with nogil=True in Cython.
    • Structural fix: Move to a multi-process architecture with gunicorn --workers 8 or Celery workers, one per core, with a Redis/RabbitMQ broker.
    • Nuclear option: Evaluate the free-threaded CPython 3.13+ build (python --disable-gil), but acknowledge it is experimental and many C extensions are not yet safe.
  • “In a previous role, we had a similar issue with a feature-engineering pipeline. Switching to ProcessPoolExecutor with 4 workers on a 4-core box dropped P99 from 850ms to 210ms. The gotcha was that our model object was 400MB and pickling it for each task killed throughput — we fixed that by loading the model once per worker process using an initializer function.”
Follow-up:
  1. “If your NumPy scoring already releases the GIL, but the feature preprocessing between calls is pure Python, how do you handle the mixed workload?”
  2. “What is the memory overhead of ProcessPoolExecutor with 8 workers each loading a 500MB model? How would you mitigate it?”
  3. “Explain what Py_BEGIN_ALLOW_THREADS actually does at the C level and why it is safe for NumPy but dangerous for code that touches Python objects.”
Scenario: Your ETL pipeline runs as a long-lived Python process consuming Kafka messages. Over 48 hours, RSS grows from 500MB to 12GB until the OOM killer terminates it. Restarting resets the cycle. Your on-call team is tired of 3 AM pages.What weak candidates say:
  • “Use del on variables” or “Call gc.collect() in a loop.”
  • Blame Python for being a memory-hungry language.
  • Suggest just restarting the service on a cron schedule (treating the symptom, not the cause).
What strong candidates say:
  • “I would approach this systematically with three tools in sequence:”
    • Step 1 — tracemalloc: Enable it in production with tracemalloc.start(25) (25 frames deep). Take snapshots every 10 minutes and compare with snapshot.compare_to(old_snapshot, 'lineno'). This shows which lines are allocating the most new memory between snapshots.
    • Step 2 — objgraph: Once I have a suspect module, use objgraph.show_most_common_types() to see which object types are accumulating. Then objgraph.show_backrefs(objgraph.by_type('SuspectClass')[:5]) to visualize what is holding references.
    • Step 3 — gc module: Check gc.garbage for uncollectable objects (those with __del__ methods in reference cycles). Run gc.set_debug(gc.DEBUG_SAVEALL) in a staging environment to catch them.
  • “The most common culprits I have seen in production:”
    • Caches without eviction: An lru_cache with maxsize=None (unbounded), or a plain dict used as a cache that grows forever. Fix: set a maxsize or use cachetools.TTLCache.
    • Closures capturing large objects: A lambda or callback inside a loop that accidentally closes over a large DataFrame. The DataFrame stays alive as long as any callback references it.
    • Circular references with __del__: Pre-Python-3.4, these were uncollectable. Post-3.4, the GC handles them, but custom __del__ methods can still prevent collection if they resurrect objects.
    • C extension leaks: If using libraries like lxml or database drivers, memory may leak in C code that tracemalloc cannot see. Use memory_profiler alongside /proc/self/smaps to compare Python-tracked vs OS-tracked RSS.
  • “At a previous company, we traced a 10GB/day leak to a logging handler that appended formatted log records to an in-memory list (someone left a MemoryHandler with no target). objgraph showed 8 million str objects. Two-line fix, saved us $400/month in oversized EC2 instances.”
Follow-up:
  1. “What is the difference between RSS, VMS, and USS, and which one should you actually monitor for a Python memory leak?”
  2. “Why might gc.collect() not reclaim memory, and why might the process RSS not drop even after Python frees objects?”
  3. “How would you detect a memory leak in a C extension that tracemalloc cannot track?”
Scenario: You have a FastAPI service handling 2,000 concurrent WebSocket connections. Under load testing at 5,000 connections, the entire event loop freezes: health check endpoints stop responding, existing connections time out, and Kubernetes kills the pod. Logs show no exceptions. CPU is at 5%.What weak candidates say:
  • “Add more asyncio tasks” or “Increase the connection limit.”
  • Confuse asyncio with threading and suggest adding locks.
  • Cannot explain what “blocking the event loop” actually means.
What strong candidates say:
  • “Classic event loop starvation. The CPU is at 5% which rules out a CPU-bound bottleneck — something is blocking the single event loop thread, preventing it from servicing other coroutines. The three most common causes:”
    • Synchronous I/O in an async handler: Someone called requests.get(), time.sleep(), or a synchronous database driver inside an async def handler instead of using aiohttp, asyncio.sleep(), or an async driver. Even one blocking call stalls every coroutine.
    • CPU-bound work on the event loop: JSON serialization of a 50MB payload, or a regex over a massive string, running directly in a coroutine without offloading.
    • Deadlock from incorrect await chains: Two coroutines waiting on each other’s asyncio.Event or asyncio.Lock, or calling loop.run_until_complete() from inside an already-running loop (common when mixing sync and async code).
  • “To diagnose, I would:”
    • Enable asyncio debug mode (PYTHONASYNCIODEBUG=1 or loop.set_debug(True)) which logs coroutines that take longer than 100ms without yielding.
    • Use py-spy to get a live stack trace of the stuck event loop thread — it will show exactly which blocking call it is sitting on.
    • Add loop.slow_callback_duration = 0.05 and watch warnings.
  • “The fix depends on the root cause:”
    • Blocking I/O: Replace with async equivalent, or wrap with asyncio.to_thread() (Python 3.9+) / loop.run_in_executor() to push it to a thread pool.
    • CPU-bound work: Offload to ProcessPoolExecutor via loop.run_in_executor().
    • Deadlock: Restructure the await chain. Never call loop.run_until_complete() from within a running loop — use await directly.
  • “I once debugged a production freeze caused by a single dns.resolver.resolve() call (synchronous dnspython) inside an async handler. Under load, DNS lookups took 2-5 seconds each, completely blocking the event loop. Replacing it with aiodns fixed the freeze and dropped P99 latency from timeout to 40ms.”
Follow-up:
  1. “What exactly happens internally when you await asyncio.sleep(0), and why do people sprinkle it in long-running coroutines?”
  2. “Explain the difference between asyncio.to_thread(), loop.run_in_executor(), and just spawning a raw thread. When would you pick each?”
  3. “Can you have a deadlock in asyncio without using any explicit locks? Describe a scenario.”
Scenario: You add a new file utils/cache.py to your Django project. Suddenly, an unrelated view that imports from utils.helpers crashes with AttributeError: module 'utils' has no attribute 'helpers'. The attribute clearly exists — utils/helpers.py is right there, unchanged. Reverting cache.py fixes it. What is happening?What weak candidates say:
  • “There is a typo in the import” or “The file is not saved.”
  • Cannot explain how Python’s import system actually resolves modules.
  • Suggest deleting __pycache__ and restarting (sometimes works, but they cannot explain why).
What strong candidates say:
  • “This is almost certainly a circular import. The import system is one of the trickiest parts of CPython. Here is what is likely happening:”
    • When Python imports utils.cache, that module’s top-level code runs. If cache.py imports something from utils.helpers at the top level, and helpers.py (or something it imports) tries to import from utils.cache, you get a partially-initialized module.
    • Python’s import system uses sys.modules as a cache. When a module is being imported, a partially-initialized module object is placed in sys.modules. If another module tries to import from it before its top-level code finishes, it gets the half-built module — which may be missing attributes that have not been defined yet.
    • The AttributeError says 'utils' has no attribute 'helpers' because the utils package __init__.py has not finished executing — it is mid-import and the helpers submodule has not been bound to the package namespace yet.
  • “To debug: add print(f'Importing {__name__}') at the top of each suspect module and read the order. Or use python -v to see the import sequence. The cycle will be obvious.”
  • “Fixes, in order of preference:”
    • Refactor: Extract the shared dependency into a new module that both cache.py and helpers.py import. Break the cycle architecturally.
    • Lazy import: Move the problematic import inside the function that uses it, so it runs at call time, not import time.
    • importlib.import_module(): Defer the import programmatically — same idea as lazy import but sometimes cleaner for dynamic cases.
    • Restructure __init__.py: If __init__.py does from .helpers import *, that is often the trigger. Keep __init__.py minimal.
  • “One subtle variant: if you name a file utils/cache.py and there is also a third-party package called cache, you can get shadowing where Python imports your local file instead of the library, or vice versa, depending on sys.path order. Always check module.__file__ to confirm you are importing what you think you are.”
Follow-up:
  1. “Walk me through the exact sequence of steps CPython takes when it encounters import utils.helpers — what does it check, and in what order?”
  2. “What is the difference between a regular package (with __init__.py) and a namespace package (PEP 420)? How does it affect import resolution?”
  3. “You mentioned sys.modules. What happens if you manually delete an entry from sys.modules and re-import the module? What are the dangers?”
Scenario: Your team built a lightweight ORM using metaclasses. Developers report that UserModel.fields contains fields from OrderModel — models are somehow sharing state. The bug is intermittent and only appears when both models are imported in the same module. Test suites pass because each test file imports only one model.What weak candidates say:
  • “I would not use metaclasses” (avoids the question entirely).
  • Cannot explain what a metaclass __new__ or __init__ does during class creation.
  • Suggest adding print statements with no systematic approach.
What strong candidates say:
  • “This is a classic mutable-default-on-the-metaclass bug. The most common cause: the metaclass stores field definitions in a shared mutable object (like a class-level list or dict on the metaclass itself) instead of creating a fresh one per class. Here is the typical pattern:”
# BUG: shared mutable state
class ModelMeta(type):
    fields = []  # This list is shared across ALL classes created by this metaclass

    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        for key, value in namespace.items():
            if isinstance(value, Field):
                mcs.fields.append(key)  # Appends to the SHARED list
        cls.fields = mcs.fields  # Every class gets the SAME list
        return cls
  • “The fix is to create a new list per class in __new__:”
# FIXED: per-class field list
class ModelMeta(type):
    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        cls.fields = []
        for key, value in namespace.items():
            if isinstance(value, Field):
                cls.fields.append(key)
        return cls
  • “To debug this in the wild, I would:”
    • Check id(UserModel.fields) == id(OrderModel.fields) — if True, they are literally the same object in memory.
    • Use UserModel.__class__ to confirm the metaclass, then inspect the metaclass’s __new__ and __init__ for shared mutable state.
    • Check if the metaclass inherits from another metaclass that introduces shared state.
    • Look for __init_subclass__ hooks that might mutate a parent class’s attributes.
  • “This same pattern bites people with Django ModelForm.Meta if you accidentally mutate fields at class creation time instead of copying. The broader lesson: any time you are in __new__ or __init__ of a metaclass, every mutable object must be explicitly copied or freshly created per class. Never assign a metaclass-level mutable to a class attribute by reference.”
Follow-up:
  1. “What is the difference between __new__ and __init__ on a metaclass versus on a regular class? When does each run?”
  2. “How does __init_subclass__ (PEP 487) reduce the need for metaclasses? Give an example where you would refactor a metaclass to use it instead.”
  3. “What happens if two metaclasses conflict — for example, your ORM metaclass and an ABC metaclass? How does Python resolve metaclass conflicts?”
Scenario: Your image-processing service uses Pillow and a custom C extension for fast EXIF parsing. In production (Ubuntu, Python 3.11, 16 workers under gunicorn), you get intermittent SIGSEGV crashes — roughly 2 per day. Core dumps show the crash in your C extension. It never reproduces locally on macOS with a single process. Your team is afraid to touch C code.What weak candidates say:
  • “Rewrite the C extension in pure Python.”
  • “Add try/except around the C call” (SIGSEGV is not a Python exception — it kills the process).
  • Have no idea how to read a core dump or use debugging tools for C extensions.
What strong candidates say:
  • “Intermittent segfaults in C extensions that only appear under multi-process/multi-worker conditions usually point to one of three things:”
    • Thread safety: If gunicorn uses gthread workers, multiple threads may call the C extension concurrently. If the C code uses static/global variables without locks, you get data races and memory corruption.
    • Fork safety: If gunicorn uses prefork workers, the C extension is initialized before forking. Some C libraries (e.g., OpenSSL, certain database drivers) are not fork-safe — they hold file descriptors, mutexes, or memory-mapped regions that become invalid after fork(). Fix: use gunicorn’s post_fork hook to reinitialize, or switch to --preload with careful init.
    • Buffer overflow / use-after-free: The EXIF parser reads variable-length metadata. A malformed JPEG with unexpected EXIF data could cause the C code to read past a buffer.
  • “My debugging approach:”
    • Step 1: Analyze the core dump with gdb python core followed by bt (backtrace). With debug symbols (python3-dbg package), this shows the exact C function and line number.
    • Step 2: Run under Valgrind in staging: valgrind --tool=memcheck python your_script.py. This catches invalid reads/writes, use-after-free, and leaks. Expect it to be 20-50x slower.
    • Step 3: If it is a data-dependent crash, collect the JPEG that triggered it from the request logs and write a regression test.
    • Step 4: Use faulthandler module (python -X faulthandler) to get a Python-level traceback even on SIGSEGV — this shows you which Python function was calling into the C code.
    • Step 5: Compile the C extension with AddressSanitizer (-fsanitize=address) and run the test suite. ASAN catches buffer overflows, use-after-free, and stack corruption with minimal overhead compared to Valgrind.
  • “In a real incident, we had a C extension that cached parsed results in a static PyObject*. It was initialized in the first worker before fork, and after fork all child processes shared the pointer. When one child’s GC collected the object, the others segfaulted on access. Fix: move initialization into post_fork and remove the static cache.”
Follow-up:
  1. “What is the difference between faulthandler.enable() and a regular Python traceback? Why can faulthandler print something useful during a segfault when a normal except block cannot?”
  2. “Explain the GIL’s role in C extension safety. If your C extension calls Py_BEGIN_ALLOW_THREADS, what are you now responsible for that the GIL previously protected?”
  3. “Your C extension needs to create Python objects. Walk me through the reference counting rules: when do you call Py_INCREF vs Py_DECREF, and what is a ‘borrowed reference’ vs a ‘new reference’?”
Scenario: Your data science team has a monorepo with 3 services. Developers use macOS (ARM), CI runs Ubuntu x86, and production is Amazon Linux 2. The requirements.txt has unpinned transitive dependencies. Every week someone’s build breaks because numpy 2.0 dropped and broke pandas, or a compiled wheel is not available for one platform. A new hire spent two days just getting the dev environment working.What weak candidates say:
  • “Just pip freeze > requirements.txt” (captures one platform’s resolved versions, breaks on another).
  • “Use Docker” as the only answer (does not address the dependency management problem itself).
  • Cannot distinguish between requirements.txt, setup.py, pyproject.toml, Pipfile, and poetry.lock.
What strong candidates say:
  • “This is a dependency resolution and lockfile problem. The Python packaging ecosystem has historically been terrible at this compared to npm/yarn or Cargo, but modern tools solve it. Here is what I would do:”
    • Step 1 — Single source of truth: Move to pyproject.toml (PEP 621) for declaring dependencies with loose constraints (e.g., numpy>=1.24,<2.0). This replaces setup.py, setup.cfg, and requirements.txt for dependency declaration.
    • Step 2 — Lockfile with platform awareness: Use uv (fastest, Rust-based), pip-tools (pip-compile), or Poetry to generate a lockfile with pinned transitive dependencies. uv lock generates a universal lockfile that works across platforms. pip-compile --generate-hashes pins versions AND validates integrity.
    • Step 3 — Per-platform resolution (if needed): For packages with compiled wheels (NumPy, scipy), you may need separate lock files per platform: requirements-linux-x86.txt, requirements-macos-arm.txt. uv handles this natively with its universal resolver.
    • Step 4 — CI enforcement: CI should install from the lockfile (uv sync --frozen or pip install -r requirements.txt --require-hashes), never resolve fresh. If lockfile is out of date, CI fails.
    • Step 5 — Docker for parity: Use a multi-stage Dockerfile where the build stage resolves/installs dependencies and the runtime stage copies the installed virtualenv. This ensures production matches CI.
  • “The specific tool choice matters:”
    • uv: My current recommendation. 10-100x faster than pip, universal lockfile, drop-in pip replacement. Handles virtualenvs, resolution, and installation.
    • Poetry: Mature, good lockfile, but slower and sometimes fights with pip-installed packages.
    • pip-tools: Minimal, just pip-compile and pip-sync. Works if you want to stay close to pip.
  • “For the monorepo specifically: use workspace support (uv workspaces or Poetry monorepo plugins) so the 3 services share a single lockfile but declare their own dependencies. This prevents version conflicts where service A needs pandas 1.5 and service B needs pandas 2.0.”
  • “At a previous company, switching from bare requirements.txt to uv lock with hashes cut our ‘broken build’ tickets from 5/week to zero and reduced CI install time from 4 minutes to 18 seconds.”
Follow-up:
  1. “What is the difference between a wheel and an sdist? Why does numpy install in 2 seconds on some platforms but takes 5 minutes compiling on others?”
  2. “Explain what --require-hashes protects against. What attack vector does it mitigate that pinned versions alone do not?”
  3. “Your data scientist needs to add a package that conflicts with an existing pinned dependency. Walk me through how you would resolve this without breaking other services.”
Scenario: Your company’s main backend is a 200,000-line Python monolith. Management wants type safety after a production incident caused by passing a str where an int was expected. You enable mypy --strict and get 14,000 errors. Three senior engineers say it is a waste of time. The intern says “just add type: ignore everywhere.” How do you proceed?What weak candidates say:
  • “Just fix all 14,000 errors” (unrealistic timeline, team will revolt).
  • “Add type: ignore to everything” (defeats the purpose).
  • Cannot explain the difference between mypy, pyright, pytype, or when to use each.
  • Think type hints are enforced at runtime by default.
What strong candidates say:
  • “Adopting strict typing on a large untyped codebase is a migration, not a switch flip. I would use an incremental approach:”
    • Phase 1 — Baseline without breaking CI (Week 1): Set up mypy in CI in non-blocking mode. Use mypy.ini with disallow_untyped_defs = false globally. Generate a baseline error count. Configure per-module overrides so new modules are strict from day one.
    • Phase 2 — Gradual strictness with per-module config (Weeks 2-4): Use mypy’s [[tool.mypy.overrides]] in pyproject.toml to enable strict mode per module. Start with the module that caused the production incident. Use monkeytype or pytype to auto-generate type stubs from runtime traces — this handles 60-70% of annotations automatically.
    • Phase 3 — CI enforcement on new code (Month 2): Use mypy --warn-unused-ignores so type: ignore comments do not linger. Block PRs that add new untyped functions to already-typed modules. Use a pre-commit hook with mypy --incremental (fast, only checks changed files).
    • Phase 4 — Ratchet mechanism (Ongoing): Track total mypy error count in CI. New PRs must not increase the count. This naturally drives the number to zero over time without requiring a dedicated sprint. Tools like mypy-baseline or a simple wc -l on mypy output make this easy.
  • “For the pushback from senior engineers, I would address their actual concerns:”
    • “It slows me down”: monkeytype run + monkeytype apply auto-annotates from tests. pyright is faster than mypy for editor integration. Show them that typed code catches bugs before code review.
    • “Python is not supposed to be typed”: Show PEP 484, 526, 544. Typing is opt-in and gradual by design. Protocols (PEP 544) give you structural typing that respects duck typing.
    • “It does not catch real bugs”: Share the production incident that triggered this initiative. Show a demo where mypy catches Optional misuse (None passed where a value is expected) — the #1 production error class in untyped Python.
  • “Tool selection matters at scale:”
    • mypy: Most mature, best ecosystem. Daemon mode (dmypy) for fast incremental checks. Use --install-types to auto-install type stubs.
    • pyright: Faster, better IDE integration (VS Code / Pylance), stricter by default. Written in TypeScript, harder to extend.
    • pytype: Google’s tool. Can infer types without annotations — great for generating initial annotations on a legacy codebase.
  • “At a 150k-line codebase, we went from 0% to 85% typed coverage in 6 months using the ratchet approach. Mypy caught 23 bugs in code review that quarter — 3 of which would have been P1 production incidents based on the code paths involved.”
Follow-up:
  1. “What is the difference between Protocol (PEP 544) and ABC? When would you choose one over the other for type checking?”
  2. “Explain TypeVar, Generic, and ParamSpec. How would you type a decorator that preserves the decorated function’s signature?”
  3. “Your codebase uses dynamic patterns like setattr, getattr, and **kwargs heavily. How does mypy handle these, and what are your options when it cannot infer types?”

Conclusion and Interview Tips

This guide covers 150+ essential Python interview questions across all major categories — basics, data structures, OOP, functions, file handling, exception handling, modules, advanced concepts, Python internals, memory management, descriptors, generators, concurrency, design patterns, architecture, testing, web development, and data science. Together, they represent the full range of what you will encounter in Python-focused technical interviews from startups through FAANG.

Key Interview Preparation Tips

  • Master Python fundamentals before diving into frameworks. An interviewer can tell within minutes whether you truly understand Python or just know Django’s API. The GIL, mutability vs immutability, and the difference between is and == are tested at every level.
  • Practice coding problems in Python specifically. Python’s idioms (list comprehensions, generator expressions, collections module) let you write cleaner solutions than direct translations from other languages. Using Counter, defaultdict, and heapq naturally signals Python fluency.
  • Build real projects and be ready to discuss the decisions you made. A deployed Flask/FastAPI service with proper error handling, logging, and tests demonstrates more than 100 solved LeetCode problems. Be prepared to explain your architecture choices and what you would change with hindsight.
  • Understand memory management and the GIL deeply. These are the two topics that most reliably separate mid-level from senior Python candidates. Know when the GIL matters (CPU-bound work), when it does not (I/O-bound work), and the practical implications for choosing between threading, multiprocessing, and asyncio.
  • Study the standard library thoroughly. Python’s “batteries included” philosophy means the standard library covers an enormous range. Knowing about functools.lru_cache, contextlib.contextmanager, itertools, pathlib, and dataclasses without needing to look them up signals deep familiarity.
  • Prepare for both theoretical and practical questions. Some interviews ask “explain the descriptor protocol,” others ask “write a function that…” Be ready for both. The strongest candidates connect theory to practice: “The descriptor protocol is what makes @property work, and here is how I used it to…”

During the Interview

  • Ask clarifying questions before coding. “Should this handle concurrent access?” or “What is the expected input size?” shows you think about context before writing code.
  • Explain your thought process continuously. “I am choosing a dictionary here because lookups are O(1) and we need to check membership frequently” is far stronger than silently writing code.
  • Write Pythonic code, not Java-in-Python. Using list comprehensions over manual loops, with statements for resource management, and f-strings over string concatenation signals Python fluency. Interviewers notice this.
  • Consider edge cases and error handling. What happens with empty input? What about None values? What if the file does not exist? Mentioning these proactively demonstrates production-level thinking.
  • Discuss time and space complexity for every solution. Even when not asked, briefly stating the complexity shows algorithmic maturity. “This is O(n) time and O(n) space due to the hash set” takes 3 seconds and makes a strong impression.
  • Be honest about gaps but show your reasoning. “I have not used asyncio in production, but I understand it uses an event loop similar to Node.js, and I would choose it over threading for I/O-bound workloads because of the GIL” shows that you can reason from principles even about unfamiliar territory.

What Separates Good from Great Python Candidates

Good CandidateGreat Candidate
Knows list vs tupleExplains when tuple’s immutability enables hashability and use as dict keys
Can write a classUnderstands __slots__, descriptors, and when to use @dataclass instead
Knows about the GILCan explain when the GIL does and does not matter, and chooses the right concurrency model accordingly
Uses try/exceptFollows EAFP over LBYL and can explain why, with concrete examples
Knows pip installUnderstands virtual environments, dependency resolution, and why poetry or uv exist
Python interviews test not just language knowledge but also problem-solving approach, code quality, and understanding of computer science fundamentals. The strongest candidates write code that is simultaneously correct, readable, and Pythonic — and can explain the reasoning behind every choice they make.
Good luck with your Python interviews! Remember that consistent practice and building real projects are the best ways to prepare.