Skip to main content

1. Python Basics

Python is a high-level, interpreted, general-purpose programming language created by Guido van Rossum. Key features include: easy-to-learn syntax, interpreted nature (no compilation), dynamic typing, object-oriented capabilities, extensive standard library, cross-platform compatibility, large community support, and open-source nature.
Python has several built-in data types: Numeric (int, float, complex), Sequence (list, tuple, range, str), Mapping (dict), Set (set, frozenset), Boolean (bool), Binary (bytes, bytearray, memoryview), and None Type (NoneType). Each serves specific purposes in data handling and manipulation.
Lists are mutable (can be modified after creation) and use square brackets []. Tuples are immutable (cannot be changed) and use parentheses (). Lists are slower than tuples but have more built-in methods. Use lists when data needs to change, tuples for fixed data that shouldn’t be modified.
Mutable types can be modified after creation: lists, dictionaries, sets, and custom objects. Immutable types cannot be changed: integers, floats, strings, tuples, and frozensets. Example: list_a = [1, 2, 3]; list_a[0] = 10 works, but string_a = 'hello'; string_a[0] = 'H' raises TypeError.
Python has two identity operators: is and is not. They check if two variables reference the same object in memory, not just equal values. Example: a = [1,2,3]; b = a; c = [1,2,3]. Here a is b returns True, but a is c returns False even though a == c returns True.
Global variables are defined outside functions and accessible throughout the module. Local variables are defined inside functions and only accessible within that function. Use global keyword inside function to modify global variable. Example: python x = 10 # global def func(): y = 5 # local global x x = 20 # modifies global Local variables shadow global ones with same name unless global is used.
In Python 2: range() returns list, xrange() returns generator (memory efficient). In Python 3: range() behaves like Python 2’s xrange() (returns range object, not list), xrange() removed. Python 3’s range() is memory efficient and lazy. Use list(range(10)) to get list if needed.
Python provides built-in functions for type conversion: int(), float(), str(), bool(), list(), tuple(), set(), dict(). These create new objects of specified type. Example: int('123') returns 123, str(42) returns '42', list('abc') returns ['a', 'b', 'c']. Implicit conversion happens in some contexts (e.g., arithmetic operations).
PEP 8 is Python’s official style guide. It defines coding conventions for readability and consistency: naming conventions (snake_case for functions/variables, PascalCase for classes), line length (79 characters), indentation (4 spaces), import organization, whitespace usage, comments. Following PEP 8 improves code maintainability, readability, and collaboration. Tools like flake8 and black help enforce PEP 8.
The Zen of Python is collection of 19 principles that guide Python’s design philosophy. Access with import this. Key principles include: “Beautiful is better than ugly”, “Simple is better than complex”, “Readability counts”, “There should be one obvious way to do it”, “Errors should never pass silently”, “In the face of ambiguity, refuse the temptation to guess”. These principles emphasize simplicity, readability, and explicit over implicit.

2. Data Structures

Lists are created using square brackets: my_list = [1, 2, 3] or list() constructor. Common operations: append() adds element at end, insert(index, item) inserts at position, remove(item) removes first occurrence, pop() removes and returns last element, extend() adds elements from another list, sort() sorts in place, reverse() reverses order.
List comprehensions provide a concise way to create lists. Syntax: [expression for item in iterable if condition]. Examples: squares = [x**2 for x in range(10)], evens = [x for x in range(20) if x % 2 == 0], matrix = [[i*j for j in range(3)] for i in range(3)].
Dictionaries are unordered collections of key-value pairs using curly braces . Keys must be immutable and unique. Common methods: get(key, default), keys(), values(), items(), update(), pop(key), popitem(), clear(), setdefault(). Example: person = {'name': 'John', 'age': 30}.
Sets are mutable unordered collections of unique elements created with or set(). Frozensets are immutable versions created with frozenset(). Sets support add(), remove(), discard() while frozensets don’t. FrozenSets can be used as dictionary keys or elements of other sets.
Sets support mathematical operations: union (|), intersection (&), difference (-), symmetric difference (^). Also available as methods: union(), intersection(), difference(), symmetric_difference(). Example: set1 = {(1, 2, 3)}; set2 = {(3, 4, 5)}; union = set1 | set2 returns {(1, 2, 3, 4, 5)}.
Lists are ordered, mutable sequences using []. Allow duplicates, indexed access, support modification. Tuples are ordered, immutable sequences using (). Allow duplicates, indexed access, cannot be modified. Sets are unordered, mutable collections of unique elements using {} or set(). No duplicates, no indexing, fast membership testing. Use lists for ordered, changeable data; tuples for fixed, ordered data; sets for unique, unordered collections.
Shallow copy (copy.copy()) creates new object but references same nested objects. Changes to nested objects affect both. Deep copy (copy.deepcopy()) creates completely independent copy including all nested objects. Changes to nested objects don’t affect original. Example:
import copy
original = [[1, 2], [3, 4]]
shallow = copy.copy(original)
deep = copy.deepcopy(original)
shallow[0][0] = 99  # affects original
deep[0][0] = 88     # doesn't affect original

3. Object-Oriented Programming

The four pillars are: Encapsulation - bundling data and methods together, hiding internal details; Abstraction - hiding complex implementation, showing only necessary features; Inheritance - creating new classes from existing ones; Polymorphism - same interface for different data types.
Use the class keyword followed by class name (PascalCase). Define __init__ method as constructor. Use self parameter to refer to instance. Example:
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age
    def greet(self):
        return f'Hello, I am {self.name}'
p = Person('John', 30)
Instance methods take self, operate on instance data. Class methods take cls, decorated with @classmethod, operate on class-level data. Static methods take neither, decorated with @staticmethod, utility functions logically grouped with class.
Inheritance allows class to acquire properties and methods from another class. Single inheritance: child class inherits from one parent. Multiple inheritance: child inherits from multiple parents. Use super() to call parent methods. Python uses Method Resolution Order (MRO) for multiple inheritance.
Magic methods are special methods with double underscores (e.g., __init__, __str__). They enable operator overloading and customize class behavior. Common ones: __init__ (constructor), __str__ (string representation), __repr__ (official representation), __len__, __add__, __eq__.
__slots__ is class variable that restricts attributes to those listed. Prevents creation of __dict__ for instances, saving memory. Useful for classes with many instances. Trade-off: can’t add new attributes dynamically. Example: python class Person: __slots__ = ['name', 'age'] def __init__(self, name, age): self.name = name self.age = age Reduces memory overhead significantly for large numbers of instances.
Property decorators provide controlled access to attributes. @property defines getter, @property_name.setter defines setter, @property_name.deleter defines deleter. Enables computed attributes, validation, encapsulation. Example:
class Circle:
    def __init__(self, radius):
        self._radius = radius
    
    @property
    def radius(self):
        return self._radius
    
    @radius.setter
    def radius(self, value):
        if value < 0:
            raise ValueError("Radius must be positive")
        self._radius = value
    
    @property
    def area(self):
        return 3.14159 * self._radius ** 2
MRO determines order in which base classes are searched when resolving methods in multiple inheritance. Python uses C3 linearization algorithm. Access with ClassName.__mro__ or ClassName.mro(). Ensures consistent, predictable method resolution. Example: class D(B, C): pass searches D → B → C → object. super() uses MRO to call next method in chain.

4. Functions and Decorators

*args allows function to accept variable number of positional arguments as tuple. **kwargs allows variable number of keyword arguments as dictionary. Useful for flexible function signatures. Example: func(1, 2, 3, name='John', age=30) where args=(1, 2, 3) and kwargs={'name': 'John', 'age': 30}.
Lambda functions are anonymous, small functions defined with lambda keyword. Limited to single expression. Syntax: lambda arguments: expression. Commonly used with map(), filter(), sorted(). Example: square = lambda x: x**2, evens = list(filter(lambda x: x % 2 == 0, numbers)).
Decorators are functions that modify behavior of other functions without changing their code. Use @decorator syntax. Take function as argument, return modified function. Used for logging, timing, authentication, caching. Example:
def my_decorator(func):
  def wrapper(*args, **kwargs):
      print('Before')
      result = func(*args, **kwargs)
      print('After')
      return result
  return wrapper 
Closure is a function that remembers values from enclosing scope even after outer function finishes executing. Inner function has access to outer function’s variables. Useful for data hiding and factory functions. Example:
  def outer(x):
    def inner(y):
      return x + y
    return inner
add_5 = outer(5)
print(add_5(10))  # 15 
Generator functions use yield instead of return to produce values lazily, one at a time. They maintain state between calls. More memory efficient for large sequences. Example:
def count_up_to(n):
    i = 1
    while i <= n:
        yield i
        i += 1
for num in count_up_to(5):
    print(num)
Function annotations provide metadata about function parameters and return values. Syntax: def func(param: type) -> return_type:. Stored in __annotations__ attribute. Not enforced by Python but useful for documentation, type checking tools (mypy), IDEs. Example:
def greet(name: str, age: int) -> str: 
  return f"{name} is {age} years old" 
Use Cases: type hints, documentation, static analysis, IDE autocomplete.
Duck typing: “If it walks like a duck and quacks like a duck, it’s a duck.” Python doesn’t check object type, only whether it has required methods/attributes. Objects are used based on their behavior, not their type. Example:
class Duck:
    def quack(self): return "Quack!"

class Person:
    def quack(self): return "I'm quacking!"

def make_sound(obj):
    return obj.quack()  # Works with any object with quack()
Enables polymorphism without explicit inheritance.

5. File Handling and I/O

Use open() function with mode: ‘r’ for read, ‘w’ for write, ‘a’ for append, ‘r+’ for read/write. Use with statement for automatic cleanup. read() reads entire file, readline() reads one line, readlines() returns list of lines. Example:
with open('file.txt', 'r') as f:
    content = f.read()
with open('file.txt', 'w') as f:
    f.write('Hello World')
‘w’ mode creates new file or truncates existing file to zero length, overwrites all content. ‘a’ mode creates new file if doesn’t exist, adds content at end of existing file without deleting. Both allow writing but handle existing content differently.
Use csv module. csv.reader() reads CSV files, csv.writer() writes. csv.DictReader() and csv.DictWriter() work with dictionaries. Example:
import csv with open('data.csv', 'r') as f: 
reader = csv.reader(f)
  for row in reader: 
    print(row) 
Use json module. json.dumps() converts Python object to JSON string. json.loads() parses JSON string to Python object. json.dump() writes to file. json.load() reads from file. Example:
import json
data = {'name': 'John', 'age': 30}
json_str = json.dumps(data)
parsed = json.loads(json_str)
Context managers handle resource allocation and cleanup automatically. with statement ensures proper acquisition and release of resources. Implements __enter__ and __exit__ methods. Commonly used for file operations, database connections, locks.

6. Exception Handling

Exception handling manages runtime errors gracefully without crashing program. Use try-except blocks. try block contains code that might raise exception. except block handles specific exceptions. Multiple except blocks possible. Prevents program termination.
Common exceptions: ZeroDivisionError (division by zero), ValueError (invalid value), TypeError (wrong type), IndexError (invalid index), KeyError (key not found in dict), FileNotFoundError (file doesn’t exist), AttributeError (invalid attribute), ImportError (import fails).
try: code that might raise exception. except: handles exceptions. else: executes if no exception occurred. finally: always executes regardless of exception, used for cleanup. Order must be: try-except-else-finally.
Create custom exception by inheriting from Exception class. Add custom attributes and methods. Raise with raise keyword. Example:
class InvalidAgeError(Exception):
  def __init__(self, age, message='Age must be >= 0'):
      self.age = age
      self.message = message
      super().__init__(self.message)
Assertions test conditions during development using assert statement. If condition false, raises AssertionError. Disabled when optimization flag used. Use for debugging, not error handling. Syntax: assert condition, message. Example: assert len(numbers) > 0, 'List cannot be empty'.

7. Modules and Packages

Module is single Python file containing functions, classes, variables. Package is directory containing multiple modules and __init__.py file. Packages organize related modules hierarchically. Import modules with import statement, packages with dot notation.
Several ways: import module imports entire module, from module import name imports specific items, from module import * imports all (not recommended), import module as alias creates alias. Example: import math, from datetime import datetime, import pandas as pd.
__init__.py makes directory a Python package. Can be empty or contain initialization code. Executed when package imported. Defines what’s exported with __all__. In Python 3.3+, namespace packages don’t require __init__.py, but it’s still recommended for clarity.
__name__ is special variable set by Python. When file executed directly, __name__ == '__main__'. When imported as module, __name__ equals module name. Common pattern: if __name__ == '__main__': allows code to run only when file executed directly, not when imported.
Virtual environments isolate Python dependencies per project.
Create with: "python -m venv env_name". 
Activate: "source env_name/bin/activate" (Linux/Mac) **or** "env_name\Scripts\activate" (Windows). 
Deactivate with "deactivate". 
Install packages with pip or peotry or uv in activated environment.

8. Advanced Python Concepts

Iterable is any object that can return iterator (implements __iter__). Iterator is object that returns next value (implements __next__). Lists, tuples, strings are iterables. iter() creates iterator from iterable. next() gets next value. StopIteration raised when exhausted.
Multithreading: Multiple threads in single process, good for I/O-bound tasks. GIL limits true parallelism. Use threading module. Multiprocessing: Separate processes, true parallelism, good for CPU-bound tasks. Use multiprocessing module. Heavier than threads.
GIL is mutex that protects Python objects, preventing multiple threads from executing Python bytecode simultaneously. One thread executes at a time. Impacts CPU-bound multithreaded programs. Doesn’t affect I/O-bound or multiprocessing. Trade-off for memory management simplicity.
Metaclasses are classes of classes. They define how classes behave. type is default metaclass. Classes are instances of metaclasses. Used for class creation customization, ORMs, API frameworks. Advanced concept, rarely needed. Define by inheriting from type.
Coroutines are functions that can pause and resume execution. Defined with async def. Use await to pause until result available. Enables asynchronous programming. Non-blocking I/O operations. Use asyncio module. More efficient than threads for I/O-bound concurrent tasks.
Comprehensions provide concise syntax for creating collections.
List comprehension: `[x**2 for x in range(10)]` returns list. 
Dict comprehension: `{k: v**2 for k, v in dict.items()}` returns dictionary. 
Set comprehension `{x**2 for x in range(10)}` returns set. 
Generator comprehension: `(x**2 for x in range(10))` returns generator (lazy evaluation, memory efficient). 
All support conditional filtering with if clause. Generator comprehensions use parentheses and don’t create full collection in memory.
Python uses automatic memory management with reference counting and cyclic garbage collector. Reference counting: objects deleted when reference count reaches zero. Cyclic GC: handles circular references that reference counting can’t resolve. Runs automatically when thresholds exceeded. Can be controlled with gc module: gc.collect() forces collection, gc.disable() disables cyclic GC. Uses generational approach: objects in different generations based on age.
Coroutines are special functions that can suspend and resume execution. Created with async def. Return coroutine object when called. Use await to pause until awaited coroutine completes. Enable cooperative multitasking. Managed by event loop (asyncio.run()). Example:
async def fetch_data():
    await asyncio.sleep(1)
    return "data"

async def main():
    result = await fetch_data()
    print(result)
More efficient than threads for I/O-bound tasks due to no context switching overhead.

9. Web Development with Python

from flask import Flask
app = Flask(__name__)

@app.route('/')
def home():
    return 'Hello, World!'

@app.route('/user/<name>')
def user(name):
    return f'Hello, {name}'

if __name__ == '__main__':
    app.run(debug=True)
Django ORM (Object-Relational Mapping) abstracts database operations. Maps Python classes to database tables. Define models as Python classes. Automatic SQL generation. Supports multiple databases. QuerySets for complex queries. Migrations handle schema changes.
Use requests library for making HTTP requests. GET, POST, PUT, DELETE methods. Handle responses, status codes, headers, JSON. Example: python import requests response = requests.get('https://api.example.com/data') if response.status_code == 200: data = response.json()
REST (Representational State Transfer) is architectural style for APIs. Uses HTTP methods (GET, POST, PUT, DELETE). Stateless, resource-based URLs. JSON commonly used. Create with Flask, Django REST Framework, or FastAPI. Define endpoints for CRUD operations.

10. Data Science and Libraries

NumPy is fundamental package for scientific computing. Provides multi-dimensional arrays (ndarray). Fast operations on arrays. Broadcasting capability. Linear algebra, random numbers, Fourier transforms. Foundation for pandas, scikit-learn. Written in C, much faster than Python lists.
pandas is data manipulation and analysis library. Two main structures: Series (1D labeled array) and DataFrame (2D labeled table). SQL-like operations, groupby, merge, pivot. Handle missing data. Time series functionality. Read/write CSV, Excel, SQL, JSON.
Several methods: isnull() and notnull() detect missing values. dropna() removes rows/columns with NaN. fillna() fills NaN with value. interpolate() fills based on interpolation. isna() checks for missing. Different strategies for different scenarios.
matplotlib is plotting library for creating static, animated, interactive visualizations. pyplot module provides MATLAB-like interface. Line plots, scatter plots, bar charts, histograms. Customizable colors, labels, legends. Save figures to files.
scikit-learn is machine learning library. Classification, regression, clustering algorithms. Model selection, preprocessing, metrics. Simple consistent API. Built on NumPy, SciPy, matplotlib. Includes datasets, dimensionality reduction, ensemble methods.

11. Additional Important Topics

Slicing extracts portion of sequence. Syntax: sequence[start:end:step]. start is inclusive, end is exclusive. Negative indices count from end. Omitting values uses defaults. Creates shallow copy. Example: lst[1:4] gets elements 1-3, lst[::-1] reverses list.
== compares values (equality). is checks if objects are same in memory (identity). Use == for value comparison. Use is primarily with None checks. Example: a = [1,2]; b = [1,2]; a == b is True, a is b is False.
Three logical operators: and (returns True if both operands are True), or (returns True if at least one is True), not (negates boolean value). Short-circuit evaluation: and stops at first False, or stops at first True.
enumerate() adds counter to iterable, returns enumerate object yielding tuples of (index, value). Useful in loops when you need both index and value. Start parameter specifies starting index (default 0). Example: for index, fruit in enumerate(fruits):
zip() combines multiple iterables element-wise into tuples. Returns iterator of tuples. Stops at shortest iterable. Use zip(*zipped) to unzip. Useful for parallel iteration. Example: for name, age in zip(names, ages):

12. Core Programming Concepts

  `append()` adds single element to end of list (element can be any type, including list). 
  `extend()` adds all elements from iterable to end of list. 
  `append()` increases length by 1, 
  `extend()` increases by number of elements in iterable.
Default arguments have preset values in function definition. Used when argument not provided in call. Must come after non-default arguments. WARNING: Don’t use mutable defaults (lists, dicts) as they persist across calls. Example: def greet(name, message='Hello'):
`remove(value)` removes first occurrence of value, raises ValueError if not found. 
`pop(index)` removes and returns item at index, default is last item.
`del` statement removes item at index or deletes entire list. All modify list in place.
Multiple ways: Old style % formatting, str.format() method, f-strings (formatted string literals). f-strings are most modern and readable (Python 3.6+). Support expressions inside . Example: f'{name} is {age} years old', f'{name.upper()} is {age * 2}'.
any() returns True if at least one element in iterable is True. all() returns True if all elements are True. Short-circuit evaluation. Empty iterable: any() returns False, all() returns True. Useful for checking conditions on sequences.

13. Python Internals and Memory Management

Python uses automatic memory management with reference counting and cyclic garbage collector. Reference counting: each object tracks number of references; deleted when count reaches zero. Cyclic GC: handles circular references that reference counting can’t resolve. Memory allocated from heap, managed by Python’s memory manager. Objects automatically deallocated when no longer referenced. Can be controlled with gc module.
Python’s memory model is “pass-by-object-reference”. Variables hold references to objects, not objects themselves. When passing to functions, references are copied, not objects. For mutable objects (lists, dicts), changes inside function affect original. For immutable objects (ints, strings, tuples), operations create new objects. Example:
def modify_list(lst):
  lst.append(4) # modifies original 

def modify_int(x): 
  x += 1 # creates new object, doesn't affect original 

my_list = [1, 2, 3] 
modify_list(my_list) # my_list becomes [1, 2, 3, 4] 
my_int = 5 
modify_int(my_int) # my_int still 5
Object interning is optimization technique where Python reuses immutable objects with same values. Small integers (-5 to 256) and short strings are interned. Example: a = 256; b = 256; a is b returns True (same object). a = 257; b = 257; a is b may return False (different objects). Interning saves memory and speeds up comparisons. Can force interning with sys.intern() for strings.
Weak references allow referencing objects without preventing garbage collection. Created with weakref module. Don’t increase reference count. Useful for caches, circular references, observer patterns. Example:
import weakref

class Data:
    pass

obj = Data()
ref = weakref.ref(obj)
print(ref())  # <__main__.Data object>
del obj
print(ref())  # None (object was garbage collected)
Allows objects to be collected when only weak references remain.
CPython: reference implementation, written in C, most common. Interprets bytecode, has GIL. PyPy: alternative implementation with JIT (Just-In-Time) compiler, often faster for long-running programs. Compatible with CPython but may have subtle differences. Jython: runs on Java Virtual Machine, can use Java libraries, no GIL. IronPython: runs on .NET Framework. Each has different performance characteristics and use cases. CPython is most widely used and compatible.
Use dis module to disassemble Python code into bytecode. Shows what CPython interpreter executes. Useful for understanding performance, debugging, learning Python internals. Example:
import dis

def add(a, b):
    return a + b

dis.dis(add)
# Output shows bytecode instructions
Helps understand how Python code is executed at low level, identify optimization opportunities, debug issues.

14. Descriptors and Magic Methods

Properties use descriptor protocol: __get__, __set__, __delete__. Descriptors are objects that define these methods. When accessing attribute, Python checks if it’s descriptor and calls appropriate method. @property creates descriptor. Example:
class Property:
    def __init__(self, getter):
        self.getter = getter
    
    def __get__(self, obj, objtype=None):
        return self.getter(obj)
Functions, methods, and properties all use descriptor protocol. Enables powerful attribute access control.
__getattr__: called only when attribute not found through normal lookup (after __dict__, class hierarchy). Last resort. __getattribute__: called for every attribute access, even if attribute exists. Must call object.__getattribute__() to avoid recursion. More powerful but dangerous. Example:
class Example: 
  def __getattr__(self, name): 
    return f"Attribute {name} not found" 
  
  def __getattribute__(self, name): # Called for ALL attribute access 
    return object.__getattribute__(self, name) 
Use __getattr__ for fallback, __getattribute__ for complete control (with caution).
Use type() metaclass to create classes dynamically. type(name, bases, dict) creates new class. Example:
# Equivalent to: class Person: pass
Person = type('Person', (), {})

# With methods
def greet(self):
    return f"Hello, I'm {self.name}"

Person = type('Person', (), {'greet': greet, 'name': 'John'})
This demonstrates that classes are objects created by metaclasses. class keyword is syntactic sugar for type() call. Understanding this explains what metaclasses are.
Monkey patching is dynamically modifying or extending code at runtime. Change classes, modules, or functions after they’re defined. Powerful but risky. Example:
class MyClass:
    def original_method(self):
    return "Original"

def new_method(self):
    return "Patched"

# Monkey patch
MyClass.original_method = new_method

obj = MyClass()
print(obj.original_method())  # "Patched"
Useful for testing (mocking), fixing bugs in third-party code, adding features. Can make code harder to understand and maintain.

15. Advanced Generators and Iterators

yield from delegates to another generator, simplifying generator composition. More than syntax sugar: handles sub-generator exceptions, sends values to sub-generator, returns value from sub-generator. Example:
def generator1():
    yield 1
    yield 2

def generator2():
    yield from generator1()  # Delegates to generator1
    yield 3

# Without yield from:
def generator2_old():
    for value in generator1():
        yield value
    yield 3
yield from also handles send(), throw(), and close() properly, making it essential for generator composition.
When generator function called, returns generator object without executing function body. Function execution frame is preserved. When next() called, function executes until yield, then pauses, preserving local state. Frame persists between calls. Example:
  def counter():
    i = 0
    while True:
      i += 1
      yield i  # Pauses here, frame preserved
  gen = counter()
  print(next(gen))  # 1 (frame created, executes to yield, pauses)
  print(next(gen))  # 2 (resumes from yield, frame still exists)
This frame persistence enables stateful generators and memory-efficient iteration over large datasets.
Use dis module to disassemble Python code into bytecode instructions. Shows what CPython interpreter executes. Useful for understanding performance, debugging, learning internals. Example:
import dis

def example():
    x = 1
    y = 2
    return x + y

dis.dis(example)
# Shows bytecode: LOAD_CONST, STORE_FAST, BINARY_ADD, etc.
Helps understand how Python code is executed, identify optimization opportunities, debug performance issues, learn Python internals.

16. Advanced Concurrency and Parallelism

GIL prevents CPU-bound parallelism, but threading is perfect for I/O-bound tasks. While one thread waits for I/O (network, disk), GIL is released, allowing other threads to run. Two real-world use cases: 1) Web scraping: multiple threads fetch different URLs simultaneously. 2) Database queries: threads wait for database responses while others process. Threading provides concurrency for I/O-bound operations where waiting time is significant.
ThreadPoolExecutor: I/O-bound tasks (network requests, file I/O, database queries). Lightweight, shared memory. Limited by GIL for CPU-bound. ProcessPoolExecutor: CPU-bound tasks (number crunching, image processing). True parallelism, separate memory. Higher overhead. asyncio: I/O-bound concurrent tasks with many connections. Single-threaded, event loop. Most efficient for high-concurrency I/O. Choose based on task type: I/O-bound → threading/asyncio, CPU-bound → multiprocessing.
Threading: Low memory overhead (shared memory), low CPU overhead (context switching). Limited by GIL for CPU-bound. Multiprocessing: High memory overhead (separate processes, duplicated memory), high CPU overhead (process creation, IPC). True parallelism. asyncio: Very low memory overhead (single thread, coroutines), very low CPU overhead (no context switching). Best for high-concurrency I/O. Choose based on resource constraints and task characteristics.
Threading: OS-managed, preemptive multitasking. Multiple threads, context switching overhead. Shared memory (need locks). async/await: Cooperative multitasking, single thread. Coroutines yield control voluntarily. No context switching overhead. More efficient for I/O-bound tasks. Example:
# Threading
import threading
import requests

def fetch(url):
    # blocking I/O
    return requests.get(url)

# async/await
import aiohttp

async def fetch(url):
    # non-blocking I/O
    return await aiohttp.get(url)
async/await is more efficient for high-concurrency I/O operations.
Use profiling tools: cProfile for function-level profiling, line_profiler for line-level, memory_profiler for memory usage. Identify bottlenecks, then optimize: use appropriate concurrency (threading/multiprocessing/asyncio), optimize algorithms, use efficient data structures, cache results, avoid premature optimization. Example:
import cProfile
cProfile.run('your_function()') 
Profile first, optimize based on data, not assumptions.
Multithreading: Web scraping (multiple URLs), file processing (multiple files), database operations (multiple queries). Multiprocessing: Data processing (large datasets), image processing, scientific computing, machine learning training. asyncio: Web servers (many concurrent connections), API clients (many requests), real-time applications (chat, gaming). Choose based on task: I/O-bound → threading/asyncio, CPU-bound → multiprocessing.

17. Design Patterns and Architecture

Single Responsibility: each class has one reason to change. Open/Closed: open for extension, closed for modification (use inheritance, composition). Liskov Substitution: subclasses must be substitutable for base classes. Interface Segregation: many specific interfaces better than one general (use Abstract Base Classes). Dependency Inversion: depend on abstractions, not concretions. Python features: ABCs, protocols, composition, dependency injection. Example: use abc.ABC for interfaces, composition over inheritance.
Composition is more flexible, avoids deep inheritance hierarchies, easier to test. Inheritance creates tight coupling. Example refactoring:
# Inheritance (bad)
class Car(Engine, Wheels, Body):
  pass

# Composition (better)
class Car:
  def __init__(self):
    self.engine = Engine()
    self.wheels = Wheels()
    self.body = Body()
Composition allows changing components at runtime, easier testing (mock components), avoids diamond problem, more flexible.
Factory pattern creates objects without specifying exact class. Use factory functions or classes. Example:
class Dog:
    def speak(self): return "Woof"

class Cat:
    def speak(self): return "Meow"

def animal_factory(animal_type):
    animals = {'dog': Dog, 'cat': Cat}
    return animals.get(animal_type, Dog)()

# Usage
pet = animal_factory('dog')
Python’s flexibility (first-class functions, dynamic typing) often makes formal Factory pattern unnecessary, but useful for complex object creation logic.
1. Module-level: Python modules are singletons. Simplest. 
2. Decorator: function decorator that wraps class. Clean syntax. 
3. Metaclass: custom metaclass controls class creation. Powerful but complex.
4. `__new__` method: override `__new__` in base class. Straightforward.
Example (decorator):
  def singleton(cls): 
    instances = {}
  
  def get_instance(): 
    if cls not in instances: 
      instances[cls] = cls() 
        return instances[cls]
    return get_instance 
**Trade-offs:** 
  module-level simplest but less flexible; 
  decorator clean but adds indirection; 
  metaclass powerful but complex; 
  `__new__` straightforward but requires inheritance.
Strategy pattern typically uses classes for algorithms. In Python, functions are first-class objects, so you can pass functions directly. Example:
# Without Strategy pattern 
def process_data(data, strategy): 
  return strategy(data) 

def sort_ascending(data): 
  return sorted(data) 
  
def sort_descending(data): 
  return sorted(data, reverse=True) 

# Usage result = process_data([3, 1, 2], sort_ascending) 
Functions as first-class objects eliminate need for Strategy pattern classes in many cases, making code simpler and more Pythonic.
Two ways:
**1) Class-based**: implement `__enter__` and `__exit__`. 
**2) Decorator**: use `@contextmanager`. 
Example:
# Class-based 
class DatabaseConnection: 
  def __enter__(self): 
    self.conn = connect() 
      return self.conn 
  
  def __exit__(self, exc_type, exc_val, exc_tb): 
    self.conn.close() 
    
# @contextmanager from contextlib 
import contextmanager

@contextmanager 
def database_connection(): 
  conn = connect() 
    try: yield 
  conn finally: 
    conn.close()
Both ensure proper resource cleanup. @contextmanager is simpler for basic cases.
  1. Resource-based URLs: `/users/123` not `/getUser?id=123`. 
  2. HTTP methods: GET (read), POST (create), PUT (update), DELETE (delete). 
  3. Stateless: each request contains all information. 
  4. JSON responses: standard format.
  5. Versioning: `/api/v1/users`. 
  6. Error handling: proper HTTP status codes.
  7. Pydantic models: validate request/response. 
  8. Use FastAPI or Flask-RESTful.
Example:
  @app.get('/users/{user_id}') 
    returns user
  
  @app.post('/users')
    creates user.
Three main approaches:
1. Refactor: move common code to third module. 
2. Lazy imports: import inside functions.
3. Dependency injection: pass dependencies as parameters. 
Example:
# Bad: circular import
# a.py imports b, b.py imports a

# Solution 1: Refactor
# common.py: shared code
# a.py and b.py import common

# Solution 2: Lazy import
def func():
  import b  # Import when needed
  return b.something()

# Solution 3: Dependency injection
def func(dependency):
  return dependency.something()
Best approach: refactor to eliminate circular dependency through better architecture.
Monkey patching is dynamically modifying code at runtime. Change classes, modules, or functions after definition. Use cases: Testing: mock external dependencies. Bug fixes: patch third-party libraries. Feature addition: extend existing code. Example:
import requests

def mock_get(url):
    return MockResponse()

# Monkey patch for testing
requests.get = mock_get
Powerful but risky: makes code harder to understand, can break with library updates. Use sparingly, primarily for testing.

18. Testing and Debugging

Mock external dependencies: databases, APIs, file systems, network calls, time-dependent code. Don’t mock code under test. Follow rule: mock what you don’t control. Example:
from unittest.mock import Mock, patch

@patch('module.external_api_call')
def test_function(mock_api):
    mock_api.return_value = {'data': 'test'}
    result = function_under_test()
    assert result == expected
Mock boundaries (I/O, external services), not business logic. Keep tests focused and fast.
Mocking: verify interactions (calls, arguments). Mocks record how they’re used. Stubbing: provide predefined responses. Stubs don’t verify interactions. In Python’s unittest.mock, Mock() can do both. Example:
# Stub: just returns value
mock_api = Mock(return_value={'data': 'test'})

# Mock: verify it was called
mock_api.assert_called_once()
mock_api.assert_called_with('expected_arg')
Mocks verify behavior, stubs provide data. Often used together: stub provides response, mock verifies usage.
Over-mocking makes tests brittle, slow, and less valuable. Problems: tests break when implementation changes (even if behavior correct), tests don’t catch real bugs, maintenance burden, false confidence. Signs: mocking everything, tests that test mocks not code, tests that break on refactoring. Solution: mock only external dependencies, test real behavior when possible, use integration tests for complex interactions. Balance: mock boundaries, test business logic.
## Debugging Python Code

### Core Debugging Tools

Use Python debugger (`pdb`) or IDE debuggers. Set breakpoints, step through code, inspect variables. Use `print()` statements strategically (or logging). Use `traceback` module to analyze exceptions. Use `inspect` module to examine objects.

### Common Techniques

**Breakpoints**: 
```python
import pdb; pdb.set_trace()
Logging: Use logging module instead of print
import logging
logging.basicConfig(level=logging.DEBUG)
logging.debug("Variable value: %s", variable)
Exception handling: Catch and log exceptions with context
try:
    # code
except Exception as e:
    logging.error("Error occurred", exc_info=True)
IDE debuggers: PyCharm, VS Code, Jupyter notebooksPost-mortem debugging: python -m pdb script.py after crash

Essential pdb Commands

  • n (next): Execute current line
  • s (step): Step into function
  • c (continue): Continue execution
  • l (list): Show current code context
  • p variable: Print variable value
  • pp variable: Pretty-print variable
  • q (quit): Exit debugger
pdb is Python’s built-in debugger. Interactive debugging tool. Start with python -m pdb script.py or insert import pdb; pdb.set_trace() in code. Commands: n (next line), s (step into function), c (continue), l (list code), p variable (print variable), pp variable (pretty print), w (where/stack trace), u (up stack), d (down stack), q (quit). Example:
  def calculate(x, y): 
    import pdb 
    pdb.set_trace() # Breakpoint 
    result = x + y 
    return result 
Essential for understanding code flow and finding bugs.
Use logging module instead of print() statements. Configure log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL. Add context: function names, line numbers, timestamps. Example:
import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

logger = logging.getLogger(__name__)

def process_data(data):
    logger.debug(f"Processing {len(data)} items")
    logger.info("Starting processing")
    # ... code ...
    logger.error("Error occurred", exc_info=True)
Logging persists, can be filtered, and provides better debugging information than print statements.
Use traceback module to analyze exceptions. traceback.print_exc() prints full traceback. traceback.format_exc() returns traceback as string. Use sys.exc_info() to get exception info. Example:
import traceback
import sys

try:
    # code that might fail
    result = 1 / 0
except Exception:
    traceback.print_exc()  # Print to stderr
    exc_type, exc_value, exc_traceback = sys.exc_info()
    print(f"Exception type: {exc_type}")
    print(f"Exception value: {exc_value}")
Use logging.exception() to log exceptions with traceback. Understand stack traces: read from bottom up to find root cause.
  1. Print debugging: strategic `print()` statements (use logging instead).
  2. Debugger: `pdb` or IDE debuggers.
  3. Logging: structured logging with context.
  4. Exception handling: catch and inspect exceptions.
  5. Assertions: verify assumptions with `assert`.
  6. Profiling: use `cProfile` to find bottlenecks.
  7. Type checking: use `mypy` to catch type errors.
  8. Unit tests**: write tests to reproduce bugs.
  9. Code review: fresh eyes catch bugs.
 10. Rubber duck debugging: explain code to find issues. Combine techniques based on problem type.
Use profiling tools:
  1. `cProfile` for function-level
  2. `line_profiler` for line-level 
  3. `memory_profiler` for memory usage. 
  4. `timeit` for micro-benchmarks.
  5. `memory_profiler` for memory leaks.
Example:
import cProfile 
profiler = cProfile.Profile() 
profiler.enable() # Your code here 
profiler.disable() 

import pstats 
stats = pstats.Stats(profiler) 
stats.sort_stats('cumulative') 
stats.print_stats(10) # Top 10 functions 
1. Identify bottlenecks, then optimize. 
2. Profile before optimizing: measure, don't guess. 
3. Focus on hot paths (code executed frequently).
Use gc module to inspect objects. Use tracemalloc to track memory allocations. Use memory_profiler for line-by-line memory usage. Check for circular references, unclosed resources, large data structures. Example:
import tracemalloc
import gc

tracemalloc.start()
# Your code
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
for stat in top_stats[:10]:
    print(stat)

# Check for uncollectable objects
print(gc.collect())
print(len(gc.garbage))
Use objgraph to visualize object references. Close file handles, database connections. Use context managers (with statement) for automatic cleanup.

Conclusion & Interview Tips

This comprehensive guide covers 150+ essential Python interview questions across all major categories including basics, data structures, OOP, functions, file handling, exception handling, modules, advanced concepts, Python internals, memory management, descriptors, generators, concurrency, design patterns, architecture, testing, web development, and data science.

Key Interview Preparation Tips

  • Master Python fundamentals before diving into frameworks
  • Practice coding problems on platforms like LeetCode and HackerRank
  • Build real projects to demonstrate practical skills
  • Understand memory management and performance implications
  • Study common algorithms and data structure implementations
  • Review Python’s standard library and popular third-party packages
  • Prepare for both theoretical and practical coding questions

During the Interview

  • Read questions carefully and ask for clarification
  • Explain your thought process while solving problems
  • Write clean, readable code with proper variable names
  • Consider edge cases and error handling
  • Discuss time and space complexity of your solutions
  • Be honest about what you don’t know but show willingness to learn

Technical Skills to Demonstrate

  • Strong understanding of Python syntax and semantics
  • Proficiency with data structures and algorithms
  • Object-oriented programming principles
  • Error handling and debugging skills
  • Knowledge of Python ecosystem and libraries
  • Problem-solving and analytical thinking
  • Code optimization and performance awareness
Python interviews often test not just language knowledge but also problem-solving approach, code quality, and understanding of computer science fundamentals. Practice explaining your code and reasoning clearly.
Good luck with your Python interviews! Remember that consistent practice and building real projects are the best ways to prepare.