> ## Documentation Index
> Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Persistence

> Spring Data JPA, Hibernate, and Transactions

# Data Persistence with Spring Data JPA

Most microservices need to store state. **Spring Data JPA** provides a repository abstraction over JPA (Hibernate), significantly reducing boilerplate code.

**Real-world analogy:** Think of JPA as a universal translator between your Java objects and your relational database. Your code speaks Java (objects, fields, methods), and your database speaks SQL (tables, columns, rows). Hibernate is the interpreter that converts between the two languages in real time. Spring Data JPA sits on top of Hibernate and acts like a personal assistant -- you describe *what* data you want (via method names like `findByPriceLessThan`), and it writes the SQL for you. You never have to learn the database's dialect directly, though understanding it makes you far more effective when things go wrong.

## 1. Dependencies

In `pom.xml` (or build.gradle):

```xml theme={null}
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>
<!-- For Production -->
<!-- <dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
</dependency> -->
```

## 2. Defining Entities

An Entity represents a table in your database.

```java theme={null}
import jakarta.persistence.*;
import lombok.Data;

@Entity
@Table(name = "products") // Maps this class to the "products" table
@Data // Lombok generates getters, setters, toString, equals, hashCode
public class Product {

    @Id // Marks this field as the primary key
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    // IDENTITY = let the database auto-increment the ID.
    // Use SEQUENCE for PostgreSQL in production -- it allows batch inserts
    // (IDENTITY forces single-row inserts because it needs the DB-generated ID back).
    private Long id;

    @Column(nullable = false) // Adds NOT NULL constraint at the DDL level
    private String name;

    private Double price;

    private boolean inStock;
}
```

**Production pitfall -- @Data on entities:** Lombok's `@Data` generates `equals()` and `hashCode()` using all fields, including the `@Id`. This breaks JPA's identity semantics: two `Product` objects representing the same DB row but loaded in different persistence contexts will have different identity if the ID is null (before persist). In production, use `@Getter @Setter @ToString` separately, and write a manual `equals()`/`hashCode()` based on the business key or use `@EqualsAndHashCode(onlyExplicitlyIncluded = true)` with `@EqualsAndHashCode.Include` on the `id` field.

## 3. The Repository Interface

This is where the magic happens. You don't need to write implementation classes.

```java theme={null}
import org.springframework.data.jpa.repository.JpaRepository;
import java.util.List;

public interface ProductRepository extends JpaRepository<Product, Long> {
    
    // Magic Method: Spring generates the SQL automatically!
    // SELECT * FROM products WHERE in_stock = ?
    List<Product> findByInStock(boolean inStock);

    // SELECT * FROM products WHERE price < ?
    List<Product> findByPriceLessThan(Double price);
}
```

## 4. Service Layer & Transactions

Business logic lives in the Service layer, not the controller.

```java theme={null}
@Service
@RequiredArgsConstructor // Lombok generates a constructor for all 'final' fields
public class ProductService {

    private final ProductRepository productRepository;

    @Transactional // Wraps this method in a database transaction (BEGIN...COMMIT/ROLLBACK)
    public Product updatePrice(Long id, Double newPrice) {
        Product product = productRepository.findById(id)
                .orElseThrow(() -> new RuntimeException("Product not found"));
        
        product.setPrice(newPrice);
        // No need to call save()!
        // Hibernate's "Dirty Checking" mechanism tracks every field change on managed entities.
        // At transaction commit, it compares the current state to a snapshot taken at load time
        // and issues an UPDATE only for changed fields.
        // This is why @Transactional matters -- without it, there is no persistence context
        // to track changes, and your setter call quietly does nothing to the database.
        return product;
    }
}
```

### @Transactional Explained

**Analogy:** A transaction is like an "undo" button for your database. You group a set of operations together and say "either all of these succeed, or pretend none of them happened." If you are transferring money between two bank accounts, you want to debit one AND credit the other. If the credit fails, the debit must be rolled back. That is a transaction.

* **Atomicity**: Either all operations in the method succeed, or none do.
* **Rollback**: If a `RuntimeException` is thrown, the transaction rolls back automatically. Checked exceptions (like `IOException`) do **not** trigger rollback by default -- this catches many developers off guard.
* **Propagation**: If one transactional method calls another, how do they relate? (Default `REQUIRED` joins the existing transaction).

**Production pitfall -- @Transactional on private methods:** Spring's `@Transactional` works via proxies. The proxy wraps your bean and intercepts method calls. But if the method is `private`, the proxy cannot intercept it, so the annotation is silently ignored. Your code runs without a transaction and you will not get an error -- only mysterious data inconsistencies in production. Always use `public` methods for `@Transactional`.

## 5. H2 Console

When using H2 (in-memory DB), you can view the data in a browser.

Add to `application.properties`:

```properties theme={null}
spring.h2.console.enabled=true
spring.h2.console.path=/h2-console
spring.datasource.url=jdbc:h2:mem:testdb
```

Access at `http://localhost:8080/h2-console`.

## 6. Projections

Sometimes you don't want the full Entity. You just want a slice of data.

```java theme={null}
// Interface based projection
public interface ProductNameOnly {
    String getName();
}

// In Repository
List<ProductNameOnly> findByNameStartingWith(String prefix);
```

Spring Data is smart enough to select only the required columns.

## 7. The N+1 Query Problem

This is the most common performance killer in Hibernate, and it has sunk more production systems than most developers realize.

**Analogy:** Imagine you are a teacher checking attendance. The N+1 approach is calling each student's parent individually to ask "Is your child here today?" -- one phone call per student. The JOIN FETCH approach is calling the school office once and getting the full attendance sheet for the entire class.

Imagine: **1** `Author` has **N** `Books`.

```java theme={null}
List<Author> authors = authorRepository.findAll(); // 1 Query: SELECT * FROM authors
for (Author a : authors) {
    // Each call triggers a LAZY load: SELECT * FROM books WHERE author_id = ?
    System.out.println(a.getBooks().size()); // N Queries (One per author!)
}
```

If you have 1000 authors, you run 1001 queries. This turns a 5ms operation into a 5-second operation.

**Solution: JOIN FETCH**
Tell Hibernate to fetch everything in ONE query.

```java theme={null}
// Single query: SELECT a.*, b.* FROM authors a JOIN books b ON a.id = b.author_id
@Query("SELECT a FROM Author a JOIN FETCH a.books")
List<Author> findAllWithBooks();
```

**Production tip:** Enable Hibernate query logging in development to catch N+1 problems early. Add `spring.jpa.properties.hibernate.generate_statistics=true` and watch for high query counts. Tools like [Hibernate Query Log](https://github.com/vladmihalcea/hibernate-query-log) or p6spy can also flag suspicious query patterns. In CI, you can even fail the build if a test exceeds a query count threshold using libraries like `datasource-proxy`.

## 8. Concurrency Control (Locking)

What if two users update the same product price at the exact same millisecond? This is the "Lost Update" problem -- one user's change silently overwrites the other's.

**Analogy:** Two people editing the same Google Doc paragraph at once. Without conflict detection, the last person to save wins and the first person's edits vanish without a trace.

### Optimistic Locking (Recommended for most cases)

Add a `@Version` field. This is a "check before you write" strategy.

```java theme={null}
@Version
private Long version; // Hibernate auto-increments this on every UPDATE
```

Hibernate checks: `UPDATE product SET price = 10, version = 2 WHERE id = 1 AND version = 1`.
If the version doesn't match (someone else updated it between your read and write), it throws `OptimisticLockException`. No database locks are held -- this is purely application-level conflict detection.

**When to use:** High-read, low-write workloads. Most CRUD APIs. Shopping carts, user profiles, product catalogs.

### Pessimistic Locking

Lock the database row so no one else can read or write it until you are done.

```java theme={null}
// PESSIMISTIC_WRITE = SELECT ... FOR UPDATE
// The row is locked until the transaction commits or rolls back.
@Lock(LockModeType.PESSIMISTIC_WRITE)
@Query("SELECT p FROM Product p WHERE p.id = :id")
Optional<Product> findByIdLocked(Long id);
```

**When to use:** High-contention, critical operations where optimistic retries would be too expensive. Bank account balance updates, inventory decrement on checkout.

**Production pitfall:** Pessimistic locks can cause deadlocks if two transactions lock rows in different orders. Always acquire locks in a consistent order (e.g., by ascending ID). Set a lock timeout to avoid infinite waits: `@QueryHints(@QueryHint(name = "jakarta.persistence.lock.timeout", value = "3000"))` (3-second timeout).

## 9. Auditing

Keep track of "Who changed what and when" automatically.

1. Add `@EnableJpaAuditing` to main class.
2. Add fields to Entity:

```java theme={null}
@EntityListeners(AuditingEntityListener.class)
public class Product {
    
    @CreatedDate
    private LocalDateTime createdAt;
    
    @LastModifiedDate
    private LocalDateTime updatedAt;
    
    @CreatedBy
    private String createdBy; // Needs AuditorAware implementation
}
```

## 10. Testing with @DataJpaTest

Don't use the full `@SpringBootTest` for DB tests (too slow). Use Slice Testing.

```java theme={null}
@DataJpaTest
class ProductRepositoryTest {

    @Autowired
    private ProductRepository repo;

    @Test
    void shouldFindInStockProducts() {
        Product p = new Product("Phone", 100.0, true);
        repo.save(p);

        List<Product> found = repo.findByInStock(true);
        assertThat(found).hasSize(1);
    }
}
```

*Note: This usually uses H2 by default. For real Postgres testing, look into **Testcontainers**.*

## 11. JPA Architecture

<div className="flex justify-center my-4">
  <div className="w-full max-w-3xl">
    ```mermaid theme={null}
    graph TD
        App[Spring Boot App] --> Repo[Spring Data Repository]
        Repo --> EM[JPA EntityManager]
        EM --> Hibernate[Hibernate Core]
        Hibernate --> JDBC[JDBC Driver]
        JDBC --> DB[(Database)]
    ```
  </div>
</div>

## 12. Deep Dive: Transaction Management

Handling transactions correctly is what separates seniors from juniors.

### Propagation Levels (`@Transactional(propagation = ...)`)

| Level                | Description                                                          | Use Case                                                           |
| :------------------- | :------------------------------------------------------------------- | :----------------------------------------------------------------- |
| `REQUIRED` (Default) | Join existing transaction. If none, create new.                      | Most business logic.                                               |
| `REQUIRES_NEW`       | **Suspend** current transaction. Create a brand new independent one. | Audit logging (save log even if main logic fails).                 |
| `MANDATORY`          | Must be called inside a transaction. Else throw Exception.           | Helper methods that shouldn't run standalone.                      |
| `SUPPORTS`           | Run in transaction if exists. Else run non-transactional.            | Read-only operations.                                              |
| `NOT_SUPPORTED`      | Suspend current transaction. Run non-transactional.                  | Sending emails/long processes (don't hold DB lock).                |
| `NESTED`             | Create a **Savepoint** within the existing transaction.              | Complex rollbacks (try sub-task, if fail, rollback only sub-task). |

### Isolation Levels (`@Transactional(isolation = ...)`)

Defines "how much" one transaction sees of another.

1. **READ\_UNCOMMITTED**: Dirty Reads allowed. (Dangerous).
2. **READ\_COMMITTED**: PostgreSQL Default. No Dirty Reads.
3. **REPEATABLE\_READ**: No Non-Repeatable Reads. (MySQL Default).
4. **SERIALIZABLE**: Full locking. Slowest but safest.

### Rollback Rules

By default, Spring **ONLY** rolls back on `RuntimeException` (Unchecked).
It does **NOT** rollback on `CheckedException` (e.g., `IOException`).

This is one of the most dangerous default behaviors in Spring. Your method throws an `IOException`, the transaction commits the partial state, and you now have corrupted data in production. Nothing in the logs tells you the transaction committed -- you only find out when a customer reports a wrong balance.

**Fix:**

```java theme={null}
@Transactional(rollbackFor = Exception.class) // Rollback for ALL exceptions, not just unchecked
public void dangerousMethod() throws IOException { ... }
```

**Production tip:** Many teams adopt a project-wide convention of always using `rollbackFor = Exception.class` on every `@Transactional` annotation. You can enforce this at the team level by creating a custom `@BusinessTransaction` meta-annotation:

```java theme={null}
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Transactional(rollbackFor = Exception.class)
public @interface BusinessTransaction {}

// Usage: cleaner and team-consistent
@BusinessTransaction
public void transferFunds(Long from, Long to, BigDecimal amount) throws InsufficientFundsException { ... }
```

## 13. High-Performance Caching

Caching is **the easiest way** to improve performance. Spring provides an abstraction over multiple caching providers.

### Enable Caching

```java theme={null}
@SpringBootApplication
@EnableCaching
public class DemoApplication {}
```

### Basic Usage

```java theme={null}
@Service
public class ProductService {

    @Cacheable("products") // Cache the result using key = id
    public Product getProduct(Long id) {
        // Expensive DB call
        return productRepository.findById(id).orElseThrow();
    }

    @CacheEvict(value = "products", key = "#id")
    public void deleteProduct(Long id) {
        productRepository.deleteById(id);
    }

    @CachePut(value = "products", key = "#product.id")
    public Product updateProduct(Product product) {
        return productRepository.save(product);
    }
}
```

**Annotations:**

* `@Cacheable`: If key exists in cache, return cached value. Else, execute method and cache the result.
* `@CacheEvict`: Remove from cache.
* `@CachePut`: Always execute method AND update cache.

### Using Redis (Production)

By default, Spring uses `ConcurrentHashMap` (in-memory). This works fine for a single instance, but the moment you scale to multiple pods, each pod has its own independent cache. User A hits Pod 1 (cache miss, loads from DB), then User A hits Pod 2 (another cache miss, loads from DB again). You have zero cache benefit under a load balancer.

For distributed systems, use **Redis** -- a shared, external cache that all pods read from.

**Dependency:**

```xml theme={null}
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
```

**Config:**

```yaml theme={null}
spring:
  cache:
    type: redis
  data:
    redis:
      host: localhost
      port: 6379
```

Spring automatically switches to Redis. No code changes needed -- the `@Cacheable` annotations work identically.

### Pitfalls

1. **Serialization Issues:** Your cached objects must be `Serializable`. If you change a field name or type in your DTO, the old cached entries cannot be deserialized and you get runtime `ClassCastException`. Use Jackson for JSON-based serialization instead of Java serialization -- it handles schema evolution gracefully.
2. **Cache Stampede:** If a popular cache entry expires, 1000 requests hit the database simultaneously. This can overwhelm the DB and cascade into a full outage. Use `@Cacheable(sync = true)` to ensure only one thread computes the value while others wait.
3. **Stale Data:** Always define a TTL (Time to Live). Without one, cached data lives forever, and your users see stale prices, stale inventory counts, or stale permissions.
4. **Cache Aside vs. Read-Through:** Spring's `@Cacheable` implements the "cache aside" pattern (application manages the cache). For write-heavy workloads, consider "write-through" or "write-behind" strategies where the cache is updated on writes, not just reads.

```java theme={null}
@Configuration
public class CacheConfig {
    @Bean
    public RedisCacheConfiguration cacheConfiguration() {
        return RedisCacheConfiguration.defaultCacheConfig()
                .entryTtl(Duration.ofMinutes(10)); // Expire after 10 min
    }
}
```

***

## Interview Deep-Dive

<AccordionGroup>
  <Accordion title="Explain how @Transactional works under the hood. What actually happens when Spring encounters this annotation, and why does calling a @Transactional method from within the same class not work?">
    **Strong Answer:**

    * At startup, Spring's `BeanPostProcessor` (specifically `InfrastructureAdvisorAutoProxyCreator`) scans every bean. If a bean or its methods carry `@Transactional`, the BPP wraps the bean in a CGLIB proxy. The ApplicationContext stores the proxy, not your original object. When external code calls a `@Transactional` method, the call hits the proxy first.
    * The proxy's `TransactionInterceptor` kicks in. It reads the annotation's attributes (propagation, isolation, rollbackFor), asks the `PlatformTransactionManager` (typically `JpaTransactionManager` for JPA) to begin a transaction, then calls `proceed()` on the actual target method. If the method completes normally, it commits. If a `RuntimeException` (unchecked) or `Error` is thrown, it rolls back. Critically, checked exceptions do NOT trigger rollback by default -- this catches many developers off guard. You need `@Transactional(rollbackFor = Exception.class)` to cover checked exceptions.
    * The self-invocation problem: when method A in `OrderService` calls `this.methodB()`, `this` refers to the raw target object, not the proxy. The proxy is an outer shell that intercepts calls from external callers. Internal calls bypass it entirely. So `methodB()`'s `@Transactional` annotation is invisible.
    * Solutions: (1) Move `methodB()` to a separate `@Service` and inject it -- the cleanest approach. (2) Self-inject the bean: inject `OrderService` into itself and call `self.methodB()`, because the injected reference is the proxy. (3) Use `AopContext.currentProxy()` after enabling `@EnableAspectJAutoProxy(exposeProxy = true)` -- this is brittle and not recommended for production.

    **Follow-up: You have a method with @Transactional(propagation = REQUIRES\_NEW) called from within another @Transactional method. What actually happens to the database connections?**

    `REQUIRES_NEW` suspends the outer transaction and creates a completely independent inner transaction. The `TransactionManager` obtains a second database connection from the pool. You now have two connections open simultaneously. If your connection pool max is 10 and you have deep `REQUIRES_NEW` nesting under concurrent load, you can exhaust the pool and deadlock -- the outer transaction holds connection 1 and waits for the inner call to return, but the inner call cannot get connection 2. I have seen this take down production systems. Size your connection pool accounting for the maximum `REQUIRES_NEW` depth multiplied by concurrent request count.
  </Accordion>

  <Accordion title="What is the N+1 query problem in Hibernate, and describe at least three different strategies to solve it with their trade-offs.">
    **Strong Answer:**

    * The N+1 problem: when you load a parent entity with a `@OneToMany` lazy collection, Hibernate executes 1 query for the parents and then N additional queries (one per parent) when you access each collection. With 1000 authors, that is 1001 queries instead of 1 or 2.
    * Solution 1: `JOIN FETCH` in JPQL -- `SELECT a FROM Author a JOIN FETCH a.books`. Single SQL JOIN query. Trade-off: Cartesian product explosion. If an author has 10 books and you load 100 authors, the result set has 1000 rows. Hibernate deduplicates, but the database still sends 1000 rows over the wire.
    * Solution 2: `@EntityGraph(attributePaths = {"books"})` on a repository method. Declarative alternative to JOIN FETCH. Same Cartesian trade-off, but cleaner and reusable across queries.
    * Solution 3: `@BatchSize(size = 50)` on the collection. Hibernate batches: `SELECT * FROM books WHERE author_id IN (?, ?, ..., ?)` with 50 IDs at a time. For 1000 authors, 20 queries instead of 1000. No result set explosion because there is no JOIN. Often the best default for large datasets.
    * Solution 4: DTO projection -- `SELECT new AuthorSummary(a.name, COUNT(b)) FROM Author a LEFT JOIN a.books b GROUP BY a.name`. No N+1 because you load flat data, not entities. Most performant for read-only use cases.

    **Follow-up: How do you detect N+1 queries in an existing application before they hit production?**

    Enable `spring.jpa.properties.hibernate.generate_statistics=true` to log query counts per session. In tests, use `datasource-proxy` library to wrap your DataSource and assert: `assertThat(queryCount).isLessThanOrEqualTo(3)`. In development, `spring.jpa.show-sql=true` with a formatter like `p6spy` exposes repeated SELECTs with different WHERE values -- the telltale N+1 signature. For production, check slow query logs sorted by frequency, not duration. A 2ms query executed 10,000 times per request is worse than a 200ms query executed once.
  </Accordion>

  <Accordion title="Explain optimistic locking vs. pessimistic locking in JPA. When would you choose each, and what are the failure modes?">
    **Strong Answer:**

    * Optimistic locking assumes conflicts are rare. A `@Version` field is included in every UPDATE's WHERE clause: `UPDATE product SET price = 10, version = 2 WHERE id = 1 AND version = 1`. If the version changed, zero rows update, and Hibernate throws `OptimisticLockException`. The application must catch and retry.
    * Pessimistic locking acquires a database-level lock: `SELECT ... FOR UPDATE`. Other transactions block until the lock is released at commit/rollback. Guarantees exclusive access but reduces throughput.
    * Choose optimistic for high-read, low-write scenarios. A product catalog read millions of times, prices changed occasionally. Zero locking overhead on reads. Conflicts are rare and cheap to handle.
    * Choose pessimistic for high-contention, high-cost-of-failure scenarios. Seat reservations where 500 people book the last 10 seats simultaneously. With optimistic locking, 490 transactions fail and retry, creating a storm. With pessimistic, transactions queue orderly.
    * Failure mode for optimistic: retry storms under contention. 100 concurrent transactions read version 1, all fail except one, all retry, 98 fail again. Exponential backoff with jitter is essential.
    * Failure mode for pessimistic: deadlocks. Transaction A locks row 1, waits for row 2. Transaction B locks row 2, waits for row 1. The database kills one. Always acquire locks in consistent order (e.g., by PK ascending).

    **Follow-up: How does optimistic locking interact with Hibernate's dirty checking and the first-level cache?**

    Hibernate's persistence context caches entity state at load time. At flush, it compares current field values to the snapshot -- dirty checking. The `@Version` field is automatically included in the UPDATE WHERE clause. The subtle case: if you `detach()` an entity, send it to a UI form, and later `merge()` it, the version check still applies. If another transaction updated the row between detach and merge, the merge throws `OptimisticLockException`. The version field becomes a concurrency token for the entire user workflow, not just one transaction. This is common in edit forms where the entity is detached for minutes before being saved.
  </Accordion>

  <Accordion title="What are the different transaction isolation levels, and describe a concrete production bug caused by choosing the wrong one.">
    **Strong Answer:**

    * `READ_UNCOMMITTED`: Dirty reads allowed. Transaction A sees uncommitted changes from B. If B rolls back, A acted on phantom data. Almost never used except analytics where approximate counts suffice.
    * `READ_COMMITTED` (PostgreSQL default): No dirty reads. But non-repeatable reads are possible -- you read a row, another transaction modifies and commits it, you re-read and get a different value within the same transaction.
    * `REPEATABLE_READ` (MySQL InnoDB default): Re-reading the same row always returns the same value within a transaction. MySQL uses MVCC snapshots. But phantom reads can occur -- range queries might return different rows if another transaction inserts matching rows.
    * `SERIALIZABLE`: Full isolation. Transactions execute as if sequential. Prevents all anomalies but kills throughput with range locks.
    * Concrete bug: A financial system uses `READ_COMMITTED` for account transfers. The transfer method reads Account A balance ($1000), reads Account B balance, then writes both. Between read and write, another transaction reads Account A (still $1000, first transaction uncommitted) and also deducts $800. Both succeed. Account A ends up at -$600. With `REPEATABLE_READ` or optimistic locking, the second transaction would fail.

    **Follow-up: How does @Transactional(isolation = SERIALIZABLE) actually get enforced?**

    Spring does not enforce isolation -- it calls `Connection.setTransactionIsolation()` and the database engine enforces it. PostgreSQL's SERIALIZABLE uses Serializable Snapshot Isolation (SSI), which allows concurrency but aborts on detected conflicts. MySQL's SERIALIZABLE acquires shared locks on all reads, blocking writers with much worse throughput. If your app is database-agnostic, test isolation behavior on each database -- do not just trust the label. The actual guarantees and performance characteristics vary dramatically between engines.
  </Accordion>
</AccordionGroup>
