​

Chapter 8: NoSQL and Caching - Firestore, Bigtable, and Memorystore

​

1. Cloud Bigtable: The Wide-Column Giant

​

1.1 Architecture: The Distributed Storage Engine

• Compute (Nodes): Bigtable nodes handle metadata management and request routing. They do not store data locally.
• Storage (Colossus): Data is stored in SSTables (Sorted String Tables) on Google’s Colossus (distributed file system).
• The Write Path (Life of a Write):
  1. Commit Log: The write is first recorded in a persistent commit log on Colossus (for durability).
  2. Memtable: The data is then written to a sorted in-memory buffer called the Memtable.
  3. Acknowledgement: The client receives a 200 OK.
  4. Flushing: When the Memtable reaches a certain size, it is flushed to Colossus as a new SSTable.
• The Read Path (Bloom Filters):
  • To avoid reading every SSTable on disk, Bigtable uses Bloom Filters.
  • A Bloom Filter is a probabilistic data structure that can tell Bigtable if a row key might be in an SSTable or definitely isn’t. This significantly reduces unnecessary I/O.
• Compaction: Background processes merge smaller SSTables into larger ones and remove deleted data (Garbage Collection).

​

1.2 Row Key Design: The Principal’s Art

​

1.2.1 Advanced Key Design Patterns

​

1.2.2 The “Walking Hotspot” Problem

• Result: Because keys are sorted, all writes for the current second hit the same Tablet. As time moves, the “heat” moves to the next tablet. This is a “walking hotspot.”
• Solution: Promote the sensor_id to the front of the key: sensor_id#timestamp. This spreads the load across as many tablets as you have sensors.

​

1.3 Schema Design: Column Families and Row Size Limits

• Column Families: Group related columns (e.g., user_profile, user_settings). Bigtable compresses data within a family. Keep families to a minimum (< 10).
• Row Size (The 100MB Limit): While Bigtable can handle rows up to 100MB, performance degrades significantly after 10MB.
  • SRE Tip: If your rows are growing too large, move high-cardinality data to a separate table or use GCS for large binary objects (blobs) and store the link in Bigtable.

​

1.3 Schema Design and Performance Tuning

​

Column Families vs. Qualifiers

• Column Family: A logical grouping of columns. You should have very few (usually < 10). Bigtable compresses data by column family.
• Column Qualifier: The individual column name. These can be millions.

​

The “Thin” vs. “Wide” Table Trade-off

• Tall/Thin: Many rows, few columns. (Ideal for Bigtable).
• Short/Wide: Few rows, thousands of columns. (Can lead to large rows that exceed the 100MB-per-row limit).

​

Performance Scaling

• Throughput: Each SSD node provides ~10,000 QPS (Reads/Writes) or ~220 MB/s.
• Latency: Sub-10ms for typical operations.

Principal Note: If you see high latency despite low CPU, check for large cells (>10MB) or large rows (>100MB). Bigtable performs best when cells are small and rows are under 100MB.

​

1.4 Garbage Collection: Managing Data Longevity

1. Max Versions: Keep only the last $N$ versions of a cell (e.g., keep the last 3 price updates).
2. Max Age: Delete data older than a specific duration (e.g., delete logs older than 30 days).
3. Complex Logic:
   • Union: Delete if EITHER condition is met (Age > 30 days OR Versions > 5).
   • Intersection: Delete only if BOTH conditions are met (Age > 30 days AND Versions > 5).

​

1.5 Key Visualizer: Diagnosing the Heatmap

• Bright Horizontal Lines: A single row key is being hammered (Hotspot).
• Diagonal Lines: Sequential keys (like timestamps) are being written. This is a “walking hotspot” that moves across nodes.
• Evenly Distributed Colors: The ideal state. Load is spread across the entire key space.

​

2. Cloud Firestore: The Serverless Document Store

​

Native vs. Datastore Mode

​

2.2 Advanced Firestore Internals

​

2.2.1 Concurrency Models

1. Optimistic Concurrency (Default): Multiple clients can attempt to update a document. If the document has changed since a client last read it, the update fails, and the client must retry.
2. Pessimistic Concurrency (Transactions): Firestore places a lock on the document during a transaction. Other clients must wait until the transaction completes.

​

2.2.2 The 500-Document Batch Limit

• SRE Tip: If you need to perform a massive migration (e.g., updating 10,000 documents), you must chunk your work into batches of 500.

​

2.2.3 Indexing Deep Dive: Single-field vs. Composite

• Single-field Indexes: Automatically created for every field in a document.
• Composite Indexes: Required for queries that use multiple fields or complex ordering (e.g., WHERE city == 'NYC' AND price > 100 ORDER BY price DESC).
• Index Selection: Firestore uses the most efficient index possible. If no composite index exists, it may use a Zig-Zag Merge of two single-field indexes, but this is significantly slower.

​

3. Cloud Memorystore: Managed Caching

​

3.1 Redis Cluster Mode (Terabyte Scaling)

• Slot Distribution: Redis uses 16,384 hash slots. Each node in the cluster is responsible for a subset of these slots.
• Scaling: You can add nodes to a cluster to increase both memory capacity and throughput. Memorystore handles the “resharding” of slots in the background.

​

3.2 Persistence: RDB vs. AOF

1. RDB (Redis Database Backup): Point-in-time snapshots of the dataset. Good for recovery with low performance overhead.
2. AOF (Append-Only File): Logs every write operation. Provides much higher durability but can impact performance.

​

4. NoSQL Selection Decision Tree

1. Is sub-millisecond latency the primary requirement?
   • Yes → Memorystore (Redis).
2. Is it a mobile/web app requiring real-time updates and offline sync?
   • Yes → Firestore (Native Mode).
3. Is it a massive (10TB+) analytical or time-series workload with sub-10ms latency?
   • Yes → Cloud Bigtable.
4. Is it a high-throughput server-side backend needing NoSQL scale?
   • Yes → Firestore (Datastore Mode).

​

4. Advanced NoSQL Internals: Sharding and Querying

​

4.1 Bigtable Tablet Splitting

• Auto-sharding: When a tablet reaches ~100-200MB, or its load (CPU/disk) exceeds a threshold, Bigtable automatically splits it into two.
• Dynamic Rebalancing: Bigtable moves these tablets between nodes in a cluster to ensure every node has a balanced load. This is why you should have at least 1GB of data per node to see the benefits of rebalancing.

​

4.2 Firestore Zig-Zag Merge

• The Algorithm: Firestore uses a Zig-Zag Merge. It takes the results of two independent single-field indexes and “merges” them by hopping between the sorted lists.
• Efficiency: While clever, this is less efficient than a Composite Index. For high-throughput apps, you must create composite indexes for frequently queried field combinations.

​

4.3 Bigtable Replication and App Profiles

• App Profiles: You define how clients connect.
  • Single-cluster routing: Best for strong consistency within a region.
  • Multi-cluster routing: Automatically routes to the nearest cluster. Provides “eventual consistency” but much higher availability (resilient to a full cluster or zonal outage).

​

5. Security in the NoSQL Layer

• Firestore Security Rules: Allow you to define access logic directly on the database. For example: allow read if request.auth != null && resource.data.owner == request.auth.uid;. This removes the need for a traditional backend for many mobile apps.
• Bigtable IAM: Access is controlled at the Instance or Table level. You cannot restrict access to specific rows using IAM alone.

​

6. Interview Preparation: Architectural Deep Dive

• Union: Data is deleted if EITHER condition is met (e.g., Age > 30 days OR Versions > 5).
• Intersection: Data is deleted only if BOTH conditions are met (e.g., Age > 30 days AND Versions > 5). Warning: GC is a background process (compaction); data is not instantly deleted from disk but becomes inaccessible immediately.

​

Implementation: The “NoSQL Architect” Lab

​

Scaling and Managing Bigtable

# 1. Create a Bigtable Instance
gcloud bigtable instances create prod-bigtable \
    --display-name="Production Bigtable" \
    --cluster-config=id=prod-cluster,zone=us-central1-b,nodes=3,storage-type=SSD

# 2. Add a second cluster for High Availability (Replication)
gcloud bigtable clusters create prod-cluster-backup \
    --instance=prod-bigtable \
    --zone=us-east1-b \
    --nodes=3

# 3. Create a Table with a Column Family
gcloud bigtable tables create sensor-data --instance=prod-bigtable --column-families=metrics

​

Pro-Tip: Key Visualizer