Skip to main content

Documentation Index

Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt

Use this file to discover all available pages before exploring further.

RESP Protocol

​
Build Your Own Redis

Target Audience: Mid-Level Engineers (2-5 years experience)
Language: Go (with Java & JavaScript alternatives)
Duration: 3-4 weeks
Difficulty: ⭐⭐⭐⭐☆

​
Why Build Redis?

Redis is the world’s most popular in-memory data store, used by companies from tiny startups to Netflix, Twitter, and GitHub. By building your own, you’ll master:
  • Network programming — TCP servers, connection handling, and the realities of partial reads and connection lifecycle
  • Protocol design — RESP (Redis Serialization Protocol) parsing, and why protocol simplicity is a feature, not a limitation
  • Data structures at scale — Hash tables, skip lists, doubly-linked lists, and when to choose each one
  • Persistence strategies — RDB snapshots vs. AOF logging, and the durability-performance trade-off that every database must make
  • Concurrent programming — Goroutines, channels, mutexes, and the art of keeping shared state consistent without destroying throughput
This is an advanced project. You should be comfortable with TCP/IP basics (what a socket is, how connections are established) and have experience with at least one systems programming language. If net.Listen("tcp", ":6379") looks unfamiliar, review the Go networking tutorial first.

​
Redis Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                           REDIS ARCHITECTURE                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│   CLIENT                     SERVER                      STORAGE            │
│   ──────                     ──────                      ───────            │
│                                                                              │
│   ┌──────────┐    RESP     ┌─────────────────┐        ┌───────────┐        │
│   │  redis   │◄───────────►│  Command Router │        │    RDB    │        │
│   │  -cli    │  Protocol   │                 │◄──────►│  Snapshot │        │
│   └──────────┘             │  ┌───────────┐  │        └───────────┘        │
│                            │  │   GET     │  │                              │
│   ┌──────────┐             │  │   SET     │  │        ┌───────────┐        │
│   │   App    │◄───────────►│  │   DEL     │  │◄──────►│    AOF    │        │
│   │  Client  │             │  │   ...     │  │        │  Append   │        │
│   └──────────┘             │  └───────────┘  │        └───────────┘        │
│                            │        │        │                              │
│                            │   ┌────▼────┐   │                              │
│                            │   │  Data   │   │                              │
│                            │   │  Store  │   │                              │
│                            │   │ (HashMap)│   │                              │
│                            │   └─────────┘   │                              │
│                            └─────────────────┘                              │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

​
What You’ll Build

​
Core Features

FeatureDescriptionSkills Learned
RESP ParserParse/serialize Redis protocolProtocol design, binary parsing
TCP ServerHandle multiple clientsNetwork programming, concurrency
String CommandsGET, SET, APPEND, INCRBasic key-value operations
ExpirationEXPIRE, TTL, PTTLTimer management, lazy deletion
List CommandsLPUSH, RPUSH, LPOP, LRANGELinked list implementation
Set CommandsSADD, SMEMBERS, SINTERSet operations
Hash CommandsHSET, HGET, HGETALLNested hash tables
Sorted SetsZADD, ZRANGE, ZRANKSkip list implementation
Pub/SubSUBSCRIBE, PUBLISHEvent-driven patterns
PersistenceRDB, AOFDurability, recovery

​
Implementation: Go

​
Project Structure

myredis/
├── cmd/
│   └── server/
│       └── main.go
├── internal/
│   ├── protocol/
│   │   ├── parser.go
│   │   ├── writer.go
│   │   └── types.go
│   ├── server/
│   │   ├── server.go
│   │   ├── client.go
│   │   └── handler.go
│   ├── store/
│   │   ├── store.go
│   │   ├── string.go
│   │   ├── list.go
│   │   ├── set.go
│   │   ├── hash.go
│   │   ├── zset.go
│   │   └── expiry.go
│   ├── persistence/
│   │   ├── rdb.go
│   │   └── aof.go
│   └── pubsub/
│       └── pubsub.go
├── go.mod
└── README.md

​
Core Implementation

package protocol

// RESP (Redis Serialization Protocol) types
// https://redis.io/docs/reference/protocol-spec/

type RESPType byte

const (
    SimpleString RESPType = '+' // +OK\r\n
    Error        RESPType = '-' // -ERR message\r\n
    Integer      RESPType = ':' // :1000\r\n
    BulkString   RESPType = '$' // $5\r\nhello\r\n
    Array        RESPType = '*' // *2\r\n$3\r\nfoo\r\n$3\r\nbar\r\n
)

// Value represents a RESP value
type Value struct {
    Type  RESPType
    Str   string
    Num   int64
    Bulk  []byte
    Array []Value
    Null  bool
}

// Common responses
var (
    OKResponse   = Value{Type: SimpleString, Str: "OK"}
    NullResponse = Value{Type: BulkString, Null: true}
    PongResponse = Value{Type: SimpleString, Str: "PONG"}
)

// NewError creates an error response
func NewError(msg string) Value {
    return Value{Type: Error, Str: msg}
}

// NewInteger creates an integer response
func NewInteger(n int64) Value {
    return Value{Type: Integer, Num: n}
}

// NewBulkString creates a bulk string response
func NewBulkString(s string) Value {
    return Value{Type: BulkString, Bulk: []byte(s)}
}

// NewArray creates an array response
func NewArray(vals []Value) Value {
    return Value{Type: Array, Array: vals}
}

​
Testing Your Redis

# Build and run
go build -o myredis ./cmd/server
./myredis -port 6379

# In another terminal, use redis-cli
redis-cli
127.0.0.1:6379> PING
PONG
127.0.0.1:6379> SET hello world
OK
127.0.0.1:6379> GET hello
"world"
127.0.0.1:6379> LPUSH mylist a b c
(integer) 3
127.0.0.1:6379> LRANGE mylist 0 -1
1) "c"
2) "b"
3) "a"

​
Advanced Topics

​
1. Persistence (RDB Snapshots)

// internal/persistence/rdb.go
package persistence

import (
    "encoding/gob"
    "os"
    "myredis/internal/store"
)

func SaveRDB(store *store.Store, filename string) error {
    file, err := os.Create(filename)
    if err != nil {
        return err
    }
    defer file.Close()

    encoder := gob.NewEncoder(file)
    // Serialize store data...
    return nil
}

​
2. Pub/Sub

// internal/pubsub/pubsub.go
package pubsub

import (
    "sync"
)

type PubSub struct {
    channels map[string]map[chan string]struct{}
    mu       sync.RWMutex
}

func (ps *PubSub) Subscribe(channel string) chan string {
    ps.mu.Lock()
    defer ps.mu.Unlock()

    if ps.channels[channel] == nil {
        ps.channels[channel] = make(map[chan string]struct{})
    }

    ch := make(chan string, 100)
    ps.channels[channel][ch] = struct{}{}
    return ch
}

func (ps *PubSub) Publish(channel, message string) int {
    ps.mu.RLock()
    defer ps.mu.RUnlock()

    subscribers := ps.channels[channel]
    for ch := range subscribers {
        select {
        case ch <- message:
        default:
            // Channel full, skip
        }
    }
    return len(subscribers)
}

​
Exercises

​
Level 1: Core Implementation

  1. Add APPEND command
  2. Implement SETEX (set with expiry)
  3. Add LINDEX command

​
Level 2: Advanced Features

  1. Implement Sorted Sets with skip list
  2. Add RDB persistence
  3. Implement MULTI/EXEC transactions

​
Level 3: Production Features

  1. Add AOF persistence
  2. Implement cluster mode
  3. Add Lua scripting support

​
What You’ve Learned

TCP server implementation in Go
Binary protocol parsing (RESP)
Concurrent data structure design
Memory management and expiration
Production-grade Go patterns

​
Next Steps

Java Implementation

See how concurrency differs in Java

JavaScript Implementation

Node.js event loop approach

Build Docker

Ready for the ultimate challenge?