Skip to main content
RESP Protocol

​
Build Your Own Redis

Target Audience: Mid-Level Engineers (2-5 years experience)
Language: Go (with Java & JavaScript alternatives)
Duration: 3-4 weeks
Difficulty: ⭐⭐⭐⭐☆

​
Why Build Redis?

Redis is the world’s most popular in-memory data store. By building your own, you’ll master:
  • Network programming — TCP servers, connection handling, event loops
  • Protocol design — RESP (Redis Serialization Protocol) parsing
  • Data structures at scale — Hash tables, skip lists, streams
  • Persistence strategies — RDB snapshots, AOF logging
  • Concurrent programming — Goroutines, channels, lock-free designs
This is an advanced project. You should be comfortable with TCP/IP basics and have experience with at least one systems programming language.

​
Redis Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                           REDIS ARCHITECTURE                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│   CLIENT                     SERVER                      STORAGE            │
│   ──────                     ──────                      ───────            │
│                                                                              │
│   ┌──────────┐    RESP     ┌─────────────────┐        ┌───────────┐        │
│   │  redis   │◄───────────►│  Command Router │        │    RDB    │        │
│   │  -cli    │  Protocol   │                 │◄──────►│  Snapshot │        │
│   └──────────┘             │  ┌───────────┐  │        └───────────┘        │
│                            │  │   GET     │  │                              │
│   ┌──────────┐             │  │   SET     │  │        ┌───────────┐        │
│   │   App    │◄───────────►│  │   DEL     │  │◄──────►│    AOF    │        │
│   │  Client  │             │  │   ...     │  │        │  Append   │        │
│   └──────────┘             │  └───────────┘  │        └───────────┘        │
│                            │        │        │                              │
│                            │   ┌────▼────┐   │                              │
│                            │   │  Data   │   │                              │
│                            │   │  Store  │   │                              │
│                            │   │ (HashMap)│   │                              │
│                            │   └─────────┘   │                              │
│                            └─────────────────┘                              │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

​
What You’ll Build

​
Core Features

FeatureDescriptionSkills Learned
RESP ParserParse/serialize Redis protocolProtocol design, binary parsing
TCP ServerHandle multiple clientsNetwork programming, concurrency
String CommandsGET, SET, APPEND, INCRBasic key-value operations
ExpirationEXPIRE, TTL, PTTLTimer management, lazy deletion
List CommandsLPUSH, RPUSH, LPOP, LRANGELinked list implementation
Set CommandsSADD, SMEMBERS, SINTERSet operations
Hash CommandsHSET, HGET, HGETALLNested hash tables
Sorted SetsZADD, ZRANGE, ZRANKSkip list implementation
Pub/SubSUBSCRIBE, PUBLISHEvent-driven patterns
PersistenceRDB, AOFDurability, recovery

​
Implementation: Go

​
Project Structure

myredis/
├── cmd/
│   └── server/
│       └── main.go
├── internal/
│   ├── protocol/
│   │   ├── parser.go
│   │   ├── writer.go
│   │   └── types.go
│   ├── server/
│   │   ├── server.go
│   │   ├── client.go
│   │   └── handler.go
│   ├── store/
│   │   ├── store.go
│   │   ├── string.go
│   │   ├── list.go
│   │   ├── set.go
│   │   ├── hash.go
│   │   ├── zset.go
│   │   └── expiry.go
│   ├── persistence/
│   │   ├── rdb.go
│   │   └── aof.go
│   └── pubsub/
│       └── pubsub.go
├── go.mod
└── README.md

​
Core Implementation

package protocol

// RESP (Redis Serialization Protocol) types
// https://redis.io/docs/reference/protocol-spec/

type RESPType byte

const (
    SimpleString RESPType = '+' // +OK\r\n
    Error        RESPType = '-' // -ERR message\r\n
    Integer      RESPType = ':' // :1000\r\n
    BulkString   RESPType = '$' // $5\r\nhello\r\n
    Array        RESPType = '*' // *2\r\n$3\r\nfoo\r\n$3\r\nbar\r\n
)

// Value represents a RESP value
type Value struct {
    Type  RESPType
    Str   string
    Num   int64
    Bulk  []byte
    Array []Value
    Null  bool
}

// Common responses
var (
    OKResponse   = Value{Type: SimpleString, Str: "OK"}
    NullResponse = Value{Type: BulkString, Null: true}
    PongResponse = Value{Type: SimpleString, Str: "PONG"}
)

// NewError creates an error response
func NewError(msg string) Value {
    return Value{Type: Error, Str: msg}
}

// NewInteger creates an integer response
func NewInteger(n int64) Value {
    return Value{Type: Integer, Num: n}
}

// NewBulkString creates a bulk string response
func NewBulkString(s string) Value {
    return Value{Type: BulkString, Bulk: []byte(s)}
}

// NewArray creates an array response
func NewArray(vals []Value) Value {
    return Value{Type: Array, Array: vals}
}

​
Testing Your Redis

# Build and run
go build -o myredis ./cmd/server
./myredis -port 6379

# In another terminal, use redis-cli
redis-cli
127.0.0.1:6379> PING
PONG
127.0.0.1:6379> SET hello world
OK
127.0.0.1:6379> GET hello
"world"
127.0.0.1:6379> LPUSH mylist a b c
(integer) 3
127.0.0.1:6379> LRANGE mylist 0 -1
1) "c"
2) "b"
3) "a"

​
Advanced Topics

​
1. Persistence (RDB Snapshots)

// internal/persistence/rdb.go
package persistence

import (
    "encoding/gob"
    "os"
    "myredis/internal/store"
)

func SaveRDB(store *store.Store, filename string) error {
    file, err := os.Create(filename)
    if err != nil {
        return err
    }
    defer file.Close()

    encoder := gob.NewEncoder(file)
    // Serialize store data...
    return nil
}

​
2. Pub/Sub

// internal/pubsub/pubsub.go
package pubsub

import (
    "sync"
)

type PubSub struct {
    channels map[string]map[chan string]struct{}
    mu       sync.RWMutex
}

func (ps *PubSub) Subscribe(channel string) chan string {
    ps.mu.Lock()
    defer ps.mu.Unlock()

    if ps.channels[channel] == nil {
        ps.channels[channel] = make(map[chan string]struct{})
    }

    ch := make(chan string, 100)
    ps.channels[channel][ch] = struct{}{}
    return ch
}

func (ps *PubSub) Publish(channel, message string) int {
    ps.mu.RLock()
    defer ps.mu.RUnlock()

    subscribers := ps.channels[channel]
    for ch := range subscribers {
        select {
        case ch <- message:
        default:
            // Channel full, skip
        }
    }
    return len(subscribers)
}

​
Exercises

​
Level 1: Core Implementation

  1. Add APPEND command
  2. Implement SETEX (set with expiry)
  3. Add LINDEX command

​
Level 2: Advanced Features

  1. Implement Sorted Sets with skip list
  2. Add RDB persistence
  3. Implement MULTI/EXEC transactions

​
Level 3: Production Features

  1. Add AOF persistence
  2. Implement cluster mode
  3. Add Lua scripting support

​
What You’ve Learned

TCP server implementation in Go
Binary protocol parsing (RESP)
Concurrent data structure design
Memory management and expiration
Production-grade Go patterns

​
Next Steps

Java Implementation

See how concurrency differs in Java

JavaScript Implementation

Node.js event loop approach

Build Docker

Ready for the ultimate challenge?