Skip to main content

​
Chapter 1: RESP Protocol

Every Redis command travels over the network as RESP (Redis Serialization Protocol). In this chapter, we’ll build a complete RESP parser and serializer - the foundation for our Redis clone.
Prerequisites: Go basics, understanding of TCP
Further Reading: Networking Fundamentals
Time: 2-3 hours
Outcome: Working RESP parser and writer

​
What is RESP?

RESP is a simple, efficient text-based protocol. Every piece of data starts with a type indicator: 
┌─────────────────────────────────────────────────────────────────────────────┐
│                        RESP DATA TYPES                                       │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│   TYPE          PREFIX    EXAMPLE                  MEANING                   │
│   ────          ──────    ───────                  ───────                   │
│                                                                              │
│   Simple String   +      +OK\r\n                   "OK"                      │
│   Error           -      -ERR unknown\r\n          Error message             │
│   Integer         :      :1000\r\n                 1000                      │
│   Bulk String     $      $5\r\nhello\r\n           "hello" (length prefix)   │
│   Array           *      *2\r\n$3\r\nfoo\r\n       ["foo", "bar"]            │
│                          $3\r\nbar\r\n                                        │
│   Null            $      $-1\r\n                   nil (null bulk string)    │
│                                                                              │
│   ALL LINES END WITH \r\n (CRLF)                                            │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
Why RESP?
  • Human-readable for debugging
  • Simple to implement
  • Efficient to parse (O(n))
  • Supports binary-safe strings

​
Real Examples

​
Client Request (SET command)

*3\r\n        ← Array of 3 elements
$3\r\n        ← Bulk string, 3 bytes
SET\r\n       ← "SET"
$4\r\n        ← Bulk string, 4 bytes
name\r\n      ← "name"
$5\r\n        ← Bulk string, 5 bytes
Alice\r\n     ← "Alice"

​
Server Response

+OK\r\n       ← Simple string "OK"

​
GET Response with Value

$5\r\n        ← Bulk string, 5 bytes
Alice\r\n     ← "Alice"

​
GET Response for Missing Key

$-1\r\n       ← Null bulk string

​
Project Setup

mkdir myredis
cd myredis
go mod init myredis

​
Project Structure

myredis/
├── cmd/
│   └── server/
│       └── main.go
├── internal/
│   └── protocol/
│       ├── types.go
│       ├── parser.go
│       └── writer.go
├── go.mod
└── README.md

​
Implementation

​
Step 1: Define RESP Types

internal/protocol/types.go
package protocol

import (
    "errors"
    "strconv"
)

// RESP type prefixes
const (
    SimpleString = '+'
    Error        = '-'
    Integer      = ':'
    BulkString   = '$'
    Array        = '*'
)

// Common errors
var (
    ErrInvalidSyntax    = errors.New("invalid RESP syntax")
    ErrUnexpectedEOF    = errors.New("unexpected end of input")
    ErrInvalidInteger   = errors.New("invalid integer")
    ErrInvalidBulkLen   = errors.New("invalid bulk string length")
    ErrInvalidArrayLen  = errors.New("invalid array length")
)

// Value represents any RESP value
type Value struct {
    Type  byte     // +, -, :, $, *
    Str   string   // For SimpleString and Error
    Num   int64    // For Integer
    Bulk  []byte   // For BulkString
    Array []Value  // For Array
    Null  bool     // True if null bulk string or null array
}

// Common responses as package-level variables for reuse
var (
    OK       = Value{Type: SimpleString, Str: "OK"}
    PONG     = Value{Type: SimpleString, Str: "PONG"}
    NullBulk = Value{Type: BulkString, Null: true}
    NullArray = Value{Type: Array, Null: true}
    Zero     = Value{Type: Integer, Num: 0}
    One      = Value{Type: Integer, Num: 1}
)

// NewError creates an error response
func NewError(msg string) Value {
    return Value{Type: Error, Str: msg}
}

// NewErrorf creates a formatted error response
func NewErrorf(format string, args ...interface{}) Value {
    return Value{Type: Error, Str: fmt.Sprintf(format, args...)}
}

// NewSimpleString creates a simple string response
func NewSimpleString(s string) Value {
    return Value{Type: SimpleString, Str: s}
}

// NewInteger creates an integer response
func NewInteger(n int64) Value {
    return Value{Type: Integer, Num: n}
}

// NewBulkString creates a bulk string response
func NewBulkString(s string) Value {
    return Value{Type: BulkString, Bulk: []byte(s)}
}

// NewBulkBytes creates a bulk string from bytes
func NewBulkBytes(b []byte) Value {
    return Value{Type: BulkString, Bulk: b}
}

// NewArray creates an array response
func NewArray(vals []Value) Value {
    return Value{Type: Array, Array: vals}
}

// IsNull returns true if this is a null value
func (v Value) IsNull() bool {
    return v.Null
}

// AsString returns the string representation of the value
func (v Value) AsString() string {
    switch v.Type {
    case SimpleString, Error:
        return v.Str
    case BulkString:
        return string(v.Bulk)
    case Integer:
        return strconv.FormatInt(v.Num, 10)
    default:
        return ""
    }
}

// AsBytes returns the value as bytes
func (v Value) AsBytes() []byte {
    switch v.Type {
    case SimpleString, Error:
        return []byte(v.Str)
    case BulkString:
        return v.Bulk
    default:
        return nil
    }
}

// AsInt returns the value as int64
func (v Value) AsInt() (int64, error) {
    switch v.Type {
    case Integer:
        return v.Num, nil
    case BulkString:
        return strconv.ParseInt(string(v.Bulk), 10, 64)
    case SimpleString:
        return strconv.ParseInt(v.Str, 10, 64)
    default:
        return 0, errors.New("cannot convert to integer")
    }
}

​
Step 2: Build the Parser

internal/protocol/parser.go
package protocol

import (
    "bufio"
    "fmt"
    "io"
    "strconv"
)

// Parser reads RESP values from a buffered reader
type Parser struct {
    reader *bufio.Reader
}

// NewParser creates a new RESP parser
func NewParser(r io.Reader) *Parser {
    return &Parser{
        reader: bufio.NewReaderSize(r, 64*1024), // 64KB buffer
    }
}

// Parse reads and returns the next RESP value
func (p *Parser) Parse() (Value, error) {
    // Read the type indicator
    typeByte, err := p.reader.ReadByte()
    if err != nil {
        return Value{}, err
    }

    switch typeByte {
    case SimpleString:
        return p.parseSimpleString()
    case Error:
        return p.parseError()
    case Integer:
        return p.parseInteger()
    case BulkString:
        return p.parseBulkString()
    case Array:
        return p.parseArray()
    default:
        // Handle inline commands (commands without RESP framing)
        p.reader.UnreadByte()
        return p.parseInline()
    }
}

// parseSimpleString reads: +OK\r\n
func (p *Parser) parseSimpleString() (Value, error) {
    line, err := p.readLine()
    if err != nil {
        return Value{}, err
    }
    return Value{Type: SimpleString, Str: line}, nil
}

// parseError reads: -ERR message\r\n
func (p *Parser) parseError() (Value, error) {
    line, err := p.readLine()
    if err != nil {
        return Value{}, err
    }
    return Value{Type: Error, Str: line}, nil
}

// parseInteger reads: :1000\r\n
func (p *Parser) parseInteger() (Value, error) {
    line, err := p.readLine()
    if err != nil {
        return Value{}, err
    }

    num, err := strconv.ParseInt(line, 10, 64)
    if err != nil {
        return Value{}, ErrInvalidInteger
    }

    return Value{Type: Integer, Num: num}, nil
}

// parseBulkString reads: $5\r\nhello\r\n or $-1\r\n (null)
func (p *Parser) parseBulkString() (Value, error) {
    // Read length line
    line, err := p.readLine()
    if err != nil {
        return Value{}, err
    }

    length, err := strconv.Atoi(line)
    if err != nil {
        return Value{}, ErrInvalidBulkLen
    }

    // Handle null bulk string
    if length < 0 {
        return Value{Type: BulkString, Null: true}, nil
    }

    // Read exactly 'length' bytes + \r\n
    buf := make([]byte, length+2)
    _, err = io.ReadFull(p.reader, buf)
    if err != nil {
        return Value{}, err
    }

    // Verify CRLF terminator
    if buf[length] != '\r' || buf[length+1] != '\n' {
        return Value{}, ErrInvalidSyntax
    }

    return Value{Type: BulkString, Bulk: buf[:length]}, nil
}

// parseArray reads: *3\r\n followed by 3 elements
func (p *Parser) parseArray() (Value, error) {
    // Read count line
    line, err := p.readLine()
    if err != nil {
        return Value{}, err
    }

    count, err := strconv.Atoi(line)
    if err != nil {
        return Value{}, ErrInvalidArrayLen
    }

    // Handle null array
    if count < 0 {
        return Value{Type: Array, Null: true}, nil
    }

    // Parse each element
    array := make([]Value, count)
    for i := 0; i < count; i++ {
        val, err := p.Parse()
        if err != nil {
            return Value{}, err
        }
        array[i] = val
    }

    return Value{Type: Array, Array: array}, nil
}

// parseInline handles commands sent without RESP framing
// e.g., "PING\r\n" or "SET foo bar\r\n"
func (p *Parser) parseInline() (Value, error) {
    line, err := p.readLine()
    if err != nil {
        return Value{}, err
    }

    // Split by spaces
    parts := splitInline(line)
    if len(parts) == 0 {
        return Value{}, ErrInvalidSyntax
    }

    // Convert to array of bulk strings
    array := make([]Value, len(parts))
    for i, part := range parts {
        array[i] = Value{Type: BulkString, Bulk: []byte(part)}
    }

    return Value{Type: Array, Array: array}, nil
}

// readLine reads until \r\n and returns the line without CRLF
func (p *Parser) readLine() (string, error) {
    line, err := p.reader.ReadString('\n')
    if err != nil {
        return "", err
    }

    // Remove \r\n
    if len(line) < 2 || line[len(line)-2] != '\r' {
        return "", ErrInvalidSyntax
    }

    return line[:len(line)-2], nil
}

// splitInline splits an inline command respecting quotes
func splitInline(line string) []string {
    var parts []string
    var current []byte
    inQuote := false
    quoteChar := byte(0)

    for i := 0; i < len(line); i++ {
        c := line[i]

        if !inQuote && (c == '"' || c == '\'') {
            inQuote = true
            quoteChar = c
        } else if inQuote && c == quoteChar {
            inQuote = false
        } else if !inQuote && c == ' ' {
            if len(current) > 0 {
                parts = append(parts, string(current))
                current = nil
            }
        } else {
            current = append(current, c)
        }
    }

    if len(current) > 0 {
        parts = append(parts, string(current))
    }

    return parts
}

​
Step 3: Build the Writer

internal/protocol/writer.go
package protocol

import (
    "bufio"
    "io"
    "strconv"
)

// Writer serializes RESP values to a buffered writer
type Writer struct {
    writer *bufio.Writer
}

// NewWriter creates a new RESP writer
func NewWriter(w io.Writer) *Writer {
    return &Writer{
        writer: bufio.NewWriter(w),
    }
}

// Write serializes a Value and writes it to the underlying writer
func (w *Writer) Write(v Value) error {
    if err := w.writeValue(v); err != nil {
        return err
    }
    return w.writer.Flush()
}

// WriteValues writes multiple values
func (w *Writer) WriteValues(vals ...Value) error {
    for _, v := range vals {
        if err := w.writeValue(v); err != nil {
            return err
        }
    }
    return w.writer.Flush()
}

func (w *Writer) writeValue(v Value) error {
    switch v.Type {
    case SimpleString:
        return w.writeSimpleString(v.Str)
    case Error:
        return w.writeError(v.Str)
    case Integer:
        return w.writeInteger(v.Num)
    case BulkString:
        return w.writeBulkString(v)
    case Array:
        return w.writeArray(v)
    default:
        return ErrInvalidSyntax
    }
}

func (w *Writer) writeSimpleString(s string) error {
    w.writer.WriteByte(SimpleString)
    w.writer.WriteString(s)
    w.writer.WriteString("\r\n")
    return nil
}

func (w *Writer) writeError(s string) error {
    w.writer.WriteByte(Error)
    w.writer.WriteString(s)
    w.writer.WriteString("\r\n")
    return nil
}

func (w *Writer) writeInteger(n int64) error {
    w.writer.WriteByte(Integer)
    w.writer.WriteString(strconv.FormatInt(n, 10))
    w.writer.WriteString("\r\n")
    return nil
}

func (w *Writer) writeBulkString(v Value) error {
    w.writer.WriteByte(BulkString)
    
    if v.Null {
        w.writer.WriteString("-1\r\n")
        return nil
    }
    
    w.writer.WriteString(strconv.Itoa(len(v.Bulk)))
    w.writer.WriteString("\r\n")
    w.writer.Write(v.Bulk)
    w.writer.WriteString("\r\n")
    return nil
}

func (w *Writer) writeArray(v Value) error {
    w.writer.WriteByte(Array)
    
    if v.Null {
        w.writer.WriteString("-1\r\n")
        return nil
    }
    
    w.writer.WriteString(strconv.Itoa(len(v.Array)))
    w.writer.WriteString("\r\n")
    
    for _, elem := range v.Array {
        if err := w.writeValue(elem); err != nil {
            return err
        }
    }
    
    return nil
}

// Flush ensures all buffered data is written
func (w *Writer) Flush() error {
    return w.writer.Flush()
}

​
Testing the Parser

internal/protocol/parser_test.go
package protocol

import (
    "strings"
    "testing"
)

func TestParseSimpleString(t *testing.T) {
    input := "+OK\r\n"
    parser := NewParser(strings.NewReader(input))
    
    val, err := parser.Parse()
    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    
    if val.Type != SimpleString || val.Str != "OK" {
        t.Errorf("expected SimpleString 'OK', got %v", val)
    }
}

func TestParseBulkString(t *testing.T) {
    input := "$5\r\nhello\r\n"
    parser := NewParser(strings.NewReader(input))
    
    val, err := parser.Parse()
    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    
    if val.Type != BulkString || string(val.Bulk) != "hello" {
        t.Errorf("expected BulkString 'hello', got %v", val)
    }
}

func TestParseArray(t *testing.T) {
    // *2\r\n$3\r\nfoo\r\n$3\r\nbar\r\n
    input := "*2\r\n$3\r\nfoo\r\n$3\r\nbar\r\n"
    parser := NewParser(strings.NewReader(input))
    
    val, err := parser.Parse()
    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    
    if val.Type != Array || len(val.Array) != 2 {
        t.Errorf("expected Array of 2 elements, got %v", val)
    }
    
    if string(val.Array[0].Bulk) != "foo" || string(val.Array[1].Bulk) != "bar" {
        t.Errorf("expected ['foo', 'bar'], got %v", val)
    }
}

func TestParseNullBulkString(t *testing.T) {
    input := "$-1\r\n"
    parser := NewParser(strings.NewReader(input))
    
    val, err := parser.Parse()
    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    
    if val.Type != BulkString || !val.Null {
        t.Errorf("expected null BulkString, got %v", val)
    }
}
Run tests: 
go test ./internal/protocol/...

​
Exercises

RESP3 (Redis 6+) adds more types:
,3.14\r\n              # Double
#t\r\n                 # Boolean true
#f\r\n                 # Boolean false
!21\r\n                # Blob error
SYNTAX error here\r\n
Extend your parser to handle these types.
Create benchmarks for your parser:
func BenchmarkParseBulkString(b *testing.B) {
    input := "$1000\r\n" + strings.Repeat("x", 1000) + "\r\n"
    
    for i := 0; i < b.N; i++ {
        parser := NewParser(strings.NewReader(input))
        parser.Parse()
    }
}
Compare with different buffer sizes.
Implement a channel-based parser that streams values:
func (p *Parser) ParseStream(ctx context.Context) <-chan Value {
    ch := make(chan Value)
    go func() {
        defer close(ch)
        for {
            val, err := p.Parse()
            if err != nil {
                return
            }
            select {
            case ch <- val:
            case <-ctx.Done():
                return
            }
        }
    }()
    return ch
}

​
Key Takeaways

Type Prefixes

Every RESP value starts with a type indicator: +, -, :, $, or *

Length-Prefixed

Bulk strings and arrays include their length for efficient parsing

Binary Safe

Bulk strings can contain any bytes, including nulls

CRLF Terminated

All lines end with \r\n for cross-platform compatibility

​
What’s Next?

In Chapter 2: TCP Server, we’ll:
  • Build a TCP server that accepts connections
  • Handle multiple clients concurrently with goroutines
  • Integrate our RESP parser

Next: TCP Server

Build the network layer for your Redis clone