> ## Documentation Index
> Fetch the complete documentation index at: https://resources.devweekends.com/llms.txt
> Use this file to discover all available pages before exploring further.

# 03. File System

> Learn how to read, write, and manage files using the Node.js fs module.

# File System Module (fs)

The `fs` module is one of the most useful built-in modules in Node.js. It allows you to work with the file system on your computer: reading files, creating files, updating files, deleting files, and renaming files.

To use it, you must first require it:

```javascript theme={null}
const fs = require('fs');
```

## Synchronous vs Asynchronous

Most methods in the `fs` module have both synchronous and asynchronous versions.

* **Asynchronous** methods take a callback function as the last argument. They are non-blocking.
* **Synchronous** methods block the execution until the operation completes. They usually end with `Sync`.

**Why this matters:** Imagine your server is a single checkout lane at a grocery store (the event loop). A synchronous file read is like one customer paying with a bag of pennies--every customer behind them waits. An asynchronous read is like taking the customer's number and calling them back when the transaction clears, so the line keeps moving.

**Recommendation:** Always use asynchronous methods in production to avoid blocking the event loop. Synchronous methods are acceptable only during startup (reading config files before the server begins accepting requests) or in CLI scripts where you are the only user.

### fs API Style Comparison

Node.js offers three ways to do file operations. Here is how they compare:

| Style           | API                          | Error handling                     | Best for                                    |
| --------------- | ---------------------------- | ---------------------------------- | ------------------------------------------- |
| **Callback**    | `fs.readFile(path, cb)`      | Error-first callback `(err, data)` | Legacy code, event-driven patterns          |
| **Synchronous** | `fs.readFileSync(path)`      | try/catch                          | Startup scripts, CLI tools, one-off scripts |
| **Promise**     | `fs.promises.readFile(path)` | async/await with try/catch         | Modern production code, anything async      |

**Decision framework:** Use `fs/promises` by default in any new code. Use synchronous methods only in two situations: (1) reading config at process startup before the server starts accepting connections, or (2) CLI scripts where blocking is acceptable. Use callback-style only when integrating with legacy callback-based code that you cannot refactor.

## Reading Files

### Asynchronous Read

```javascript theme={null}
// readFile loads the ENTIRE file into memory at once.
// For small config files this is fine; for a 2GB log file, this will crash your process.
// For large files, use streams instead (covered in Chapter 06).
fs.readFile('example.txt', 'utf8', (err, data) => {
  if (err) {
    console.error(err);
    return;  // Always return after error handling to prevent executing the rest
  }
  console.log(data);
});
```

Note: If you don't specify the encoding ('utf8'), you will get a raw Buffer object instead of a string. This is a common source of confusion--you try to log the contents and see `<Buffer 48 65 6c 6c 6f>` instead of readable text.

### Synchronous Read

```javascript theme={null}
try {
  const data = fs.readFileSync('example.txt', 'utf8');
  console.log(data);
} catch (err) {
  console.error(err);
}
```

## Writing Files

### Asynchronous Write

`fs.writeFile()` replaces the file and content if it exists. If the file doesn't exist, a new file, containing the specified content, will be created.

```javascript theme={null}
const content = 'Hello, this is new content!';

fs.writeFile('output.txt', content, err => {
  if (err) {
    console.error(err);
    return;
  }
  console.log('File written successfully');
});
```

### Appending to Files

To add content to the end of a file without replacing it, use `fs.appendFile()`.

```javascript theme={null}
fs.appendFile('output.txt', '\nThis is appended text.', err => {
  if (err) throw err;
  console.log('Content appended!');
});
```

## Directories

### Creating a Directory

```javascript theme={null}
if (!fs.existsSync('./new-folder')) {
  fs.mkdir('./new-folder', err => {
    if (err) throw err;
    console.log('Directory created');
  });
}
```

### Reading a Directory

```javascript theme={null}
fs.readdir('./', (err, files) => {
  if (err) console.log(err);
  else console.log(files); // Returns an array of filenames
});
```

### Removing Files and Directories

```javascript theme={null}
// Remove a file
fs.unlink('./output.txt', (err) => {
  if (err) throw err;
  console.log('File deleted');
});

// Remove an empty directory
fs.rmdir('./empty-folder', (err) => {
  if (err) throw err;
  console.log('Directory removed');
});

// Remove directory recursively (Node.js 14.14+)
fs.rm('./folder-with-files', { recursive: true, force: true }, (err) => {
  if (err) throw err;
  console.log('Directory removed recursively');
});
```

## Promise-based API (fs/promises)

Modern Node.js provides a promise-based API that works beautifully with async/await.

```javascript theme={null}
const fs = require('fs/promises');  // Note: import from 'fs/promises', not 'fs'
const path = require('path');

async function fileOperations() {
  try {
    // With fs/promises, every operation returns a Promise.
    // This lets us use async/await instead of nested callbacks,
    // making the code read top-to-bottom like synchronous code
    // while remaining non-blocking under the hood.
    
    // Read file
    const content = await fs.readFile('input.txt', 'utf8');
    console.log(content);

    // Write file
    await fs.writeFile('output.txt', 'New content');

    // Append to file
    await fs.appendFile('output.txt', '\nMore content');

    // Get file stats
    const stats = await fs.stat('output.txt');
    console.log('File size:', stats.size);
    console.log('Is file:', stats.isFile());
    console.log('Is directory:', stats.isDirectory());
    console.log('Created:', stats.birthtime);
    console.log('Modified:', stats.mtime);

    // Rename file
    await fs.rename('output.txt', 'renamed.txt');

    // Copy file
    await fs.copyFile('renamed.txt', 'backup.txt');

    // Create directory
    await fs.mkdir('./new-folder', { recursive: true });

    // Read directory
    const files = await fs.readdir('./');
    console.log('Files:', files);

  } catch (error) {
    console.error('Error:', error.message);
  }
}

fileOperations();
```

<Tip>
  Always use `fs/promises` with async/await for cleaner, more maintainable code in modern Node.js applications.
</Tip>

## Watching Files for Changes

```javascript theme={null}
const fs = require('fs');

// Watch a file for changes
fs.watch('config.json', (eventType, filename) => {
  console.log(`Event: ${eventType}, File: ${filename}`);
  if (eventType === 'change') {
    console.log('Config file was modified!');
    // Reload configuration
  }
});

// Watch a directory
fs.watch('./src', { recursive: true }, (eventType, filename) => {
  console.log(`${eventType}: ${filename}`);
});
```

## Working with JSON Files

```javascript theme={null}
const fs = require('fs/promises');

// Read and parse JSON
async function readJSON(filepath) {
  const data = await fs.readFile(filepath, 'utf8');
  return JSON.parse(data);
}

// Write JSON with formatting
async function writeJSON(filepath, data) {
  const json = JSON.stringify(data, null, 2); // Pretty print with 2 spaces
  await fs.writeFile(filepath, json);
}

// Usage
async function main() {
  // Read existing data
  const config = await readJSON('./config.json');
  console.log('Current config:', config);

  // Modify and save
  config.updatedAt = new Date().toISOString();
  await writeJSON('./config.json', config);
}
```

## Practical Example: File-based Logger

```javascript theme={null}
const fs = require('fs/promises');
const path = require('path');

class FileLogger {
  constructor(logDir = './logs') {
    this.logDir = logDir;
    this.init();
  }

  async init() {
    // Ensure log directory exists
    await fs.mkdir(this.logDir, { recursive: true });
  }

  getLogFileName() {
    const date = new Date().toISOString().split('T')[0];
    return path.join(this.logDir, `app-${date}.log`);
  }

  formatMessage(level, message) {
    const timestamp = new Date().toISOString();
    return `[${timestamp}] [${level.toUpperCase()}] ${message}\n`;
  }

  async log(level, message) {
    const logFile = this.getLogFileName();
    const formatted = this.formatMessage(level, message);
    await fs.appendFile(logFile, formatted);
  }

  async info(message) {
    await this.log('info', message);
  }

  async error(message) {
    await this.log('error', message);
  }

  async warn(message) {
    await this.log('warn', message);
  }

  // Get recent logs
  async getRecentLogs(lines = 50) {
    const logFile = this.getLogFileName();
    try {
      const content = await fs.readFile(logFile, 'utf8');
      const logLines = content.trim().split('\n');
      return logLines.slice(-lines);
    } catch (err) {
      return [];
    }
  }

  // Clean old logs
  async cleanOldLogs(daysToKeep = 30) {
    const files = await fs.readdir(this.logDir);
    const cutoffDate = new Date();
    cutoffDate.setDate(cutoffDate.getDate() - daysToKeep);

    for (const file of files) {
      const filePath = path.join(this.logDir, file);
      const stats = await fs.stat(filePath);
      
      if (stats.mtime < cutoffDate) {
        await fs.unlink(filePath);
        console.log(`Deleted old log: ${file}`);
      }
    }
  }
}

module.exports = FileLogger;
```

## Performance: readFile vs Streams vs readline

Choosing the right file-reading approach depends on file size and what you need to do with the data:

| Approach                | Memory usage                    | Speed                                        | Use when                                                         |
| ----------------------- | ------------------------------- | -------------------------------------------- | ---------------------------------------------------------------- |
| `fs.readFile()`         | Entire file in RAM              | Fast for small files                         | File is under \~50MB, you need it all at once                    |
| `fs.createReadStream()` | \~64KB buffer (configurable)    | Constant memory regardless of file size      | File is large or you are processing chunks (compression, upload) |
| `readline` interface    | One line in memory at a time    | Slightly slower due to line parsing overhead | Processing text line-by-line (log parsing, CSV)                  |
| `fs.read()` (low-level) | You control buffer size exactly | Maximum control, most code                   | Binary file formats, random access, seek operations              |

**Benchmark intuition (rough numbers on modern hardware):**

* `readFileSync` on a 1MB file: \~2ms
* `readFileSync` on a 100MB file: \~80ms (and blocks the event loop the entire time)
* `createReadStream` on a 100MB file: \~80ms total, but the event loop stays responsive throughout
* `readFileSync` on a 1GB file: \~800ms of blocked event loop -- every request to your server waits

**Edge case -- reading the same file concurrently:** If 100 requests all call `readFile` on the same 10MB file simultaneously, you will use \~1GB of RAM (100 copies of the file). Node.js does not deduplicate concurrent reads of the same file. For frequently-accessed files, read once at startup and cache the result, or use a stream that pipes directly to the response.

## Common Pitfalls with File Operations

<Warning>
  **Race conditions with fs.existsSync:** A common anti-pattern is checking if a file exists and then operating on it. Between your check and your operation, another process could delete or modify the file. Instead, just perform the operation and handle the error:

  ```javascript theme={null}
  // Anti-pattern -- race condition between check and operation
  if (fs.existsSync('file.txt')) {
    fs.unlinkSync('file.txt');  // File might be gone by now!
  }

  // Better -- just try and handle the error
  try {
    await fs.unlink('file.txt');
  } catch (err) {
    if (err.code !== 'ENOENT') throw err;  // Ignore "file not found", rethrow others
  }
  ```
</Warning>

## Common fs Error Codes

When file operations fail, the error object includes a `code` property. Knowing these codes lets you handle failures precisely:

| Error code | Meaning                   | Typical cause                                                | How to handle                                              |
| ---------- | ------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------- |
| `ENOENT`   | No such file or directory | Path is wrong, file was deleted                              | Create the file/directory, or return 404                   |
| `EACCES`   | Permission denied         | Process lacks read/write permissions                         | Check file permissions, run with correct user              |
| `EEXIST`   | File already exists       | Trying to create something that already exists               | Use `{ recursive: true }` for mkdir, or catch and continue |
| `EISDIR`   | Is a directory            | Tried to read/write a directory as a file                    | Check path, use readdir instead                            |
| `EMFILE`   | Too many open files       | Exhausted file descriptor limit                              | Use streams with backpressure, increase ulimit             |
| `ENOSPC`   | No space left on device   | Disk is full                                                 | Alert ops, clean up, use disk space monitoring             |
| `EPERM`    | Operation not permitted   | OS-level restriction (e.g., deleting a root file on Windows) | Run with elevated permissions or change approach           |

**Edge case -- EMFILE in production:** The default file descriptor limit on many Linux systems is 1024. A busy server that opens files for logging, uploads, and static assets can hit this limit. Symptoms: random "EMFILE: too many open files" errors that seem intermittent because they depend on how many files are open at that moment. Fix: increase the limit with `ulimit -n 65536` in your startup script, and ensure you always close file handles (streams auto-close on end/error, but manual `fs.open` calls need explicit `fs.close`).

## Summary

* Use **asynchronous methods** in production (avoid `Sync` methods except at startup)
* Prefer **`fs/promises`** with async/await for modern code--it eliminates callback nesting
* Use **`fs.watch()`** to monitor file/directory changes (but note it is platform-dependent and can fire duplicate events)
* **`fs.stat()`** provides file metadata (size, dates, type)
* Always handle errors--file not found (`ENOENT`), permission denied (`EACCES`), and disk full (`ENOSPC`) are the most common
* For large files, use **Streams** instead of `readFile` (covered in detail in Chapter 06)
* Use `{ recursive: true }` with `fs.mkdir` to safely create nested directories without checking existence first