Skip to main content

Windows & Time Operations

Module Duration: 4-5 hours Focus: Windowing in Apache Flink Prerequisites: DataStream API, Event Time concepts

Introduction

Windows are essential for bounded computations over unbounded streams. They divide the infinite stream into finite chunks based on time or count, enabling aggregations, joins, and pattern detection.

Window Types

Tumbling Windows

Non-overlapping, fixed-size windows.

Sliding Windows

Overlapping windows with fixed size and slide interval.

Session Windows

Dynamic windows based on activity gaps.

Global Windows

All elements go to a single window (requires custom trigger).

Count Windows

Windows based on element count instead of time.

Window Functions

ReduceFunction

Incrementally combines elements.

AggregateFunction

More efficient than reduce, with separate accumulator.

ProcessWindowFunction

Access to all window elements and metadata.

Combining Incremental and Full-Window Processing

Triggers

Control when windows fire.

Evictors

Remove elements from window before/after function application.

Allowed Lateness

Real-World Examples

Example 1: Real-Time Analytics Dashboard

Example 2: User Session Analysis

Example 3: Anomaly Detection with Windows

Window Joins

Tumbling Window Join

Interval Join

CoGroup

More flexible than join - access to all elements from both sides.

Performance Optimization

1. Choose Right Window Function

2. Window Size Tuning

3. State Management

Best Practices

1. Always Handle Late Data

2. Use Appropriate Watermark Strategy

3. Monitor Window Latency

Exercises

Exercise 1: Implement Session Timeout Detection

Detect when users have been inactive for too long.

Exercise 2: Top-N per Window

Find top N items in each window.

Summary

In this module, you learned:
  • Window types: tumbling, sliding, session, global, count
  • Window functions: reduce, aggregate, process
  • Triggers for custom window firing logic
  • Evictors for element removal
  • Handling late data with allowed lateness
  • Real-world examples: analytics, sessions, anomaly detection
  • Window joins and CoGroup
  • Performance optimization techniques
  • Production best practices

Next Steps

Module 6: Table API & Flink SQL

Learn declarative stream processing with SQL

Additional Resources