Flink Sliding Window Deep Dive: Principles, Use Cases & I...

Sliding Window Core Concepts

Sliding window is a more generalized form of fixed window, achieving dynamic window movement through introducing slide interval. It consists of two key parameters:

• Window size: Determines the data range contained in each window
• Slide interval: Controls the movement frequency of the window

When slide interval is less than window size, windows will overlap. For example, setting 10-minute window + 5-minute slide interval, system computes data from past 10 minutes every 5 minutes.

Window Overlap Mechanism

• Sliding windows have overlapping parts, each event may be included in multiple windows
• Overlap degree is determined by window size and slide interval
• If window size is 5 minutes and slide interval is 1 minute, overlap part is 4 minutes

Typical Application Scenarios

• Moving averages (e.g., stock 5-day moving average)
• Active users statistics for recent period
• Real-time traffic monitoring
• Sensor data smoothing

Time-driven Implementation Example

WindowedStream<Tuple2<String, Integer>, Tuple, TimeWindow> timeWindow = keyedStream
    .timeWindow(Time.seconds(10), Time.seconds(5));
timeWindow.apply(new MyTimeWindowFunction()).print();

Event-driven Implementation Example

WindowedStream<Tuple2<String, Integer>, Tuple, GlobalWindow> globalWindow = keyedStream
    .countWindow(3, 2);
globalWindow.apply(new MyCountWindowFuntion()).print();

Session Window

Session window consists of a series of events combined with a specified time gap timeout, similar to Web application sessions. A new window is generated when no new data is received for a period of time.

WindowedStream<Tuple2<String, Integer>, Tuple, TimeWindow> window = keyedStream
    .window(ProcessingTimeSessionWindows.withGap(Time.seconds(10)));