flink窗口函数之增量聚合与全量聚合

  • Z时代
  • 2024-01-10
  • 分类：综合

编程

flink 窗口函数后会跟进聚集函数，聚合函数又分为增量聚合与全量聚合

增量聚合：本时间窗口范围内的数据聚合

全量聚合：本次时间窗口聚合结果+ 历史聚合结果 聚合后生成新的历史聚合结果

public class InterctiveReduceFunction implements ReduceFunction<Tuple2<String, Long>> {
    private static final Logger logger = Logger.getLogger(InterctiveReduceFunction.class);
    @Override
    public Tuple2<String, Long> reduce(Tuple2<String, Long> value1, Tuple2<String, Long> value2) throws Exception {
        value1.setFields(value1.f0, value1.f1 + value2.f1);
        return value1;
    }
}

public class InterctiveWindowFunction implements WindowFunction<Tuple2<String, Long>, Tuple3<Integer, String, Long>, String, TimeWindow> {
    private static final Logger logger = Logger.getLogger(InterctiveWindowFunction.class);
    public void apply(String key, TimeWindow timeWindow, Iterable<Tuple2<String, Long>> iterable, Collector<Tuple3<Integer, String, Long>> collector) throws Exception {
        long count = 0;
        for (Tuple2<String, Long> value : iterable) {
            count += value.f1;
        }
        collector.collect(Tuple3.of(DealMidInteractive.BhvType, key, count));
    }
}

 

第一个函数是 增量聚合，第二个是全量聚合

 

 

 

以上是 flink窗口函数之增量聚合与全量聚合 的全部内容， 来源链接： utcz.com/z/519194.html

回到顶部