DataStream Api

source

1. 基于文件 env.readTextFile 逐行读取文件
2. 基于Socketenv.socketTextStream
3. 基于集合env.fromCollection(collection)
4. 基于Flink提供的连接器，kafka env.addSource(new FlinkKafkaConsumer<>(topic,deserializerScehma,props))
5. 自定义数据源，继承sourceFucntion/PararelleSourceFunction/RichSourceFunction重写open.run.close，cancel等方法

transformation

1. Map DataStream->DataStream
2. FlatMap DataStream->DataStream
3. Filter DataStream->DataStream
4. KeyBy DataStream->KeyedStream 按照指定的key进行逻辑分区，相同的key会被分发到相同的任务中
5. reduce KeyedStream->DataStream
6. fold 有初始值合并 KeyedStream->DataStream
7. aggredations sum等聚合操作
8. Window KeyedStream->WindowStream
9. Window All 所有记录都会汇集到同一个任务中
10. Window Apply/Reduce/Flod/ aggregations windowedStream->DataStream
11. Union DataStream->DataStream
12. WindowJoin
13. IntervalJoin
14. Window CoGroup,Connect,CoMap,CoFlatMap
15. Split DataStream->SplitDataStream 流分割
16. select SplitStream->DataStream
17. iterate DataStream->IterativeStream

sink

1. writeAsText()
2. print()
3. 自定义输出 继承RichSinkFunction,重写open.invoke,close方法

DataSource Api

source

1. 基于集合 fromCollection
2. 基于文件 readTextFile
3. 也可自定义

transformation

1. Map
2. FlatMap
3. MapPartition
4. Filter
5. reduce
6. ReduceGroup
7. Aggregate
8. Distinct
9. Join
10. OuterJoin
11. CoGroup
12. Cross
13. Union
14. Reblance
15. Hash-Partition
16. Range-Partition
17. CustomPartitioning
18. First-N
19. partitionByHash
20. sortPartition

sink

writeAsText(),writeAsCSV,print()等

Flink Table Api&SQL API

1.概述：Table API 和SQL实现了流批统一
2.table API编程
获取table执行环境

StreamTableEnvironment tenv = StreamTableEnvironment.create(env)

将流式数据转换成table

Table table = tenv.fromDataStream(data,$("name"),$("age"))

进行查询

table.select($("name"))