编程知识 cdmana.com

Flink1.11 + hive batch flow integrated digital warehouse

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":" Reading guide "},{"type":"text","text":": Flink from 1.9.0 Start offering and Hive Integrated features , With the iteration of several versions , In the latest Flink 1.11 in , And Hive The function of integration is further deepened , And began to try to stream computing scenarios with Hive Integration . This article is mainly shared in Flink 1.11 Middle docking Hive New features , And how to use Flink Yes Hive Digital warehouse for real-time transformation , In order to achieve the goal of batch flow integration . The main contents include :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flink And Hive Introduction to the background of integration "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flink 1.11 New features in "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" make Hive Batch flow integrated warehouse "}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"01 Flink And Hive Introduction to the background of integration "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Why do it Flink and Hive What about the integrated functions ? The original intention was that we wanted to dig Flink Ability in batch processing . as everyone knows ,Flink It's been a successful engine in stream computing , There are also a lot of users . stay Flink In the design concept of , Batch computing is a special case of stream processing . That means , If Flink Do well in stream computing , In fact, its architecture can also support batch computing scenarios . In the case of batch computing ,SQL It's a very important entry point . Because the students who do data analysis , They are more used to using SQL Development , Instead of writing DataStream perhaps DataSet Such a program ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}

版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢

Scroll to Top