Got it

Application Development of Flink

Latest reply: Mar 14, 2022 12:30:53 677 6 5 0 0

Flink is a unified computing framework that supports both batch processing and stream processing. It provides a stream data processing engine that supports data distribution and parallel computing. Flink features stream processing and is a top open-source stream processing engine in the industry.

Flink provides high-concurrency pipeline data processing, millisecond-level latency, and high reliability, making it suitable for low-latency data processing.


This pic shows the technology stack of Flink.

Flink


The following lists the key features of Flink in the current version:

  • DataStream

  • Checkpoint

  • Window

  • Job Pipeline

  • Configuration Table

For details about other Flink features, see https://ci.apache.org/projects/flink/flink-docs-stable/ .


Flink architecture:

architecture of Flink

  • Client

    Flink client is used to submit jobs (streaming jobs) to Flink.

  • TaskManager

    TaskManager (also called worker) is a service execution node of Flink. It executes specific tasks. A Flink system could have multiple TaskManagers. These TaskManagers are equivalent to each other.

  • JobManager

    JobManager (also called master) is a management node of Flink. It manages all TaskManagers and schedules tasks submitted by users to specific TaskManagers. In high-availability (HA) mode, multiple JobManagers are deployed. Among these JobManagers, one of which is selected as the leader, and the others are standby.

Flink provides the following features:

  • Low latency

    Millisecond-level processing capability.

  • Exactly once

    Asynchronous snapshot mechanism, ensuring that all data is processed only once.

  • High availability

    Leader/Standby JobManagers, preventing single point of failure (SPOF).

  • Scale out

    Manual scale out supported by TaskManagers.


See more: https://forum.huawei.com/enterprise/en/online-courses-flink-application-development/thread/438885-899


  • x
  • convention:

MahMush
Moderator Author Created Jan 25, 2022 05:11:01

great content
View more
  • x
  • convention:

Anno7
Moderator Author Created Jan 25, 2022 05:18:48

interesting
View more
  • x
  • convention:

user_4358465
Created Feb 19, 2022 08:24:58

Well explained post. Thank you for sharing!
View more
  • x
  • convention:

little_fish
little_fish Created Mar 14, 2022 00:36:18 (0) (0)
 
Saqibaz
Created Mar 13, 2022 04:45:04

Interesting
View more
  • x
  • convention:

SamB
Moderator Created Mar 14, 2022 12:30:53

Thanks for sharing
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.