Got it

The difference between checkpoint and savepoint

Created: May 6, 2022 09:25:33Latest reply: May 8, 2022 02:15:43 276 3 1 0 0
  HiCoins as reward: 0 (problem unresolved)

Hello, friend!

Do you know the describe the checkpoint and savepoint? 

What's the difference between checkpoint and savepoint?

Thanks in advance!

  • x
  • convention:

Featured Answers
olive.zhao
Admin Created May 6, 2022 09:34:45

Hello, Helen!

Have a nice day!

1. What is a checkpoint?

Apache Flink offers a lightweight fault tolerance mechanism based on distributed checkpoints. A checkpoint is an automatic, asynchronous snapshot of task/operator state. Flink generates checkpoint barriers at intervals on the input data set and uses barriers to divide the data during the interval into the corresponding checkpoints. When an application error occurs, the states of all operators can be restored from the previous snapshot to ensure data consistency.

For applications with small state, these snapshots are very light-weight and can be drawn frequently without impacting the performance much. During checkpointing, the state is stored at a configurable place (such as the JobManager node or HDFS).

2.  What is savepoint?

The checkpoint can be retained in an external medium when a job is cancelled. Flink also has savepoint, another mechanism to restore job data. 

Similar to checkpoints, savepoints allow saving state to external media. If a job fails, it can be restored from an external source. 

3. What are the differences between savepoints and checkpoints? 


Checkpoint

Checkpoint

Triggering and 

management

Automatically triggered and managed by Flink

Manually triggered and managed by users

Purpose

Quickly restores tasks from failures, for example, timeout due to network jitter

Backs up data as planned, for example, by modifying code or adjusting concurrency.

Features

  • Lightweight

  • Automatic recovery from failures

  • State is cleared by default after a job is stopped

  • Persistent

  • Stored in a standard format and allows code or configuration changes.

  • Manually restores data from savepoints.

Hope this helps!

View more
  • x
  • convention:

All Answers
olive.zhao
olive.zhao Admin Created May 6, 2022 09:34:45

Hello, Helen!

Have a nice day!

1. What is a checkpoint?

Apache Flink offers a lightweight fault tolerance mechanism based on distributed checkpoints. A checkpoint is an automatic, asynchronous snapshot of task/operator state. Flink generates checkpoint barriers at intervals on the input data set and uses barriers to divide the data during the interval into the corresponding checkpoints. When an application error occurs, the states of all operators can be restored from the previous snapshot to ensure data consistency.

For applications with small state, these snapshots are very light-weight and can be drawn frequently without impacting the performance much. During checkpointing, the state is stored at a configurable place (such as the JobManager node or HDFS).

2.  What is savepoint?

The checkpoint can be retained in an external medium when a job is cancelled. Flink also has savepoint, another mechanism to restore job data. 

Similar to checkpoints, savepoints allow saving state to external media. If a job fails, it can be restored from an external source. 

3. What are the differences between savepoints and checkpoints? 


Checkpoint

Checkpoint

Triggering and 

management

Automatically triggered and managed by Flink

Manually triggered and managed by users

Purpose

Quickly restores tasks from failures, for example, timeout due to network jitter

Backs up data as planned, for example, by modifying code or adjusting concurrency.

Features

  • Lightweight

  • Automatic recovery from failures

  • State is cleared by default after a job is stopped

  • Persistent

  • Stored in a standard format and allows code or configuration changes.

  • Manually restores data from savepoints.

Hope this helps!

View more
  • x
  • convention:

user_4147187
user_4147187 MVE Created May 8, 2022 02:15:43

Hello, here are some ideas for reference only.

Checkpoint: Checkpoint barriers are the same as normal records. They are handled by operators, but do not participate in computations; rather, they trigger checkpoint-related behavior. It flows between operators. When the data source reading the input stream encounters a checkpoint barrier, it stores its position in the input stream in a stable storage. If the input stream is from a message transmission system (Kafka or MapR Streams), this location is the offset.


Save points work like checkpoints, except that checkpoints are triggered automatically, and savepoints require command line or web console triggers. As with checkpoints, savepoints are saved in stable storage, and users can restart jobs from the savepoint without starting from scratch.


Hope to help you.


View more
  • x
  • convention:

olive.zhao
olive.zhao Created May 10, 2022 07:44:15 (0) (0)
Thanks for your sharing!  

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.