Got it

How does deduplication work in the cloud?

Created: Jan 26, 2022 20:55:50Latest reply: Feb 19, 2022 08:11:31 280 8 1 0 0
  Rewarded HiCoins: 0 (problem resolved)

I want to know, how does deduplication work in the cloud?

Thanks

  • x
  • convention:

Featured Answers

Best answer

Recommended answer

olive.zhao
Admin Created Jan 27, 2022 00:57:44

Hello, friend!

Deduplication in cloud and other storage platforms is the process of removing duplicate or duplicate data from a data stream to reduce the amount of physical data stored on a device or system.

In primary storage, deduplication helps reduce the amount of physical space consumed by removing the same blocks of data and using metadata to correlate logical copies of the data with the actual data. In the public cloud, the deduplication capabilities of storage platforms are not exposed to users.

For example, this is how it works in FusionStorage.

deduplication

1. The account writes objects to bucket 1 and bucket 2. The contents of object A and object D are the same, and those of object C, object E, and object F are the same.

2. The system periodically scans objects in buckets and modifies the metadata mapping relationships of duplicate objects.

  • For object A and object D, the system maps the metadata (metadata D) of object D to object A.

  • For object C, object E, and object F, the system maps the metadata (metadata E and F) of object E and object F to object C.

3. The system deletes duplicate data.

  • Because object A and object D are identical, the system deletes data of object D. When a client sends a read request for object D, the system returns the data of object A in accordance with the mapping relationship of metadata D.

  • Because object C, object E, and object F are identical, the system deletes data of object E and object F. When a client sends a read request for object E or object F, the system returns the data of object C in accordance with the mapping relationship of metadata E or F.

If the account wants to delete objects, the deletion is implemented as follows:

  • When metadata A and D are mapped to object A and the account initiates a request to delete object D, the system will delete metadata D and the mapping relationship between metadata D and object A.

  • After object D is deleted, if the account initiates a request to delete object A, the system will delete both metadata and data of object A.

Hope this helps!

View more
  • x
  • convention:

S_Noch
S_Noch Created Jan 27, 2022 08:26:02 (0) (0)
great  
wissal
wissal Created Jan 27, 2022 08:39:34 (0) (0)
Thanks so much for your help  
chantha
chantha Created Feb 1, 2022 05:36:26 (0) (0)
 
All Answers
ariase88
ariase88 Admin Created Jan 26, 2022 20:56:39

Thanks for contacting the Huawei community!

We are checking your question and will provide an answer to you shortly...
View more
  • x
  • convention:

olive.zhao
olive.zhao Admin Created Jan 27, 2022 00:57:44

Hello, friend!

Deduplication in cloud and other storage platforms is the process of removing duplicate or duplicate data from a data stream to reduce the amount of physical data stored on a device or system.

In primary storage, deduplication helps reduce the amount of physical space consumed by removing the same blocks of data and using metadata to correlate logical copies of the data with the actual data. In the public cloud, the deduplication capabilities of storage platforms are not exposed to users.

For example, this is how it works in FusionStorage.

deduplication

1. The account writes objects to bucket 1 and bucket 2. The contents of object A and object D are the same, and those of object C, object E, and object F are the same.

2. The system periodically scans objects in buckets and modifies the metadata mapping relationships of duplicate objects.

  • For object A and object D, the system maps the metadata (metadata D) of object D to object A.

  • For object C, object E, and object F, the system maps the metadata (metadata E and F) of object E and object F to object C.

3. The system deletes duplicate data.

  • Because object A and object D are identical, the system deletes data of object D. When a client sends a read request for object D, the system returns the data of object A in accordance with the mapping relationship of metadata D.

  • Because object C, object E, and object F are identical, the system deletes data of object E and object F. When a client sends a read request for object E or object F, the system returns the data of object C in accordance with the mapping relationship of metadata E or F.

If the account wants to delete objects, the deletion is implemented as follows:

  • When metadata A and D are mapped to object A and the account initiates a request to delete object D, the system will delete metadata D and the mapping relationship between metadata D and object A.

  • After object D is deleted, if the account initiates a request to delete object A, the system will delete both metadata and data of object A.

Hope this helps!

View more
  • x
  • convention:

S_Noch
S_Noch Created Jan 27, 2022 08:26:02 (0) (0)
great  
wissal
wissal Created Jan 27, 2022 08:39:34 (0) (0)
Thanks so much for your help  
chantha
chantha Created Feb 1, 2022 05:36:26 (0) (0)
 
S_Noch
S_Noch Created Jan 27, 2022 08:26:10

How does deduplication work in the cloud?-4632101-1
View more
  • x
  • convention:

Saqibaz
Saqibaz Created Jan 28, 2022 10:32:36

Thanks for sharing
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 19, 2022 08:11:31

Thanks Olive for sharing this key information.
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.