How to understand the EC rules
- x
- convention：
- Emotion(0)
|
Hello! The erasure code technology uses the erasure code algorithm to encode the original data for redundancy and store the data and redundancy to achieve fault tolerance. The basic idea is that the original data elements of the n block are calculated to obtain m redundant elements (check blocks). For an element of the n+m block, when any m block element error (including original data and redundant data) occurs, the original n block data may be recovered by using a corresponding reconstruction algorithm. The process of generating the checksum is encoded as (encoding), and the process of restoring the lost data block is called decoding (decoding). The disk usage is n/(n+m). Compared with the multi-copy method, the method based on erasure code has the advantages of low redundancy and high disk utilization.
Working PrincipleUnlike the conventional RAID technology which implements redundant data calculation and data storage together, Erasure Code enables file systems to flexibly store data based on load balancing and reliability functions. Figure 1 shows the working principle of Erasure Code Figure 1 Working principle of Erasure Code  Protection Level Definition Disk Usage Min. Number of Nodes Alternative Protection Level (When Nodes Are Insufficient) N+1 Each data stripe contains N original data strips and one redundant data strip to ensure data integrity when one disk fails or one storage node fails. 66% to 95% 3 - N+2 Each data stripe contains N original data strips and two redundant data strips to ensure data integrity when two disks fail or two storage nodes fail. 66% to 89% 6 N+2:1 N+3 Each data stripe contains N original data strips and three redundant data strips to ensure data integrity when three disks fail or three storage nodes fail. 66% to 84% 9 N+3:1 N+4 Each data stripe contains N original data strips and four redundant data strips to ensure data integrity when four disks fail or four storage nodes fail. 60% to 80% 10 N+4:1 N+2:1 Each data stripe contains N original data strips and two redundant data strips to ensure data integrity when two disks fail or one storage node fails. 66% to 90% 3 - N+3:1 Each data stripe contains N original data strips and three redundant data strips to ensure data integrity when three disks fail or one storage node fails. 57% to 84% 3 - Note 1: A protection level is expressed in the format of N+M or N+M:B, where N indicates the original data count (ODC), M indicates the redundant data count (RDC), and B indicates the number of redundant data nodes (RDNs). Note 2: N+1 is applicable to scenarios that have low reliability requirements and not recommended. If the value must be set to +1, contact technical support engineers to confirm risks in advance. If the system is configured with 3 nodes and the redundancy ratio is configured to +1, the system will automatically adjust the redundancy ratio to +2:1. After the subsequent system expansion, the redundancy will be inherited. Note 3: To prevent data loss caused by a dual-disk failure, ensure that M is not less than 2. Note 4: Disk usage keeps unchanged or improves as the number of nodes increases in a node pool. When a disk or node in a node pool fails, the reliability and security of data in other node pools are not affected. For details, see Physical Domain Division. Note 5: The cluster has multiple nodes that are disconnected. If the total number of nodes whose data has not been completely reconstructed and nodes that are disconnected is greater than the maximum number of faulty nodes allowed by the data protection level, some of data is unavailable. Note 6: For C72, the minimum number of nodes should be reduced to half as shown in the table. |
|
|
Hello! The erasure code technology uses the erasure code algorithm to encode the original data for redundancy and store the data and redundancy to achieve fault tolerance. The basic idea is that the original data elements of the n block are calculated to obtain m redundant elements (check blocks). For an element of the n+m block, when any m block element error (including original data and redundant data) occurs, the original n block data may be recovered by using a corresponding reconstruction algorithm. The process of generating the checksum is encoded as (encoding), and the process of restoring the lost data block is called decoding (decoding). The disk usage is n/(n+m). Compared with the multi-copy method, the method based on erasure code has the advantages of low redundancy and high disk utilization.
Working PrincipleUnlike the conventional RAID technology which implements redundant data calculation and data storage together, Erasure Code enables file systems to flexibly store data based on load balancing and reliability functions. Figure 1 shows the working principle of Erasure Code Figure 1 Working principle of Erasure Code Protection Level Definition Disk Usage Min. Number of Nodes Alternative Protection Level (When Nodes Are Insufficient) N+1 Each data stripe contains N original data strips and one redundant data strip to ensure data integrity when one disk fails or one storage node fails. 66% to 95% 3 - N+2 Each data stripe contains N original data strips and two redundant data strips to ensure data integrity when two disks fail or two storage nodes fail. 66% to 89% 6 N+2:1 N+3 Each data stripe contains N original data strips and three redundant data strips to ensure data integrity when three disks fail or three storage nodes fail. 66% to 84% 9 N+3:1 N+4 Each data stripe contains N original data strips and four redundant data strips to ensure data integrity when four disks fail or four storage nodes fail. 60% to 80% 10 N+4:1 N+2:1 Each data stripe contains N original data strips and two redundant data strips to ensure data integrity when two disks fail or one storage node fails. 66% to 90% 3 - N+3:1 Each data stripe contains N original data strips and three redundant data strips to ensure data integrity when three disks fail or one storage node fails. 57% to 84% 3 - Note 1: A protection level is expressed in the format of N+M or N+M:B, where N indicates the original data count (ODC), M indicates the redundant data count (RDC), and B indicates the number of redundant data nodes (RDNs). Note 2: N+1 is applicable to scenarios that have low reliability requirements and not recommended. If the value must be set to +1, contact technical support engineers to confirm risks in advance. If the system is configured with 3 nodes and the redundancy ratio is configured to +1, the system will automatically adjust the redundancy ratio to +2:1. After the subsequent system expansion, the redundancy will be inherited. Note 3: To prevent data loss caused by a dual-disk failure, ensure that M is not less than 2. Note 4: Disk usage keeps unchanged or improves as the number of nodes increases in a node pool. When a disk or node in a node pool fails, the reliability and security of data in other node pools are not affected. For details, see Physical Domain Division. Note 5: The cluster has multiple nodes that are disconnected. If the total number of nodes whose data has not been completely reconstructed and nodes that are disconnected is greater than the maximum number of faulty nodes allowed by the data protection level, some of data is unavailable. Note 6: For C72, the minimum number of nodes should be reduced to half as shown in the table. |
|
Comment
- Politically sensitive content
- Content concerning pornography, gambling, and drug abuse
- Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
