Introduction
This section describes the definition, purposes, and benefits of the DST feature.
Definition
DST is a technology that automatically migrates data among multiple storage tiers based on data access frequency. The frequently accessed data is dynamically migrated to high-performance storage tiers, and infrequently accessed data is dynamically migrated to low-performance storage tiers. This technology optimizes the overall storage performance for applications.
Purposes
-
Optimizing storage performance
Infrequently accessed data is automatically migrated to a low-speed storage array to release space of a high-speed storage array. This ensures fast access to hotspot data, improving the system's storage performance.
-
Reducing the total cost of storage
Infrequently accessed data is stored on a storage array of a low cost and low performance, leveraging cost advantages of storage arrays that offer different performance levels.
-
Enhancing data availability
Data is stored on storage arrays that offer different performance levels. The system gives priority to data accesses that require high I/O performance. This improves the storage system's data access mode, accelerates the process of saving data, and enhances availability of hotspot data.
-
Implementing automatic data migration
Based on factors such as data access frequency and data retention period, a user can determine the optimal storage policy to control data migration rules, eliminating costly and complex manual management.
Benefits
Table 1 describes the benefits provided by the DST feature for customers.
Table 1 Benefits of the DST feature
|
Item |
Benefit |
|---|---|
|
Data access mode |
Data is stored on storage arrays that offer different performance levels. The system gives priority to data accesses that require high I/O performance, improving the storage system's data access mode. |
|
Load balancing |
Frequently accessed data and infrequently accessed data are stored on storage arrays that offer different performance levels, preventing system resource contention caused by concurrent data accesses. |
|
Storage cost |
There are two ways to decrease the total cost of storage.
|
|
Automatic migration |
Based on factors such as data access frequency and data retention period, a user can determine the optimal storage policy to control data migration rules. |
Specifications
This section describes the specifications of the DST feature.
Table 1 describes the specifications of the DST feature.
Table 1 Specifications of the DST feature
|
Parameter |
Description |
|---|---|
|
Maximum file system capacity |
255 TB |
|
Maximum number of files |
|
|
Maximum number of LUNsa supported by a storage unit |
|
|
Maximum number of tiers |
2 |
|
a: LUN is short for Logical Unit Number. |
|
Availability
This section describes how to obtain the DST feature.
License Requirement
To use the DST feature, you need to purchase a license separately.
Applicable Versions
|
Product |
Applicable Version |
|---|---|
|
OceanStor S2600T/S5500T/S5600T/S5800T |
V100R005 |
Principle Description
This section describes the implementation principle of the DST feature.
Basic Concepts
-
Virtual storage volume
Virtual storage volumes are virtual disks that represent the range of addressable disk blocks used by a file system. A collection of volumes is called a volume set.
-
Single-volume file system
A single-volume file system is a file system created on a separate virtual storage volume.
-
Multi-volume file system
On the storage system, a multi-volume file system is a file system that occupies two or more virtual storage volumes, that is, a volume set. A multi-volume file system provides a single namespace that makes volumes transparent to applications and users.
-
Placement level
To facilitate management and control of the storage location of a single file, each volume of the file system on the storage system has a unique volume label, namely, placement level. A placement level is a DST property of a specific volume in a volume set of the multi-volume file system. The property represents a storage tier, namely, tier-1 storage or tier-2 storage. On the storage system, you can specify the initial storage tier to determine whether data written to the file system is by default stored on tier-1 storage or tier-2 storage.
As shown in Figure 1, a volume set of a multi-volume file system contains two placement levels, tier 1 and tier 2, where tier1 represents tier-1 storage created on a high-speed disk array (such as SSDs, Fibre Channel disks, and SAS disks) for storing hotspot data, and tier2 represents tier-2 storage typically created on a low-performance disk array (such as SATA disks) for storing non-hotspot data.
-
DST policy
A DST policy includes a file placement policy and a DST schedule.
- A file placement policy is a file placement rule defined by a user on the storage system. It determines the condition of file migration and placement. Based on a file placement policy, files can be stored on and migrated between any volumes of a volume set. A file placement policy specifies the initial storage position of a file and defines a file migration rule based on the access time limit, average access frequency, and access period.
- A DST schedule is defined by a user for carrying out a file placement policy automatically.
-
Access time limit
A file's access time limit is the interval between the time when a file placement policy is forcibly carried out and the time when an application program accesses the file last time. The access time limit is expressed in days.
-
Access period
A file's access period is the interval at which the file is accessed by an application program. The access period is expressed in days.
-
Average access frequency
A file's average access frequency is the frequency at which a file is accessed within the specified period. On the storage system, the average access frequency is the number of file read or write requests handled within the specified period divided by the number of specified period.
-
File change log
A file change log (FCL) records information about the changes made to a file, such as creating, deleting, and extending a file and the amount of a file's I/O activities (number of bytes being read or written and number of I/O reads or writes).
Working Principle
The working principle of DST is based on access to partial data. Infrequently accessed data is automatically migrated to a low-level storage tier so that high-cost storage space is released for frequently accessed data, achieving better cost-effectiveness.
- As indicated by 1 in Figure 2, after tier-2 storage is configured for tier-1 storage of the storage system, the single-volume file system is automatically upgraded to a multi-volume file system that includes tier-1 storage (typically created on a high-performance storage array equipped with SSDs, Fibre Channel disks, or/and SAS disks) and tier-2 storage (typically created on a low-performance storage array equipped with SATA disks).
- When a client accesses a shared file on the storage system through a file access protocol such as NFS and CIFS, the FCL on the storage system automatically records the amount of I/O activities. As indicated by 2 in Figure 2, when the file placement policy is carried out manually or automatically, the storage system migrates data that is not accessed for a long time to low-performance tier-2 storage according to the file placement policy.
- As indicated by 3 in Figure 2, when data on tier-2 storage becomes hotspot data again, the storage system migrates the data to tier-1 storage according to the DST policy.
Data migrations are transparent to applications and users. That is, regardless of whether data is migrated to tier-1 storage or tier-2 storage, all files are a part of the same namespace. These files are accessed and processed as if they occupy a single volume without affecting users' viewing the directory structure of the source file system. In addition, there is not function or latency effect. For example, when an application program accesses a migrated file, the latency in accessing the first byte does not occur.
Figure 2 Working principle of the DST feature
http://localhost:7890/pages/3118G2D8/07/3118G2D8/07/resources/dita/nas/DST/figure/dstfea/os_dstfea_prcp_fig02.pngProcessing Procedure
-
-
A single-volume file system is upgraded to a multi-volume file system.
The file system created on the storage system is a single-volume file system. As shown in Figure 3, after tier-2 storage is created for the file system, the storage system automatically creates a volume set, and the original single-volume file system becomes a part of the volume set and its placement level is tier1. Data disks allocated to the volume set become a newly added part of the volume set and the placement level of this part is tier2.
Figure 3 Upgrading a single-volume file system to a multi-volume file system
http://localhost:7890/pages/3118G2D8/07/3118G2D8/07/resources/dita/nas/DST/figure/dstfea/os_dstfea_prcp_fig03.png -
A file placement policy is set.
When the storage system has the DST feature enabled, the storage position of a file changes depending on its access frequency. On a storage system, data on tier-1 storage can be migrated to tier-2 storage only when the data's access time limit exceeds the specified access time limit; data on tier-2 storage can be migrated to tier-1 storage only when the data's access frequency exceeds the specified average access frequency within the specified access period.
-
Based on the access time limit, non-hotspot files are migrated from tier-1 storage to tier-2 storage.
A file's access time limit is the interval between the time when a file placement policy is forcibly carried out and the time when an application program accesses the file last time. The access time limit is expressed in days. In the example shown in Figure 3, the specified access time limit is three days. If a file on tier-1 storage of the file system is never accessed within three days, the file is migrated to tier-2 storage after the file placement policy is carried out.
-
Based on the average access frequency and access period, hotspot files are migrated from tier-2 storage to tier-1 storage.
A file's access frequency is the number of file read or write requests handled within the specified period divided by the number of specified period. In the example shown in Figure 3, the specified access frequency is five within two days. When the policy is forcibly carried out, the files whose data was read or written completely for more than 10 times in the past two days are migrated from tier-2 storage to tier-1 storage because the files' access frequency exceeds 5 within two days.
Information about the access time limit and access frequency is recorded in an FCL. Based on an FCL, the storage system can accurately calculate or collect statistics about the access time limit and access frequency, ensuring that files are migrated according to user-defined placement policy.
-
Based on the access time limit, non-hotspot files are migrated from tier-1 storage to tier-2 storage.
-
-
Files are migrated according to the file placement policy.
As shown in Figure 4, after the file placement policy is carried out, non-hotspot files File3, File5, and File6 that are not accessed within three days are automatically migrated to tier-2 storage.
Figure 4 Migrating files according to the file placement policy
http://localhost:7890/pages/3118G2D8/07/3118G2D8/07/resources/dita/nas/DST/figure/dstfea/os_dstfea_prcp_fig04.pngWhen the file placement policy is carried out the next time, non-hotspot files that are accessed more than 10 times within the latest two days are automatically migrated to tier-1 storage because they meet the required average access frequency.
