RDD storage modes

4

1. Users can select different storage levels to cache an RDD for reuse. (There are 11 storage levels for storing an RDD.)
2. The current RDD is stored in the memory by default. When the memory is insufficient, the RDD overflows to the disk.

Other related questions:
What is the concept of RDD?
Resilient Distributed Dataset (RDD) is a core concept of Spark. It indicates a read-only distributed dataset that can be partitioned. Partial or all data of this dataset can be cached in the memory and reused in subsequent computations.

How is an RDD generated?
1. An RDD compatible with Hadoop can be created from the Hadoop file system or other storage systems that are compatible with Hadoop. HDFS is an example. 2. A new RDD can be converted from a parent RDD. 3. An RDD can be converted from a collection.

How is an RDD formed?
1. An RDD compatible with Hadoop can be created from the Hadoop file system or other storage systems that are compatible with Hadoop. HDFS is an example. 2. A new RDD can be converted from a parent RDD. 3. An RDD can be converted from a collection.

Generation process of RDDs
1. An RDD compatible with Hadoop can be created from the Hadoop file system or other storage systems that are compatible with Hadoop. HDFS is an example. 2. A new RDD can be converted from a parent RDD. 3. An RDD can be converted from a collection.

Method used to enter to the debug mode on the storage system
You can enter the debug mode on the storage system as follows: 1. Fault location and rectification a. Log in to the storage system, enter the debug mode, and enter cli_main 0 0 admin to check whether you can go to the admin mode. If you cannot enter the admin mode, check whether much Func OSP_SocketServerThread: Socket has TimeOut command,waiting return! information is displayed in the message log. b. If you can enter the admin mode, enter osp_usr in debug mode and check whether Func OSP_SocketServerThread: Socket has TimeOut command,waiting return! is displayed in the message log. c. Enter ps -aux | grep os_ to check whether the os_exeshell process exists. If the process does not exist, refer to the following solution. If the process exists, contact Huawei R&D engineers. 2. Solution If you cannot enter the admin mode by entering cli_mian 0 0 admin and information Func OSP_SocketServerThread: Socket has TimeOut command,waiting return! is displayed, go to the OSM/bin directory, run the os_exeshell & command and the osp_usr command, and then log out. Log in to the storage system again. You can enter the admin mode.

If you have more questions, you can seek help from following ways:
To iKnow To Live Chat
Scroll to top