Failed to Start Spark During the Cluster Installation

8

During the cluster installation, the system displays a message indicating that the Spark service fails to be started.Procedure:1.Use PuTTY to log in to each node as user root.
2.Run the following commands to go to /usr/lib64/ and check whether the libhadoop.so file exists:cd /usr/lib64,find libhadoop*.If the following information is displayed, the file exists:libhadoop.so ,libhadoop.so.1.0.0
3.Run the following command to delete the file:rm -f libhadoop.so.1.0.0,rm -f libhadoop.so
4.Restart the Spark service. The fault is rectified.

Other related questions:
What can I do if the embedded GK fails to start after the SMC2.0 is installed?
Since C02, the authentication password of the embedded GK is changed. The default password is changed from 123456 to Change_Me. You need to open the gkcfg.ini file in D:\Huawei Technologies Co., Ltd\Switch System\SwitchCentre, set GKM-AUTH-PWD to Change_Me, and restart the SC.

Advantages of Spark
1. Spark improves the data processing capability by as much as 10 to 100 times over the capabilities of MapReduce. It does this by using distributed memory computing and a Directed Acyclic Graph (DAG) engine. 2. Spark supports multiple development languages including Scala, Java, and Python. It supports dozens of highly abstract operators. This flexibility facilitates the construction of distributed data processing applications. 3. Spark provides one-stop data processing capability by working with SQL, Streaming, MLlib, and GraphX to form data processing stacks. 4. Spark can run in standalone, Mesos, or Yarn mode. It can access HDFS, HBase, and Hive data sources. It supports smooth swift from MapReduce. All of these functions allow Spark to easily fit into the Hadoop ecosystem.

The AP fails to start
Perform the following operations to handle AP start failure: 1. Observe the status of the AP power indicator or the information output through the serial port to determine whether the AP is not powered on or the AP cannot start after power-on. - If the AP can be powered on but cannot be logged in, contact Huawei technical support. - If the AP cannot be powered on, go to step 2. 2. Check the AP's power supply mode. - If the DC power adapter is used to supply power to the AP, check the source of the power adapter. - If the power adapter is supplied with the AP, go to step 3. - If the power adapter is purchased from another vendor, check whether its rated output voltage and current meet the requirements for Huawei AP products. If not, replace the power adapter. It is recommended that you use the power adapter supplied with the AP. - If PoE is used to supply power to the AP, see "Why Does an AP's PoE Fail to Be Powered On" in Huawei support website. - For AC power supply, use a multimeter to check whether the cable connection is normal and whether the output voltage of the power supply meets AP requirements. You can also perform a cross test using other AC power supply devices. If the output voltage of the power supply does not meet AP requirements, contact Huawei technical support. 3. Perform a cross test on the AP using the DC power adapter. - If the AP cannot be powered on, contact Huawei technical support. - If the DC power adapter is faulty, replace it.

The AC fails to start
Two scenarios for AC start failure are as follows: - The AC can be powered on but cannot start, and the serial port does not display information. Rectify this fault by performing the following steps: 1. Check whether the power module functions properly. a. Measure the input voltage. Use a multimeter to measure the input voltage of the power module, and check whether the input voltage is in the operating voltage range of the power module. b. Remove and reinstall the power module. Unplug and plug the power module and its power cable to ensure that they are securely installed. c. Perform a cross test of the power module. Move the power module to another AC or replace it with another power module that can work normally in another AC, to determine whether the power module or the AC itself is faulty. If the test proves that the power module is faulty, send the power module back for repair. d. If the fault still occurs on the AC after the cross test, contact Huawei technical support. 2. Check whether there is obvious yellow mark on the interface. If the color of the connector turns yellowish, the fault may be caused by lightning strikes or surge current. The internal CPU chip may be damaged. In this case, dial Huawei customer service hotline and return the AC for repair (select the cause: F009 unexpected damage). Note: To protect the AC from lightning strikes or surge voltage, install a surge protector or a voltage stabilizer to ensure a stable running environment. 3. If there is no obvious mark, contact Huawei technical support. - The AC can be powered on and the serial port can display information, but the AC cannot start. a. If information can be displayed through the serial port, press Ctrl+B to enter the BootROM menu during the startup. Then, press CTRL+E to enter the test menu to check whether the SDRAM and flash memory are faulty. If so, dial Huawei customer service hotline and return the AC for repair. b. If not, dial Huawei customer service hotline or contact Huawei maintenance engineers to determine whether to upgrade the AC software or change other configurations. c. If the fault persists, contact Huawei technical support personnel.

Why OceanStor ReplicationDirector Fails to Start After Installed On Virtual Machines?
Set the memory and CPU resources allocated to the OceanStor ReplicationDirector virtual machine to fixed sizes instead of dynamic allocation. For details about how to set sizes of memory and CPU resources, see the OceanStor ReplicationDirector standard configuration. Restart the virtual machine and shut down all virtual machines that are not in use.

If you have more questions, you can seek help from following ways:
To iKnow To Live Chat
Scroll to top