Got it

How to accessing the Spark UI

Created: May 25, 2022 07:28:42Latest reply: May 25, 2022 07:37:17 185 1 0 0 0
  Rewarded HiCoins: 0 (problem resolved)

Hello, friend!

I use the Huawei cloud stack, and I want to access the Spark UI, can you help me?

Thanks in advance!

  • x
  • convention:

Featured Answers
olive.zhao
Admin Created May 25, 2022 07:37:17

Hello, friend!

The Spark web UI is used to view the running status of Spark applications. Google Chrome is recommended for better user experience.

Spark has two web UIs.

  • Spark UI: used to display the status of running applications.

    The UI includes the following parts: Jobs, Stages, Storage, Environment, Executors, SQL, and JDBC/ODBC Server. The Streaming application has the Streaming tab in addition to the preceding parts.

  • History Server UI: used to display the status of Spark applications that are complete or incomplete.

    The UI includes the application ID, application name, start time, end time, execution time, and owner information.

Spark UI

1. Access the component management page.

  • For versions earlier than MRS 3.x, click the cluster name to go to the cluster details page and choose Components.

  • For MRS 3.x or later, log in to FusionInsight Manager. Choose Cluster > Name of the desired cluster Services.

2. Select Yarn. In the Yarn Summary area, click ResourceManager in ResourceManager Web UI to access the web UI.

Figure 1 ResourceManager web UI

yarn ui

3. Locate the Spark application. Click ApplicationMaster in the last column of the application information. The Spark UI is displayed.

Figure 2 ApplicationMaster

ApplicationMaster

Figure 3 Spark UI

spark UI


History Server

1. Access the component management page.

  • For versions earlier than MRS 3.x, click the cluster name to go to the cluster details page and choose Components.

  • For MRS 3.x or later, log in to FusionInsight Manager. Choose Cluster Name of the desired cluster > Services.

2. Select Spark. In the Spark Summary area, click JobHistory corresponding to Spark Web UI to access the web UI.

Figure 4 ResourceManager web UI

ResourceManager web UI

Figure 5 Spark History Server

Spark History Server

Hope this helps!

View more
  • x
  • convention:

All Answers
olive.zhao
olive.zhao Admin Created May 25, 2022 07:37:17

Hello, friend!

The Spark web UI is used to view the running status of Spark applications. Google Chrome is recommended for better user experience.

Spark has two web UIs.

  • Spark UI: used to display the status of running applications.

    The UI includes the following parts: Jobs, Stages, Storage, Environment, Executors, SQL, and JDBC/ODBC Server. The Streaming application has the Streaming tab in addition to the preceding parts.

  • History Server UI: used to display the status of Spark applications that are complete or incomplete.

    The UI includes the application ID, application name, start time, end time, execution time, and owner information.

Spark UI

1. Access the component management page.

  • For versions earlier than MRS 3.x, click the cluster name to go to the cluster details page and choose Components.

  • For MRS 3.x or later, log in to FusionInsight Manager. Choose Cluster > Name of the desired cluster Services.

2. Select Yarn. In the Yarn Summary area, click ResourceManager in ResourceManager Web UI to access the web UI.

Figure 1 ResourceManager web UI

yarn ui

3. Locate the Spark application. Click ApplicationMaster in the last column of the application information. The Spark UI is displayed.

Figure 2 ApplicationMaster

ApplicationMaster

Figure 3 Spark UI

spark UI


History Server

1. Access the component management page.

  • For versions earlier than MRS 3.x, click the cluster name to go to the cluster details page and choose Components.

  • For MRS 3.x or later, log in to FusionInsight Manager. Choose Cluster Name of the desired cluster > Services.

2. Select Spark. In the Spark Summary area, click JobHistory corresponding to Spark Web UI to access the web UI.

Figure 4 ResourceManager web UI

ResourceManager web UI

Figure 5 Spark History Server

Spark History Server

Hope this helps!

View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.