Got it
Huawei Enterprise Support Community
Community Forums Cloud & Big Data
Spark Error

Spark Error

Created: Jul 11, 2019 07:37:39Latest reply: Jul 11, 2019 08:01:34 1208 1 1 0 1
  Rewarded HiCoins： 0 (problem resolved)
View the author 1#

Hello, everyone!

I have an error about spark, can you help me?

This is the error information.


1

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
19/04/08 23:47:19 ERROR ContextCleaner: Error cleaning broadcast 11700946
 org.apache.spark.rpc.RpcTimeoutException: Futures timed out after [120 seconds]. This timeout is controlled by spark.rpc.askTimeout
   at org.apache.spark.rpc.RpcTimeout.org$apache$spark$rpc$RpcTimeout$$createRpcTimeoutException(RpcTimeout.scala:47)
   at org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:62)
   at org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:58)
   at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
   at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:76)
   at org.apache.spark.rpc.RpcEndpointRef.askSync(RpcEndpointRef.scala:92)
   at org.apache.spark.rpc.RpcEndpointRef.askSync(RpcEndpointRef.scala:76)
   at org.apache.spark.storage.BlockManagerMaster.removeBroadcast(BlockManagerMaster.scala:148)
   at org.apache.spark.broadcast.TorrentBroadcast$.unpersist(TorrentBroadcast.scala:321)
   at org.apache.spark.broadcast.TorrentBroadcastFactory.unbroadcast(TorrentBroadcastFactory.scala:45)
   at org.apache.spark.broadcast.BroadcastManager.unbroadcast(BroadcastManager.scala:66)
   at org.apache.spark.ContextCleaner.doCleanupBroadcast(ContextCleaner.scala:238)
   at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$1.apply(ContextCleaner.scala:194)
   at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$1.apply(ContextCleaner.scala:185)
   at scala.Option.foreach(Option.scala:257)
   at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:185)
   at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1302)
   at org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:178)
   at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:73)
 Caused by: java.util.concurrent.TimeoutException: Futures timed out after [120 seconds]
   at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
   at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
   at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)


Like (1) Dislike (0) Favorite(1) Share Report
Previous: Kafka producer error
Next: 11 Advantages of Cloud Computing and How Your Business Can Benefit From Them
Featured Answers

Best answer

(2) (0)
TingtingGG
Created Jul 11, 2019 08:01:34

Good day!

The log also reported java.lang.OutOfMemoryError: Java heap space.

Analysis

From the above log analysis, it is because the spark memory is not enough, resulting in gc, gc will make the executor and driver communication interrupt.

Solution

1. Increase the hardware resources, modify the executor memory;

2. Increase the degree of concurrency;

3. Modify spark-defaults.conf, increase the executor communication timeout time spark.executor.heartbeatInterval.
View more
  • x
  • convention：
All Answers
(2) (0)
2#
TingtingGG
TingtingGG Created Jul 11, 2019 08:01:34

Good day!

The log also reported java.lang.OutOfMemoryError: Java heap space.

Analysis

From the above log analysis, it is because the spark memory is not enough, resulting in gc, gc will make the executor and driver communication interrupt.

Solution

1. Increase the hardware resources, modify the executor memory;

2. Increase the degree of concurrency;

3. Modify spark-defaults.conf, increase the executor communication timeout time spark.executor.heartbeatInterval.
View more
  • x
  • convention：
Back to list

Comment

Advanced
B Color Image Link Quote Code Smilies
You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login
Huawei Website Support About Huawei Privacy User Agreement Terms of Use Cookies Cookie Settings
Huawei Enterprise Huawei Carrier Forum Training & Certification

Contact Us: e_online@huawei.com Copyright © 2022 Huawei Technologies Co., Ltd. All rights reserved.

APP

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.