Got it

The number of reported data groups exceeds the threshold

305 0 12 0 0

Symptom

When a user runs a query command in the Spark-shell, the system displays a message indicating that the number of groups exceeds the upper limit. However, the same SQL statement can be queried in Spark-SQL and spark-beeline.

Solution

When a table is created, null characters are not converted to null characters. As a result, an array violation exception occurs when the spark-shell query field is empty. Add 'serialization.null.format' = 'or ROW FORMAT delimited fields terminated by' \t 'lines terminated by' \ n 'null defined as'' in the table creation statement to solve the problem.


Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.