Got it
Huawei Enterprise Support Community
Community Forums Cloud & Big Data
HBase A large amount of d...

HBase A large amount of data is written slowly.

Created: Jul 29, 2019 09:28:21Latest reply: Jul 29, 2019 09:34:17 3193 2 0 0 0
  Rewarded HiCoins： 4 (problem resolved)
View the author 1#

A column family is used to write 30000 data records per second in 200 columns. The mutate is used to add data. The size of the clientbuffer cache is 10M. Four test machines, 128G memory, and 60G are allocated to the HBase. How to optimize the data?

Like (0) Dislike (0) Favorite(0) Share Report
Previous: Can you add the function of "Insert Code Box" in the edit box?
Next: Does the flink sql support checkpoints?
Featured Answers

Best answer

(1) (0)
TingtingGG
Created Jul 29, 2019 09:34:17

Good day!

The file can be written in the bulkload mode. The mr program is used to produce the hfile file. The hfile file is imported by using the bulkload file, which is very fast.
View more
  • x
  • convention：
All Answers
(1) (0)
2#
tanjingmeng
tanjingmeng Created Jul 29, 2019 09:33:15

In which mode is the HBase used? If the data volume is large, bukload is recommended.
View more
  • x
  • convention：
(1) (0)
3#
TingtingGG
TingtingGG Created Jul 29, 2019 09:34:17

Good day!

The file can be written in the bulkload mode. The mr program is used to produce the hfile file. The hfile file is imported by using the bulkload file, which is very fast.
View more
  • x
  • convention：
Back to list

Comment

Advanced
B Color Image Link Quote Code Smilies
You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login
Huawei Website Support About Huawei Privacy User Agreement Terms of Use Cookies Cookie Settings
Huawei Enterprise Huawei Carrier Forum Training & Certification

Contact Us: e_online@huawei.com Copyright © 2022 Huawei Technologies Co., Ltd. All rights reserved.

APP

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.