Got it

what should i do if Data Is Loaded in BulkLoad Mode, and the Region Is Not Automatically Split

Created: Jun 30, 2019 09:26:52Latest reply: Jun 30, 2019 09:31:24 194 2 0 0 0
  Rewarded HiCoins: 0 (problem resolved)

what should i do if Data Is Loaded in BulkLoad Mode, and the Region Is Not Automatically Split

Featured Answers
Lwx387225
Created Jun 30, 2019 09:31:24

Hi!

Solution

1. Automatic region splitting is triggered by the flushRegion Region or compaction. If data is written in BulkLoad mode only, flushRegion is not invoked. For this reason, automatic region splitting is triggered by compaction only.

2. Compaction operations are classified into major compaction and minor compaction. Both of them are triggered by the scheduled tasks of CompactionChecker. The minor compaction is checked once every 10 x 1000 seconds (the number of storefiles is not less than 3) and the frequency of major compaction is controlled by hbase.hregion.majorcompaction. The default frequency is once a day.

3. During compaction, the system checks whether the storefile size is greater than the hbase.hstore.compaction.max.size. If the file size is smaller than hbase.hstore.compaction.max.size, the storefile is filtered out and not compacted. In some scenarios, compaction cannot be triggered, or the number of the remaining storefiles is greater than or equal to hbase.hstore.blockingStoreFiles (default value: 7). Automatic slitting is not implemented later.
View more
  • x
  • convention:

user_3426907
user_3426907 Created Jun 30, 2019 15:54:54 (0) (0)
Thanks  
All Answers
Hi!

Solution

1. Automatic region splitting is triggered by the flushRegion Region or compaction. If data is written in BulkLoad mode only, flushRegion is not invoked. For this reason, automatic region splitting is triggered by compaction only.

2. Compaction operations are classified into major compaction and minor compaction. Both of them are triggered by the scheduled tasks of CompactionChecker. The minor compaction is checked once every 10 x 1000 seconds (the number of storefiles is not less than 3) and the frequency of major compaction is controlled by hbase.hregion.majorcompaction. The default frequency is once a day.

3. During compaction, the system checks whether the storefile size is greater than the hbase.hstore.compaction.max.size. If the file size is smaller than hbase.hstore.compaction.max.size, the storefile is filtered out and not compacted. In some scenarios, compaction cannot be triggered, or the number of the remaining storefiles is greater than or equal to hbase.hstore.blockingStoreFiles (default value: 7). Automatic slitting is not implemented later.
View more
  • x
  • convention:

user_3426907
user_3426907 Created Jun 30, 2019 15:54:54 (0) (0)
Thanks  

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.