Got it

Active/Standby Switchover of the HMasters Fails

Latest reply: Mar 11, 2022 09:30:36 2867 2 1 0 0

Hello, everyone!

Do you know how to solve the HMaster active/standby switchover failure? Don't worry, the post will share with you.

Symptom

During active/standby switchover of HMasters, the standby HMaster fails to become active in 10 minutes. In this case, both HMasters are standby, and the following information is generated in the run log of the HMaster:

2015-11-23 08:48:47,643 | INFO  | MASTER_SERVER_OPERATIONS-linux-174:21300-3 | Skip assigning region in transition on other 
server{7630007b8b816b171535a056ae828fcf state=CLOSED, ts=1448239727400, server=linux-223,21302,1448101490432} | 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:268)

Possible Causes

Services are running during the startup process, which results in an increase in the Region. The increase in the Region triggers the Region split operation, which divides a Region into multiple Regions. When the new HMaster goes online, the RegionServer registered by meta region on the ZooKeeper needs to be re-allocated and the metatable needs to be re-loaded. The HMaster checks the Region status on all RegionServers and places the Regions online. Then the HMaster performs the OFFLINE > PENDING_OPEN > OPENING > OPEN process and executes the split tasks that have not been completed. If there is a large number of Regions, the time required for placing them online is long. If the time exceeds the value specified by hbase.master.initializationmonitor.timeout, the HMaster expected to become active will fail to go online, resulting in an active/standby switchover failure.

Procedure

1. On FusionInsight Manager, click Services HBase Service Configuration, set Type to All. Search and modify the value of hbase.master.initializationmonitor.timeout .

 2. Click Services > HBase > Instances, select the HMaster instances and choose More Actions > Restart instance. Restart the HMaster instances.

That's all, thanks!

  • x
  • convention:

olive.zhao
Admin Created Dec 24, 2021 01:23:29

Thanks for your sharing!
View more
  • x
  • convention:

olive.zhao
Admin Created Mar 11, 2022 09:30:36

I find a post about introducing the HMasters

The HMaster in the HBase allocates regions; migrates the corresponding meta from a failed region server to another region server. The HMaster HA feature is brought in to prevent HBase functions from being affected by the HMaster SPOF.

[FI Components] HMaster HA feature


View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.