Got it

HCIA-Big Data | HBase Performance Tuning

Latest reply: Apr 2, 2022 19:04:53 715 18 6 0 0

Hello, friend!

This post will share with you the HBase performance tuning and common HBase shell commands.

HBase Performance Tuning

Row Key

Row keys are stored in alphabetical order. Therefore, when designing row keys, you need to fully use the sorting feature to store the data that is frequently read together and the data that may be accessed recently.

For example, if the data that is recently written to the HBase table is most likely to be accessed, you can use the timestamp as a part of the row key. Because the data is sorted in alphabetical order, you can use Long.MAX_VALUE -timestamp as the row key, in this way, newly written data can be quickly hit when being read.

Unlike the relational database, there is only one key in HBase row key. Row key are stored in alphabetical order. Therefore when designing the row keys or fully using the sorting feature to store the data frequently read together or accessed recently, it can improve the efficiency a lot by this mechanism.

Creating HBase Secondary Index

HBase has only one index for row keys.

There are three methods for accessing rows in the HBase table:

  • Access through a single rowkey.

  • Access through a row key interval.

  • Full table scan.

Hindex Secondary Index

Hindex is a Java-based HBase secondary index developed by Huawei and is compatible with Apache HBase 0.94.8. The current features are as follows:

  • Multiple table indexes.

  • Multiple column indexes.

  • Index based on some column values.

This is enhanced feature is called secondary index, we use row key to locate the whole line together with the column family as well as the column qualifiers, etc.

But if we have a record say a guy called Zhangsan and we want to find all the information corresponding to him.

This brings great inconvenience for querying right. So in Huawei, we made a secondary index that associates the column to be searched with row key into an index table. To be more specific, we made the column of names as the index and the row key being the value. So we inverted the key-value pairs and then we can search the row key with names.

Does that solve the problem we mentioned before? Such an easy idea makes great improvement how smart it is. And now the column becomes the key and the original row ket becomes the value, we use the new key to find the row key then use the row key to find the other information, so we only need to query twice now.

Common HBase Shell Commands

create: creating Hive tables.

list: listing all tables in HBase.

put: adding data to a specified cell in a table, row, or column.

scan: browsing information about a table.

get: obtaining the value of a cell based on the table name, row, column, timestamp, time range, and version number.

enable/disable: enabling or disabling a table.

drop: deleting a table.

Summary of HBase-related posts

Title

[FI Components] Basic Principle about Hbase

[HBase Emergency Failure Recovery]Data in a User Table Is Deleted or Abnormal

[HBase Emergency Failure Recovery]The Service Becomes Abnormal After HBase Table Data Is Manually Deleted

Spark Applications Fail to Access HBase in Another Cluster

"Failed to find any Kerberos tgt" Was Reported When the Spark Application Used AddResource to Access HBase Across Clusters

Apache Hive vs. Apache Hbase

Apache Phoenix: An SQL Driver for Hbase

HDFS vs Hbase

Install Phoenix on MRS HBase cluster and connect to superset

HBase REST API Invoking Example

HBase Full Migration Procedure

HBase Thrift API Invoking Example

Spark Reads Hive and Writes HBase Samples

Use Flume to consume Kafka topic data and store it in Hbase

What is the difference between the HBase and traditional databases?

That's all, thanks!


The post is synchronized to: HCIA-Big Data

  • x
  • convention:

user_3915171
Created Mar 24, 2022 05:00:04

thanks
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Mar 24, 2022 06:48:23 (0) (0)
 
Saqibaz
Created Mar 24, 2022 05:20:27

Thanks for sharing
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Mar 24, 2022 06:48:29 (0) (0)
 
zj5000
Created Mar 24, 2022 06:02:49

Thanks for your sharing!
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Mar 24, 2022 06:48:36 (0) (0)
 
user_4358465
Created Mar 25, 2022 04:53:15

Excellent share on HBase performance tuning and common HBase shell commands
View more
  • x
  • convention:

little_fish
little_fish Created Mar 29, 2022 08:41:48 (0) (0)
 
VinceD
Moderator Created Apr 2, 2022 16:04:04

thanks for sharing.
View more
  • x
  • convention:

VinceD
VinceD Created Apr 2, 2022 16:04:18 (0) (0)
 
azkasaqib
azkasaqib Created Apr 2, 2022 18:13:39 (0) (0)
 
azkasaqib
Created Apr 2, 2022 18:13:47

Thanks for Sharing
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Apr 6, 2022 05:11:10 (0) (0)
 
AliBinHussain
Created Apr 2, 2022 18:14:16

Nice work
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Apr 6, 2022 05:11:18 (0) (0)
Thanks!  
AliBinHussain
Created Apr 2, 2022 18:14:24

HCIA-Big Data | HBase Performance Tuning-4834775-1
View more
  • x
  • convention:

gabo.lr
MVE Created Apr 2, 2022 19:04:53

Well explained
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Apr 6, 2022 05:11:26 (0) (0)
Thanks!  

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.