1.1 Scenarios

You can quickly learn and master the HBase development process and know key interface functions in a typical application scenario.

Applicable Versions

FusionInsight HD V100R002C70, FusionInsight HD V100R002C80.

Scenario Description

Assume that users develop an application to manage information about users who use service A in an enterprise, as shown in Table 1-1. The service A operation procedures are as follows:

l  Create a user information table.

l  Add users' educational backgrounds and titles to the table.

l  Query user names and addresses by user ID.

l  Query information by user name.

l  Query information about users whose age ranges from 20 to 29.

l  Collect the number of users and their maximum, minimum, and average age.

l  Deregister users and delete user data from the table.

l  Delete the user information table after service A ends.

Table 1-1 User information

Data Planning

Proper design of table structures, RowKeys, and column names enable you to use HBase advantages. In the sample project, a unique ID is used as the RowKey, and columns are stored in the infocolumn family.

1.2 Development Guidelines

Function Description

Determine functions to be developed based on the preceding scenario. Table 1-2 lists functions to be developed.

Table 1-2 Functionsto be developed in HBase

Key Design Principles

HBase is a distributed database system based on the lexicographic order of RowKey. The RowKey design has great impact on performance, so the RowKey must be designed based on specific services.

1.3 Sample Code Description

Creating the Configuration Object

Creating the Configuration Object

l  Overview

HBase obtains configuration items by using the login method, including user login information and security authentication information.

l  Sample code

The following code snippet is in the initmethod in the TestMainclass of the com.huawei.bigdata.hbase.examplespacket.

private static void init() throws IOException { 
    // Default load from conf directory 
    conf = HBaseConfiguration.create(); 
    String userdir = System.getProperty("user.dir") + File.separator + "conf" + File.separator; 
    conf.addResource(new Path(userdir + "core-site.xml")); 
    conf.addResource(new Path(userdir + "hdfs-site.xml")); 
    conf.addResource(new Path(userdir + "hbase-site.xml")); 
  }

Creating the Connection Object

l  Overview

HBase creates a Connection object using the ConnectionFactory.createConnection(configuration)method. The transferred parameter is the Configuration created in the previous step.

Connection encapsulates the connections between underlying applications and servers and ZooKeeper. Connection is instantiated using the ConnectionFactory class. Creating Connection is a heavyweight operation. Connection is thread-safe. Therefore, multiple client threads can share one Connection.

In a typical scenario, a client program uses a Connection, and each thread obtains its own Admin or Table instance and invokes the operation interface provided by the Admin or Table object. You are not advised to cache or pool Table and Admin. The lifecycle of Connection is maintained by invokers who can release resources by invoking close().

l  Sample code

The following code snippet is an example of logging in to and creating a connection, and creating a table. The example is in the HBaseSample method of the HBaseSample class in the com.huawei.bigdata.hbase.examplespackage.

  private TableName tableName = null; 
  private Configuration conf = null; 
  private Connection conn = null; 
 
  public HBaseSample(Configuration conf) throws IOException { 
    this.conf = conf; 
    this.tableName = TableName.valueOf("hbase_sample_table"); 
    this.conn = ConnectionFactory.createConnection(conf); 
  }

Avoid invoking login code repeatedly.

Creating a Table

l  Overview

HBase allows you to create a table using the createTable method of the org.apache.hadoop.hbase.client.Adminobject. You need to specify a table name and a column family name. You can create a table by using either of the following methods, but the latter one is recommended:

−          Quickly create a table. A newly created table contains only one region, which will be automatically split into multiple new regions as data increases.

−          Create a table using pre-assigned regions. You can pre-assign multiple regions before creating a table. This mode accelerates data write at the beginning of massive data writing.

The column name and column family name of a table consist of letters, digits, and underscores (_), excluding any special characters.

l  Sample code

The following code snippets are in the testCreateTablemethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testCreateTable() { 
             LOG.info("Entering testCreateTable."); 
             // Specify the table descriptor. 
             HTableDescriptor htd = new HTableDescriptor(tableName); (1) 
             // Set the column family name to info. 
             HColumnDescriptor hcd = new HColumnDescriptor("info"); (2) 
             // Set data encoding methods, HBase provides DIFF,FAST_DIFF,PREFIX 
             // and PREFIX_TREE 
             hcd.setDataBlockEncoding(DataBlockEncoding.FAST_DIFF); 
             // Set compression methods, HBase provides two default compression 
             // methods:GZ and SNAPPY 
           // GZ has the highest compression rate,but low compression and 
             // decompression effeciency,fit for cold data 
             // SNAPPY has low compression rate, but high compression and 
             // decompression effeciency,fit for hot data. 
             // it is advised to use SNAANPPY 
             hcd.setCompressionType(Compression.Algorithm.SNAPPY);// 
             htd.addFamily(hcd);  (3) 
             Admin admin = null;  
             try { 
               // Instantiate an Admin object. 
               admin = conn.getAdmin();  (4)  
               if (!admin.tableExists(tableName)) { 
                 LOG.info("Creating table..."); 
                 admin.createTable(htd);//  (5) 
                 LOG.info(admin.getClusterStatus()); 
                 LOG.info(admin.listNamespaceDescriptors()); 
                 LOG.info("Table created successfully."); 
               } else { 
                 LOG.warn("table already exists"); 
               } 
             } catch (IOException e) { 
                 LOG.error("Create table failed " ,e); 
             } finally { 
               if (admin != null) { 
                 try { 
                   // Close the Admin object. 
                   admin.close(); 
                 } catch (IOException e) { 
                   LOG.error("Failed to close admin " ,e); 
                 } 
               } 
             } 
             LOG.info("Exiting testCreateTable."); 
  }    

1.     Create a table descriptor.

2.     Create a column family descriptor.

3.     Add the column family descriptor to the table descriptor.

4.     Obtain the Admin object. You can use the Admin object to create a table and a column family, check whether the table exists, modify the table structure and column family structure, and delete the table.

5.     Invoke the Admin object to create a table

l  Precautions

−          Note [1] Use the following code to set the compression mode for a column family:

//Set the encoding algorithm. HBase provides four encoding algoriths, such as DIFF, FAST_DIFF, PREFIX, and PREFIX_TREE.
 hcd.setDataBlockEncoding(DataBlockEncoding.FAST_DIFF);   
 
 //Set the file compression method. HBase provides the GZ and SNAPPY compression algorithms by default. 
 //The GZ compression rate is high with low compression and decompression performance. This method is applicable to clod data.   
 //The SNAPPY compression rate is low with high compression and decompression performance. This method is applicable to hot data. 
 //You are advised to enable the SNAPPY compression be default.  
 hcd.setCompressionType(Compression.Algorithm.SNAPPY);

−          Note [2] Create a table by specifying the start and end RowKeys or by pre-assigning regions using RowKey arrays. The code snippet is as follows:

 // Create a preplanned region table 
 byte[][] splits = new byte[4][];   
 splits[0] = Bytes.toBytes("A");   
 splits[1] = Bytes.toBytes("H");   
 splits[2] = Bytes.toBytes("O");   
 splits[3] = Bytes.toBytes("U");   
 admin.createTable(htd, splits);

Deleting a Table

l  Overview

HBase allows you to modify a table using the modifyTable method of org.apache.hadoop.hbase.client.Admin.

l  Sample code

The following code snippets are in the testModifyTablemethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

 public void testModifyTable() {  
    LOG.info("Entering testModifyTable.");  
 
    // Specify the column family name.  
    byte[] familyName = Bytes.toBytes("education");  
    Admin admin = null;  
    try {  
      // Instantiate an Admin object.  
      admin = conn.getAdmin();  
      // Obtain the table descriptor.  
      HTableDescriptor htd = admin.getTableDescriptor(tableName);  
      // Check whether the column family is specified before modification.  
      if (!htd.hasFamily(familyName)) {  
        // Create the column descriptor.  
        HColumnDescriptor hcd = new HColumnDescriptor(familyName);  
        htd.addFamily(hcd);  
        // Disable the table to get the table offline before modifying  
        // the table.  
        admin.disableTable(tableName);//Note[1]  
        // Submit a modifyTable request.  
        admin.modifyTable(tableName, htd);  
        // Enable the table to get the table online after modifying the  
        // table.  
        admin.enableTable(tableName);  
      }  
      LOG.info("Modify table successfully.");  
    } catch (IOException e) {  
      LOG.error("Modify table failed " ,e);  
    } finally {  
      if (admin != null) {  
        try {  
          // Close the Admin object.  
          admin.close();  
        } catch (IOException e) {  
          LOG.error("Close admin failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting testModifyTable.");  
  }

l  Precautions

Note [1] modifyTable takes effect only when a table is disabled.

Inserting Data

l  Overview

HBase is a column-based database. A row of data may have multiple column families, and a column family may contain multiple columns. When writing data, you must specify the columns (including the column family names and column names) to which data is written. HBase allows you to insert data (a row of data or data sets) using the putmethod of HTable.

l  Sample code

The following code snippets are in the testPutmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testPut() {  
    LOG.info("Entering testPut.");  
    // Specify the column family name.  
    byte[] familyName = Bytes.toBytes("info");  
    // Specify the column name.  
    byte[][] qualifiers = { Bytes.toBytes("name"), Bytes.toBytes("gender"),  
        Bytes.toBytes("age"), Bytes.toBytes("address") };  
    Table table = null;  
    try {  
      // Instantiate an HTable object.  
      table = conn.getTable(tableName);  
      List<Put> puts = new ArrayList<Put>();  
      // Instantiate a Put object.  
      Put put = new Put(Bytes.toBytes("012005000201"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Zhang San"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Male"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("19"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Shenzhen, Guangdong"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000202"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Li Wanting"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Female"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("23"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Shijiazhuang, Hebei"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000203"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Wang Ming"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Male"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("26"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Ningbo, Zhejiang"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000204"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Li Gang"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Male"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("18"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Xiangyang, Hubei"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000205"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Zhao Enru"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Female"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("21"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Shangrao, Jiangxi"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000206"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Chen Long"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Male"));  
  put.addColumn(familyName, qualifiers[2], Bytes.toBytes("32"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Zhuzhou, Hunan"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000207"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Zhou Wei"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Female"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("29"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Nanyang, Henan"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000208"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Yang Yiwen"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Female"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("30"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Kaixian, Chongqing"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000209"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Xu Bing"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Male"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("26"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Weinan, Shaanxi"));  
      puts.add(put);  
      put = new Put(Bytes.toBytes("012005000210"));  
      put.addColumn(familyName, qualifiers[0], Bytes.toBytes("Xiao Kai"));  
      put.addColumn(familyName, qualifiers[1], Bytes.toBytes("Male"));  
      put.addColumn(familyName, qualifiers[2], Bytes.toBytes("25"));  
      put.addColumn(familyName, qualifiers[3], Bytes.toBytes("Dalian, Liaoning"));  
      puts.add(put);  
      // Submit a put request.  
      table.put(puts);  
 
      LOG.info("Put successfully.");  
    } catch (IOException e) {  
      LOG.error("Put failed " ,e);  
    } finally {  
      if (table != null) {  
        try {  
          // Close the HTable object.  
          table.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting testPut.");  
  }

l  Precautions

Multiple threads are not allowed to use the same HTable instance at the same time. HTable is a non-thread-safe class. If an HTable instance is used by multiple threads at the same time, exceptions will occur.

Deleting Data

l  Overview

HBase allows you to delete data (a row of data or data sets) using the delete method of a Table instance.

l  Sample code

The following code snippets are in the testDeletemethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testDelete() {  
    LOG.info("Entering testDelete.");  
 
    byte[] rowKey = Bytes.toBytes("012005000201");  
 
    Table table = null;  
    try {  
      // Instantiate an HTable object.  
      table = conn.getTable(tableName);  
 
      // Instantiate an Delete object.  
      Delete delete = new Delete(rowKey);  
 
      // Submit a delete request.  
      table.delete(delete);  
 
      LOG.info("Delete table successfully.");  
    } catch (IOException e) {  
      LOG.error("Delete table failed " ,e);  
    } finally {  
      if (table != null) {  
        try {  
          // Close the HTable object.  
          table.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting testDelete.");  
  }  

If the secondary index is set for the column family of the cell to be deleted, the index data is also deleted.

Reading Data Using a Get Object

l  Overview

Before reading data from a table, instantiate the Table instance of the table and create a Get object. You can also set parameters for the Get object, such as the column family name and column name. Query results are stored in the Result object that stores multiple cells.

l  Sample code

The following code snippets are in the testGetmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testGet() {  
    LOG.info("Entering testGet.");  
    // Specify the column family name.  
    byte[] familyName = Bytes.toBytes("info");  
    // Specify the column name.  
    byte[][] qualifier = { Bytes.toBytes("name"), Bytes.toBytes("address") };  
    // Specify RowKey.  
    byte[] rowKey = Bytes.toBytes("012005000201");  
    Table table = null;  
    try {  
      // Create the Table instance.  
      table = conn.getTable(tableName);  
      // Instantiate a Get object.  
      Get get = new Get(rowKey);  
      // Set the column family name and column name.  
      get.addColumn(familyName, qualifier[0]);  
      get.addColumn(familyName, qualifier[1]);  
      // Submit a get request.  
      Result result = table.get(get);  
      // Print query results.  
      for (Cell cell : result.rawCells()) {  
        LOG.info(Bytes.toString(CellUtil.cloneRow(cell)) + ":"  
            + Bytes.toString(CellUtil.cloneFamily(cell)) + ","  
            + Bytes.toString(CellUtil.cloneQualifier(cell)) + ","  
            + Bytes.toString(CellUtil.cloneValue(cell)));  
      }  
      LOG.info("Get data successfully.");  
    } catch (IOException e) {  
      LOG.error("Get data failed " ,e);  
    } finally {  
      if (table != null) {  
        try {  
          // Close the HTable object.  
          table.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting testGet.");  
  }

Reading Data Using a Scan Object

l  Overview

Before reading data from a table, instantiate the Table instance of the table, and then create a Scan object and set parameters for the Scan object based on search criteria. To improve query efficiency, you are advised to specify StartRow and StopRow. Query results are stored in the ResultScanner object, where each row of data is stored as a Result object that stores multiple cells.

l  Sample code

The following code snippets are in the testScanDatamethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testScanData() {  
    LOG.info("Entering testScanData.");  
    Table table = null;   
    // Instantiate a ResultScanner object.  
    ResultScanner rScanner = null;  
    try {  
      // Create the Configuration instance.  
      table = conn.getTable(tableName);  
      // Instantiate a Get object.  
      Scan scan = new Scan();  
      scan.addColumn(Bytes.toBytes("info"), Bytes.toBytes("name"));  
      // Set the cache size.  
      scan.setCaching(1000);  
 
      // Submit a scan request.  
      rScanner = table.getScanner(scan);  
      // Print query results.  
      for (Result r = rScanner.next(); r != null; r = rScanner.next()) {  
        for (Cell cell : r.rawCells()) {  
          LOG.info(Bytes.toString(CellUtil.cloneRow(cell)) + ":"  
              + Bytes.toString(CellUtil.cloneFamily(cell)) + ","  
              + Bytes.toString(CellUtil.cloneQualifier(cell)) + ","  
              + Bytes.toString(CellUtil.cloneValue(cell)));  
        }  
      }  
      LOG.info("Scan data successfully.");  
    } catch (IOException e) {  
      LOG.error("Scan data failed " ,e);  
    } finally {  
      if (rScanner != null) {  
        // Close the scanner object.  
        rScanner.close();  
      }  
      if (table != null) {  
        try {  
          // Close the HTable object.  
          table.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting testScanData.");  
  } 

l  Precautions

a.        You are advised to specify StartRow and StopRow to ensure good performance with a specified Scan scope.

b.        You can set Batchand Caching.

l  Batch

Batchindicates the maximum number of records returned each time when the next API is invoked using Scan. This parameter is related to the number of columns read each time.

l  Caching

Cachingindicates the maximum number of nextrecords returned for a remote procedure call (RPC) request. This parameter is related to the number of rows read by an RPC.

Using a Filter

l  Overview

HBase Filter is used to filter data during Scan and Get. You can specify the filter criteria, such as filtering by RowKey, column name, or column value.

l  Sample code

The following code snippets are in the testSingleColumnValueFiltermethod in the HBaseSampleclass of the com.huawei.bigdata.hbase.examplespacket.

    ResultScanner rScanner = null;  
 
    try {  
 
      table = conn.getTable(tableName);  
 
      Scan scan = new Scan();  
      scan.addColumn(Bytes.toBytes("info"), Bytes.toBytes("name"));  
      // Set the filter criteria.  
      SingleColumnValueFilter filter = new SingleColumnValueFilter(  
          Bytes.toBytes("info"), Bytes.toBytes("name"), CompareOp.EQUAL,  
          Bytes.toBytes("Xu Bing"));  
      scan.setFilter(filter);  
      // Submit a scan request.  
      rScanner = table.getScanner(scan);  
      // Print query results.  
      for (Result r = rScanner.next(); r != null; r = rScanner.next()) {  
        for (Cell cell : r.rawCells()) {  
          LOG.info(Bytes.toString(CellUtil.cloneRow(cell)) + ":"  
              + Bytes.toString(CellUtil.cloneFamily(cell)) + ","  
              + Bytes.toString(CellUtil.cloneQualifier(cell)) + ","  
              + Bytes.toString(CellUtil.cloneValue(cell)));  
        }  
      }  
      LOG.info("Single column value filter successfully.");  
    } catch (IOException e) {  
      LOG.error("Single column value filter failed " ,e);  
    } finally {  
        if (rScanner != null) {   
            // Close the scanner object.  
            rScanner.close();  
          }  
      if (table != null) {  
        try {  
          // Close the HTable object.  
          table.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting testSingleColumnValueFilter.");  
  }

l  Precautions

Currently, secondary indexes do not support the comparators that use objects of the SubstringComparator class as filters.

For example, the following sample code is not supported:

Scan scan = new Scan();  filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL);  filterList.addFilter(new SingleColumnValueFilter(Bytes  .toBytes(columnFamily), Bytes.toBytes(qualifier),  CompareOp.EQUAL, new SubstringComparator(substring)));  scan.setFilter(filterList);

Creating a Secondary Index

l  Overview

You can manage HBase secondary indexes using methods provided in org.apache.hadoop.hbase.hindex.client.HIndexAdmin. This class provides methods of creating an index.

Secondary indexes cannot be modified. If you want to modify a secondary index, delete the old one and create an index.

l  Sample code

The following code snippets are in the createIndexmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void createIndex() {  
    LOG.info("Entering createIndex.");  
 
    String indexName = "index_name";  
    // Create hindex instance  
    TableIndices tableIndices = new TableIndices();  
    IndexSpecification iSpec = new IndexSpecification(indexName);  
    iSpec.addIndexColumn(new HColumnDescriptor("info"), "name", ValueType.String);//Note[1]  
    tableIndices.addIndex(iSpec);  
 
    HIndexAdmin iAdmin = null;  
    Admin admin = null;  
    try {  
 
      admin = conn.getAdmin();  
      iAdmin = new IndexAdmin(conf);  
 
      // add index to the table  
      iAdmin.addIndices(tableName, tableIndices);  
      LOG.info("Create index successfully.");  
    } catch (IOException e) {  
      LOG.error("Create index failed " ,e);  
    } finally {  
      if (admin != null) {  
          try {  
            admin.close();  
        } catch (IOException e) {  
            LOG.error("Close admin failed " ,e);  
        }  
      }  
      if (iAdmin != null) {  
        try {  
          // Close IndexAdmin Object  
          iAdmin.close();  
        } catch (IOException e) {  
          LOG.error("Close admin failed " ,e);  
        }  
      }  
    }  
    LOG.info("Exiting createIndex.");  
  }

By default, newly created level-2 indexes are disabled. To enable a specified level-2 index, see the following code snippets. The following code snippets are in the enableIndexmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examples packet.

public void enableIndex() {  
    LOG.info("Entering createIndex.");  
 
    // Name of the index to be enabled  
    String indexName = "index_name";  
 
    List<String> indexNameList = new ArrayList<String>();  
    indexNameList.add(indexName);  
    HIndexAdmin iAdmin = null;  
    try {  
      iAdmin = HIndexClient.newHIndexAdmin(conn.getAdmin());  
      // Alternately, enable the specified indices  
      iAdmin.enableIndices(tableName, indexNameList);  
      System.out.println("Successfully enable indices " + indexNameList + " of the table " + tableName);  
    } catch (IOException e) {  
      System.out.println("Failed to enable indices " + indexNameList + " of the table " + tableName + "." + e);  
    } finally {  
      if (iAdmin != null) {  
        try {  
          iAdmin.close();  
        } catch (IOException e) {  
          LOG.error("Close admin failed ", e);  
        }  
      }  
    }  
  }

l  Precautions

Note [1]: Create a combination index.

HBase supports creation of secondary indexes on multiple fields, for example, the name and age columns.

HIndexSpecification iSpecUnite = new HIndexSpecification(indexName);    iSpecUnite.addIndexColumn(new HColumnDescriptor("info"), "name", ValueType.String);    iSpecUnite.addIndexColumn(new HColumnDescriptor("info"), "age", ValueType.String);

l  Related operations

Creating an index by running commands

You can also use the TableIndexertool to create an index in an existing user table.

The <table_name> user table must exist.

hbase org.apache.hadoop.hbase.hindex.mapreduce.TableIndexer -Dtablename.to.index=<table_name> -Dindexspecs.to.add='IDX1=>cf1:[q1->datatype];cf2:[q1->datatype],[q2->datatype],[q3->datatype]#IDX2=>cf1:[q5->datatype]'

"#" is used to separate indexes. ";" is used to separate column families. "," is used to separate columns.

tablename.to.index: indicates the name of a user table where an index is created.

indexspecs.to.add: indicates the user table column corresponding to an index.

The parameters in the command are described as follows:

l  IDX1: indicates the index name.

l  cf1: indicates the column family name.

l  q1: indicates the column name.

l  datatype: indicates the data type. The data type can be Integer, String, Double, Float, Long, Short, Byte, or Char.

Deleting an Index

l  Overview

You can manage HBase secondary indexes using methods provided in org.apache.hadoop.hbase.hindex.client.HIndexAdmin. This class provides methods of querying and deleting an index.

l  Sample code

The following code snippets are in the dropIndexmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void dropIndex() {  
    LOG.info("Entering dropIndex.");  
    String indexName = "index_name";  
    List<String> indexNameList = new ArrayList<String>();  
  indexNameList.add(indexName);  
 
    IndexAdmin iAdmin = null;  
    try {  
      // Instantiate HIndexAdmin Object  
      iAdmin = HIndexClient.newHIndexAdmin(conn.getAdmin());  
      // Delete Secondary Index  
      iAdmin.dropIndex(tableName, indexNameList);  
 
      LOG.info("Drop index successfully.");  
    } catch (IOException e) {  
      LOG.error("Drop index failed.");  
    } finally {  
      if (iAdmin != null) {  
        try {   
          // Close Secondary Index  
          iAdmin.close();  
      } catch (IOException e) {  
          LOG.error("Close admin failed.");  
        }  
      }  
    }  
    LOG.info("Exiting dropIndex.");  
  }

Querying Data Based on Secondary Indexes

l  Overview

In user tables with secondary indexes, you can use Filter to query data. The data query performance is higher than that in user tables without secondary indexes.

The secondary index usage rules are as follows:

−          For scenarios in which a single index is created for one or more columns:

l  When you use this column for AND or OR filtering, the index is used to improve the query performance.

For example, Filter_Condition(IndexCol1) AND/OR Filter_Condition(IndexCol2).

l  When you use "Index Column AND Non-Index Column" for filtering in the query, the index is used to improve the query performance.

Filter_Condition(IndexCol1) AND Filter_Condition(IndexCol2) AND Filter_Condition(NonIndexCol1).

l  When you use "Index Column OR Non-Index Column" for filtering in the query, the index is not used and the query performance is not improved.

Filter_Condition(IndexCol1) AND/OR Filter_Condition(IndexCol2) OR Filter_Condition(NonIndexCol1).

−          For scenarios in which a combination index is created for multiple columns:

l  When the columns used for query are all or part of the columns of the combination index and are in the same sequence with the combination index, the index is used to improve the query performance.

For example, a combination index is created for C1, C2, and C3 columns. The index takes effect in the following scenarios:

Filter_Condition(IndexCol1) AND Filter_Condition(IndexCol2) AND Filter_Condition(IndexCol3)

Filter_Condition(IndexCol1) AND Filter_Condition(IndexCol2)

Filter_Condition(IndexCol1)

The index does not take effect in the following scenarios:

Filter_Condition(IndexCol2) AND Filter_Condition(IndexCol3)

Filter_Condition(IndexCol1) AND Filter_Condition(IndexCol3)

Filter_Condition(IndexCol2)

Filter_Condition(IndexCol3)

l  When you use "Index Column AND Non-Index Column" for filtering in the query, the index is used to improve the query performance.

For example:

Filter_Condition(IndexCol1) AND Filter_Condition(NonIndexCol1)

Filter_Condition(IndexCol1) AND Filter_Condition(IndexCol2) AND Filter_Condition(NonIndexCol1)

l  When you use "Index Column OR Non-Index Column" for filtering in the query, the index is not used and the query performance is not improved.

For example:

Filter_Condition(IndexCol1) OR Filter_Condition(NonIndexCol1)

(Filter_Condition(IndexCol1) AND Filter_Condition(IndexCol2))OR ( Filter_Condition(NonIndexCol1))

l  When multiple columns are used for query, a value range can be specified only for the last column in the combination index and the other columns can only be set to a specified value.

For example, a combination index is created for C1, C2, and C3 columns. In range query, a value range can be set only for C3 and the filter criterion is "C1 = XXX, C2 = XXX, and C3 = value range".

−          For a user table that has a secondary index, you can query data by using Filter. The query results are the same as those without index, and the data query performance is higher than the data query performance of the user table without secondary index.

l  Sample code

The following code snippets are in the testScanDataByIndexmethod in the HBaseSample class of the com.huawei.hadoop.hbase.examplepacket.

Example: Query data using secondary indexes

public void testScanDataByIndex() {  
    LOG.info("Entering testScanDataByIndex.");  
    Table table = null;  
    ResultScanner scanner = null;  
    try {  
      table = conn.getTable(tableName);  
 
      // Create a filter for indexed column.  
      Filter filter = new SingleColumnValueFilter(Bytes.toBytes("info"), Bytes.toBytes("name"),  
          CompareOp.EQUAL, "Li Gang".getBytes());  
      Scan scan = new Scan();  
      scan.setFilter(filter);  
      scanner = table.getScanner(scan);  
      LOG.info("Scan indexed data.");  
 
      for (Result result : scanner) {  
        for (Cell cell : result.rawCells()) {  
          LOG.info(Bytes.toString(CellUtil.cloneRow(cell)) + ":"  
              + Bytes.toString(CellUtil.cloneFamily(cell)) + ","  
              + Bytes.toString(CellUtil.cloneQualifier(cell)) + ","  
              + Bytes.toString(CellUtil.cloneValue(cell)));  
        }  
      }  
      LOG.info("Scan data by index successfully.");  
    } catch (IOException e) {  
      LOG.error("Scan data by index failed.");  
    } finally {  
      if (scanner != null) {  
        // Close the scanner object.  
        scanner.close();  
      }  
      try {  
        if (table != null) {  
          table.close();  
        }  
      } catch (IOException e) {  
        LOG.error("Close table failed.");  
      }  
    }  
 
    LOG.info("Exiting testScanDataByIndex.");  
  }

l  Precautions

A secondary index must be created for the name field in advance.

l  Related operations

Query a table using a secondary index.

The query example is as follows:

Add an index to the name column of the infocolumn family in hbase_sample_table. On the client, run the following command:

hbase org.apache.hadoop.hbase.hindex.mapreduce.TableIndexer -Dtablename.to.index=hbase_sample_table-Dindexspecs.to.add='IDX1=>info:[name->String]'

Query info:name. Run the following command on the hbase shell client:

scan 'hbase_sample_table',{FILTER=>"SingleColumnValueFilter(family,qualifier,compareOp,comparator,filterIfMissing,latestVersionOnly)"}

Use APIs to perform complex queries on the Hbase shell client.

Parameter description:

l  family: indicates the column family where the column to be queried locates, such as info.

l  qualifier: indicates the column to be queried, such as name.

l  compareOp: indicates the comparison operator, such as = and >.

l  comparator: indicates the target value to be queried, such as binary:Zhang San.

l  filterIfMissing: indicates whether a row is filtered if the column does not exist in this row. The default value is false.

l  latestVersionOnly: indicates whether only values of the latest version are to be queried. The default value is false.

For example:

scan 'hbase_sample_table',{FILTER=>"SingleColumnValueFilter('info','name',=,'binary:Zhang San',true,true)"}

Writing Data into a MOB Table

l  Overview

Similar to writing common HBase data, writing HBase medium object storage (MOB) data is transparent to users. To use the HBase MOB function, you need to add the related configuration items to thehbase-site.xml file. For details, see the section "Configuring the MOB" in Service Operation Guide. In addition, you need to enable the MOB function for the specified column family. The example code is as follows.

l  Sample code

The following code snippets are in the testCreateMOBTablemethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testCreateMOBTable() {  
      LOG.info("Entering testCreateMOBTable.");  
 
    Admin admin = null;  
    try {  
      // Create Admin instance  
      admin = conn.getAdmin();  
      HTableDescriptor tabDescriptor = new HTableDescriptor(tableName);  
      HColumnDescriptor mob = new HColumnDescriptor("mobcf");  
      // Open mob function  
      mob.setMobEnabled(true);  
      // Set mob threshold  
      mob.setMobThreshold(10L);  
      tabDescriptor.addFamily(mob);  
      admin.createTable(tabDescriptor);  
      LOG.info("MOB Table is created successfully.");  
    } catch (Exception e) {  
      LOG.error("MOB Table is created failed " ,e);  
    } finally {  
        if (admin != null) {  
            try {  
              // Close the Admin object.  
              admin.close();  
            } catch (IOException e) {  
              LOG.error("Close admin failed " ,e);  
            }  
          }  
    }  
    LOG.info("Exiting testCreateMOBTable.");  
  }

Example: Using the Put interface to write the MOB data

public void testMOBDataInsertion() {  
      LOG.info("Entering testMOBDataInsertion.");  
 
  Table table = null;  
    try {  
      // set row name to "row"  
      Put p = new Put(Bytes.toBytes("row"));  
      byte[] value = new byte[1000];  
      // set the column value of column family mobcf with the value of "cf1"  
      p.addColumn(Bytes.toBytes("mobcf"), Bytes.toBytes("cf1"), value);  
      // get the table object represent table tableName  
      table = conn.getTable(tableName);  
      // put data  
      table.put(p);  
      LOG.info("MOB data inserted successfully.");  
    } catch (Exception e) {  
        LOG.error("MOB data inserted failed " ,e);  
    } finally {  
        if(table != null){  
            try {  
                table.close();  
            } catch (Exception e2) {  
                LOG.error("Close table failed " ,e);  
            }  
        }  
    }  
    LOG.info("Exiting testMOBDataInsertion.");  
  }

l  Precautions

Multiple threads are not allowed to use the same HTable instance at the same time. HTable is a non-thread-safe class. If an HTable instance is used by multiple threads at the same time, exceptions will occur.

Reading MOB Data

l  Overview

Similar to reading common HBase data, reading HBase MOB data is transparent to users. To use the HBase MOB function, you need to add the related configuration items to the hbase-site.xmlfile. For details, see the section "Configuring the MOB" in Service Operation Guide. In addition, you need to enable the MOB function for the specified column family.

l  Sample code

The following code snippets are in the testMOBDataReadmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testMOBDataRead() {  
      LOG.info("Entering testMOBDataRead.");  
      ResultScanner scanner = null;  
      Table table = null;  
      Admin admin = null;  
    try {  
 
      // get table object representing table tableName  
      table = conn.getTable(tableName);  
      admin = conn.getAdmin();  
      admin.flush(table.getName());  
      Scan scan = new Scan();  
      // get table scanner  
      scanner = table.getScanner(scan);  
      for (Result result : scanner) {  
        byte[] value = result.getValue(Bytes.toBytes("mobcf"),  
            Bytes.toBytes("cf1"));  
        String string = Bytes.toString(value);  
        LOG.info("value:" + string);  
      }  
      LOG.info("MOB data read successfully.");  
    } catch (Exception e) {  
        LOG.error("MOB data read failed " ,e);  
    } finally {  
        if(scanner != null){  
            scanner.close();  
        }  
        if (table != null) {  
            try {  
              // Close table object  
              table.close();  
            } catch (IOException e) {  
              LOG.error("Close table failed " ,e);  
            }  
          }  
          if (admin != null) {  
              try {  
                // Close the Admin object.  
                admin.close();  
              } catch (IOException e) {  
                LOG.error("Close admin failed " ,e);  
              }  
        }  
    }  
    LOG.info("Exiting testMOBDataRead.");  
  }

Multi-Point Region Splitting

l  Overview

You can perform multi-point splitting by using org.apache.hadoop.hbase.client.HBaseAdmin. Note that the splitting operation takes effect on empty regions only.

In this example, the multi-point splitting is performed on an HBase table by using multiSplit. The table will be split into five regions: "-∞~A", "A~D", "D~F", "F~H", and "H~+∞".

l  Sample code

The following code snippets are in the testMultiSplitmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

public void testMultiSplit() {  
    LOG.info("Entering testMultiSplit.");  
 
    Table table = null;  
    Admin admin = null;  
    try {  
      admin = conn.getAdmin();  
 
      // initilize a HTable object   
      table = conn.getTable(tableName);  
      Set<HRegionInfo> regionSet = new HashSet<HRegionInfo>();  
      List<HRegionLocation> regionList = conn.getRegionLocator(tableName).getAllRegionLocations();  
      for(HRegionLocation hrl : regionList){  
          regionSet.add(hrl.getRegionInfo());  
      }  
      byte[][] sk = new byte[4][];  
      sk[0] = "A".getBytes();  
      sk[1] = "D".getBytes();  
      sk[2] = "F".getBytes();  
      sk[3] = "H".getBytes();  
      for (HRegionInfo regionInfo : regionSet) {  
        ((HBaseAdmin)admin).multiSplit(regionInfo.getRegionName(), sk);  
      }  
      LOG.info("MultiSplit successfully.");  
    } catch (Exception e) {  
      LOG.error("MultiSplit failed.");  
    } finally {  
      if (table != null) {  
        try {  
          // Close table object  
          table.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed " ,e);  
        }  
      }  
      if (admin != null) {  
          try {  
            // Close the Admin object.  
            admin.close();  
          } catch (IOException e) {  
              LOG.error("Close admin failed " ,e);  
          }  
      }  
    }  
    LOG.info("Exiting testMultiSplit.");   
  }

Note that the splitting operation takes effect on empty regions only.

ACL Security Configuration

l  Overview

Access rights control is mature in relational databases. HBase provides a simple access rights control feature. This feature is implemented in read (R), write (W), creation (C), execution (X), and administration (A) operations.

The Access Control List (ACL) method is defined in the org.apache.hadoop.hbase.security.access.AccessControlClienttool class.

In the current version, permissions can be assigned to columns on the FusionInsight GUI.

l  Sample code

The following code snippets are in the grantACLmethod in the HBaseSample class of the com.huawei.bigdata.hbase.examplespacket.

−          Java sample code for authorizing an ACL

  public void grantACL() {  
    LOG.info("Entering grantACL.");  
    String user = "huawei";  
    String permissions = "RW";  
    TableName tableName = TableName.valueOf("test_table");  
    String familyName = "info";  
    String qualifierName = "name";  
    Connection conn = null;  
    Admin hAdmin = null;  
    Table aclTable = null;  
    try {  
      conn = ConnectionFactory.createConnection(conf);  
      hAdmin = conn.getAdmin();  
      // Create ACL Instance  
      aclTable = conn.getTable(AccessControlLists.ACL_TABLE_NAME);  
      Permission perm = new Permission(Bytes.toBytes(permissions));  
      HTableDescriptor ht = hAdmin.getTableDescriptor(tableName);  
      // Determine whether the table exists  
      if (hAdmin.tableExists(aclTable.getName())) {  
        // Determine whether ColumnFamily exists  
        if (ht.hasFamily(Bytes.toBytes(familyName))) {  
          // grant permission  
          AccessControlClient.grant(conn, tableName, user, Bytes.toBytes(familyName),  
            (qualifierName == null ? null : Bytes.toBytes(qualifierName)), perm.getActions());  
        } else {  
          // grant permission  
          AccessControlClient.grant(conn, tableName, user, null, null, perm.getActions());  
        }  
      }  
      LOG.info("Grant ACL successfully.");  
    } catch (Throwable e) {  
      LOG.error("Grant ACL failed ", e);  
    } finally {  
      if (aclTable != null) {  
        try {  
          // Close  
          aclTable.close();  
        } catch (IOException e) {  
          LOG.error("Close table failed ", e);  
        }  
      }  
      if (hAdmin != null) {  
        try {  
          // Close Admin Object  
          hAdmin.close();  
        } catch (IOException e) {  
          LOG.error("Close admin failed ", e);  
        }  
      }  
      if (conn != null) {  
        try {  
          // Close HBase connection  
          conn.close();  
        } catch (IOException e) {  
          LOG.error("Close connection failed ", e);  
        }  
      }  
    }  
    LOG.info("Exiting grantACL.");  
  }

−          Java sample code for retrieving ACL permissions

  public void getACL() {  
    LOG.info("Entering getACL.");  
    Connection conn = null;  
    Admin hAdmin = null;  
    Table aclTable = null;  
    TableName tableName = TableName.valueOf("test_table");  
    try {  
      conn = ConnectionFactory.createConnection(conf);  
      hAdmin = conn.getAdmin();  
      // Create ACL Instance  
      aclTable = conn.getTable(AccessControlLists.ACL_TABLE_NAME);  
      // Determine whether acl table exists  
      if (hAdmin.tableExists(aclTable.getName())) {  
        // Get user permission  
        List<UserPermission> userPerms =  
            AccessControlClient.getUserPermissions(conn, tableName.getNameAsString());  
        if (userPerms.isEmpty()) {  
          System.out.println("Table permissions size is " + userPerms.size());  
        } else {  
          System.out  
              .println("Table permission size is " + userPerms.size() + ". They are as follows,");  
          for (UserPermission perm : userPerms) {  
            System.out.println(perm);  
          }  
        }  
      }  
      LOG.info("ACL permission are retrieved successfully.");  
    } catch (Throwable e) {  
      LOG.error("ACL permission retrieval failed", e);  
    } finally {  
      if (aclTable != null) {  
        try {  
          // Close  
          aclTable.close();  
        } catch (IOException e) {   
          LOG.error("Close table failed ", e);  
        }  
      }  
      if (hAdmin != null) {  
        try {  
          // Close Admin Object  
          hAdmin.close();  
        } catch (IOException e) {  
          LOG.error("Close admin failed ", e);  
        }  
      }  
      if (conn != null) {  
        try {  
          // Close HBase connection  
          conn.close();  
        } catch (IOException e) {  
          LOG.error("Close connection failed ", e);  
        }  
      }  
    }  
    LOG.info("Exiting getACL.");  
  }

−          Shell commands:

Command line  
# Grant permissions   
grant <user> <permissions>[ <table>[ <column family>[ <column qualifier> ] ] ]   
 
# Revoke permissions   
revoke <user> <permissions> [ <table> [ <column family> [ <column qualifier> ] ] ]   
 
# Set the owner of a table   
alter <table> {owner => <user>}   
 
# Display the permissions list   
user_permission <table>  # displays existing permissions
For example:
grant 'user1', 'RWC'   
grant 'user2', 'RW', 'tableA'  
 
user_permission  
user_permission 'tableA'

For example:

grant 'user1', 'RWC'   grant 'user2', 'RW', 'tableA'    user_permission  user_permission 'tableA'

HBase Full-Text Index

l  Overview

The full-text index function enables you to create tables and indexes using the createTable method of org.apache.luna.client.LunaAdminand specify table names, column family names, requests for creating indexes, and the directory path for storing mapping files. You can add indexes to existing tables using addCollection. You can obtain tables to perform a scan operation by using the getTable method of org.apache.luna.client.LunaAdmin.

The column name and column family name of a table consist of letters, digits, and underscores (_), excluding any special characters.

HBase tables with a full-text index have the following limitations:

1. Multiple instances are not supported.

2. Disaster recovery, backup, and restoration are not supported.

3. Rows and column families cannot be deleted.

4. Tight consistency is not supported by the query on Solr.

l  Sample code

The following code snippets are in the testFullTextScanmethod in the LunaSample class of the com.huawei.bigdata.hbase.examplespacket.

public static void testFullTextScan() throws Exception {  
    /**  
     * Create create request of Solr. Specify collection name, confset name,  
     * number of shards, and number of replication factor.  
     */  
    Create create = new Create();  
    create.setCollectionName(COLLECTION_NAME);  
    create.setConfigName(CONFSET_NAME);  
    create.setNumShards(NUM_OF_SHARDS);  
    create.setReplicationFactor(NUM_OF_REPLICATIONFACTOR);  
    /**  
     * Create mapping. Specify index fields(mandatory) and non-index  
     * fields(optional).  
     */  
    List<ColumnField> indexedFields = new ArrayList<ColumnField>();  
    indexedFields.add(new ColumnField("name", "f:n"));  
    indexedFields.add(new ColumnField("cat", "f:t"));  
    indexedFields.add(new ColumnField("features", "f:d"));  
    Mapping mapping = new Mapping(indexedFields);  
    /**  
     * Create table descriptor of HBase.  
     */  
    HTableDescriptor desc = new HTableDescriptor(HBASE_TABLE);  
    desc.addFamily(new HColumnDescriptor(TABLE_FAMILY));  
    /**  
     * Create table and collection at the same time.  
     */  
    LunaAdmin admin = null;  
    try {  
      admin = new AdminSingleton().getAdmin();  
      admin.deleteTable(HBASE_TABLE);  
      if (!admin.tableExists(HBASE_TABLE)) {  
        admin.createTable(desc, Bytes.toByteArrays(new String[] { "0", "1", "2", "3", "4" }),  
            create, mapping);  
      }  
      /**  
       * Put data.  
       */  
      Table table = admin.getTable(HBASE_TABLE);  
      int i = 0;  
      while (i < 5) {  
        byte[] row = Bytes.toBytes(i + "+sohrowkey");  
        Put put = new Put(row);  
        put.addColumn(TABLE_FAMILY, Bytes.toBytes("n"), Bytes.toBytes("ZhangSan" + i));  
        put.addColumn(TABLE_FAMILY, Bytes.toBytes("t"), Bytes.toBytes("CO" + i));  
        put.addColumn(TABLE_FAMILY, Bytes.toBytes("d"), Bytes.toBytes("Male, Leader of M.O" + i));  
        table.put(put);  
        i++;  
      }  
 
      /**  
       * Scan table.  
       */  
      Scan scan = new Scan();  
      SolrQuery query = new SolrQuery();  
      query.setQuery("name:ZhangSan1 AND cat:CO1");  
      Filter filter = new FullTextFilter(query, COLLECTION_NAME);  
      scan.setFilter(filter);  
      ResultScanner scanner = table.getScanner(scan);  
      LOG.info("-----------------records----------------");  
      for (Result r = scanner.next(); r != null; r = scanner.next()) {  
        for (Cell cell : r.rawCells()) {  
          LOG.info(Bytes.toString(CellUtil.cloneRow(cell)) + ":"  
              + Bytes.toString(CellUtil.cloneFamily(cell)) + ","  
              + Bytes.toString(CellUtil.cloneQualifier(cell)) + ","  
              + Bytes.toString(CellUtil.cloneValue(cell)));   
        }  
      }  
      LOG.info("-------------------end------------------");  
      /**  
       * Delete collection.  
       */  
      admin.deleteCollection(HBASE_TABLE, COLLECTION_NAME);  
 
      /**  
       * Delete table.  
       */  
      admin.deleteTable(HBASE_TABLE);  
    } catch (IOException e) {  
      e.printStackTrace();  
    } finally {  
      /**  
       * When everything done, close LunaAdmin.  
       */  
      admin.close();  
    }  
  }

1.     Create an index request.

2.     Create a table descriptor.

3.     Obtain the LunaAdmin object. The LunaAdmin provides functions, such as creating tables and indexes, adding indexes, checking whether tables exist, checking whether indexes exist, deleting indexes, and deleting tables.

4.     Invoke LunaAdmin to create a table.

5.     Insert data into a table.

6.     Create full-text index criteria and set FullTextFilter for query.

7.     Delete an index.

8.     Delete a table.

9.     Disable the admin resource.

l  Precautions:

−          Tables and indexes to be created must be unique.

−          You can use LunaAdminonly to obtain tables to perform a scan operation.

1.4 Obtaining Sample Code

Using the FusionInsight Client

Obtain the sample project hdfs-example in the HDFSdirectory in the FusionInsight_Services_ClientConfig file decompressed from the client.

Using the Maven Project

Download the code from the GitHub website to the local computer. The GitHub URL of the sample code is as follows: http://xxx.xxx.xxx/xxxxxxxx/components/hbase.

1.5 Application Commissioning

1.5.1 Commissioning an Application in Windows

1.5.1.1 Compiling and Running an Application

Scenario

After the code development is complete, you can run the application in the Windows development environment.

If IBM JDK is used in the Windows environment, the application cannot be run in  the Windows environment.

Procedure

In the development environment (such as Eclipse), right-click TestMain.java, and choose Run as > Java Application to run the application project.

1.5.1.2 Viewing Windows Commissioning Results

Scenario

After an HBase application is run, you can check the running result through one of the following methods:

l  Viewing the Eclipse running result.

l  Viewing HBase logs.

l  Logging in to the HBase WebUI, see More Information >  External Interfaces > WebUI.

l  Using HBase Shell command, see More Information >  External Interfaces > Shell.

Procedure

l  The following information is displayed in the running results:

2015-09-09 14:11,496 INFO  [main] example.TestMain: Entering testCreateTable.
2015-09-09 14:11,894 INFO  [main] example.TestMain: Creating table...
2015-09-09 14:11,545 INFO  [main] example.TestMain: Master: 10-1-131-140,21300,1441784082485
Number of backup masters: 1
  10-1-131-130,21300,1441784098969
Number of live region servers: 3
  10-1-131-150,21302,1441784158435
  10-1-131-130,21302,1441784126506
  10-1-131-140,21302,1441784118303
Number of dead region servers: 0
Average load: 1.0
Number of requests: 0
Number of regions: 3
Number of regions in transition: 0
2015-09-09 14:11,562 INFO  [main] example.TestMain: [Lorg.apache.hadoop.hbase.NamespaceDescriptor;@11c6af6
2015-09-09 14:11,562 INFO  [main] example.TestMain: Table created successfully.
2015-09-09 14:11,563 INFO  [main] example.TestMain: Exiting testCreateTable.
2015-09-09 14:11,563 INFO  [main] example.TestMain: Entering testMultiSplit.
2015-09-09 14:11,630 INFO  [main] example.TestMain: MultiSplit successfully.
2015-09-09 14:11,630 INFO  [main] example.TestMain: Exiting testMultiSplit.
2015-09-09 14:11,630 INFO  [main] example.TestMain: Entering testPut.
2015-09-09 14:11,148 INFO  [main] example.TestMain: Put successfully.
2015-09-09 14:11,148 INFO  [main] example.TestMain: Exiting testPut.
2015-09-09 14:11,148 INFO  [main] example.TestMain: Entering createIndex.

In the Windows environment, the following exception occurs but does not affect services.

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

l  Log description

The log level is INFO by default and more detailed information can be viewed by changing the log level (DEBUG,INFO, WARN, ERROR, and FATL). You can modify the log4j.propertiesfile to change log levels, for example:

hbase.root.logger=INFO,console
log4j.logger.org.apache.zookeeper=INFO
#log4j.logger.org.apache.hadoop.fs.FSNamesystem=DEBUG
log4j.logger.org.apache.hadoop.hbase=INFO
# Make these two classes DEBUG-level. Make them DEBUG to see more zk debug.
log4j.logger.org.apache.hadoop.hbase.zookeeper.ZKUtil=INFO
log4j.logger.org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher=INFO

1.5.2 Commissioning an Application in Linux

1.5.2.1 Compiling and Running an Application When a Client Is Installed

Scenario

In a Linux environment where an HBase client is installed, you can upload the JAR package to the Linux and then run an application after the code development is complete.

Prerequisites

l  You have installed the HBase client.

l  If the host where the client is installed is not a node in the cluster, set the mapping between the host name and the IP address in the hosts file on the node where the client locates. The host name and IP address must be in one-to-one mapping.

Procedure

                              Step 1     Export a JAR package.

1.        Right-click the example project and choose Exportfrom the shortcut menu.

Figure 1-1 Exporting the JAR packages

2.        Select JAR file and click Next.

Figure 1-2 Selecting JAR file

3.        Select the src and confdirectories, and export the JAR package to the specified location. Click Nexttwice.

Figure 1-3 Selecting the path for exporting the JAR package

4.        Click Browse, select Main class, and click OK.

Figure 1-4 Selecting the class of the main function of the JAR package

5.        Click Finish.

                              Step 2     Run the JAR package.

1.        Before running the JAR package on the Linux client, run the following command to go to the client directory:

cd $BIGDATA_CLIENT_HOME

$BIGDATA_CLIENT_HOME is the installation directory of the HBase client.

2.        Then run the following command:

source $BIGDATA_CLIENT_HOME/bigdata_env

After the multi-instance function is enabled, run the following command to switch to the client of the specified service instance before performing application development for the HBase service instance. For example, for HBase2, run the source /opt/client/HBase2/component_env command.

3.        Upload the JAR package generated in the application development environment to the directory in the client running environment, for example, $BIGDATA_CLIENT_HOME/HBase/hbase/lib, and upload the krb5.conf and user.keytab file to the $BIGDATA_CLIENT_HOME/HBase/hbase/confdirectory on the client. Go to the $BIGDATA_CLIENT_HOME/HBase/hbasedirectory and run the following command to run the JAR package. The task is complete.

hbase com.huawei.bigdata.hbase.examples.TestMain

In the preceding command, hbase com.huawei.bigdata.hbase.examples.TestMain is used as an example.

----End

1.5.2.2 Compiling and Running an Application When No Client Is Installed

Scenario

In a Linux environment where no HBase client is installed, you can upload the JAR package to the Linux and then run an application after the code development is complete.

Prerequisites

l  A JDK has been installed in the Linux environment. The version of the JDK must be consistent with that of the JDK used by the JAR package exported by Eclipse.

l  If the Linux host is not a node in the cluster, set the mapping between the host name and the IP address in the hostsfile on the node. The host name and IP address must be in one-to-one mapping.

Procedure

                              Step 1     Export a JAR package.

1.        Right-click the example project and choose Exportfrom the shortcut menu.

Figure 1-5 Exporting the JAR packages

2.        Select JAR file and click Next.

Figure 1-6 Selecting JAR file

3.        Select the src and confdirectories, and export the JAR package to the specified location. Click Nexttwice.

Figure 1-7 Selecting the path for exporting the JAR package

4.        Click Browse, select Main class, and click OK.

Figure 1-8 Selecting the class of the main function of the JAR package

5.        Click Finish.

                              Step 2     Prepare for the required JAR packages and configuration files.

1.        In the Linux environment, create a directory, for example, /opt/test, and create subdirectories conf and lib. Upload the JAR packages in the sample project and the JAR packages exported in Step 1 to the lib directory in Linux. Upload the conf configuration files in the sample project to the confdirectory in Linux.

2.        In /opt/test, create the run.shscript, modify the following content, and save the file:

#!/bin/sh
BASEDIR=`cd $(dirname $0);pwd`
cd ${BASEDIR}
for file in ${BASEDIR}/lib/*.jar
do
i_cp=$i_cp:$file
echo "$file"
done
for file in ${BASEDIR}/conf/*
do
i_cp=$i_cp:$file
done
java -cp .${i_cp} com.huawei.bigdata.hbase.examples.TestMain

In the preceding content, com.huawei.bigdata.hbase.examples.TestMainis used as an example.

                              Step 3     Go to /opt/test and run the following commands to run the JAR packages:

sh run.sh

----End

1.5.2.3 Viewing Linux Commissioning Results

Scenario

After an HBase application is run, you can check the running result through one of the following methods:

l  Viewing the command output.

l  Viewing HBase logs.

l  Logging in to the HBase WebUI, see More Information >  External Interfaces > WebUI.

l  Using HBase Shell command, see More Information >  External Interfaces > Shell.

Procedure

The following information is displayed in the running results:

2280 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Entering testCreateTable.
3091 [main] WARNcom.huawei.hadoop.hbase.example.HBaseSample- table already exists
3091 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Exiting testCreateTable.
3091 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Entering testPut.
3264 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Put successfully.
3264 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Exiting testPut.
3264 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Entering testGet.
3283 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- 012005000201:info,address,Shenzhen, Guangdong
3283 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- 012005000201:info,name,yugeZhang San
3283 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Get data successfully.
3283 [main] INFOcom.huawei.hadoop.hbase.example.HBaseSample- Exiting testGet.
3283 [main] INFOorg.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation- Closing zookeeper sessionid=0xd000035eba278e9
3297 [main] INFOorg.apache.zookeeper.ZooKeeper- Session: 0xd000035eba278e9 closed
3297 [main-EventThread] INFOorg.apache.zookeeper.ClientCnxn- EventThread shut down
-----------finish HBase -------------------