【Problem Description】:
When the Hive on Hbase table is queried, the select limit operation is executed, and the execution is successful. Execute the select * or select count(*) operation to report the insufficient permissions, as follows:
Error: java.io.IOException: org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘Username‘(table=cs_rec_workrecrela, action=READ)
org.apache.hadoop.hbase.security.access.AccessController.internalPreRead(AccessController.java:1765)
.....
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2157)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
java.lang.Thread.run(Thread.java:745) (state=,code=0)
[Measurement method]:
1) On the HBASE page, find the node where the region of the corresponding table is located.
2) Execute the id Username command in these nodes in sequence, as shown in the following figure, if the node of the user cannot be found, the faulty node is
【solution】:
1) If the regionserver instance process of the abnormal node is stopped on the FI page, the region on the regionserver will be migrated to other nodes.
2) Contact the operating system manufacturer to repair the SSSD service
【problem analysis】:
1. First run the show grant on table command to view the permissions of the table; check the hiveserver node, the user's ownership and permissions; confirm the following, found no problem;
a: show grant on table The results are as follows:
b: The fimanager page to view user permissions is as follows:
The c:hiveserver node views the user rights as follows:
2. Use the select limit ; command to query part of the data of the table, you can return the result normally;
3. Run the select count(*) command to check the running status of the mr task. It is found that a total of 35 map tasks are started. 34 of the tasks are normal. Only one of the mr tasks fails for four consecutive executions. The entire mr task fails to exit:
4. Execute the scan limit operation in hbase to return the result normally.
5. View the number of regions in the table on the hbase native page, and scan the data of each region in turn. It is also found that when scanning a region, an exception with no permission is reported.
6. View the node corresponding to the region from the hbase native page, and check the user status on the node.
a) Query the node hostname corresponding to the region on the hbase native page.
b) Query relevant user information on the node and find that the user information cannot be queried:
7. The OS service sssd of the operating system is faulty, and the user information cannot be queried. The restart is also abnormal.
[Reference analysis ideas]:
View the running status of the MR task corresponding to the execution of the select count(*) operation. It is found that the operation starts 35 Map tasks, of which 34 map tasks are executed successfully, and only one map runs continuously failed four times, causing the task to fail. The reason for the failure of the task is insufficient authority; from the operation of the map task, the user who submits the sql itself has the permission of the related table, otherwise all the maps will fail to run, so it is suspected that the authentication failure of the individual node must first be What you do is find out which node caused the authentication failure.
When the regions in which the table is searched are distributed on those regionservers, the data of each region is scanned by finding the region of the table and the startkey of each region on the hbase sound page. Sure enough, the regionserver node of one of the region distributions was found to be abnormal due to the sssd service failure of the os caused by the client scanning the region.
