Got it

[FI Components] Working Principle of Solr

Latest reply: Apr 11, 2022 13:53:43 1217 21 12 0 0

Dear all,

This post tells you the working principle of Solr and the relationship with other components in Fusioninsight HD.

Descending-order Indexing

The traditional search (which uses the ascending-order indexing, as shown in Figure 1) starts from keypoints and then uses the keypoints to find the specific information that meets the search criteria. In the traditional mode, values are found according to keys. During search based on the ascending-order indexing, keywords are found by document number.


Figure 1 Ascending-order indexing

Solr Ascending-order indexing

The Solr (Lucene) search uses the descending-order indexing mode (as shown in Figure 2). In this mode, keys are found according to values. Values in the full-text search indicate the keywords that need to be searched. Places where the keywords are stored are called dictionaries. Keys indicate document number lists, with which users can find the documents that contain the search keywords (values), as shown in the following figure. During search based on the descending-order indexing, document numbers are found by keyword and then documents are found by document number.


Figure 2 Descending-order indexing

Solr Descending-order indexing

Distributed Indexing Operation Procedure

Figure 3 describes the Solr distributed indexing operation procedure.


Figure 3 Distributed indexing operation procedure

Distributed indexing operation procedure Solr

The procedure is as follows:


1. When initiating a document indexing request, the Client obtains the SolrServer cluster information of SolrCloud from the ZooKeeper cluster, and then obtains any SolrServer that contains the Collection information according to the Collection information in the request.

2. The Client sends the document indexing request to a Replica of the related Shard in the Collection of the SolrServer.

3. If the Replica is not the Leader Replica, the Replica will forward the document indexing request to the Leader Replica in the same Shard.

4. After indexing documents locally, the Leader Replica routes the document indexing request to other Replicas for processing.

5. If the target Shard of the document indexing is not the Shard of this request, the Leader Replica of the Shard will forward the document indexing request to the Leader Replica of the target Shard.

6. After indexing documents locally, the Leader Replica of the target Shard routes the document indexing request to other Replicas of the Shard of the request for processing.

Distributed Search Operation Procedure

Figure 4 describes the Solr distributed search operation procedure.


Figure 4 Distributed search operation procedure

Distributed search operation procedure Solr

The procedure is as follows:


1. When initiating a search request, the Client obtains the SolrServer cluster information using ZooKeeper and then randomly selects a SolrServer that contains the Collection.

2. The Client sends the search request to any Replica (which does not need to be the Leader Replica) of the related Shard in the Collection of the SolrServer for processing.

3. The Replica starts a distributed query, converts the query into multiple subqueries based on the number of Shards of the Collection (there are two Shards in Figure 5, Shard 1 and Shard 2), and distributes each subquery to any Replica (which does not need to be the Leader Replica) of the related Shard for processing.

4. After each subquery is completed, the query results are returned.

5. After receiving the results of each subquery, the Replicas that receives a query request for the first time combines the query results and then sends the final results to the Client.


Relationship Between Solr and HDFS

Solr is a project of the Apache Software Foundation and a major component in the ecosystem of the Apache Hadoop project. Solr can use the Hadoop Distributed File System (HDFS) as its index file storage system. Solr is located on the structured storage layer. The HDFS provides highly reliable support for the storage of Solr. All index data files of Solr can be stored in the HDFS.


Relationship Between Solr and HBase

HBase stores massive data. It is a distributed column-oriented storage system built on the HDFS. Indexing for HBase data by Solr is the process of writing HBase data into the HDFS and creating indexes for HBase data. The index ID corresponds to the HBase data according to rowkey. Ensure that each piece of index data is unique and each piece of HBase data is unique, implementing full-text search for HBase data.


Thank you.

The post is synchronized to: FusionInsight Components

  • x
  • convention:

stephen.xu
Admin Created Sep 10, 2020 07:44:10

Working Principle of Solr
View more
  • x
  • convention:

little_fish
little_fish Created Sep 10, 2020 07:44:55 (0) (0)
yes  
ChanKaFai
Created Sep 10, 2020 07:59:21

good
View more
  • x
  • convention:

little_fish
little_fish Created Sep 18, 2020 14:21:29 (0) (0)
 
carter12
Created Sep 10, 2020 07:59:35

thanks
View more
  • x
  • convention:

little_fish
little_fish Created Sep 18, 2020 14:21:36 (0) (0)
 
andrew890
Created Sep 18, 2020 12:35:41

interesting content
View more
  • x
  • convention:

little_fish
little_fish Created Sep 18, 2020 14:21:44 (0) (0)
 
gabo.lr
MVE Created Sep 18, 2020 13:07:06

Thank you for sharing!
View more
  • x
  • convention:

little_fish
little_fish Created Sep 18, 2020 14:21:51 (0) (0)
 
user_4083247
Created Dec 27, 2020 08:06:32

Order
View more
  • x
  • convention:

little_fish
little_fish Created Apr 16, 2021 01:39:12 (0) (0)
Solr  
VinceD
Moderator Created Apr 15, 2021 08:02:40

interesting...
View more
  • x
  • convention:

little_fish
little_fish Created Apr 16, 2021 01:38:56 (0) (0)
Thank you.  
Unicef
MVE Created Apr 15, 2021 11:59:37

Well done
View more
  • x
  • convention:

little_fish
little_fish Created Apr 16, 2021 01:38:46 (0) (0)
Thanks  
titusmahwe
Created Apr 17, 2021 07:44:37

Your energy and determination towards Big data and cloud computing is awesome. Nice read.[FI Components] Working Principle of Solr-3887265-1
View more
  • x
  • convention:

little_fish
little_fish Created Apr 17, 2021 07:57:35 (0) (0)
And storage.  
12
Back to list

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.