Got it

[FI Components] Basic concepts for Elasticsearch

Latest reply: Apr 5, 2022 14:24:19 2645 10 2 0 0

Hello all,

I will share with you the Elasticsearch cluster solution today, it consists of the EsMaster and EsClient, EsNode1, EsNode2, EsNode3, EsNode4, EsNode5, EsNode6, EsNode7, EsNode8, and EsNode9 processes, as shown in Figure 1Table 1 describes the modules.

Figure 1 Elasticsearch Structure
https://support.huawei.com/hedex/pages/EDOC1100094385YEI0715H/04/EDOC1100094385YEI0715H/04/resources/en-us_image_0165585769.png


Table 1 Module description

Module

Description

Client

Communicates with the EsClient and EsNode instance processes in the Elasticsearch cluster over HTTP or HTTPS to perform distributed collection and search.

EsMaster

EsMaster is the master node of Elasticsearch. It manages the cluster, such as determining shard allocation and tracing cluster nodes.

EsNode1-9

EsNode1-9 are data nodes of Elasticsearch. They store index data; and add, delete, modify, query, and aggregate documents.

EsClient

EsClient is the coordinator node of Elasticsearch. It processes routing requests, searches for data, and dispatching indexes. EsClient does not store data or manage clusters.

ZooKeeper cluster

Provides heartbeat mechanism for processes in Elasticsearch clusters.

Basic Concepts

  • Index: An index is a logical namespace in Elasticsearch, consisting of one or multiple shards. Apache Lucene is used to read and write data in the index. It is similar to the relational database (RDB) instance database. One Elasticsearch instance can contain multiple indexes.

  • Type: The type is similar to the table in the database. One index corresponds to one document type.

  • Document: A document is a basic unit of information that can be indexed. This document refers to JSON data at the top-level structure or obtained by serializing the root object. The document is similar to the row in the database. A type contains multiple documents.

  • Mapping: A mapping is used to restrict the type of a field and can be automatically created based on data. The mapping is similar to the schema in the database.

  • Field: The field is the minimum unit of a document. The field is similar to the column in the database. Each document contains multiple fields.

  • EsMaster: The master node that temporarily manages some cluster-level changes, such as creating or deleting indexes, and adding or removing nodes. The master node does not participate in document level change or search. When the traffic increases, the master node does not become the bottleneck of the cluster.

  • EsNode: Elasticsearch node. A node is an Elasticsearch instance.

  • ESClient: The coordinator node of Elasticsearch. It processes routing requests, searches for data, and dispatching indexes. EsClient does not store data or manage clusters.

  • Shard: The shard is the smallest work unit in Elasticsearch. The document is stored and referenced in the shard.

  • Primary Shard: Each document in the index belongs to a primary shard. The number of primary shards determines the maximum data that can be stored in the index.

  • Replica Shard: A replica shard is a copy of the primary shard. It prevents data loss caused by hardware faults and provides read requests, such as searching for or retrieving documents from other shards.

  • Recovery: Indicates data restoration or data redistribution. When a node is added or deleted, ElasticSearch redistributes index shards based on the load of the corresponding physical server. When a faulty node is restarted, data restoration is also performed.

  • Gateway: Indicates the storage mode of an ElasticSearch index snapshot. By default, ElasticSearch stores an index in the memory. When the memory is full, ElasticSearch persistently saves the index to the local hard disk. A gateway stores index snapshots. When the corresponding ElasticSearch cluster is stopped and then restarted, the index backup data is read from the gateway. ElasticSearch supports multiple types of gateways, including local file systems (default), distributed file systems, Hadoop HDFS, and Amazon S3 cloud storage.

  • Transport: Indicates the interaction mode between ElasticSearch internal nodes or clusters and the ElasticSearch client. By default, TCP is used for interaction. In addition, HTTP (JSON format), Thrift, Servlet, Memcached, and ZeroMQ transmission protocols (integrated through plug-ins) are supported.

  • ZooKeeper Cluster: It is mandatory in Elasticsearch and provides functions such as storage of security authentication information.

This is what I want to talk about today, thank you!

The post is synchronized to: FusionInsight Components

  • x
  • convention:

GADS
Created Feb 27, 2020 19:12:15

Thanks for sharing.
View more
  • x
  • convention:

little_fish
Admin Created Feb 28, 2020 06:34:57

Thank you.
View more
  • x
  • convention:

kasamon
Created Jun 16, 2020 00:07:17

Thanks for taking the time to share it ¡
View more
  • x
  • convention:

little_fish
little_fish Created Jun 16, 2020 06:13:22 (0) (0)
What‘s this? ¡  
albertsilva
Created Jun 26, 2020 11:16:09

thank you for sharing
View more
  • x
  • convention:

albertsilva
Created Jun 26, 2020 11:16:15

Experienced
View more
  • x
  • convention:

gabo.lr
MVE Created Jun 26, 2020 12:50:46

Thank you for sharing!
View more
  • x
  • convention:

Unicef
MVE Created Jun 26, 2020 13:32:04

Learn a lot
View more
  • x
  • convention:

JackJ
Created Apr 5, 2022 14:24:19

Good one
View more
  • x
  • convention:

little_fish
little_fish Created Apr 6, 2022 00:35:43 (0) (0)
 

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.