Got it

Data Write to Elasticsearch Is Slow Due to put-mapping Timeout

Latest reply: Nov 21, 2021 15:35:44 519 2 2 0 0

Hello all, 

This post mainly talks about "Data Write to Elasticsearch Is Slow Due to put-mapping Timeout".

Applicable Version

6.5.x

Context and Symptom

The customer reports that data writing to the Elasticsearch on FusionInsight HD is slow.

1. The data writing to an Elasticsearch cluster is slow, and the client reports a write rejection error.

Elasticsearch cluster

2. Check the Elasticsearch run logs. It is found that the cluster times out during the put-mapping operation.

Elasticsearch run logs

Error information: failed to process cluster event (put-mapping) within 30s

Cause Analysis

The possible causes are as follows:

1. The number of indexes and shards is too large.

Indexes are created by day. A large number of indexes are created during data writing, and the number of shards in the cluster has reached 70,000+.

2. This is the most common case when an index is automatically created by writing data.

An index is created automatically when a cluster writes data in bulk mode.

3. The refresh interval for writing a large amount of data is too short.

4. The number of index fields is too large (dozens of or hundreds of fields).

More than 100 fields are written into the index.

According to the feedback information, the possible cases 1, 2, and 4 meet the requirements.

Because there are too many index fields and not all fields are set in the index template, Elasticsearch performs dynamic mapping on data. When a large amount of data is written and new fields are added, a large number of put_mapping operations are performed, causing EsMaster blocking, and the running of the entire Elasticsearch cluster is affected.

Solution

1. Control the number of indexes. If the number of indexes created by day is too large, you can create indexes by month. If indexes are created by month, increase the number of index shards. Control the number of shards. The number of shards in a single cluster must be less than 50,000.

2. You are advised to create indexes in advance instead of automatically creating indexes when data is written. (To be improved on the service side)

3. Control the number of fields. (To be improved on the service side based on the actual service requirements)

4. Use dynamic mapping. Because dynamic mapping cannot be disabled due to service requirements. You are advised to use the mapping rules as precisely as possible to prevent violent operations, such as using wildcard characters (*).

Any solutions will be appreciated!

  • x
  • convention:

olive.zhao
Admin Created Nov 16, 2021 08:50:18

Thanks for your sharing!
View more
  • x
  • convention:

user_4358465
Created Nov 21, 2021 15:35:44

Excellent solution post!
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.