Got it

Multiple Elasticsearch Instances Are Faulty Because the value of index.max_result_window Is Too Large

Latest reply: Nov 21, 2021 15:42:16 366 2 3 0 0

Hello all, 

In this case, mainly talks about "Multiple Elasticsearch Instances Are Faulty Because the value of index.max_result_window Is Too Large".

Applicable Version

6.5.x

Context and Symptom

A large amount of data is returned for the query request. As a result, multiple instances report the heap memory threshold exceeded alarm, some Elasticsearch instances are suspended, two Elasticsearch instances are taken offline, and the primary shard is down.

Cause Analysis

1. The ISV services continuously initiate a request for querying 10 million records at a time. As a result, the heap memory of the Elasticsearch instances is quickly used up and the instances are suspended.

2. The default value of index.max_result_window is 10000. It is suspected that the services have adjusted the value. Check the index settings and find that the services have changed the value to 2 billion.

Solution

1. Restart the Elasticsearch cluster to release memory. The restoration takes about 50 minutes. The translog restoration of a primary shard takes 45 minutes, which is too long.

2. Set the maximum number of records that can be returned in a query using the index settings to 10000.

curl -XPUT --negotiate -k -u : 'https://ip:port/Index name/_settings?pretty' -H 'Content-Type: application/json' -d '{"index.max_result_window": "10000"}'

3. The ISV adjusts the service query logic to query full data in scroll mode to avoid excessive query results.

Any solutions will be appreciated!

The post is synchronized to: Huawei Cloud Computing Case

  • x
  • convention:

olive.zhao
Admin Created Nov 15, 2021 02:40:06

Thanks for your sharing!
View more
  • x
  • convention:

user_4358465
Created Nov 21, 2021 15:42:16

Informative post. Thanks for sharing
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.