Got it

Restoring the Damaged ES Lucene Segment Files

Latest reply: Dec 2, 2021 05:27:43 312 3 3 0 0

Hello everyone, 

I believe you can learn how to deal with Restoring the Damaged ES Lucene Segment Files from my case.

The disk of an instance in the Elasticsearch cluster on the live network is faulty for one month. After the disk is restored one month later, the data files of the Elasticsearch instance are restored from the disk. After the disk is restored, the instance cannot be started and all shards on the instance are in the UNASSIGN state. Some indexes at this site are empty. Before locating the problem, you are advised to check the number of indexes involved, the data volume of these indexes, and whether there are some empty indexes.

[Problem Description]

Restoring the Damaged ES Lucene Segment Files.


The kernel code of Lucene provides a CheckIndex tool. The tool can detect whether the Lucene segment file is damaged. If the segment file is damaged, the tool can perform a parameter repair. The repair principle is to remove the damaged segment file. Run the following commands to create an empty file (data may be lost):

Usage: java org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-crossCheckTermVectors] [-segment X] [-segment Y] [-dir-impl X]

 -exorcise: actually write a new segments_N file, removing any problematic segments
 -fast: just verify file checksums, omitting logical integrity checks
 -crossCheckTermVectors: verifies that term vectors match postings; THIS IS VERY SLOW!
 -codec X: when exorcising, codec to write the new segments_N file with
 -verbose: print additional details
 -segment X: only check the specified segments.  This can be specified multiple
             times, to check more than one segment, eg '-segment _2 -segment _a'.
             You can't use this with the -exorcise option
 -dir-impl X: use a specific FSDirectory implementation. If no package is specified the org.apache.lucene.store package will be used.

**WARNING**: -exorcise *LOSES DATA*. This should only be used on an emergency basis as it will cause
documents (perhaps many) to be permanently removed from the index.  Always make
a backup copy of your index before running this!  Do not run this tool on an index
that is actively being written to.  You have been warned!

Run without -exorcise, this tool will open the index, report version information
and report any exceptions it hits and what action it would take if -exorcise were
specified.  With -exorcise, this tool will remove any segments that have issues and
write a new segments_N file.  This means all documents contained in the affected
segments will be removed.

I hope it is helpful to you!


  • x
  • convention:

thibay
Created Mar 31, 2021 14:28:41

Thanks for sharing
View more
  • x
  • convention:

Unicef
MVE Created Jul 21, 2021 15:41:28

Well done
View more
  • x
  • convention:

MahMush
Moderator Author Created Dec 2, 2021 05:27:43

it is very useful ...please defined further on it
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.