This post provides an in-depth understanding of the Elasitcsearch storage. Please have a read below.
Elasticsearch path configuration
Elasticsearch has many path configurations.
path.home: home directory of the user who runs the Elasticsearch process. The default value is the JAVA system variable user.dir. The default value of this variable is the home directory of the process owner.
path.conf: contains the configuration file directory. The path is usually set by the JAVA system variable es.config because the value needs to be parsed before the configuration file is found.
path.plugins: contains the file directory of the Elasticsearch plug-in subdirectory. This directory supports symbolic links. When an executable program runs multiple Elasticsearch instances, the feature can selectively disable and enable a series of plug-ins for an Elasticsearch instance.
The path.work: has been used to store temporary files generated during the running of the Elasticsearch and has been discarded.
Directory for storing path.log: logs. You are advised to isolate the directory from the storage space of the data directory to prevent one space from being exhausted.
Directory for storing path.data: Elasticsearch data files.
In this article, we will look at the contents of the data directory (path.data) in detail and study the use of these files.
Where do these files come from?
Because the bottom layer of Elasticsearch uses Lucene to index and query data, files in the data directory are written by both Elasticseach and Lucene.
The division of labor is clear: Lucene writes and maintains the Lucene index file. Elasticsearch writes and maintains the metadata files of features built on Lucene, such as field mapping, index setting and other cluster metadata (the underlying Lucene does not support the user-side or supporting features supported by Elasticsearch).
Before studying the Lucene index file, let's look at the data directory outer structure created by Elasticsearch.
Node data
Let's start with the empty data directory of the Elasticsearch cluster. The directory structure is as follows:
The node.lock file is used to ensure that only one Elasticsearch instance is reading and writing to the data directory at a time.
global-0.st is an interesting file. The global-prefix indicates that this is a global state file, and the .st suffix indicates that it is a state file containing metadata.
As you can guess, the binary file contains the global metadata of the cluster and the numbers behind the prefix represent the version of the cluster metadata (a strict version mechanism provided by the cluster).
