This post refers to an import fault. Let's find the process of how to tackle it below.
Impact
LibrA supports three data import modes to adapt to different service scenarios.
In video cloud scenarios, a large number of records are imported to the database in a unit time, and users have high requirements for timeliness in data loading. Therefore, data import typically invokes the CopyManger interface to consume data from Kafka and then load the data to LibrA in microbatch mode. In this case, the setting of a microbatch window size greatly affects data loading efficiency and cluster resource consumption.
Inspection method
Investigate application developers and do code walkthrough if necessary to learn about the data import mode and the microbatch window size.
Optimization method
According to experience, it is recommended that the size of a micro-batch window for CopyManger loading be set to a value meeting either of the following conditions:
l The number of records reaches 10,000.
l The loading interval reaches 60s.
