Create Partial Table Indexing for Search Sources

Due to the growing number of data and the growing variety of requirements for their processing, now we have to move away from data processing at the time of the request and increasingly shift the main work on its implementation or the implementation of its main aspects to pre-stored and prepared results. In many ways, DBMS thus try to solve performance problems by increasing memory consumption, but in many ways, it is necessary to think about saving the latter, while preferably preserving the results of methods based on a similar approach – indexing, hashing, neural network algorithms. The article discusses a method for improving the efficiency of solving search problems for large tables. The proposed method is based on partial indexing of elements near convergence centers and the introduction of the concept of metadata for these centers. Such clustering with stored metadata for the centers, near which the next intermediate nodes are formed, allows you to reduce the memory costs for indexing, because, firstly, with this approach there is no need for nested indexing, which can lead to serious spatial costs. Secondly, such an approach can make it possible to use one indexing for different combinations of the presence of columns in the search image, without losing most of the search efficiency during indexing. Such a combination, if used correctly, can make it possible to efficiently process tables with different search needs, for different groups of columns, for which storing indexing for each large type of query or group of queries can naturally lead to serious memory consumption costs as well as loss of performance when working with large arrays of memory, which also increases far from linearly.