Search

In order to overcome the inefficiency and resource consumption of full-text search in relational databases, a light full-text search model with auxiliary cache is developed. Specially, we utilize the MySQL as the data storage layer and the Redis as the index cache layer. We first design a full-index cache mechanism by the Redis-based inverted indexes construction methods to augment the efficient memory processing capability of relational databases. In addition, an increment-index synchronization mechanism is implemented to fit the dynamic update of relation database. For hot data, an index update optimization mechanism is provided to guarantee the fast response and accuracy of full-text search. The proposed Redis-based auxiliary cache method has also been put into practical industrial applications and achieved promising results. Finally, we evaluate our method from index space occupation, time consumption and the accuracy of retrieval results. The experimental results show that the proposed model outperforms MySQL Full-Text method 2–3 times and surpasses ElasticSearch 12 times in space resource consumption. [ABSTRACT FROM AUTHOR]

Corpus retrieval is an important research direction in the field of corpora, which can help learners or applications search for relevant content and improve learning and application efficiency. Traditional retrieval methods have problems such as high development costs, complex structural design, and low data coverage. This article conducts research based on Lucene technology, leveraging its advantages such as inverted index structure, incremental indexing, object-oriented design, text analysis interfaces, and query engine services, to provide a complete solution for developing high-performance large-scale corpus full-text retrieval systems. Firstly, this article studies the foundation of Lucene technology, including system architecture, functional packages, and data flow analysis. Next, study probability retrieval models and directly model the relevance of user needs. Then, this article conducts system design and implementation, including system function design, inverted index design, and retrieval function implementation. Finally, this article conducts system testing and challenges, including building a testing environment, functional testing, and reliability testing. [ABSTRACT FROM AUTHOR]

List intersection plays a pivotal role in various domains such as search engines, database systems, and social networks. Efficient indexes and query strategies can significantly enhance the efficiency of list intersection. Existing inverted index-based algorithms fail to utilize the length information of documents and require excessive list intersections, resulting in lower efficiency. To address this issue, in this paper, we propose the LDRpV (Length-based Document Reordering plus Verification) algorithm. LDRpV filters out documents that are unlikely to satisfy the intersection results by reordering documents based on their length, thereby reducing the number of candidates. Additionally, to minimize the number of list intersection operations, an intersection and verification strategy is designed, where only the first m lists are intersected, and the resulting candidate set is directly verified. This approach effectively improves the efficiency of list intersection. Experimental results on four real datasets demonstrate that LDRpV can achieve a maximum efficiency improvement of 46.69% compared to the most competitive counterparts. [ABSTRACT FROM AUTHOR]

Abstract Peptide spectrum matching is the process of linking mass spectrometry data with peptide sequences. An experimental spectrum can match thousands of candidate peptides with variable modifications leading to an exponential increase in candidates. Completing the search within a limited time is a key challenge. Traditional searches expedite the process by restricting peptide mass errors and variable modifications, but this limits interpretive capability. To address this challenge, we propose Dear‐PSM, a peptide search engine that supports full database searching. Dear‐PSM does not restrict peptide mass errors, matching each spectrum to all peptides in the database and increasing the number of variable modifications per peptide from the conventional 3–20. Leveraging inverted index technology, Dear‐PSM creates a high‐performance index table of experimental spectra and utilizes deep learning algorithms for peptide validation. Through these techniques, Dear‐PSM achieves a speed breakthrough 7 times faster than mainstream search engines on a regular desktop computer, with a remarkable 240‐fold reduction in memory consumption. Benchmark test results demonstrate that Dear‐PSM, in full database search mode, can reproduce over 90% of the results obtained by mainstream search engines when handling complex mass spectrometry data collected from different species using various instruments. Furthermore, it uncovers a substantial number of new peptides and proteins. Dear‐PSM has been publicly released on the GitHub repository https://github.com/jianweishuai/Dear‐PSM.

Users of cloud storage often outsource their data to cloud servers to save local storage resources.However, cloud storage data is out of the user’s physical control, which may lead to malicious theft or disclosure of private data on cloud.At present, dynamic searchable encryption schemes are mostly based on symmetric searchable encryption, which need to establish a secure key sharing channel in advance, and cannot be directly applied to data sharing in cloud storage scenarios.For the secure sharing scenario of multi-party data in cloud storage, a dynamic multi-keyword searchable encryption scheme was proposed.The forward index was constructed by a cuckoo filter to enable data owners to perform documents and index dynamic updates.The conjunctive multi-keywords search was supported by combining bilinear pairing with Lagrangian interpolation polynomials.To reduce the computational overhead in the ciphertext retrieval phase, a combination of inverted index and forward index was constructed to improve the retrieval efficiency of cloud server.This scheme is provably secure with the indistinguishability in adaptively chosen keyword attack under decision linear Diffie-Hellman problem.Simulation experiments were conducted to analyze the execution efficiency of the scheme for keyword search and index update in different datasets.The results show that the scheme effectively avoids the linear correlation between the retrieval time and the number of ciphertexts, and reduces the computational overhead in the update operation with a large amount of data.

List intersection plays a pivotal role in various domains such as search engines, database systems, and social networks. Efficient indexes and query strategies can significantly enhance the efficiency of list intersection. Existing inverted index-based algorithms fail to utilize the length information of documents and require excessive list intersections, resulting in lower efficiency. To address this issue, in this paper, we propose the LDRpV (Length-based Document Reordering plus Verification) algorithm. LDRpV filters out documents that are unlikely to satisfy the intersection results by reordering documents based on their length, thereby reducing the number of candidates. Additionally, to minimize the number of list intersection operations, an intersection and verification strategy is designed, where only the first m lists are intersected, and the resulting candidate set is directly verified. This approach effectively improves the efficiency of list intersection. Experimental results on four real datasets demonstrate that LDRpV can achieve a maximum efficiency improvement of 46.69% compared to the most competitive counterparts.

Spatial keyword query has attracted the attention of many researchers. Most of the existing spatial keyword indexes do not consider the differences in keyword distribution, so their efficiencies are not high when data are skewed. To this end, this paper proposes a novel association rule mining based spatial keyword index, ARM-SQ, whose inverted lists are materialized by the frequent item sets mined by association rules; thus, intersections of long lists can be avoided. To prevent excessive space costs caused by materialization, a depth-based materialization strategy is introduced, which maintains a good balance between query and space costs. To select the right frequent item sets for answering a query, the authors further implement a benefit-based greedy frequent item set selection algorithm, BGF-Selection. The experimental results show that this algorithm significantly outperforms the existing algorithms, and its efficiency can be an order of magnitude higher than SFC-Quad. [ABSTRACT FROM AUTHOR]

The document indexing process aims to store documents in a manner that facilitates the process of retrieving specific documents efficiently in terms of accuracy and time complexity. Many information retrieval systems encounter security issues and execution time to retrieve relevant documents. In addition, these systems lead to ample storage. Therefore, it requires combining confidentiality with the indexed document, and a separate process is performed to encrypt the documents. Hence, a new indexing structure named tree browser (TB) was proposed in this paper to be applied to index files of the large document set in an encrypted manner. This method represents the keywords in a variable-length binary format before being stored in the index. This binary format provides additional encryption to the information stored and reduces the index size. The proposed method (TB) is applied to the WebKB dataset. This dataset is related to web page documents (semi-structured documents). The experimental results demonstrated that the storage size is reduced by using TB-tree to 48.5 MB, while the traditional index is 307 MB. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Approximate string searches have been widely applied in many fields, such as bioinformatics, text retrieval, search engines, and location-based services (LBS). However, the approximate string search results from third-party servers may be incorrect due to the possibility of malicious third parties or compromised servers. In this paper, we design an authenticated index structure (AIS) for string databases, which is based on the Merkle hash tree (MHT) method and the q-gram inverted index. Our AIS can facilitate verification object (VO) construction for approximate string searches with edit distance thresholds. We design an efficient algorithm named GS2 for VO construction at the server side and search result verification at the user side. We also introduce an optimization method called GS2-opt that can reduce VO size dramatically. Finally, we conduct extensive experiments on real datasets to show that our proposed methods are efficient and promising. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Network & Information Security is the property of Beijing Xintong Media Co., Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Indexing is one of the key components of any search tool to be optimized for searching documents. Among the existing indexing techniques, inverted indexing is one of the best methods used at a larger scale for various applications. Under this method, the index is designed using a signature file, hash tree and B-tree to retrieve the required document in efficient time. B-tree is popular due to its searching efficiency, but its performance degrades with increasing data set size. The wavelet tree has become a popular and versatile data structure in the last decade, used in various domains such as sequences, indexing, compression, and grid-point with surprising results. This study proposes a parallel wavelet tree algorithm with hybridization of the Map-Reduce concept to construct an index for textual search. The proposed algorithm reduces the index construction time considerably. Experiments show that the proposed algorithm takes a reasonable trade-off with existing indexing approaches. For large data sets, index construction time has been reduced with respect to other existing state-of-art schemes. Also, results show that the algorithm performs well when the data-set scales up to up-to-the full utilization of available cores. It is possible due to the use of multiple threads working in parallel. Our experiment demonstrated consistent performance with 2-core, 4-core, 8-core, 12-core and results of 16-core show increase in index construction time due to parallel overhead when the data-set in not sufficiently large. [ABSTRACT FROM AUTHOR]

We explore the field of searchable encryption (SE) and present a comprehensive survey of relevant literature. Since the existing survey articles are primarily written for the security experts, we present the discussion to assist the general practitioners (not from security background) in identifying an appropriate SE scheme for their application of interest. We initiate with the brief overview of SE along with its application-oriented criteria. By analyzing various SE schemes, we derive five significant characteristics – key structure, search structure, search functionality, support to reader/writers, and reader's capability. Based on these characteristics, we categorize the existing SE schemes and showcase the significant features offered by each scheme. We explore numerous schemes based on symmetric/asymmetric key structures, simple/inverted search structure, single/multi-keyword search functionality, single/multiple reader/writer support, and verification functionality owned by data reader. A most promising part of the survey is the comparative analysis of the existing schemes under specific category in terms of tables showing efficiency and security. We hope that this survey is indeed beneficial for the general practitioners to pick an appropriate SE scheme better suited to the selected application. [ABSTRACT FROM AUTHOR]

Multi-source skyline query based on road network distance is widely used in map services. However, the existing skyline query methods are inefficient for complex road network distance calculation, and with the increasing of the number of the query point, the query result set becomes too large to provide users with tidy and effective query results. In order to improve the validity of the query results and the query efficiency, this paper proposed an inverted index skyline query algorithm based on the minimum aggregation distance, which established QG-tree index for road network to improve the calculation efficiency of aggregation distance. And it established an inverted index for the points of interest set to retrieve the points of interest combined with pruning strategy, which reduced the overhead of aggregation distance calculation and the dominantion determination, and effectively improved the query efficiency. Finally, experiments on the real road network show that the efficiency of the proposed algorithm is 1～3 orders of magnitude faster than that of the existing algorithms DSR and N3 S, and it can effectively handle the multi-source skyline query problem in the road network. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Basrah Researches (Sciences) is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The increase in the amounts of information used to analyze data is problematic since the memory necessary to store and process it is getting quite big. The interval inverted index representation was developed to reduce the required memory to store data, and Frag-Cubing is one of the most popular algorithms. In this paper, we propose two new data cubing algorithms: 3iCubing and M3iCubing. 3iCubing is a Frag-Cubing-based algorithm that uses the interval inverted index representation, while M3iCubing uses both a normal and interval inverted index data representation. The algorithms were compared using synthetic and real data sets in indexation and querying operations, both runtime and memory-wise. The experimental evaluation shows that 3iCubing can considerably reduce the memory needed to index a data set, reducing around 25% of the memory used by Frag-Cubing. Moreover, the results show that the interval inverted index representation is dependent on the data skewness to reduce the memory consumption, having positive results with highly skewed and real-world data sets.

With the exponential growth of the amount of information on Internet web pages, the search efficiency and accuracy of search engines become a serious challenge. Inverted index as the core index structure of the search engines, its organization and storage have a significant impact on the performance of the search engines. To address the problem that the classical fast inversion algorithm (FAST-INV) cannot build inverted indexes quickly in the face of large-scale document data. This paper proposes two new inverted index construction algorithms: an improvement upon FAST-INV algorithm named FASTER-INV and a new algorithm AC-INV based on Aho-Corasick (AC) automaton for inverted index construction. Firstly, aiming at the redundancy of four information documents in FAST-INV, FASTER-INV is proposed to reduce two unnecessary information documents to build an inverted index. FASTER-INV cuts down redundant information while optimizing the memory space cost. Then this paper further proposes AC-INV, which combines the process of constructing pairs and inverted indexing. AC-INV saves significant memory occupation while ensuring the integrity of information. In addition, it eliminates the time of constructing information documents and improves the algorithm's scalability. Finally, experiments have been conducted on the Chinese AI and Law challenge dataset. The experimental results show that FASTER-INV and AC-INV proposed in this paper are better than FAST-INV. FASTER-INV's speed has increased by 1.11~1.14 times, and the memory has saved about 10%. AC-INV's speed has increased by 1.33~1.43 times, and the memory saved about 35%. [ABSTRACT FROM AUTHOR]

The traditional ciphertext retrieval scheme has some problems, such as low retrieval performance, lack of single keyword retrieval mode and limitation of single machine resources in traditional single server architecture. At the same time, for searchable encryption, it needs to balance the data security and retrieval efficiency. In this paper, a multi-keyword parallel ciphertext retrieval system based on inverted index is proposed. The system adopts different index encryption methods to improve the performance of ciphertext retrieval. Through the segmentation of ciphertext inverted index, the block retrieval of inverted index is realized, which overcomes the limitation of single machine resources and improves the retrieval efficiency. By combining the characteristics of distribution, the traditional single-machine retrieval architecture is extended and multi-keyword parallel retrieval is realized. The experimental results show that compared with SSE-1 scheme, the proposed scheme can improve the efficiency of retrieval, update and other operations on the premise of ensuring the security of ciphertext data, achieve multi-keyword retrieval, and dynamically expand the distributed architecture of the system. Finally, it can improve the system load capacity. [ABSTRACT FROM AUTHOR]

Approximate string searches have been widely applied in many fields, such as bioinformatics, text retrieval, search engines, and location-based services (LBS). However, the approximate string search results from third-party servers may be incorrect due to the possibility of malicious third parties or compromised servers. In this paper, we design an authenticated index structure (AIS) for string databases, which is based on the Merkle hash tree (MHT) method and the q-gram inverted index. Our AIS can facilitate verification object (VO) construction for approximate string searches with edit distance thresholds. We design an efficient algorithm named GS2 for VO construction at the server side and search result verification at the user side. We also introduce an optimization method called GS2-opt that can reduce VO size dramatically. Finally, we conduct extensive experiments on real datasets to show that our proposed methods are efficient and promising.

Summary: It is prevalent nowadays for data owners to outsource their data to the cloud. Most of the traditional searchable schemes are proposed under the honest and curious model, lacking verification of integrity and correctness of the retrieval results. Because the cloud server is not completely trustworthy and there may be improper execution of the users' retrieval requests, it is necessary to verify the retrieval results returned by the cloud server. In order to address these issues, we propose a verifiable ranked ciphertext retrieval scheme (VRCRS) based on bilinear mapping. For the purpose of ranking the keywords and improving the security of ciphertext retrieval scheme, we utilize the lucene search engine toolkit to score keywords, and improve the traditional inverted index structure for building a secure inverted index structure. By adding dummy words into the dictionary of keywords, the proposed scheme can resist the keywords statistics attack of malicious server. To ensure the correctness and integrity of returned ciphertext, we apply bilinear mapping to generate validation tags for keywords. The theoretical analysis and experimental results show that the proposed scheme is secure and efficient. VRCRS can actually identify the illegal behavior of cloud server such as tampering and forgery. [ABSTRACT FROM AUTHOR]

Malicious binary files are a serious threat to industrial information systems. Because of their large number, an automatic assistant tool becomes essential for analysis, and finding similar files would be a great help. In this paper, we present a fast, scalable, and multifaceted search scheme to find similar binary malware files. We use a content-defined chunking algorithm to convert a file into a feature set for the first time. The proposed scheme uses MinHash to reduce any feature set of any file to a fixed size, which significantly improves search accuracy, processing speed, and space utilization. We theoretically prove that the new scheme returns similar files in jaccard index order. Through implementation and experiments with 12 million malicious files, we confirm that the search speed is increased by 600%, space is reduced by 90%, and the accuracy is increased by 400% at least, compared with the state-of-the-art of Elasticsearch.

The rapid growth in the number of scholarly documents on the Web and in other digital platforms makes it challenging for researchers to find research publications most relevant to their information needs. This challenge has been mitigated to a greater extent by the major scholarly retrieval systems, such as Google Scholar, Semantic Scholar, PubMed, CiteSeerX, and others. The reason for the success of these retrieval solutions lies in the advances in ranking approaches. However, the existing studies advocate for the fact that we are still far from the method’s effectiveness ceiling, leaving ample room for further improvement to meet the scholarly needs of users. The existing methods adopt different approaches; some use classical Information Retrieval (IR), others use semantics-aware methods, including Knowledge Graph (KG) to support scholarly search. However, we hypothesize that combining the best of both worlds can further improve search relevance. In this context, this work incorporates inverted index from the classical IR with BM25 as the weighting scheme, combined with Citation Networks Analysis (CNA) for the baseline search results, which are then re-ranked by passing the selected entities from the top-k initial search results as the search query to the KG. This way, not only the textual content but also the structural semantics of the research publications are well exploited in the retrieval processes. The goal is to exploit IR and KG-based retrieval techniques to gain insights into the behavior of both textual and structured information in the strategic ranking of scholarly articles. The proposed solution has been evaluated using the ACL Anthology Network (AAN) dataset. The results show that the proposed technique can comparatively improve the retrieval performance in terms of Normalized Discounted Cumulative Gain (nDCG) and precision rates.

Peptide spectrum matching is the process of linking mass spectrometry data with peptide sequences. An experimental spectrum can match thousands of candidate peptides with variable modifications leading to an exponential increase in candidates. Completing the search within a limited time is a key challenge. Traditional searches expedite the process by restricting peptide mass errors and variable modifications, but this limits interpretive capability. To address this challenge, we propose Dear-PSM, a peptide search engine that supports full database searching. Dear-PSM does not restrict peptide mass errors, matching each spectrum to all peptides in the database and increasing the number of variable modifications per peptide from the conventional 3-20. Leveraging inverted index technology, Dear-PSM creates a high-performance index table of experimental spectra and utilizes deep learning algorithms for peptide validation. Through these techniques, Dear-PSM achieves a speed breakthrough 7 times faster than mainstream search engines on a regular desktop computer, with a remarkable 240-fold reduction in memory consumption. Benchmark test results demonstrate that Dear-PSM, in full database search mode, can reproduce over 90% of the results obtained by mainstream search engines when handling complex mass spectrometry data collected from different species using various instruments. Furthermore, it uncovers a substantial number of new peptides and proteins. Dear-PSM has been publicly released on the GitHub repository https://github.com/jianweishuai/Dear-PSM., Competing Interests: The authors declare no competing interests., (© 2024 The Author(s). Smart Medicine published by Wiley‐VCH GmbH on behalf of Wenzhou Institute, University of Chinese Academy of Sciences.)

Applications for document similarity detection are widespread in diverse communities, including institutions and corporations. However, currently available detection systems fail to take into account the private nature of material or documents that have been outsourced to remote servers. None of the existing solutions can be described as lightweight techniques that are compatible with lightweight client implementation, and this deficiency can limit the effectiveness of these systems. For instance, the discovery of similarity between two conferences or journals must maintain the privacy of the submitted papers in a lightweight manner to ensure that the security and application requirements for limited-resource devices are fulfilled. This paper considers the problem of lightweight similarity detection between document sets while preserving the privacy of the material. The proposed solution permits documents to be compared without disclosing the content to untrusted servers. The fingerprint set for each document is determined in an efficient manner, also developing an inverted index that uses the whole set of fingerprints. Before being uploaded to the untrusted server, this index is secured by the Paillier cryptosystem. This study develops a secure, yet efficient method for scalable encrypted document comparison. To evaluate the computational performance of this method, this paper carries out several comparative assessments against other major approaches. [ABSTRACT FROM AUTHOR]

In order to solve the problems of the current art circulation information is not transparent and the centralized trading authority is asymmetric, this paper proposed a art blockchain certificate traceability model based on three chains. First, in view of the large amount of data in a single piece of artwork information, this paper proposed a blockchain-based symmetric searchable encryption algorithm, and used the interplanetary file system to securely store artwork circulation records on-chain and off-chain. Secondly, for the characteristics of the increasing effective data of artwork information, this paper proposed a blockchain-based inverted fast query and traceability algorithm to help data users realize fast traceability and positioning. At the same time, this paper proposed for the first time that the false information alliance chain stores false art circulation records to realize the processing of false information. Finally, experiments verify the feasibility of the model for big data security storage and fast query traceability. Experiments show that the model is feasible and the query method is efficient. [ABSTRACT FROM AUTHOR]