Search

This paper addresses the need for improved precision in existing Retrieval-Augmented Generation (RAG) methods that primarily focus on enhancing recall. We propose a multi-layer knowledge pyramid approach within the RAG framework to achieve a better balance between precision and recall. The knowledge pyramid consists of three layers: Ontologies, Knowledge Graphs (KGs), and chunk-based raw text. We employ cross-layer augmentation techniques for comprehensive knowledge coverage and dynamic updates of the Ontology schema and instances. To ensure compactness, we utilize cross-layer filtering methods for knowledge condensation in KGs. Our approach, named PolyRAG, follows a waterfall model for retrieval, starting from the top of the pyramid and progressing down until a confident answer is obtained. We introduce two benchmarks for domain-specific knowledge retrieval, one in the academic domain and the other in the financial domain. The effectiveness of the methods has been validated through comprehensive experiments by outperforming 19 SOTA methods. An encouraging observation is that the proposed method has augmented the GPT-4, providing 395\% F1 gain by improving its performance from 0.1636 to 0.8109.

Accurate human localization is crucial for various applications, especially in the Metaverse era. Existing high precision solutions rely on expensive, tag-dependent hardware, while vision-based methods offer a cheaper, tag-free alternative. However, current vision solutions based on stereo vision face limitations due to rigid perspective transformation principles and error propagation in multi-stage SVD solvers. These solutions also require multiple high-resolution cameras with strict setup constraints. To address these limitations, we propose a probabilistic approach that considers all points on the human body as observations generated by a distribution centered around the body's geometric center. This enables us to improve sampling significantly, increasing the number of samples for each point of interest from hundreds to billions. By modeling the relation between the means of the distributions of world coordinates and pixel coordinates, leveraging the Central Limit Theorem, we ensure normality and facilitate the learning process. Experimental results demonstrate human localization accuracy of 95% within a 0.3m range and nearly 100% accuracy within a 0.5m range, achieved at a low cost of only 10 USD using two web cameras with a resolution of 640x480 pixels.

Text-to-video retrieval answers user queries through search by concepts and embeddings. Limited by the size of the concept bank and the amount of training data, answering queries in the wild is not always effective due to the out-of-vocabulary problem. Furthermore, neither concept-based nor embedding-based search can perform reasoning to consolidate the search results for complex queries mixed with logical and spatial constraints. To address these problems, we leverage large language models (LLM) to paraphrase the query by text-to-text (T2T), text-to-image (T2I), and image-to-text (I2T) transformations. These transformations rephrase abstract concepts into simple words to address the out-of-vocabulary problem. Furthermore, the complex relationship in a query can be decoupled into simpler sub-queries, yielding better retrieval performance when fusing the search results of these sub-queries. To address the LLM hallucination problem, this paper also proposes a novel consistency-based verification strategy to filter the paraphrased queries that are factually incorrect. Extensive experiments are conducted for ad-hoc video search and known-item search on the TRECVid datasets. We provide empirical insights into how traditionally difficult-to-answer queries can be resolved by query paraphrasing.

Large Language Models (LLMs) show promise for natural language generation in healthcare, but risk hallucinating factually incorrect information. Deploying LLMs for medical question answering necessitates reliable uncertainty estimation (UE) methods to detect hallucinations. In this work, we benchmark popular UE methods with different model sizes on medical question-answering datasets. Our results show that current approaches generally perform poorly in this domain, highlighting the challenge of UE for medical applications. We also observe that larger models tend to yield better results, suggesting a correlation between model size and the reliability of UE. To address these challenges, we propose Two-phase Verification, a probability-free Uncertainty Estimation approach. First, an LLM generates a step-by-step explanation alongside its initial answer, followed by formulating verification questions to check the factual claims in the explanation. The model then answers these questions twice: first independently, and then referencing the explanation. Inconsistencies between the two sets of answers measure the uncertainty in the original response. We evaluate our approach on three biomedical question-answering datasets using Llama 2 Chat models and compare it against the benchmarked baseline methods. The results show that our Two-phase Verification method achieves the best overall accuracy and stability across various datasets and model sizes, and its performance scales as the model size increases.

Identifying and managing high-emitters among heavy-duty diesel vehicles is a key to mitigating urban air pollution, as a small number of such vehicles could contribute a significant amount of total transport emissions. On-board monitoring (OBM) systems can directly monitor the real-time emission performance of heavy-duty vehicles on road and have become part of the future emissions compliance framework. The challenge, however, lies in the frequent unavailability of OBM data, affecting the effective screening of high-emitting vehicles. This work proposes to bridge the gap by integrating OBM data with remote sensing data to create a comprehensive monitoring system. OBM data is used to characterize the detailed real-world NOx emission performance of both normally-behaving vehicles and high-emitters at various vehicle operating conditions. Remote sensing data is employed to screen out candidate high-emitting vehicles based on thresholds determined by OBM data. Finally, the dynamic NOx emission reduction potential across all roads is mapped by combining the trajectory data for each vehicle with the emission data. A case study in Chengdu, China, utilizing emission and traffic data from heavy-duty vehicles for transporting construction waste (a.k.a. slag trucks), reveals the national threshold for identifying high-emitters via remote sensing might be too lenient, particularly in the medium speed range. An emission reduction of 18.8% in the China V slag truck fleet could be achieved by implementing this novel method in practice in Chengdu. This approach establishes a reliable and ongoing scheme for pinpointing high-emitters through multi-source data, which allows local authorities to develop more robust and targeted strategies to mitigate urban air pollution from heavy-duty diesel vehicles., Comment: 17 pages, 4 figures

Aligning a user query and video clips in cross-modal latent space and that with semantic concepts are two mainstream approaches for ad-hoc video search (AVS). However, the effectiveness of existing approaches is bottlenecked by the small sizes of available video-text datasets and the low quality of concept banks, which results in the failures of unseen queries and the out-of-vocabulary problem. This paper addresses these two problems by constructing a new dataset and developing a multi-word concept bank. Specifically, capitalizing on a generative model, we construct a new dataset consisting of 7 million generated text and video pairs for pre-training. To tackle the out-of-vocabulary problem, we develop a multi-word concept bank based on syntax analysis to enhance the capability of a state-of-the-art interpretable AVS method in modeling relationships between query words. We also study the impact of current advanced features on the method. Experimental results show that the integration of the above-proposed elements doubles the R@1 performance of the AVS method on the MSRVTT dataset and improves the xinfAP on the TRECVid AVS query sets for 2016-2023 (eight years) by a margin from 2% to 77%, with an average about 20%., Comment: Accepted in ICMR2024

Answering query with semantic concepts has long been the mainstream approach for video search. Until recently, its performance is surpassed by concept-free approach, which embeds queries in a joint space as videos. Nevertheless, the embedded features as well as search results are not interpretable, hindering subsequent steps in video browsing and query reformulation. This paper integrates feature embedding and concept interpretation into a neural network for unified dual-task learning. In this way, an embedding is associated with a list of semantic concepts as an interpretation of video content. This paper empirically demonstrates that, by using either the embedding features or concepts, considerable search improvement is attainable on TRECVid benchmarked datasets. Concepts are not only effective in pruning false positive videos, but also highly complementary to concept-free search, leading to large margin of improvement compared to state-of-the-art approaches., Comment: accepted in ACMMM 2020

The reconstruction and prediction of full-state flows from sparse data are of great scientific and engineering significance yet remain challenging, especially in applications where data are sparse and/or subjected to noise. To this end, this study proposes a deep-learning assisted non-intrusive reduced order model (named DCDMD) for high-dimensional flow prediction from sparse data. Based on the compressed sensing (CS)-Dynamic Mode Decomposition (DMD), the DCDMD model is distinguished by two novelties. Firstly, a sparse matrix is defined to overcome the strict random distribution condition of sensor locations in CS, thus allowing flexible sensor deployments and requiring very few sensors. Secondly, a deep-learning-based proxy is invoked to acquire coherent flow modes from the sparse data of high-dimensional flows, thereby addressing the issue of defining sparsity and the stringent incoherence condition in the conventional CSDMD. The two advantageous features, combined with the fact that the model retains flow physics in the online stage, lead to significant enhancements in accuracy and efficiency, as well as superior insensitivity to data noises (i.e., robustness), in both reconstruction and prediction of full-state flows. These are demonstrated by three benchmark examples, i.e., cylinder wake, weekly-mean sea surface temperature and isotropic turbulence in a periodic square area., Comment: 36 Pages, 23 Figures, 5 Tables

Interconnected complex systems usually undergo disruptions due to internal uncertainties and external negative impacts such as those caused by harsh operating environments or regional natural disaster events. To maintain the operation of interconnected network systems under both internal and external challenges, design for resilience research has been conducted from both enhancing the reliability of the system through better designs and improving the failure recovery capabilities. As for enhancing the designs, challenges have arisen for designing a robust system due to the increasing scale of modern systems and the complicated underlying physical constraints. To tackle these challenges and design a resilient system efficiently, this study presents a generative design method that utilizes graph learning algorithms. The generative design framework contains a performance estimator and a candidate design generator. The generator can intelligently mine good properties from existing systems and output new designs that meet predefined performance criteria. While the estimator can efficiently predict the performance of the generated design for a fast iterative learning process. Case studies results based on power systems from the IEEE dataset have illustrated the applicability of the proposed method for designing resilient interconnected systems.

Cross-modal representation learning has become a new normal for bridging the semantic gap between text and visual data. Learning modality agnostic representations in a continuous latent space, however, is often treated as a black-box data-driven training process. It is well-known that the effectiveness of representation learning depends heavily on the quality and scale of training data. For video representation learning, having a complete set of labels that annotate the full spectrum of video content for training is highly difficult if not impossible. These issues, black-box training and dataset bias, make representation learning practically challenging to be deployed for video understanding due to unexplainable and unpredictable results. In this paper, we propose two novel training objectives, likelihood and unlikelihood functions, to unroll semantics behind embeddings while addressing the label sparsity problem in training. The likelihood training aims to interpret semantics of embeddings beyond training labels, while the unlikelihood training leverages prior knowledge for regularization to ensure semantically coherent interpretation. With both training objectives, a new encoder-decoder network, which learns interpretable cross-modal representation, is proposed for ad-hoc video search. Extensive experiments on TRECVid and MSR-VTT datasets show the proposed network outperforms several state-of-the-art retrieval models with a statistically significant performance margin., Comment: accepted in ACM Transactions on Information Systems

Establishing cleaner energy generation therefore improving the sustainability of the power system is a crucial task in this century, and one of the key strategies being pursued is to shift the dependence on fossil fuel to renewable technologies such as wind, solar, and nuclear. However, with the increasing number of heterogeneous components included, the complexity of the hybrid energy system becomes more significant. And the complex system imposes a more stringent requirement of the contingency plan to enhance the overall system resilience. Among different strategies to ensure a reliable system, intentional islanding is commonly applied in practical applications for power systems and attracts abundant interest in the literature. In this study, we propose a hierarchical spectral clustering-based intentional islanding strategy at the transmission level with renewable generations. To incorporate the renewable generation that relies on the inverter technology, the frequency measurements are considered to represent the transient response. And it has been further used as embedded information in the clustering algorithm along with other important electrical information from the system to enrich the modeling capability of the proposed framework. To demonstrate the effectiveness of the islanding strategy, the modified IEEE-9 bus and IEEE-118 bus systems coupled with wind farms are considered as the test cases.

Networked systems are widely applicable in real-world scenarios such as social networks, infrastructure networks, and biological networks. Among those applications, we are interested in social networks due to their complexity and popularity. One crucial task on the social network is to recommend new content based on special characteristics of the graph structure. In this project, we aim to enhance the recommender systems by preventing the recommendations from leaning towards contents from closed communities. To counteract the bias, we will consider information dissemination across network as a metric to assess the recommendation for contents e.g. new connections and news feed. We use academic collaboration network and user-item interaction datasets from Yelp to simulate an environment for connection recommendations and to validate the proposed algorithm.