Search

This article presents the design of an open-API-based explainable AI (XAI) service to provide feature contribution explanations for cloud AI services. Cloud AI services are widely used to develop domain-specific applications with precise learning metrics. However, the underlying cloud AI services remain opaque on how the model produces the prediction. We argue that XAI operations are accessible as open APIs to enable the consolidation of the XAI operations into the cloud AI services assessment. We propose a design using a microservice architecture that offers feature contribution explanations for cloud AI services without unfolding the network structure of the cloud models. We can also utilize this architecture to evaluate the model performance and XAI consistency metrics showing cloud AI services trustworthiness. We collect provenance data from operational pipelines to enable reproducibility within the XAI service. Furthermore, we present the discovery scenarios for the experimental tests regarding model performance and XAI consistency metrics for the leading cloud vision AI services. The results confirm that the architecture, based on open APIs, is cloud-agnostic. Additionally, data augmentations result in measurable improvements in XAI consistency metrics for cloud AI services., Comment: Published in: IEEE Transactions on Cloud Computing ( Volume: 12, Issue: 2, April-June 2024)

Transformer-based models have achieved state-of-the-art performance in various computer vision tasks, including image and video analysis. However, Transformer's complex architecture and black-box nature pose challenges for explainability, a crucial aspect for real-world applications and scientific inquiry. Current Explainable AI (XAI) methods can only provide one-dimensional feature importance, either spatial or temporal explanation, with significant computational complexity. This paper introduces STAA (Spatio-Temporal Attention Attribution), an XAI method for interpreting video Transformer models. Differ from traditional methods that separately apply image XAI techniques for spatial features or segment contribution analysis for temporal aspects, STAA offers both spatial and temporal information simultaneously from attention values in Transformers. The study utilizes the Kinetics-400 dataset, a benchmark collection of 400 human action classes used for action recognition research. We introduce metrics to quantify explanations. We also apply optimization to enhance STAA's raw output. By implementing dynamic thresholding and attention focusing mechanisms, we improve the signal-to-noise ratio in our explanations, resulting in more precise visualizations and better evaluation results. In terms of computational overhead, our method requires less than 3\% of the computational resources of traditional XAI methods, making it suitable for real-time video XAI analysis applications. STAA contributes to the growing field of XAI by offering a method for researchers and practitioners to analyze Transformer models.

Polaritonic crystals (PoCs) have experienced significant advancements through involving hyperbolic polaritons in anisotropic materials such as $\alpha$-MoO$_{\rm 3}$, offering a promising approach for nanoscale light control and improved light-matter interactions. Notably, twisted bilayer $\alpha$-MoO$_{\rm 3}$ enables tunable iso-frequency contours (IFCs), especially generating flat IFCs at certain twist angles, which could enhance mode selectivity in their PoCs through the highly collimated and canalized polaritons. This study unveils the selective excitation of Bloch modes in PoCs with square-lattice structures on twisted bilayer $\alpha$-MoO$_{\rm 3}$ with canalized phonon polaritons. Through the optimization of the square lattice design, there is an effective redistribution of canalized polaritons into the reciprocal lattices of PoCs. Fine-tuning the periodicity and orientation of the hole lattice enables momentum matching between flat IFCs and co-linear reciprocal points, allowing precise and directional control over desired Bragg resonances and Bloch modes. This research establishes a versatile platform for tunable polaritonic devices and paves the way for advanced photonic applications., Comment: 20 pages, 4 figures or other essential info

With the evolution of large language models, traditional Transformer models become computationally demanding for lengthy sequences due to the quadratic growth in computation with respect to the sequence length. Mamba, emerging as a groundbreaking architecture in the field of generative AI, demonstrates remarkable proficiency in handling elongated sequences with reduced computational and memory complexity. Nevertheless, the existing training framework of Mamba presents inefficiency with variable-length sequence inputs. Either single-sequence training results in low GPU utilization, or batched processing of variable-length sequences to a maximum length incurs considerable memory and computational overhead. To address this problem, we analyze the performance of bottleneck operators in Mamba under diverse tensor shapes and proposed PackMamba, a high-throughput Mamba that efficiently handles variable-length sequences. Diving deep into state-space models (SSMs), we modify the parallel operators to avoid passing information between individual sequences while maintaining high performance. Experimental results on an NVIDIA A100 GPU demonstrate throughput exceeding the baseline single-sequence processing scheme: 3.06x speedup on the 1.4B model and 2.62x on the 2.8B model.

Large Language Models (LLMs) like GPT and LLaMA are revolutionizing the AI industry with their sophisticated capabilities. Training these models requires vast GPU clusters and significant computing time, posing major challenges in terms of scalability, efficiency, and reliability. This survey explores recent advancements in training systems for LLMs, including innovations in training infrastructure with AI accelerators, networking, storage, and scheduling. Additionally, the survey covers parallelism strategies, as well as optimizations for computation, communication, and memory in distributed LLM training. It also includes approaches of maintaining system reliability over extended training periods. By examining current innovations and future directions, this survey aims to provide valuable insights towards improving LLM training systems and tackling ongoing challenges. Furthermore, traditional digital circuit-based computing systems face significant constraints in meeting the computational demands of LLMs, highlighting the need for innovative solutions such as optical computing and optical networks.

In this study, we propose the early adoption of Explainable AI (XAI) with a focus on three properties: Quality of explanation, the explanation summaries should be consistent across multiple XAI methods; Architectural Compatibility, for effective integration in XAI, the architecture styles of both the XAI methods and the models to be explained must be compatible with the framework; Configurable operations, XAI explanations are operable, akin to machine learning operations. Thus, an explanation for AI models should be reproducible and tractable to be trustworthy. We present XAIport, a framework of XAI microservices encapsulated into Open APIs to deliver early explanations as observation for learning model quality assurance. XAIport enables configurable XAI operations along with machine learning development. We quantify the operational costs of incorporating XAI with three cloud computer vision services on Microsoft Azure Cognitive Services, Google Cloud Vertex AI, and Amazon Rekognition. Our findings show comparable operational costs between XAI and traditional machine learning, with XAIport significantly improving both cloud AI model performance and explanation stability., Comment: Accepted at the ICSE'24 conference, NIER track

Large Language Models (LLMs) have presented impressive performance across several transformative tasks. However, it is non-trivial to efficiently utilize large-scale cluster resources to develop LLMs, often riddled with numerous challenges such as frequent hardware failures, intricate parallelization strategies, and imbalanced resource utilization. In this paper, we present an in-depth characterization study of a six-month LLM development workload trace collected from our GPU datacenter Acme. Specifically, we investigate discrepancies between LLMs and prior task-specific Deep Learning (DL) workloads, explore resource utilization patterns, and identify the impact of various job failures. Our analysis summarizes hurdles we encountered and uncovers potential opportunities to optimize systems tailored for LLMs. Furthermore, we introduce our system efforts: (1) fault-tolerant pretraining, which enhances fault tolerance through LLM-involved failure diagnosis and automatic recovery. (2) decoupled scheduling for evaluation, which achieves timely performance feedback via trial decomposition and scheduling optimization.

The opacity of AI models necessitates both validation and evaluation before their integration into services. To investigate these models, explainable AI (XAI) employs methods that elucidate the relationship between input features and output predictions. The operations of XAI extend beyond the execution of a single algorithm, involving a series of activities that include preprocessing data, adjusting XAI to align with model parameters, invoking the model to generate predictions, and summarizing the XAI results. Adversarial attacks are well-known threats that aim to mislead AI models. The assessment complexity, especially for XAI, increases when open-source AI models are subject to adversarial attacks, due to various combinations. To automate the numerous entities and tasks involved in XAI-based assessments, we propose a cloud-based service framework that encapsulates computing components as microservices and organizes assessment tasks into pipelines. The current XAI tools are not inherently service-oriented. This framework also integrates open XAI tool libraries as part of the pipeline composition. We demonstrate the application of XAI services for assessing five quality attributes of AI models: (1) computational cost, (2) performance, (3) robustness, (4) explanation deviation, and (5) explanation resilience across computer vision and tabular cases. The service framework generates aggregated analysis that showcases the quality attributes for more than a hundred combination scenarios., Comment: 2024 IEEE International Conference on Software Services Engineering (SSE)

Accurate segmentation of surgical instrument tip is an important task for enabling downstream applications in robotic surgery, such as surgical skill assessment, tool-tissue interaction and deformation modeling, as well as surgical autonomy. However, this task is very challenging due to the small sizes of surgical instrument tips, and significant variance of surgical scenes across different procedures. Although much effort has been made on visual-based methods, existing segmentation models still suffer from low robustness thus not usable in practice. Fortunately, kinematics data from the robotic system can provide reliable prior for instrument location, which is consistent regardless of different surgery types. To make use of such multi-modal information, we propose a novel visual-kinematics graph learning framework to accurately segment the instrument tip given various surgical procedures. Specifically, a graph learning framework is proposed to encode relational features of instrument parts from both image and kinematics. Next, a cross-modal contrastive loss is designed to incorporate robust geometric prior from kinematics to image for tip segmentation. We have conducted experiments on a private paired visual-kinematics dataset including multiple procedures, i.e., prostatectomy, total mesorectal excision, fundoplication and distal gastrectomy on cadaver, and distal gastrectomy on porcine. The leave-one-procedure-out cross validation demonstrated that our proposed multi-modal segmentation method significantly outperformed current image-based state-of-the-art approaches, exceeding averagely 11.2% on Dice., Comment: Accepted to IROS 2023

This paper addresses the problem of controlling a continuum manipulator (CM) in free or obstructed environments with no prior knowledge about the deformation behavior of the CM and the stiffness and geometry of the interacting obstructed environment. We propose a versatile data-driven priori-model-independent (PMI) control framework, in which various control paradigms (e.g. CM's position or shape control) can be defined based on the provided feedback. This optimal iterative algorithm learns the deformation behavior of the CM in interaction with an unknown environment, in real time, and then accomplishes the defined control objective. To evaluate the scalability of the proposed framework, we integrated two different CMs, designed for medical applications, with the da Vinci Research Kit (dVRK). The performance and learning capability of the framework was investigated in 11 sets of experiments including PMI position and shape control in free and unknown obstructed environments as well as during manipulation of an unknown deformable object. We also evaluated the performance of our algorithm in an ex-vivo experiment with a lamb heart.The theoretical and experimental results demonstrate the adaptivity, versatility, and accuracy of the proposed framework and, therefore, its suitability for a variety of applications involving continuum manipulators., Comment: 28 pages, 15 Figures

We collect a sample of 78 Fermi detected blazars with broad line region (BLR) data and the quasi-simultaneous multi-wave band data. By analyzing the sample, we find: (1) For whole blazar sample, there exist significant correlations between radio, $\gamma$-ray luminosity and BLR luminosity. The slope of the best-fit linear regression equation is close to the theoretical value 1. Our results provided another support of jet-disk symbiosis. (2) For FSRQ sample, we find a significant correlation between radio luminosity and optical luminosity, and weaker correlations between X-ray luminosity and radio, optical luminosity. It indicates that the X-ray band radiation mechanism is slightly different from radio and optical bands. The X-ray emission is diluted by inverse Compton scattering (IC). There is a significant correlation between BLR luminosity and $\gamma$-ray luminosity, but no correlations between X-ray luminosity and BLR, $\gamma$-ray luminosity, which suggests that the seed soft photons are different for IC process. From these results we suggest that the X-ray band emission is diluted by synchrotron self-Compton (SSC) process, $\gamma$-ray band emission is produced by external Compton (EC) process., Comment: 10 pages, 1 figure, accepted for publication in Ap&SS

Bolometric luminosity is a basic physical parameter that is widely used in the study of blazars. Due to the lack of simultaneous full wavelength data, several estimators of the bolometric luminosity are being used in practice. In this paper, we study and evaluate the reliability and significance of six estimators, the $5GHz$ luminosity, the 1keV luminosity, the $\gamma$-ray luminosity, the $5GHz$ luminosity+the 1keV luminosity, the $5GHz$ luminosity+the $\gamma$-ray luminosity and the 1keV luminosity+the $\gamma$-ray luminosity, by analyzing the linear correlations between the integrated bolometric luminosity and them. Our main results are as follows. (i) All the six estimators are reliable in the sense that they are all significant correlated with the bolometric luminosity. (ii) Ranking from the higher significance of the reliability to lower one the six estimators are the the $5GHz$ luminosity+the $\gamma$-ray luminosity, the 1keV luminosity+the $\gamma$-ray luminosity, $\gamma$-ray luminosity, the $5GHz$ luminosity+the 1keV luminosity, the $5GHz$ luminosity and the 1keV luminosity. (iii) We suggest that the bolometric luminosity can be well estimated by the $\gamma$-ray luminosity using the best linear equation that given in this paper for Fermi FSRQs. (iv) According to the linear regressions obtained in the analysis, we provide calibration for each estimator., Comment: 10 pages, 4 figures, accepted for publication in Ap&SS