Search

The increasing popularity of deep learning models has created new opportunities for developing AI-based recommender systems. Designing recommender systems using deep neural networks requires careful architecture design, and further optimization demands extensive co-design efforts on jointly optimizing model architecture and hardware. Design automation, such as Automated Machine Learning (AutoML), is necessary to fully exploit the potential of recommender model design, including model choices and model-hardware co-design strategies. We introduce a novel paradigm that utilizes weight sharing to explore abundant solution spaces. Our paradigm creates a large supernet to search for optimal architectures and co-design strategies to address the challenges of data multi-modality and heterogeneity in the recommendation domain. From a model perspective, the supernet includes a variety of operators, dense connectivity, and dimension search options. From a co-design perspective, it encompasses versatile Processing-In-Memory (PIM) configurations to produce hardware-efficient models. Our solution space's scale, heterogeneity, and complexity pose several challenges, which we address by proposing various techniques for training and evaluating the supernet. Our crafted models show promising results on three Click-Through Rates (CTR) prediction benchmarks, outperforming both manually designed and AutoML-crafted models with state-of-the-art performance when focusing solely on architecture search. From a co-design perspective, we achieve 2x FLOPs efficiency, 1.8x energy efficiency, and 1.5x performance improvements in recommender models., Comment: Accepted in ACM Transactions on Recommender Systems. arXiv admin note: substantial text overlap with arXiv:2207.07187

Multipartite quantum scenarios are a significant and challenging resource in quantum information science. Tensors provide a powerful framework for representing multipartite quantum systems. In this work, we introduce the role of tensor-generated matrices that can broadly be defined as the relationships between an $m$-th order $n$-dimensional tensor and an $n$-dimensional square matrix. Through these established connections, we demonstrate that the classification of the tensor-generated matrix as an $H$-matrix implies the original tensor is also an $H$-tensor. We also explore various similar properties exhibited by both the original tensor and the tensor-generated matrix, including weak irreducibility, weakly chained diagonal dominance, and (strong) symmetry. These findings provide a method to transform intricate tensor problems into matrices in specific contexts, which is especially pertinent due to the NP-hard complexity of the majority of tensor problems. Subsequently, we explore the application of tensor-generated matrices in analyzing the classicality of spin states. Leveraging the tensor representation, we introduce classicality criteria for (strongly) symmetric spin-$j$ states, which potentially provide fresh perspectives on the study of multipartite quantum resources. Finally, we extend classical matrix eigenvalue inclusion sets to higher-order tensor $H$-eigenvalues, a task that is typically challenging for higher-order tensors. Consequently, we propose representative tensor $H$-eigenvalue inclusion sets, such as modified Brauer's Ovals of Cassini sets, Ostrowski sets, and $S$-type inclusion sets.

Separability from the spectrum is a significant and ongoing research topic in quantum entanglement. In this study, we investigate properties related to absolute separability from the spectrum in qudits-qudits states in the bipartite states space $\mathcal{H}_{mn}=\mathcal{H}_m \otimes \mathcal{H}_n$. Firstly, we propose the necessary and sufficient conditions for absolute separable states in the Hilbert space $\mathcal{H}_{4n}$. These conditions are equivalent to the positive semidefiniteness of twelve matrices resulting from the symmetric matricizations of eigenvalues. Furthermore, we demonstrate that this sufficient condition can be extended to the general $\mathcal{H}_{mn}$ case, improving existing conclusions in the literature. These sufficient conditions depend only on the first few leading and last few leading eigenvalues, significantly reducing the complexity of determining absolute separable states. On the other hand, we also introduce additional sufficient conditions for determining that states in $\mathcal{H}_{mn}$ are not absolutely separable. These conditions only depend on $2m-1$ eigenvalues of the mixed states. Our sufficient conditions are not only simple and easy to implement. As applications, we derive distance bounds for eigenvalues and purity bounds for general absolutely separable states., Comment: 12 pages

Neural Architecture Search (NAS) has demonstrated its efficacy in computer vision and potential for ranking systems. However, prior work focused on academic problems, which are evaluated at small scale under well-controlled fixed baselines. In industry system, such as ranking system in Meta, it is unclear whether NAS algorithms from the literature can outperform production baselines because of: (1) scale - Meta ranking systems serve billions of users, (2) strong baselines - the baselines are production models optimized by hundreds to thousands of world-class engineers for years since the rise of deep learning, (3) dynamic baselines - engineers may have established new and stronger baselines during NAS search, and (4) efficiency - the search pipeline must yield results quickly in alignment with the productionization life cycle. In this paper, we present Rankitect, a NAS software framework for ranking systems at Meta. Rankitect seeks to build brand new architectures by composing low level building blocks from scratch. Rankitect implements and improves state-of-the-art (SOTA) NAS methods for comprehensive and fair comparison under the same search space, including sampling-based NAS, one-shot NAS, and Differentiable NAS (DNAS). We evaluate Rankitect by comparing to multiple production ranking models at Meta. We find that Rankitect can discover new models from scratch achieving competitive tradeoff between Normalized Entropy loss and FLOPs. When utilizing search space designed by engineers, Rankitect can generate better models than engineers, achieving positive offline evaluation and online A/B test at Meta scale., Comment: Wei Wen and Kuang-Hung Liu contribute equally

The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate architecture fabrication. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures. The supernet incorporates versatile choice of operators and dense connectivity to minimize human efforts for finding priors. The scale and heterogeneity in NASRec impose several challenges, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our crafted models, NASRecNet, show promising results on three Click-Through Rates (CTR) prediction benchmarks, indicating that NASRec outperforms both manually designed models and existing NAS methods with state-of-the-art performance. Our work is publicly available at https://github.com/facebookresearch/NasRec., Comment: Proceedings of the ACM Web Conference 2023 (WWW'23)

Dynamic graphs with ordered sequences of events between nodes are prevalent in real-world industrial applications such as e-commerce and social platforms. However, representation learning for dynamic graphs has posed great computational challenges due to the time and structure dependency and irregular nature of the data, preventing such models from being deployed to real-world applications. To tackle this challenge, we propose an efficient algorithm, Efficient Dynamic Graph lEarning (EDGE), which selectively expresses certain temporal dependency via training loss to improve the parallelism in computations. We show that EDGE can scale to dynamic graphs with millions of nodes and hundreds of millions of temporal events and achieve new state-of-the-art (SOTA) performance.

In this work, a hybrid integration of few-layer transition metal dichalcogenides (TMDCs) and ferroelectric polymer is designed to achieve passive control of optical properties in-situ. The electrical polarization in ferroelectric P(VDF-TrFE) polymer can regulate the photoluminescence (PL) in few-layer TMDCs. The total PL intensity is substantially suppressed or enhanced under opposite polarization in bilayer WSe2. This is because electrons transfer between valley K and {\Lambda} in the conduction band induced by the built-in electric field in P(VDF-TrFE) polymer. This charge transfer further changes the competing dynamics between direct and indirect exciton recombination path and overall optical radiation efficiency. We also illustrate that the engineered PL originates from external electric field dependent transferred electron effect. The theoretical result matches the experimental data well. This work demonstrates a device platform in which passive regulation is achieved using 2D TMDCs modulated by polarized ferroelectric materials.