Search

While thin film lithium niobate (TFLN) is known for efficient signal generation, on-chip signal amplification remains challenging from fully integrated optical communication circuits. Here we demonstrate the first continuous-wave-pump optical parametric amplification (OPA) using an x-cut domain-engineered TFLN waveguide, with high gain over the telecom band up to 13.9 dB, and test it for high signal-to-noise ratio signal amplification using a commercial optical communication module pair. Fabricated in wafer scale using common process as devices including modulators, this OPA device marks an important step in TFLN photonic integration.

Recent developments in multimodal methodologies have marked the beginning of an exciting era for models adept at processing diverse data types, encompassing text, audio, and visual content. Models like GPT-4V, which merge computer vision with advanced language processing, exhibit extraordinary proficiency in handling intricate tasks that require a simultaneous understanding of both textual and visual information. Prior research efforts have meticulously evaluated the efficacy of these Vision Large Language Models (VLLMs) in various domains, including object detection, image captioning, and other related fields. However, existing analyses have often suffered from limitations, primarily centering on the isolated evaluation of each modality's performance while neglecting to explore their intricate cross-modal interactions. Specifically, the question of whether these models achieve the same level of accuracy when confronted with identical task instances across different modalities remains unanswered. In this study, we take the initiative to delve into the interaction and comparison among these modalities of interest by introducing a novel concept termed cross-modal consistency. Furthermore, we propose a quantitative evaluation framework founded on this concept. Our experimental findings, drawn from a curated collection of parallel vision-language datasets developed by us, unveil a pronounced inconsistency between the vision and language modalities within GPT-4V, despite its portrayal as a unified multimodal model. Our research yields insights into the appropriate utilization of such models and hints at potential avenues for enhancing their design.

In this paper, we introduce MIO, a novel foundation model built on multimodal tokens, capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner. While the emergence of large language models (LLMs) and multimodal large language models (MM-LLMs) propels advancements in artificial general intelligence through their versatile capabilities, they still lack true any-to-any understanding and generation. Recently, the release of GPT-4o has showcased the remarkable potential of any-to-any LLMs for complex real-world tasks, enabling omnidirectional input and output across images, speech, and text. However, it is closed-source and does not support the generation of multimodal interleaved sequences. To address this gap, we present MIO, which is trained on a mixture of discrete tokens across four modalities using causal multimodal modeling. MIO undergoes a four-stage training process: (1) alignment pre-training, (2) interleaved pre-training, (3) speech-enhanced pre-training, and (4) comprehensive supervised fine-tuning on diverse textual, visual, and speech tasks. Our experimental results indicate that MIO exhibits competitive, and in some cases superior, performance compared to previous dual-modal baselines, any-to-any model baselines, and even modality-specific baselines. Moreover, MIO demonstrates advanced capabilities inherent to its any-to-any feature, such as interleaved video-text generation, chain-of-visual-thought reasoning, visual guideline generation, instructional image editing, etc., Comment: Technical Report. Codes and models are available in https://github.com/MIO-Team/MIO

Recent studies have demonstrated the potential to control paraphrase generation, such as through syntax, which has broad applications in various downstream tasks. However, these methods often require detailed parse trees or syntactic exemplars, countering human-like paraphrasing behavior in language use. Furthermore, an inference gap exists, as control specifications are only available during training but not during inference. In this work, we propose a new setup for controlled paraphrase generation. Specifically, we represent user intent as action tokens, embedding and concatenating them with text embeddings, thus flowing together into a self-attention encoder for representation fusion. To address the inference gap, we introduce an optional action token as a placeholder that encourages the model to determine the appropriate action independently when users' intended actions are not provided. Experimental results show that our method successfully enables precise action-controlled paraphrasing and preserves or even enhances performance compared to conventional uncontrolled methods when actions are not given. Our findings promote the concept of action-controlled paraphrasing for a more user-centered design., Comment: Work in Progress

In text-to-image generation, using negative prompts, which describe undesirable image characteristics, can significantly boost image quality. However, producing good negative prompts is manual and tedious. To address this, we propose NegOpt, a novel method for optimizing negative prompt generation toward enhanced image generation, using supervised fine-tuning and reinforcement learning. Our combined approach results in a substantial increase of 25% in Inception Score compared to other approaches and surpasses ground-truth negative prompts from the test set. Furthermore, with NegOpt we can preferentially optimize the metrics most important to us. Finally, we construct Negative Prompts DB (https://huggingface.co/datasets/mikeogezi/negopt_full), a publicly available dataset of negative prompts.

Self-injection locking scheme has the potential to narrow the linewidth of lasers in a compact setup. Here, we report a narrow linewidth laser source near 1 {\mu}m by self-injection locking scheme using a Fabry-Perot (FP) hollow resonator with a high-quality factor (Q>10^8). The measured fundamental linewidth of the laser is 41 Hz, and a coarse tuning range over 5.5 nm is achieved by changing the driving current of the laser source. Meanwhile, a fine-tuning range of 373 MHz is achieved without mode hops by changing the voltage applied to the PZT on the resonator. More importantly, benefiting from the low thermal refractive noise and low thermal expansion of the FP hollow resonator, the beat-note linewidth and the frequency Allan deviation are measured to be 510.3 Hz in and 10^-11 (1s averaging time), respectively, by using a fully stabilized frequency comb as reference. Such a high-performance laser is fully integrated with a palm-sized package (52.3 mL) for field-deployable applications.

Optical frequency comb based on microresonator (microcomb) is an integrated coherent light source and has the potential to promise a high-precision frequency standard, and self-reference and long-term stable microcomb is the key to this realization. Here, we demonstrated a 0.7-octave spectrum Kerr comb via dispersion engineering in a thin film lithium niobate microresonator, and the single soliton state can be accessed passively with long-term stability over 3 hours. With such a robust broadband coherent comb source using thin film lithium niobate, fully stabilized microcomb can be expected for massive practical applications., Comment: 8 pages, 4 figures

Recent advancements in multimodal techniques open exciting possibilities for models excelling in diverse tasks involving text, audio, and image processing. Models like GPT-4V, blending computer vision and language modeling, excel in complex text and image tasks. Numerous prior research endeavors have diligently examined the performance of these Vision Large Language Models (VLLMs) across tasks like object detection, image captioning and others. However, these analyses often focus on evaluating the performance of each modality in isolation, lacking insights into their cross-modal interactions. Specifically, questions concerning whether these vision-language models execute vision and language tasks consistently or independently have remained unanswered. In this study, we draw inspiration from recent investigations into multilingualism and conduct a comprehensive analysis of model's cross-modal interactions. We introduce a systematic framework that quantifies the capability disparities between different modalities in the multi-modal setting and provide a set of datasets designed for these evaluations. Our findings reveal that models like GPT-4V tend to perform consistently modalities when the tasks are relatively simple. However, the trustworthiness of results derived from the vision modality diminishes as the tasks become more challenging. Expanding on our findings, we introduce "Vision Description Prompting," a method that effectively improves performance in challenging vision-related tasks.

We describe the systems of the University of Alberta team for the SemEval-2023 Visual Word Sense Disambiguation (V-WSD) Task. We present a novel algorithm that leverages glosses retrieved from BabelNet, in combination with text and image encoders. Furthermore, we compare language-specific encoders against the application of English encoders to translated texts. As the contexts given in the task datasets are extremely short, we also experiment with augmenting these contexts with descriptions generated by a language model. This yields substantial improvements in accuracy. We describe and evaluate additional V-WSD methods which use image generation and text-conditioned image segmentation. Overall, the results of our official submission rank us 18 out of 56 teams. Some of our unofficial results are even better than the official ones. Our code is publicly available at https://github.com/UAlberta-NLP/v-wsd.

On-chip mode division (de)multiplexing plays a significant role in the integrated devices to greatly improve communication capacity. One of the pursuing goals of mode division (de)multiplexer is how to realize such devices with larger bandwidth and better fabrication tolerance. However, integrated broadband mode division (de)multiplexing devices were rarely reported up to now. In this work, we experimentally realize a broadband mode division demultiplexing device with a high degree of fabrication tolerance based on lithium niobate-on-insulator waveguide array. By taking advantage of the fact that different modes in the waveguide array have different group velocity, we experimentally confirmed that different modes in the waveguide array have different transverse displacements. The implementation of the integrated broadband mode division demultiplexer may have future applications in information processing technology, quantum communication as well as quantum computing.

Textual adversarial attacks can discover models' weaknesses by adding semantic-preserved but misleading perturbations to the inputs. The long-lasting adversarial attack-and-defense arms race in Natural Language Processing (NLP) is algorithm-centric, providing valuable techniques for automatic robustness evaluation. However, the existing practice of robustness evaluation may exhibit issues of incomprehensive evaluation, impractical evaluation protocol, and invalid adversarial samples. In this paper, we aim to set up a unified automatic robustness evaluation framework, shifting towards model-centric evaluation to further exploit the advantages of adversarial attacks. To address the above challenges, we first determine robustness evaluation dimensions based on model capabilities and specify the reasonable algorithm to generate adversarial samples for each dimension. Then we establish the evaluation protocol, including evaluation settings and metrics, under realistic demands. Finally, we use the perturbation degree of adversarial samples to control the sample validity. We implement a toolkit RobTest that realizes our automatic robustness evaluation framework. In our experiments, we conduct a robustness evaluation of RoBERTa models to demonstrate the effectiveness of our evaluation framework, and further show the rationality of each component in the framework. The code will be made public at \url{https://github.com/thunlp/RobTest}., Comment: Accepted to Findings of ACL 2023

Large Language Models (LLMs) have demonstrated exceptional natural language understanding abilities and have excelled in a variety of natural language processing (NLP)tasks in recent years. Despite the fact that most LLMs are trained predominantly in English, multiple studies have demonstrated their comparative performance in many other languages. However, fundamental questions persist regarding how LLMs acquire their multi-lingual abilities and how performance varies across different languages. These inquiries are crucial for the study of LLMs since users and researchers often come from diverse language backgrounds, potentially influencing their utilization and interpretation of LLMs' results. In this work, we propose a systematic way of qualifying the performance disparities of LLMs under multilingual settings. We investigate the phenomenon of across-language generalizations in LLMs, wherein insufficient multi-lingual training data leads to advanced multi-lingual capabilities. To accomplish this, we employ a novel back-translation-based prompting method. The results show that GPT exhibits highly translating-like behaviour in multilingual settings., Comment: Paper accepted to EMNLP 2023

Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP, aimed at addressing limitations in existing frameworks while aligning with the ultimate goals of artificial intelligence. This paradigm considers language models as agents capable of observing, acting, and receiving feedback iteratively from external entities. Specifically, language models in this context can: (1) interact with humans for better understanding and addressing user needs, personalizing responses, aligning with human values, and improving the overall user experience; (2) interact with knowledge bases for enriching language representations with factual knowledge, enhancing the contextual relevance of responses, and dynamically leveraging external information to generate more accurate and informed responses; (3) interact with models and tools for effectively decomposing and addressing complex tasks, leveraging specialized expertise for specific subtasks, and fostering the simulation of social behaviors; and (4) interact with environments for learning grounded representations of language, and effectively tackling embodied tasks such as reasoning, planning, and decision-making in response to environmental observations. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept. We then provide a systematic classification of iNLP, dissecting its various components, including interactive objects, interaction interfaces, and interaction methods. We proceed to delve into the evaluation methodologies used in the field, explore its diverse applications, scrutinize its ethical and safety issues, and discuss prospective research directions. This survey serves as an entry point for researchers who are interested in this rapidly evolving area and offers a broad view of the current landscape and future trajectory of iNLP., Comment: 110 pages

Drone-based quantum link has the potential to realize mobile quantum network, and entanglement distribution has been demonstrated using one and two drones. Here we report the first drone-based quantum key distribution (QKD), with average secure key rate larger than 8 kHz using decoy-state BB84 protocol with polarization coding. Compact acquisition, pointing, and tracking (APT) system and QKD modules are developed and loaded on a home-made octocopter, within takeoff weight of 30 kg. A robust link is established between the flying octocopter and a ground station separated 200 meters away and real-time QKD is performed for 400 seconds. This work shows potential of drone-based quantum communication for the future mobile quantum networks.

Free-space optical communication (FSO) can achieve fast, secure and license-free communication without need for physical cables, making it a cost-effective, energy-efficient and flexible solution when the fiber connection is unavailable. To establish FSO connection on-demand, it is essential to build portable FSO devices with compact structure and light weight. Here, we develop a miniaturized FSO system and realize 9.16 Gbps FSO between two nodes that is 1 km apart, using a commercial single-mode-fiber-coupled optical transceiver module without optical amplification. Using our 4-stage acquisition, pointing and tracking (APT) systems, the tracking error is within 3 {\mu}rad and results an average link loss of 13.7 dB, which is the key for this high-bandwidth FSO demonstration without optical amplification. Our FSO link has been tested up to 4 km, with link loss of 18 dB that is limited by the foggy weather during the test. Longer FSO distances can be expected with better weather condition and optical amplification. With single FSO device weight of only 9.5 kg, this result arouses massive applications of field-deployable high-speed wireless communication., Comment: 7 pages, 5 figures

The quantum computation of molecular response properties on near-term quantum hardware is a topic of significant interest. While computing time-domain response properties is in principle straightforward due to the natural ability of quantum computers to simulate unitary time evolution, circuit depth limitations restrict the maximum time that can be simulated and hence the extraction of frequency-domain properties. Computing properties directly in the frequency domain is therefore desirable, but the circuits require large depth when the typical hardware gate set consisting of single- and two-qubit gates is used. Here, we report the experimental quantum computation of the response properties of diatomic molecules directly in the frequency domain using a three-qubit iToffoli gate, enabling a reduction in circuit depth by a factor of two. We show that the molecular properties obtained with the iToffoli gate exhibit comparable or better agreement with theory than those obtained with the native CZ gates. Our work is among the first demonstrations of the practical usage of a native multi-qubit gate in quantum simulation, with diverse potential applications to the simulation of quantum many-body systems on near-term digital quantum computers., Comment: 10 pages, 6 figures

Microresonator-based optical frequency comb (microcomb) has the potential to revolutionize the accuracy of frequency synthesizer in radar and communication applications. However, fundamental limit exists for low noise microcomb generation, especially in low size, weight, power and cost (SWaP-C) package. Here we resolve this limit, by the demonstration of an automated turnkey microcomb, operating close to its low quantum-limited phase noise, within a compact setup size of 85 mm * 90 mm * 25 mm. High quality factor fiber Fabry-Perot resonator (FFPR), with Q up to 4.0 * 10^9, is the key for both low quantum noise and pump noise limit, in the diode-pump case in a self-injection locking scheme. Low phase noise of -80 and -105 dBc/Hz at 100 Hz, -106 and -125 dBc/Hz at 1 kHz, -133 and -148 dBc/Hz at 10 kHz is achieved at 10.1 GHz and 1.7 GHz repetition frequencies, respectively. With the simultaneous automated turnkey, low-noise and direct-diode-pump capability, our microcomb is ready to be used as a low-noise frequency synthesizer with low SWaP-C and thus field deployability.

Despite of the superb performance on a wide range of tasks, pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. In this paper, we present RoChBERT, a framework to build more Robust BERT-based models by utilizing a more comprehensive adversarial graph to fuse Chinese phonetic and glyph features into pre-trained representations during fine-tuning. Inspired by curriculum learning, we further propose to augment the training dataset with adversarial texts in combination with intermediate samples. Extensive experiments demonstrate that RoChBERT outperforms previous methods in significant ways: (i) robust -- RoChBERT greatly improves the model robustness without sacrificing accuracy on benign texts. Specifically, the defense lowers the success rates of unlimited and limited attacks by 59.43% and 39.33% respectively, while remaining accuracy of 93.30%; (ii) flexible -- RoChBERT can easily extend to various language models to solve different downstream tasks with excellent performance; and (iii) efficient -- RoChBERT can be directly applied to the fine-tuning stage without pre-training language model from scratch, and the proposed data augmentation method is also low-cost., Comment: Accepted by Findings of EMNLP 2022

Self-testing, which refers to device independent characterization of the state and the measurement, enables the security of quantum information processing task certified independently of the operation performed inside the devices. Quantum states lie in the core of self-testing as key resources. However, for the different entangled states, usually different measurement settings should be taken in self-testing recipes. This may lead to the redundancy of measurement resources. In this work, we use fixed two-binary measurements and answer the question that what states can be self-tested with the same settings. By investigating the structure of generalized tilted-CHSH Bell operators with sum of squares decomposition method, we show that a family of two-qubit entangled states can be self-tested by the same measurement settings. The robustness analysis indicates that our scheme is feasible for practical experiment instrument. Moreover, our results can be applied to various quantum information processing tasks.

Text editing, such as grammatical error correction, arises naturally from imperfect textual data. Recent works frame text editing as a multi-round sequence tagging task, where operations -- such as insertion and substitution -- are represented as a sequence of tags. While achieving good results, this encoding is limited in flexibility as all actions are bound to token-level tags. In this work, we reformulate text editing as an imitation game using behavioral cloning. Specifically, we convert conventional sequence-to-sequence data into state-to-action demonstrations, where the action space can be as flexible as needed. Instead of generating the actions one at a time, we introduce a dual decoders structure to parallel the decoding while retaining the dependencies between action tokens, coupled with trajectory augmentation to alleviate the distribution shift that imitation learning often suffers. In experiments on a suite of Arithmetic Equation benchmarks, our model consistently outperforms the autoregressive baselines in terms of performance, efficiency, and robustness. We hope our findings will shed light on future studies in reinforcement learning applying sequence-level action generation to natural language processing., Comment: Accepted to Findings of EMNLP 2022

Optical interferometers are pillars of modern precision metrology, but their resolution is limited by the wavelength of the light source, which cannot be infinitely reduced. Magically, this limitation can be circumvented by using an entangled multiphoton source because interference produced by an N-photon amplitude features a reduced de Broglie wavelength {\lambda}/N. However, the extremely low efficiency in multiphoton state generation and coincidence counts actually negates the potential of using multiphoton states in practical measurements. Here, we demonstrate a novel interferometric technique based on structured nonlinear optics, i.e., parametric upconversion of a structured beam, capable of superresolution measurement in real time. The main principle relies in that the orbital angular momentum (OAM) state and associated intramodal phase within the structured beam are both continuously multiplied in cascading upconversion to mimic the superresolved phase evolution of a multiphoton amplitude. Owing to the use of bright sensing beams and OAM mode projection, up to a 12-photon de Broglie wavelength with almost perfect visibility is observed in real time and, importantly, by using only a low-cost detector. Our results open the door to real-time superresolution interferometric metrology and provide a promising way toward multiphoton superiority in practical applications.