Search

We introduce the Yi model family, a series of language and multimodal models that demonstrate strong multi-dimensional capabilities. The Yi model family is based on 6B and 34B pretrained language models, then we extend them to chat models, 200K long context models, depth-upscaled models, and vision-language models. Our base models achieve strong performance on a wide range of benchmarks like MMLU, and our finetuned chat models deliver strong human preference rate on major evaluation platforms like AlpacaEval and Chatbot Arena. Building upon our scalable super-computing infrastructure and the classical transformer architecture, we attribute the performance of Yi models primarily to its data quality resulting from our data-engineering efforts. For pretraining, we construct 3.1 trillion tokens of English and Chinese corpora using a cascaded data deduplication and quality filtering pipeline. For finetuning, we polish a small scale (less than 10K) instruction dataset over multiple iterations such that every single instance has been verified directly by our machine learning engineers. For vision-language, we combine the chat language model with a vision transformer encoder and train the model to align visual representations to the semantic space of the language model. We further extend the context length to 200K through lightweight continual pretraining and demonstrate strong needle-in-a-haystack retrieval performance. We show that extending the depth of the pretrained checkpoint through continual pretraining further improves performance. We believe that given our current results, continuing to scale up model parameters using thoroughly optimized data will lead to even stronger frontier models.

ESGReveal is an innovative method proposed for efficiently extracting and analyzing Environmental, Social, and Governance (ESG) data from corporate reports, catering to the critical need for reliable ESG information retrieval. This approach utilizes Large Language Models (LLM) enhanced with Retrieval Augmented Generation (RAG) techniques. The ESGReveal system includes an ESG metadata module for targeted queries, a preprocessing module for assembling databases, and an LLM agent for data extraction. Its efficacy was appraised using ESG reports from 166 companies across various sectors listed on the Hong Kong Stock Exchange in 2022, ensuring comprehensive industry and market capitalization representation. Utilizing ESGReveal unearthed significant insights into ESG reporting with GPT-4, demonstrating an accuracy of 76.9% in data extraction and 83.7% in disclosure analysis, which is an improvement over baseline models. This highlights the framework's capacity to refine ESG data analysis precision. Moreover, it revealed a demand for reinforced ESG disclosures, with environmental and social data disclosures standing at 69.5% and 57.2%, respectively, suggesting a pursuit for more corporate transparency. While current iterations of ESGReveal do not process pictorial information, a functionality intended for future enhancement, the study calls for continued research to further develop and compare the analytical capabilities of various LLMs. In summary, ESGReveal is a stride forward in ESG data processing, offering stakeholders a sophisticated tool to better evaluate and advance corporate sustainability efforts. Its evolution is promising in promoting transparency in corporate reporting and aligning with broader sustainable development aims.

Objective. Algorithmic differentiation (AD) can be a useful technique to numerically optimize design and algorithmic parameters by, and quantify uncertainties in, computer simulations. However, the effectiveness of AD depends on how "well-linearizable" the software is. In this study, we assess how promising derivative information of a typical proton computed tomography (pCT) scan computer simulation is for the aforementioned applications. Approach. This study is mainly based on numerical experiments, in which we repeatedly evaluate three representative computational steps with perturbed input values. We support our observations with a review of the algorithmic steps and arithmetic operations performed by the software, using debugging techniques. Main results. The model-based iterative reconstruction (MBIR) subprocedure (at the end of the software pipeline) and the Monte Carlo (MC) simulation (at the beginning) were piecewise differentiable. Jumps in the MBIR function arose from the discrete computation of the set of voxels intersected by a proton path. Jumps in the MC function likely arose from changes in the control flow that affect the amount of consumed random numbers. The tracking algorithm solves an inherently non-differentiable problem. Significance. The MC and MBIR codes are ready for the integration of AD, and further research on surrogate models for the tracking subprocedure is necessary., Comment: 27 pages, 11 figures

The Modular X- and Gamma-ray Sensor (MXGS) is an imaging and spectral X- and Gamma-ray instrument mounted on the starboard side of the Columbus module on the International Space Station. Together with the Modular Multi-Spectral Imaging Assembly (MMIA) (Chanrion et al. this issue) MXGS constitutes the instruments of the Atmosphere-Space Interactions Monitor (ASIM) (Neubert et al. this issue). The main objectives of MXGS are to image and measure the spectrum of X- and $\gamma$-rays from lightning discharges, known as Terrestrial Gamma-ray Flashes (TGFs), and for MMIA to image and perform high speed photometry of Transient Luminous Events (TLEs) and lightning discharges. With these two instruments specifically designed to explore the relation between electrical discharges, TLEs and TGFs, ASIM is the first mission of its kind.

This paper presents the experimental results of $17~MeV$ proton irradiation on a $60~GHz$ low power, half-duplex transceiver (TRX) chip implemented in $65~nm$ CMOS technology. It supports short range point-to-point data rate up to $6~Gbps$ by employing on-off keying (OOK). To investigate the irradiation hardness for high energy physics applications, two TRX chips were irradiated with total ionizing doses (TID) of $74~kGy$ and $42~kGy$ and fluence of $1.4~\times$10$^{14}~ N_{eq}/cm^2$ and $0.8~\times$10$^{14}~N_{eq}/cm^2$ for RX and TX modes, respectively. The chips were characterized by pre- and post-irradiation analogue voltage measurements on different circuit blocks as well as through the analysis of wireless transmission parameters like bit error rate (BER), eye diagram, jitter etc. Post-irradiation measurements have shown certain reduction in performance but both TRX chips have been found operational through over the air measurements at $5~Gbps$. Moreover, very small shift in the carrier frequency was observed after the irradiation., Comment: Accepted on 25th March 2019, 6 pages