Search

Employing Large Language Models (LLMs) to assess the quality of generated responses, such as prompting instruct-tuned models or fine-tuning judge models, has become a widely adopted evaluation method. It is also known that such evaluators are vulnerable to biases, such as favoring longer responses. While it is important to overcome this problem, the specifics of these biases remain under-explored. In this work, we qualitatively identify six types of biases inherent in various judge models. We propose EvalBiasBench as a meta-evaluation collection of hand-crafted test cases for each bias type. Additionally, we present de-biasing dataset construction methods and the associated preference dataset OffsetBias. Experimental results demonstrate that fine-tuning on our dataset significantly enhances the robustness of judge models against biases and improves performance across most evaluation scenarios. We release our datasets and the fine-tuned judge model to public., Comment: Work in Progress

We demonstrate an efficient and accurate, general-purpose first-principles blueprint for calculating anharmonic vibrational free energy and predicting structural phase transition temperatures of solids. Thermodynamic integration is performed without molecular dynamics using only interatomic force constants to model analogues of the true potential and generate their thermal ensembles. By replacing \textit{ab initio} molecular dynamics (AIMD) with statistical sampling of ensemble configurations and trading density-functional theory (DFT) energy calculations on each configuration for a set of matrix operations, our approach enables a faster thermodynamic integration by 4 orders of magnitude over the traditional route via AIMD. Experimental phase transition temperatures of a variety of strongly anharmonic materials with dynamical instabilities including shape-memory alloys are recovered to largely within 25% error. Such a combination of speed and accuracy enables the method to be deployed at a large-scale for predictive mapping of phase transition temperatures.

One of the challenges of developing a summarization model arises from the difficulty in measuring the factual inconsistency of the generated text. In this study, we reinterpret the decoder overconfidence-regularizing objective suggested in (Miao et al., 2021) as a hallucination risk measurement to better estimate the quality of generated summaries. We propose a reference-free metric, HaRiM+, which only requires an off-the-shelf summarization model to compute the hallucination risk based on token likelihoods. Deploying it requires no additional training of models or ad-hoc modules, which usually need alignment to human judgments. For summary-quality estimation, HaRiM+ records state-of-the-art correlation to human judgment on three summary-quality annotation sets: FRANK, QAGS, and SummEval. We hope that our work, which merits the use of summarization models, facilitates the progress of both automated evaluation and generation of summary., Comment: 9 pages (+ 21 pages of Appendix), AACL 2022

The temperature dependence of experimental charge carrier mobility is commonly used as a predictor of the dominant carrier scattering mechanism in semiconductors, particularly in thermoelectric applications. In this work, we critically evaluate whether this practice is well founded. A review of 47 state-of-the-art mobility calculations reveals no correlation between the major scattering mechanism and the temperature trend of mobility. Instead, we demonstrate that the phonon frequencies are the prevailing driving forces behind the temperature dependence and can cause it to vary between $T^{-1}$ to $T^{-3}$ even for an idealised material. To demonstrate this, we calculate the mobility of 23,000 materials and review their temperature dependence, including separating the contributions from deformation, polar, and impurity scattering mechanisms. We conclusively demonstrate that a temperature dependence of $T^{-1.5}$ is not a reliable indicator of deformation potential scattering. Our work highlights the potential pitfalls of predicting the major scattering type based on the experimental mobility temperature trend alone.

Observation is made that a linear dispersion in any dimension under acoustic-phonon-deformation-potential scattering theoretically prescribes a constant charge transport distribution, required for the boxcar profile known to maximize the thermoelectric figure of merit. A linear dispersion squeezed by two transport gaps for optimized bandwidth under scattering by phonon deformation then theoretically constitutes a globally optimal qualitative band structure that may arise in real systems.

Band convergence is considered a clear benefit to thermoelectric performance because it increases the charge carrier concentration for a given Fermi level, which typically enhances charge conductivity while preserving the Seebeck coefficient. However, this advantage hinges on the assumption that interband scattering of carriers is weak or insignificant. With first-principles treatment of electron-phonon scattering in CaMg$_{2}$Sb$_{2}$-CaZn$_{2}$Sb$_{2}$ Zintl system and full Heusler Sr$_{2}$SbAu, we demonstrate that the benefit of band convergence can be intrinsically negated by interband scattering depending on the manner in which bands converge. In the Zintl alloy, band convergence does not improve weighted mobility or the density-of-states effective mass. We trace the underlying reason to the fact that the bands converge at one k-point, which induces strong interband scattering of both the deformation-potential and the polar-optical kinds. The case contrasts with band convergence at distant k-points (as in the full Heusler), which better preserves the single-band scattering behavior thereby successfully leading to improved performance. Therefore, we suggest that band convergence as thermoelectric design principle is best suited to cases in which it occurs at distant k-points.

Understanding how to optimize electronic band structures for thermoelectrics is a topic of long-standing interest in the community. Prior models have been limited to simplified bands and/or scattering models. In this study, we apply more rigorous scattering treatments to more realistic model band structures - upward-parabolic bands that inflect to an inverted parabolic behavior - including cases of multiple bands. In contrast to common descriptors (e.g., quality factor and complexity factor), the degree to which multiple pockets improve thermoelectric performance is bounded by interband scattering and the relative shapes of the bands. We establish that extremely anisotropic `flat-and-dispersive' bands, although best-performing in theory, may not represent a promising design strategy in practice. Critically, we determine optimum bandwidth, dependent on temperature and lattice thermal conductivity, from perfect transport cutoffs that can in theory significantly boost $zT$ beyond the values attainable through intrinsic band structures alone. Our analysis should be widely useful as the thermoelectric research community eyes $zT>3$.

The electronic transport behaviour of materials determines their suitability for technological applications. We develop an efficient method for calculating carrier scattering rates of solid-state semiconductors and insulators from first principles inputs. The present method extends existing polar and non-polar electron-phonon coupling, ionized impurity, and piezoelectric scattering mechanisms formulated for isotropic band structures to support highly anisotropic materials. We test the formalism by calculating the electronic transport properties of 16 semiconductors and comparing the results against experimental measurements. The present work is amenable for use in high-throughput computational workflows and enables accurate screening of carrier mobilities, lifetimes, and thermoelectric power., Comment: 11 pages, 4 figures (SI: 21 pages, 14 figures)

We report first-principles density-functional study of electron-phonon interactions and thermoelectric transport properties of full-Heusler compounds Sr$_{2}$BiAu and Sr$_{2}$SbAu. Our results show that ultrahigh intrinsic bulk thermoelectric performance across a wide range of temperatures is physically possible and point to the presence of multiply degenerate and highly dispersive carrier pockets as the key factor for achieving it. Sr$_{2}$BiAu, which features ten energy-aligned low effective mass pockets (six along $\Gamma-X$ and four at $L$), is predicted to deliver $n$-type $zT=0.4-4.9$ at $T=100-700$~K. Comparison with the previously investigated Ba$_{2}$BiAu compound shows that the additional $L$-pockets in Sr$_{2}$BiAu significantly increase its low-temperature power factor to a maximum value of $12$~mW~m$^{-1}$~K$^{-2}$ near $T=300$~K. However, at high temperatures the power factor of Sr$_{2}$BiAu drops below that of Ba$_{2}$BiAu because the $L$ states are heavier and subject to strong scattering by phonon deformation as opposed to the lighter $\Gamma-X$ states that are limited by polar-optical scattering. Sr$_{2}$SbAu is predicted to deliver lower $n$-type of $zT=3.4$ at $T=750$~K due to appreciable misalignment between the $L$ and $\Gamma-X$ carrier pockets, generally heavier scattering, and slightly higher lattice thermal conductivity. Soft acoustic modes, responsible for low lattice thermal conductivity, also increase vibrational entropies and high-temperature stability of the Heusler compounds, suggesting that their experimental synthesis may be feasible. The dominant intrinsic defects are found to be Au vacancies, which drive the Fermi level towards the conduction band and work in favor of $n$-doping.

Electron-phonon interaction (EPI) is presumably detrimental for thermoelectric performance in semiconductors because it limits carrier mobility. Here we show that enhanced EPI with strong energy dependence offers an intrinsic pathway to significant increase in the Seebeck coefficient and the thermoelectric power factor, particularly in the context of two-dimensional (2D) graphene-like Dirac bands. The increase is realized by enabling electron energy filtering through preferential scattering of electron/hole carriers. We prove this concept by implementing state-of-the-art first-principles computational methods with explicit treatment of EPI for a 2D gapless MoS$_{2}$ allotrope, which has both massless Dirac bands and a heavy-fermion state that acts as the filter. We determine that the optimal location of the heavy state and hence the onset of the filtering process is at the Dirac point. Our study opens a new avenue for attaining ultrahigh power factors via engineering the EPI in graphene-like semimetals or identifying new compounds that intrinsically possess the featured electronic structure.

Using first-principles density-functional theory calculations, we predict the potential for unprecedented thermoelectric efficiency $zT=5$ at 800 K in $n$-type Ba$_{2}$BiAu full-Heusler compound. Such a high efficiency arises from an intrinsically ultralow lattice thermal conductivity coupled with a very high power factor reaching 7 mW m$^{-1}$ K$^{-2}$ at 500 K. The high power factor originates from a light, sixfold degenerate conduction band pocket along the $\Gamma$-X direction. Weak acoustic phonon scattering and sixfold multiplicity combine to yield high mobility and high Seebeck coefficient. In contrast, the flat-and-dispersive (a.k.a. low-dimensional) valence band of Ba$_{2}$BiAu fail to generate a high power factor due to strong acoustic phonon scattering. The Lorenz numbers at optimal doping are smaller than the Wiedemann-Franz value, an integral feature for $zT$ enhancement as electrons are the majority heat carriers.