Search

Class imbalance remains a major challenge in machine learning, especially in multi-class problems with long-tailed distributions. Existing methods, such as data resampling, cost-sensitive techniques, and logistic loss modifications, though popular and often effective, lack solid theoretical foundations. As an example, we demonstrate that cost-sensitive methods are not Bayes consistent. This paper introduces a novel theoretical framework for analyzing generalization in imbalanced classification. We propose a new class-imbalanced margin loss function for both binary and multi-class settings, prove its strong $H$-consistency, and derive corresponding learning guarantees based on empirical loss and a new notion of class-sensitive Rademacher complexity. Leveraging these theoretical results, we devise novel and general learning algorithms, IMMAX (Imbalanced Margin Maximization), which incorporate confidence margins and are applicable to various hypothesis sets. While our focus is theoretical, we also present extensive empirical results demonstrating the effectiveness of our algorithms compared to existing baselines.

QCD jets are considered important probes for quark gluon plasma created in collisions of nuclei at high energies. Their parton showers are significantly altered if they develop inside of a deconfined medium. Hadronization of jets is also thought to be affected by the presence of quarks and gluons. We present a systematic study of the effects of a thermal bath of partons on the hadronization of parton showers. We use the JETSCAPE framework to create parton showers both in vacuum and in a brick of quark gluon plasma. The brick setup allows important parameters, like the size of the plasma as well as the collective flow of partons, to be varied systematically. We hadronize the parton showers using Hybrid Hadronization, which permits shower partons to form strings with thermal partons, or to recombine directly with thermal partons as well as with each other. We find a sizeable amount of interaction of shower partons with thermal partons during hadronization, indicating a natural continuation of the interaction of jet and medium during this stage. The observed effects grow with the size of the medium. Collective flow easily transfers from the thermal partons onto the emerging jet hadrons. We also see a significant change in hadron chemistry as expected in the presence of quark recombination processes., Comment: 12 pages, 6 figures

An investigation of high transverse momentum (high-$p_T$) photon triggered jets in proton-proton ($p$-$p$) and ion-ion ($A$-$A$) collisions at $\sqrt{s_{NN}} = 0.2$ and $5.02~\mathrm{TeV}$ is carried out, using the multistage description of in-medium jet evolution. Monte Carlo simulations of hard scattering and energy loss in heavy-ion collisions are performed using parameters tuned in a previous study of the nuclear modification factor ($R_{AA}$) for inclusive jets and high-$p_T$ hadrons. We obtain a good reproduction of the experimental data for photon triggered jet $R_{AA}$, as measured by the ATLAS detector, the distribution of the ratio of jet to photon $p_T$ ($X_{\rm J \gamma}$), measured by both CMS and ATLAS, and the photon-jet azimuthal correlation as measured by CMS. We obtain a moderate description of the photon triggered jet $I_{AA}$, as measured by STAR. A noticeable improvement in the comparison is observed when one goes beyond prompt photons and includes bremsstrahlung and decay photons, revealing their significance in certain kinematic regions, particularly at $X_{J\gamma} > 1$. Moreover, azimuthal angle correlations demonstrate a notable impact of non-prompt photons on the distribution, emphasizing their role in accurately describing experimental results. This work highlights the success of the multistage model of jet modification to straightforwardly predict (this set of) photon triggered jet observables. This comparison, along with the role played by non-prompt photons, has important consequences on the inclusion of such observables in a future Bayesian analysis., Comment: 16 pages, 10 figures

The JETSCAPE Collaboration reports a new determination of the jet transport parameter $\hat{q}$ in the Quark-Gluon Plasma (QGP) using Bayesian Inference, incorporating all available inclusive hadron and jet yield suppression data measured in heavy-ion collisions at RHIC and the LHC. This multi-observable analysis extends the previously published JETSCAPE Bayesian Inference determination of $\hat{q}$, which was based solely on a selection of inclusive hadron suppression data. JETSCAPE is a modular framework incorporating detailed dynamical models of QGP formation and evolution, and jet propagation and interaction in the QGP. Virtuality-dependent partonic energy loss in the QGP is modeled as a thermalized weakly-coupled plasma, with parameters determined from Bayesian calibration using soft-sector observables. This Bayesian calibration of $\hat{q}$ utilizes Active Learning, a machine--learning approach, for efficient exploitation of computing resources. The experimental data included in this analysis span a broad range in collision energy and centrality, and in transverse momentum. In order to explore the systematic dependence of the extracted parameter posterior distributions, several different calibrations are reported, based on combined jet and hadron data; on jet or hadron data separately; and on restricted kinematic or centrality ranges of the jet and hadron data. Tension is observed in comparison of these variations, providing new insights into the physics of jet transport in the QGP and its theoretical formulation., Comment: 20 pages, 10 figures, 2 tables, submitted to PRC; updated acknowledgements

A new framework, called x-scape, for the combined study of both hard and soft transverse momentum sectors in high energy proton-proton ($p$-$p$) and proton-nucleus ($p$-$A$) collisions is set up. A dynamical initial state is set up using the 3d-Glauber model with transverse locations of hotspots within each incoming nucleon. A hard scattering that emanates from two colliding hotspots is carried out using the Pythia generator. Initial state radiation from the incoming hard partons is carried out in a new module called I-matter, which includes the longitudinal location of initial splits. The energy-momentum of both the initial hard partons and their associated beam remnants is removed from the hot spots, depleting the energy-momentum available for the formation of the bulk medium. Outgoing showers are simulated using the matter generator, and results are presented for both cases, allowing for and not allowing for energy loss. First comparisons between this hard-soft model and single inclusive hadron and jet data from $p$-$p$ and minimum bias $p$-$Pb$ collisions are presented. Single hadron spectra in $p$-$p$ are used to carry out a limited (in number of parameters) Bayesian calibration of the model. Fair comparisons with data are indicative of the utility of this new framework. Theoretical studies of the correlation between jet $p_T$ and event activity at mid and forward rapidity are carried out., Comment: 18 pages, 15 figures

We present a detailed study of surrogate losses and algorithms for multi-label learning, supported by $H$-consistency bounds. We first show that, for the simplest form of multi-label loss (the popular Hamming loss), the well-known consistent binary relevance surrogate suffers from a sub-optimal dependency on the number of labels in terms of $H$-consistency bounds, when using smooth losses such as logistic losses. Furthermore, this loss function fails to account for label correlations. To address these drawbacks, we introduce a novel surrogate loss, multi-label logistic loss, that accounts for label correlations and benefits from label-independent $H$-consistency bounds. We then broaden our analysis to cover a more extensive family of multi-label losses, including all common ones and a new extension defined based on linear-fractional functions with respect to the confusion matrix. We also extend our multi-label logistic losses to more comprehensive multi-label comp-sum losses, adapting comp-sum losses from standard classification to the multi-label learning. We prove that this family of surrogate losses benefits from $H$-consistency bounds, and thus Bayes-consistency, across any general multi-label loss. Our work thus proposes a unified surrogate loss framework benefiting from strong consistency guarantees for any multi-label loss, significantly expanding upon previous work which only established Bayes-consistency and for specific loss functions. Additionally, we adapt constrained losses from standard classification to multi-label constrained losses in a similar way, which also benefit from $H$-consistency bounds and thus Bayes-consistency for any multi-label loss. We further describe efficient gradient computation algorithms for minimizing the multi-label logistic loss.

We present a comprehensive study of surrogate loss functions for learning to defer. We introduce a broad family of surrogate losses, parameterized by a non-increasing function $\Psi$, and establish their realizable $H$-consistency under mild conditions. For cost functions based on classification error, we further show that these losses admit $H$-consistency bounds when the hypothesis set is symmetric and complete, a property satisfied by common neural network and linear function hypothesis sets. Our results also resolve an open question raised in previous work (Mozannar et al., 2023) by proving the realizable $H$-consistency and Bayes-consistency of a specific surrogate loss. Furthermore, we identify choices of $\Psi$ that lead to $H$-consistent surrogate losses for any general cost function, thus achieving Bayes-consistency, realizable $H$-consistency, and $H$-consistency bounds simultaneously. We also investigate the relationship between $H$-consistency bounds and realizable $H$-consistency in learning to defer, highlighting key differences from standard classification. Finally, we empirically evaluate our proposed surrogate losses and compare them with existing baselines.

Recent research has introduced a key notion of $H$-consistency bounds for surrogate losses. These bounds offer finite-sample guarantees, quantifying the relationship between the zero-one estimation error (or other target loss) and the surrogate loss estimation error for a specific hypothesis set. However, previous bounds were derived under the condition that a lower bound of the surrogate loss conditional regret is given as a convex function of the target conditional regret, without non-constant factors depending on the predictor or input instance. Can we derive finer and more favorable $H$-consistency bounds? In this work, we relax this condition and present a general framework for establishing enhanced $H$-consistency bounds based on more general inequalities relating conditional regrets. Our theorems not only subsume existing results as special cases but also enable the derivation of more favorable bounds in various scenarios. These include standard multi-class classification, binary and multi-class classification under Tsybakov noise conditions, and bipartite ranking.

We present a detailed study of cardinality-aware top-$k$ classification, a novel approach that aims to learn an accurate top-$k$ set predictor while maintaining a low cardinality. We introduce a new target loss function tailored to this setting that accounts for both the classification error and the cardinality of the set predicted. To optimize this loss function, we propose two families of surrogate losses: cost-sensitive comp-sum losses and cost-sensitive constrained losses. Minimizing these loss functions leads to new cardinality-aware algorithms that we describe in detail in the case of both top-$k$ and threshold-based classifiers. We establish $H$-consistency bounds for our cardinality-aware surrogate loss functions, thereby providing a strong theoretical foundation for our algorithms. We report the results of extensive experiments on CIFAR-10, CIFAR-100, ImageNet, and SVHN datasets demonstrating the effectiveness and benefits of our cardinality-aware algorithms., Comment: arXiv admin note: substantial text overlap with arXiv:2403.19625

This paper presents a novel approach for unsupervised video summarization using reinforcement learning. It aims to address the existing limitations of current unsupervised methods, including unstable training of adversarial generator-discriminator architectures and reliance on hand-crafted reward functions for quality evaluation. The proposed method is based on the concept that a concise and informative summary should result in a reconstructed video that closely resembles the original. The summarizer model assigns an importance score to each frame and generates a video summary. In the proposed scheme, reinforcement learning, coupled with a unique reward generation pipeline, is employed to train the summarizer model. The reward generation pipeline trains the summarizer to create summaries that lead to improved reconstructions. It comprises a generator model capable of reconstructing masked frames from a partially masked video, along with a reward mechanism that compares the reconstructed video from the summary against the original. The video generator is trained in a self-supervised manner to reconstruct randomly masked frames, enhancing its ability to generate accurate summaries. This training pipeline results in a summarizer model that better mimics human-generated video summaries compared to methods relying on hand-crafted rewards. The training process consists of two stable and isolated training steps, unlike adversarial architectures. Experimental results demonstrate promising performance, with F-scores of 62.3 and 54.5 on TVSum and SumMe datasets, respectively. Additionally, the inference stage is 300 times faster than our previously reported state-of-the-art method.

Abernethy et al. (2011) showed that Blackwell approachability and no-regret learning are equivalent, in the sense that any algorithm that solves a specific Blackwell approachability instance can be converted to a sublinear regret algorithm for a specific no-regret learning instance, and vice versa. In this paper, we study a more fine-grained form of such reductions, and ask when this translation between problems preserves not only a sublinear rate of convergence, but also preserves the optimal rate of convergence. That is, in which cases does it suffice to find the optimal regret bound for a no-regret learning instance in order to find the optimal rate of convergence for a corresponding approachability instance? We show that the reduction of Abernethy et al. (2011) does not preserve rates: their reduction may reduce a $d$-dimensional approachability instance $I_1$ with optimal convergence rate $R_1$ to a no-regret learning instance $I_2$ with optimal regret-per-round of $R_2$, with $R_{2}/R_{1}$ arbitrarily large (in particular, it is possible that $R_1 = 0$ and $R_{2} > 0$). On the other hand, we show that it is possible to tightly reduce any approachability instance to an instance of a generalized form of regret minimization we call improper $\phi$-regret minimization (a variant of the $\phi$-regret minimization of Gordon et al. (2008) where the transformation functions may map actions outside of the action set). Finally, we characterize when linear transformations suffice to reduce improper $\phi$-regret minimization problems to standard classes of regret minimization problems in a rate preserving manner. We prove that some improper $\phi$-regret minimization instances cannot be reduced to either subclass of instance in this way, suggesting that approachability can capture some problems that cannot be phrased in the language of online learning.

This paper presents a comprehensive analysis of the growth rate of $H$-consistency bounds (and excess error bounds) for various surrogate losses used in classification. We prove a square-root growth rate near zero for smooth margin-based surrogate losses in binary classification, providing both upper and lower bounds under mild assumptions. This result also translates to excess error bounds. Our lower bound requires weaker conditions than those in previous work for excess error bounds, and our upper bound is entirely novel. Moreover, we extend this analysis to multi-class classification with a series of novel results, demonstrating a universal square-root growth rate for smooth comp-sum and constrained losses, covering common choices for training neural networks in multi-class classification. Given this universal rate, we turn to the question of choosing among different surrogate losses. We first examine how $H$-consistency bounds vary across surrogates based on the number of classes. Next, ignoring constants and focusing on behavior near zero, we identify minimizability gaps as the key differentiating factor in these bounds. Thus, we thoroughly analyze these gaps, to guide surrogate loss selection, covering: comparisons across different comp-sum losses, conditions where gaps become zero, and general conditions leading to small gaps. Additionally, we demonstrate the key role of minimizability gaps in comparing excess error bounds and $H$-consistency bounds.

We present a detailed study of top-$k$ classification, the task of predicting the $k$ most probable classes for an input, extending beyond single-class prediction. We demonstrate that several prevalent surrogate loss functions in multi-class classification, such as comp-sum and constrained losses, are supported by $H$-consistency bounds with respect to the top-$k$ loss. These bounds guarantee consistency in relation to the hypothesis set $H$, providing stronger guarantees than Bayes-consistency due to their non-asymptotic and hypothesis-set specific nature. To address the trade-off between accuracy and cardinality $k$, we further introduce cardinality-aware loss functions through instance-dependent cost-sensitive learning. For these functions, we derive cost-sensitive comp-sum and constrained surrogate losses, establishing their $H$-consistency bounds and Bayes-consistency. Minimizing these losses leads to new cardinality-aware algorithms for top-$k$ classification. We report the results of extensive experiments on CIFAR-100, ImageNet, CIFAR-10, and SVHN datasets demonstrating the effectiveness and benefit of these algorithms.

Learning to defer with multiple experts is a framework where the learner can choose to defer the prediction to several experts. While this problem has received significant attention in classification contexts, it presents unique challenges in regression due to the infinite and continuous nature of the label space. In this work, we introduce a novel framework of regression with deferral, which involves deferring the prediction to multiple experts. We present a comprehensive analysis for both the single-stage scenario, where there is simultaneous learning of predictor and deferral functions, and the two-stage scenario, which involves a pre-trained predictor with a learned deferral function. We introduce new surrogate loss functions for both scenarios and prove that they are supported by $H$-consistency bounds. These bounds provide consistency guarantees that are stronger than Bayes consistency, as they are non-asymptotic and hypothesis set-specific. Our framework is versatile, applying to multiple experts, accommodating any bounded regression losses, addressing both instance-dependent and label-dependent costs, and supporting both single-stage and two-stage methods. A by-product is that our single-stage formulation includes the recent regression with abstention framework (Cheng et al., 2023) as a special case, where only a single expert, the squared loss and a label-independent cost are considered. Minimizing our proposed loss functions directly leads to novel algorithms for regression with deferral. We report the results of extensive experiments showing the effectiveness of our proposed algorithms.

We present a detailed study of $H$-consistency bounds for regression. We first present new theorems that generalize the tools previously given to establish $H$-consistency bounds. This generalization proves essential for analyzing $H$-consistency bounds specific to regression. Next, we prove a series of novel $H$-consistency bounds for surrogate loss functions of the squared loss, under the assumption of a symmetric distribution and a bounded hypothesis set. This includes positive results for the Huber loss, all $\ell_p$ losses, $p \geq 1$, the squared $\epsilon$-insensitive loss, as well as a negative result for the $\epsilon$-insensitive loss used in squared Support Vector Regression (SVR). We further leverage our analysis of $H$-consistency for regression and derive principled surrogate losses for adversarial regression (Section 5). This readily establishes novel algorithms for adversarial regression, for which we report favorable experimental results in Section 6.

Prompt photons are created in the early stages of heavy ion collisions and traverse the QGP medium without any interaction. Therefore, photon-triggered jets can be used to study the jet quenching in the QGP medium. In this work, photon-triggered jets are studied through different jet and jet substructure observables for different collision systems and energies using the JETSCAPE framework. Since the multistage evolution used in the JETSCAPE framework is adequate to describe a wide range of experimental observables simultaneously using the same parameter tune, we use the same parameters tuned for jet and leading hadron studies. The same isolation criteria used in the experimental analysis are used to identify prompt photons for better comparison. For the first time, high-accuracy JETSCAPE results are compared with multi-energy LHC and RHIC measurements to better understand the deviations observed in prior studies. This study highlights the importance of multistage evolution for the simultaneous description of experimental observables through different collision systems and energies using a single parameter tune., Comment: 4 pages, 5 figures. Proceedings of Quark Matter 2023 - XXXth International Conference on Ultra-relativistic Nucleus-Nucleus Collisions, Houston, TX, 3-9 September 2023

The rise of Large Language Models (LLMs) has revolutionized our comprehension of intelligence bringing us closer to Artificial Intelligence. Since their introduction, researchers have actively explored the applications of LLMs across diverse fields, significantly elevating capabilities. Cybersecurity, traditionally resistant to data-driven solutions and slow to embrace machine learning, stands out as a domain. This study examines the existing literature, providing a thorough characterization of both defensive and adversarial applications of LLMs within the realm of cybersecurity. Our review not only surveys and categorizes the current landscape but also identifies critical research gaps. By evaluating both offensive and defensive applications, we aim to provide a holistic understanding of the potential risks and opportunities associated with LLM-driven cybersecurity.

The JETSCAPE Collaboration reports the first multi-messenger study of the QGP jet transport parameter $\hat{q}$ using Bayesian inference, incorporating all available hadron and jet inclusive yield and jet substructure data from RHIC and the LHC. The theoretical model utilizes virtuality-dependent in-medium partonic energy loss coupled to a detailed dynamical model of QGP evolution. Tension is observed when constraining $\hat{q}$ for different kinematic cuts of the inclusive hadron data. The addition of substructure data is shown to improve the constraint on $\hat{q}$, without inducing tension with the constraint due to inclusive observables. These studies provide new insight into the mechanisms of jet interactions in matter, and point to next steps in the field for comprehensive understanding of jet quenching as a probe of the QGP., Comment: 4 pages, 2 figures. Proceedings of Quark Matter 2023 - XXXth International Conference on Ultra-relativistic Nucleus-Nucleus Collisions, Houston, TX, 3-9 September 2023

Considerable information about the early-stage dynamics of heavy-ion collisions is encoded in the rapidity dependence of measurements. To leverage the large amount of experimental data, we perform a systematic analysis using three-dimensional hydrodynamic simulations of multiple collision systems -- large and small, symmetric and asymmetric. Specifically, we perform fully 3D multi-stage hydrodynamic simulations initialized by a parameterized model for rapidity-dependent energy deposition, which we calibrate on the hadron multiplicity and anisotropic flow coefficients. We utilize Bayesian inference to constrain properties of the early- and late- time dynamics of the system, and highlight the impact of enforcing global energy conservation in our 3D model.

In this talk we review jet production in a large variety of collision systems using the JETSCAPE event generator and Hybrid Hadronization. Hybrid Hadronization combines quark recombination, applicable when distances between partons in phase space are small, and string fragmentation appropriate for dilute parton systems. It can therefore smoothly describe the transition from very dilute parton systems like $e^++e^-$ to full $A+A$ collisions. We test this picture by using JETSCAPE to generate jets in various systems. Comparison to experimental data in $e^++e^-$ and $p+p$ collisions allows for a precise tuning of vacuum baseline parameters in JETSCAPE and Hybrid Hadronization. Proceeding to systems with jets embedded in a medium, we study in-medium hadronization for jet showers. We quantify the effects of an ambient medium, focusing in particular on the dependence on the collective flow and size of the medium. Our results clarify the effects we expect from in-medium hadronization of jets on observables like fragmentation functions, hadron chemistry and jet shape., Comment: 6 pages, 4 figures, Hard Probes 2023 conference, accepted for publication in Proceedings of Science: version 2, references added, typos fixed

We present a study of surrogate losses and algorithms for the general problem of learning to defer with multiple experts. We first introduce a new family of surrogate losses specifically tailored for the multiple-expert setting, where the prediction and deferral functions are learned simultaneously. We then prove that these surrogate losses benefit from strong $H$-consistency bounds. We illustrate the application of our analysis through several examples of practical surrogate losses, for which we give explicit guarantees. These loss functions readily lead to the design of new learning to defer algorithms based on their minimization. While the main focus of this work is a theoretical analysis, we also report the results of several experiments on SVHN and CIFAR-10 datasets., Comment: ISAIM 2024

We study the key framework of learning with abstention in the multi-class classification setting. In this setting, the learner can choose to abstain from making a prediction with some pre-defined cost. We present a series of new theoretical and algorithmic results for this learning problem in the predictor-rejector framework. We introduce several new families of surrogate losses for which we prove strong non-asymptotic and hypothesis set-specific consistency guarantees, thereby resolving positively two existing open questions. These guarantees provide upper bounds on the estimation error of the abstention loss function in terms of that of the surrogate loss. We analyze both a single-stage setting where the predictor and rejector are learned simultaneously and a two-stage setting crucial in applications, where the predictor is learned in a first stage using a standard surrogate loss such as cross-entropy. These guarantees suggest new multi-class abstention algorithms based on minimizing these surrogate losses. We also report the results of extensive experiments comparing these algorithms to the current state-of-the-art algorithms on CIFAR-10, CIFAR-100 and SVHN datasets. Our results demonstrate empirically the benefit of our new surrogate losses and show the remarkable performance of our broadly applicable two-stage abstention algorithm., Comment: ALT 2024

Learning with abstention is a key scenario where the learner can abstain from making a prediction at some cost. In this paper, we analyze the score-based formulation of learning with abstention in the multi-class classification setting. We introduce new families of surrogate losses for the abstention loss function, which include the state-of-the-art surrogate losses in the single-stage setting and a novel family of loss functions in the two-stage setting. We prove strong non-asymptotic and hypothesis set-specific consistency guarantees for these surrogate losses, which upper-bound the estimation error of the abstention loss function in terms of the estimation error of the surrogate loss. Our bounds can help compare different score-based surrogates and guide the design of novel abstention algorithms by minimizing the proposed surrogate losses. We experimentally evaluate our new algorithms on CIFAR-10, CIFAR-100, and SVHN datasets and the practical significance of our new surrogate losses and two-stage abstention algorithms. Our results also show that the relative performance of the state-of-the-art score-based surrogate losses can vary across datasets., Comment: AISTATS 2024

Understanding the modification of jets and high-$p_T$ probes in small systems requires the integration of soft and hard physics. We present recent developments in extending the JETSCAPE framework to build an event generator, which includes correlations between soft and hard partons, to study jet observables in small systems. The multi-scale physics of the collision is separated into different stages. Hard scatterings are first sampled at binary collision positions provided by the Glauber geometry. They are then propagated backward in space-time following an initial-state shower to obtain the initiating partons' energies and momenta before the collision. These energies and momenta are then subtracted from the incoming colliding nucleons for soft-particle production, modeled by the 3D-Glauber + hydrodynamics + hadronic transport framework. This new hybrid approach (X-SCAPE) includes non-trivial correlations between jet and soft particle productions in small systems. We calibrate this framework with the final state hadrons' $p_T$-spectra from low to high $p_T$ in $p$-$p$, and and then compare with the spectra in $p$-$Pb$ collisions from the LHC. We also present results for additional observables such as the distributions of event activity as a function of the hardest jet $p_T$ in forward and mid-rapidity for both $p$-$p$ and $p$-$Pb$ collisions., Comment: 6 pages, 6 figures, Proceedings of Hard Probes 2023, 26-31 March 2023 Aschaffenburg, Germany

We study parton energy-momentum exchange with the quark gluon plasma (QGP) within a multistage approach composed of in-medium DGLAP evolution at high virtuality, and (linearized) Boltzmann Transport formalism at lower virtuality. This multistage simulation is then calibrated in comparison with high $p_T$ charged hadrons, D-mesons, and the inclusive jet nuclear modification factors, using Bayesian model-to-data comparison, to extract the virtuality-dependent transverse momentum broadening transport coefficient $\hat{q}$. To facilitate this undertaking, we develop a quantitative metric for validating the Bayesian workflow, which is used to analyze the sensitivity of various model parameters to individual observables. The usefulness of this new metric in improving Bayesian model emulation is shown to be highly beneficial for future such analyses., Comment: 24 pages, 16 figures

Shower development dynamics for a jet traveling through the quark-gluon plasma (QGP) is a multiscale process, where the heavy flavor mass is an important scale. During the high virtuality portion of the jet evolution in the QGP, emission of gluons from a heavy flavor is modified owing to heavy quark mass. Medium-induced radiation of heavy flavor is sensitive to microscopic processes (e.g. diffusion), whose virtuality dependence is phenomenologically explored in this study. In the lower virtuality part of shower evolution, i.e. when the mass is comparable to the virtuality of the parton, scattering and radiation processes of heavy quarks differ from light quarks. The effects of these mechanisms on shower development in heavy flavor tagged showers in the QGP is explored here. Furthermore, this multiscale study examines dynamical pair production of heavy flavor (via virtual gluon splittings) and their subsequent evolution in the QGP, which is not possible otherwise. A realistic event-by-event simulation is performed using the JETSCAPE framework. Energy-momentum exchange with the medium proceeds using a weak coupling recoil approach. Using leading hadron and open heavy flavor observables, differences in heavy versus light quark energy-loss mechanisms are explored, while the importance of heavy flavor pair production is highlighted along with future directions to study., Comment: 6 pages, 3 figures, contribution to the Hard Probes 2023 proceedings

We utilize event-by-event Monte Carlo simulations within the JETSCAPE framework to examine scale-dependent jet-medium interactions in heavy-ion collisions. The reduction in jet-medium interaction during the early high-virtuality stage, where the medium is resolved at a short distance scale, is emphasized as a key element in explaining multiple jet observables, particularly substructures, simultaneously. By employing the MATTER+LBT setup, which incorporates this explicit reduction of medium effects at high virtuality, we investigate jet substructure observables, such as Soft Drop groomed observables. When contrasted with existing data, our findings spotlight the significant influence of the reduction at the early high-virtuality stages. Furthermore, we study the substructure of gamma-tagged jets, providing predictive insights for future experimental analyses. This broadens our understanding of the various contributing factors involved in modifying jet substructures., Comment: 6 pages, 4 figures, contribution to the Hard Probes 2023 proceedings

We introduce a novel framework of ranking with abstention, where the learner can abstain from making prediction at some limited cost $c$. We present a extensive theoretical analysis of this framework including a series of $H$-consistency bounds for both the family of linear functions and that of neural networks with one hidden-layer. These theoretical guarantees are the state-of-the-art consistency guarantees in the literature, which are upper bounds on the target loss estimation error of a predictor in a hypothesis set $H$, expressed in terms of the surrogate loss estimation error of that predictor. We further argue that our proposed abstention methods are important when using common equicontinuous hypothesis sets in practice. We report the results of experiments illustrating the effectiveness of ranking with abstention.

In many applications, the labeled data at the learner's disposal is subject to privacy constraints and is relatively limited. To derive a more accurate predictor for the target domain, it is often beneficial to leverage publicly available labeled data from an alternative domain, somewhat close to the target domain. This is the modern problem of supervised domain adaptation from a public source to a private target domain. We present two $(\epsilon, \delta)$-differentially private adaptation algorithms for supervised adaptation, for which we make use of a general optimization problem, recently shown to benefit from favorable theoretical learning guarantees. Our first algorithm is designed for regression with linear predictors and shown to solve a convex optimization problem. Our second algorithm is a more general solution for loss functions that may be non-convex but Lipschitz and smooth. While our main objective is a theoretical analysis, we also report the results of several experiments first demonstrating that the non-private versions of our algorithms outperform adaptation baselines and next showing that, for larger values of the target sample size or $\epsilon$, the performance of our private algorithms remains close to that of the non-private formulation.

We study a problem of best-effort adaptation motivated by several applications and considerations, which consists of determining an accurate predictor for a target domain, for which a moderate amount of labeled samples are available, while leveraging information from another domain for which substantially more labeled samples are at one's disposal. We present a new and general discrepancy-based theoretical analysis of sample reweighting methods, including bounds holding uniformly over the weights. We show how these bounds can guide the design of learning algorithms that we discuss in detail. We further show that our learning guarantees and algorithms provide improved solutions for standard domain adaptation problems, for which few labeled data or none are available from the target domain. We finally report the results of a series of experiments demonstrating the effectiveness of our best-effort adaptation and domain adaptation algorithms, as well as comparisons with several baselines. We also discuss how our analysis can benefit the design of principled solutions for fine-tuning.

Cross-entropy is a widely used loss function in applications. It coincides with the logistic loss applied to the outputs of a neural network, when the softmax is used. But, what guarantees can we rely on when using cross-entropy as a surrogate loss? We present a theoretical analysis of a broad family of loss functions, comp-sum losses, that includes cross-entropy (or logistic loss), generalized cross-entropy, the mean absolute error and other cross-entropy-like loss functions. We give the first $H$-consistency bounds for these loss functions. These are non-asymptotic guarantees that upper bound the zero-one loss estimation error in terms of the estimation error of a surrogate loss, for the specific hypothesis set $H$ used. We further show that our bounds are tight. These bounds depend on quantities called minimizability gaps. To make them more explicit, we give a specific analysis of these gaps for comp-sum losses. We also introduce a new family of loss functions, smooth adversarial comp-sum losses, that are derived from their comp-sum counterparts by adding in a related smooth term. We show that these loss functions are beneficial in the adversarial setting by proving that they admit $H$-consistency bounds. This leads to new adversarial robustness algorithms that consist of minimizing a regularized smooth adversarial comp-sum loss. While our main purpose is a theoretical analysis, we also present an extensive empirical analysis comparing comp-sum losses. We further report the results of a series of experiments demonstrating that our adversarial robustness algorithms outperform the current state-of-the-art, while also achieving a superior non-adversarial accuracy., Comment: ICML 2023

Time Series Classification (TSC) is an important and challenging task for many visual computing applications. Despite the extensive range of methods developed for TSC, relatively few utilized Deep Neural Networks (DNNs). In this paper, we propose two novel attention blocks (Global Temporal Attention and Temporal Pseudo-Gaussian augmented Self-Attention) that can enhance deep learning-based TSC approaches, even when such approaches are designed and optimized for a specific dataset or task. We validate this claim by evaluating multiple state-of-the-art deep learning-based TSC models on the University of East Anglia (UEA) benchmark, a standardized collection of 30 Multivariate Time Series Classification (MTSC) datasets. We show that adding the proposed attention blocks improves base models' average accuracy by up to 3.6%. Additionally, the proposed TPS block uses a new injection module to include the relative positional information in transformers. As a standalone unit with less computational complexity, it enables TPS to perform better than most of the state-of-the-art DNN-based TSC methods. The source codes for our experimental setups and proposed attention blocks are made publicly available.

Blackwell's celebrated approachability theory provides a general framework for a variety of learning problems, including regret minimization. However, Blackwell's proof and implicit algorithm measure approachability using the $\ell_2$ (Euclidean) distance. We argue that in many applications such as regret minimization, it is more useful to study approachability under other distance metrics, most commonly the $\ell_\infty$-metric. But, the time and space complexity of the algorithms designed for $\ell_\infty$-approachability depend on the dimension of the space of the vectorial payoffs, which is often prohibitively large. Thus, we present a framework for converting high-dimensional $\ell_\infty$-approachability problems to low-dimensional pseudonorm approachability problems, thereby resolving such issues. We first show that the $\ell_\infty$-distance between the average payoff and the approachability set can be equivalently defined as a pseudodistance between a lower-dimensional average vector payoff and a new convex set we define. Next, we develop an algorithmic theory of pseudonorm approachability, analogous to previous work on approachability for $\ell_2$ and other norms, showing that it can be achieved via online linear optimization (OLO) over a convex set given by the Fenchel dual of the unit pseudonorm ball. We then use that to show, modulo mild normalization assumptions, that there exists an $\ell_\infty$-approachability algorithm whose convergence is independent of the dimension of the original vectorial payoff. We further show that that algorithm admits a polynomial-time complexity, assuming that the original $\ell_\infty$-distance can be computed efficiently. We also give an $\ell_\infty$-approachability algorithm whose convergence is logarithmic in that dimension using an FTRL algorithm with a maximum-entropy regularizer., Comment: To appear at ALT 2023

We present predictions and postdictions for a wide variety of hard jet-substructure observables using a multistage model within the JETSCAPE framework. The details of the multistage model and the various parameter choices are described in [A. Kumar et al., arXiv:2204.01163]. A novel feature of this model is the presence of two stages of jet modification: a high virtuality phase [modeled using the modular all twist transverse-scattering elastic-drag and radiation model (MATTER)], where modified coherence effects diminish medium-induced radiation, and a lower virtuality phase [modeled using the linear Boltzmann transport model (LBT)], where parton splits are fully resolved by the medium as they endure multiple scattering induced energy loss. Energy-loss calculations are carried out on event-by-event viscous fluid dynamic backgrounds constrained by experimental data. The uniform and consistent descriptions of multiple experimental observables demonstrate the essential role of modified coherence effects and the multistage modeling of jet evolution. Using the best choice of parameters from [A. Kumar et al., arXiv:2204.01163], and with no further tuning, we present calculations for the medium modified jet fragmentation function, the groomed jet momentum fraction $z_g$ and angular separation $r_g$ distributions, as well as the nuclear modification factor of groomed jets. These calculations provide accurate descriptions of published data from experiments at the Large Hadron Collider. Furthermore, we provide predictions from the multistage model for future measurements at the BNL Relativistic Heavy Ion Collider., Comment: 20 pages, 12 figures

We explore jet-medium interactions at various scales in high-energy heavy-ion collisions using the JETSCAPE framework. The physics of the multi-stage modeling and the coherence effect at high virtuality is discussed through the results of multiple jet and high-$p_{\mathrm{T}}$ particle observables, compared with experimental data. Furthermore, we investigate the jet-medium interaction involved in the hadronization process., Comment: 6 pages, 5 figures, contribution to the Quark Matter 2022 proceedings