Search

Supervisory signals are a critical resource for training learning to rank models. In many real-world search and retrieval scenarios, these signals may not be readily available or could be costly to obtain for some queries. The examples include domains where labeling requires professional expertise, applications with strong privacy constraints, and user engagement information that are too scarce. We refer to these scenarios as sparsely supervised queries which pose significant challenges to traditional learning to rank models. In this work, we address sparsely supervised queries by proposing a novel meta learning to rank framework which leverages fast learning and adaption capability of meta-learning. The proposed approach accounts for the fact that different queries have different optimal parameters for their rankers, in contrast to traditional learning to rank models which only learn a global ranking model applied to all the queries. In consequence, the proposed method would yield significant advantages especially when new queries are of different characteristics with the training queries. Moreover, the proposed meta learning to rank framework is generic and flexible. We conduct a set of comprehensive experiments on both public datasets and a real-world e-commerce dataset. The results demonstrate that the proposed meta-learning approach can significantly enhance the performance of learning to rank models with sparsely labeled queries., Comment: Accepted at TOIS

It is an essential product requirement of Yahoo Mail to distinguish between personal and machine-generated emails. The old production classifier in Yahoo Mail was based on a simple logistic regression model. That model was trained by aggregating features at the SMTP address level. We propose building deep learning models at the message level. We built and trained four individual CNN models: (1) a content model with subject and content as input; (2) a sender model with sender email address and name as input; (3) an action model by analyzing email recipients' action patterns and correspondingly generating target labels based on senders' opening/deleting behaviors; (4) a salutation model by utilizing senders' "explicit salutation" signal as positive labels. Next, we built a final full model after exploring different combinations of the above four models. Experimental results on editorial data show that our full model improves the adjusted-recall from 70.5% to 78.8% compared to the old production model, while at the same time lifts the precision from 94.7% to 96.0%. Our full model also significantly beats the state-of-the-art Bert model at this task. This full model has been deployed into the current production system (Yahoo Mail 6)., Comment: This paper is accepted by AAAI'22

A/B testing, also known as controlled experiment, bucket testing or splitting testing, has been widely used for evaluating a new feature, service or product in the data-driven decision processes of online websites. The goal of A/B testing is to estimate or test the difference between the treatment effects of the old and new variations. It is a well-studied two-sample comparison problem if each user's response is influenced by her treatment only. However, in many applications of A/B testing, especially those in HIVE of Yahoo and other social networks of Microsoft, Facebook, LinkedIn, Twitter and Google, users in the social networks influence their friends via underlying social interactions, and the conventional A/B testing methods fail to work. This paper considers the network A/B testing problem and provide a general framework consisting of five steps: data sampling, probabilistic model, parameter inference, computing average treatment effect and hypothesis test. The framework performs well for network A/B testing in simulation studies.

In modeling spatial extremes, the dependence structure is classically inferred by assuming that block maxima derive from max-stable processes. Weather stations provide daily records rather than just block maxima. The point process approach for univariate extreme value analysis, which uses more historical data and is preferred by some practitioners, does not adapt easily to the spatial setting. We propose a two-step approach with a composite likelihood that utilizes site-wise daily records in addition to block maxima. The procedure separates the estimation of marginal parameters and dependence parameters into two steps. The first step estimates the marginal parameters with an independence likelihood from the point process approach using daily records. Given the marginal parameter estimates, the second step estimates the dependence parameters with a pairwise likelihood using block maxima. In a simulation study, the two-step approach was found to be more efficient than the pairwise likelihood approach using only block maxima. The method was applied to study the effect of El Ni\~{n}o-Southern Oscillation on extreme precipitation in California with maximum daily winter precipitation from 35 sites over 55 years. Using site-specific generalized extreme value models, the two-step approach led to more sites detected with the El Ni\~{n}o effect, narrower confidence intervals for return levels and tighter confidence regions for risk measures of jointly defined events., Comment: Published at http://dx.doi.org/10.1214/14-AOAS804 in the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Parametric max-stable processes are increasingly used to model spatial extremes. Starting from the fact that the dependence structure of a max-stable process is completely characterized by an extreme-value copula, a class of goodness-of-fit tests is proposed based on the comparison between a nonparametric and a parametric estimator of the corresponding unknown multivariate Pickands dependence function. Because of the high-dimensional setting under consideration, these functional estimators are only compared at a specific set of points at which they coincide, up to a multiplicative constant, with estimators of the extremal coefficients. The nonparametric estimators of the Pickands dependence function used in this work are those recently studied by Gudendorf and Segers. The parametric estimators rely on the use of the {\em pairwise pseudo-likelihood} which extends the concept of pairwise (composite) likelihood to a rank-based context. Approximate $p$-values for the resulting margin-free tests are obtained by means of a {\em one- or two-level parametric bootstrap}. Conditions for the asymptotic validity of these resampling procedures are given based on the work of Genest and R\'emillard. The finite-sample performance of the tests is investigated in dimension 10 under the Smith, Schlather and geometric Gaussian models. An application of the tests to rainfall data is finally presented., Comment: 28 pages, 3 figures, 5 tables