Search

Your search keyword '"Lazaric, Alessandro"' showing total 157 results
Start Over Author "Lazaric, Alessandro"
157 results on '"Lazaric, Alessandro"'

151. A Truthful Learning Mechanism for Contextual Multi-Slot Sponsored Search Auctions with Externalities.

Author

Gatti, Nicola, Lazaric, Alessandro, and Trovò, Francesco

Subjects
INTERNET auctions, INTERNET advertising, CLICK through rate, MACHINE learning, COMPUTER simulation, SIMULATION methods & models
Abstract

Sponsored search auctions constitute one of the most successful applications of microeconomic mechanisms. In mechanism design, auctions are usually designed to incentivize advertisers to bid their truthful valuations and, at the same time, to assure both the advertisers and the auctioneer a non-negative utility. Nonetheless, in sponsored search auctions, the click-through-rates (CTRs) of the advertisers are often unknown to the auctioneer and thus standard incentive compatible mechanisms cannot be directly applied and must be paired with an effective learning algorithm for the estimation of the CTRs. This introduces the critical problem of designing a learning mechanism able to estimate the CTRs as the same time as implementing a truthful mechanism with a revenue loss as small as possible compared to an optimal mechanism designed with the true CTRs. Previous works showed that in single-slot auctions the problem can be solved using a suitable exploration-exploitation mechanism able to achieve a per-step regret of order O(T-1/3) (where T is the number of times the auction is repeated). In this paper we extend these results to the general case of contextual multi-slot auctions with position- and ad-dependent externalities. In particular, we prove novel upper-bounds on the revenue loss w.r.t. to a VCG auction and we report numerical simulations investigating their accuracy in predicting the dependency of the regret on the number of rounds T, the number of slots K, and the number of advertisements n. [ABSTRACT FROM AUTHOR]

Published
2012

152. Reinforcement Learning in Complex Environments Through Multiple Adaptive Partitions.

Author

Carbonell, Jaime G., Siekmann, Jörg, Basili, Roberto, Pazienza, Maria Teresa, Bonarini, Andrea, Lazaric, Alessandro, and Restelli, Marcello

Abstract

The application of Reinforcement Learning (RL) algorithms to learn tasks for robots is often limited by the large dimension of the state space, which may make prohibitive its application on a tabular model. In this paper, we describe LEAP (Learning Entities Adaptive Partitioning), a model-free learning algorithm that uses overlapping partitions which are dynamically modified to learn near-optimal policies with a small number of parameters. Starting from a coarse aggregation of the state space, LEAP generates refined partitions whenever it detects an incoherence between the current action values and the actual rewards from the environment. Since in highly stochastic problems the adaptive process can lead to over-refinement, we introduce a mechanism that prunes the macrostates without affecting the learned policy. Through refinement and pruning, LEAP builds a multi-resolution state representation specialized only where it is actually needed. In the last section, we present some experimental evaluation on a grid world and a complex simulated robotic soccer task. [ABSTRACT FROM AUTHOR]

Published
2007
Full Text
View/download PDF

153. Finite-Sample Analysis of Least-Squares Policy Iteration.

Author

Lazaric, Alessandro, Ghavamzadeh, Mohammad, and Munos, Rémi

Subjects
*LEAST squares, *ITERATIVE methods (Mathematics), *MACHINE learning, *COMPUTER algorithms, *MARKOV processes, *ERROR analysis in mathematics, *PERFORMANCE evaluation
Abstract

In this paper, we report a performance bound for the widely used least-squares policy iteration (LSPI) algorithm. We first consider the problem of policy evaluation in reinforcement learning, that is, learning the value function of a fixed policy, using the least-squares temporal-difference (LSTD) learning method, and report finite-sample analysis for this algorithm. To do so, we first derive a bound on the performance of the LSTD solution evaluated at the states generated by the Markov chain and used by the algorithm to learn an estimate of the value function. This result is general in the sense that no assumption is made on the existence of a stationary distribution for the Markov chain. We then derive generalization bounds in the case when the Markov chain possesses a stationary distribution and is β-mixing. Finally, we analyze how the error at each policy evaluation step is propagated through the iterations of a policy iteration method, and derive a performance bound for the LSPI algorithm. [ABSTRACT FROM AUTHOR]

Published
2012

157. Conservative and Greedy Approaches to Classification-based Policy Iteration

Author

Mohammad Ghavamzadeh, Alessandro Lazaric, Sequential Learning (SEQUEL), Laboratoire d'Informatique Fondamentale de Lille (LIFL), Université de Lille, Sciences et Technologies-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lille, Sciences Humaines et Sociales-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lille, Sciences Humaines et Sociales-Centre National de la Recherche Scientifique (CNRS)-Laboratoire d'Automatique, Génie Informatique et Signal (LAGIS), Université de Lille, Sciences et Technologies-Centrale Lille-Centre National de la Recherche Scientifique (CNRS)-Centrale Lille-Centre National de la Recherche Scientifique (CNRS)-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria), European Project: 270327,EC:FP7:ICT,FP7-ICT-2009-6,COMPLACS(2011), Université de Lille, Sciences et Technologies-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lille, Sciences Humaines et Sociales-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lille, Sciences Humaines et Sociales-Centre National de la Recherche Scientifique (CNRS)-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire d'Automatique, Génie Informatique et Signal (LAGIS), Université de Lille, Sciences et Technologies-Centrale Lille-Centre National de la Recherche Scientifique (CNRS)-Centrale Lille-Centre National de la Recherche Scientifique (CNRS), Lazaric, Alessandro, and Composing Learning for Artificial Cognitive Systems - COMPLACS - - EC:FP7:ICT2011-03-01 - 2015-02-28 - 270327 - VALID

Subjects
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML], General Medicine, [STAT.ML] Statistics [stat]/Machine Learning [stat.ML]
Abstract

The existing classification-based policy iteration (CBPI) algorithms can be divided into two categories: direct policy iteration (DPI) methods that directly assign the output of the classifier (the approximate greedy policy w.r.t.~the current policy) to the next policy, and conservative policy iteration (CPI) methods in which the new policy is a mixture distribution of the current policy and the output of the classifier. The conservative policy update gives CPI a desirable feature, namely the guarantee that the policies generated by this algorithm improve at each iteration. We provide a detailed algorithmic and theoretical comparison of these two classes of CBPI algorithms. Our results reveal that in order to achieve the same level of accuracy, CPI requires more iterations, and thus, more samples than the DPI algorithm. Furthermore, CPI may converge to suboptimal policies whose performance is not better than DPI's.

Published
2012

Catalog

Books, media, physical & digital resources
See catalog results