Nader H. Bshouty Claudio Gentile (Eds.)

Learning Theory

20th Annual Conference on Learning Theory, COLT 2007 San Diego, CA, USA, June 13-15, 2007 Proceedings

4y Spri ringer

Table of Contents

Invited Presentat ions

Property Testing: A Learning Theory Perspective 1 Dana Ron

Spectral Algorithms for Learning and Clustering 3 Santosh S. Vempala

Unsupervised, Semisupervised and Active Learning I

Minimax Bounds for Active Learning 5 Rui M. Castro and Robert D. Nowak

Stability of fc-Means Clustering 20 Shai Ben-David, David Pal, and Hans Ulrich Simon

Margin Based Active Learning 35 Maria-Florina Balcan, Andrei Broder, and Tong Zhang

Unsupervised, Semisupervised and Active Learning II

Learning Large-Alphabet and Analog Circuits with Value Injection Queries 51

Dana Angluin, James Aspnes, Jiang Chen, and Lev Reyzin

Teaching Dimension and the Complexity of Active Learning 66 Steve Hanneke

Multi-view Regression Via Canonical Correlation Analysis 82 Sham M. Kakade and Dean P. Foster

Statistical Learning Theory

Aggregation by Exponential Weighting and Sharp Oracle Inequalities . . . 97 Arnak S. Dalalyan and Alexandre B. Tsybakov

Occam's Hammer 112 Gilles Blanchard and Frangois Fleuret

Resampling-Based Confidence Regions and Multiple Tests for a Correlated Random Vector 127

Sylvain Arlot, Gilles Blanchard, and Etienne Roquain

X Table of Contents

Suboptimality of Penalized Empirical Risk Minimization in Classification 142

Guülaume Lecue

Transductive Rademacher Complexity and Its Applications 157 Ran El-Yaniv and Dmitry Pechyony

Inductive Inference

U-Shaped, Iterative, and Iterative-with-Counter Learning 172 John Case and Samuel E. Moelius III

Mind Change Optimal Learning of Bayes Net Structure 187 Oliver Schulte, Wei Luo, and Russell Greiner

Learning Correction Grammars 203 Lorenzo Cariucci, John Case, and Sanjay Jain

Mitotic Classes 218 Sanjay Jain and Frank Stephan

Online and Reinforcement Learning I

Regret to the Best vs. Regret to the Average 233 Eyal Even-Dar, Michael Kearns, Yishay Mansour, and Jennifer Wortman

Strategies for Prediction Under Imperfect Monitoring 248 Gabor Lugosi, Shie Mannor, and Gilles Stoltz

Bounded Parameter Markov Decision Processes with Average Reward Criterion 263

Ambuj Tewari and Peter L. Bartlett

Online and Reinforcement Learning II

On-Line Estimation with the Multivariate Gaussian Distribution 278 Sanjoy Dasgupta and Daniel Hsu

Generalised Entropy and Asymptotic Complexities of Languages 293 Yuri Kalnishkan, Vladimir Vovk, and Michael V. Vyugin

Q-Learning with Linear Function Approximation 308 Francisco S. Melo and M. Isabel Ribeiro

Regularized Learning, Kernel Methods, SVM

How Good Is a Kernel When Used as a Similarity Measure? 323 Nathan Srebro

Table of Contents X I

Gaps in Support Vector Optimization 336 Nikolas List, Don Hush, Clint Scovel, and Ingo Steinwart

Learning Languages with Rational Kernels 349 Corinna Cortes, Leonid Kontorovich, and Mehryar Mohri

Generalized SMO-Style Decomposition Algorithms 365 Nikolas List

Learning Algorithms and Limitations on Learning

Learning Nested Halfspaces and Uphill Decision Trees 378 Adam, Tauman Kalai

An EfScient Re-scaled Perceptron Algorithm for Conic Systems 393 Alexandre Belloni, Robert M. Freund, and Santosh S. Vempala

A Lower Bound for Agnostically Learning Disjunctions 409 Adam R. Klivans and Alexander A. Sherstov

Sketching Information Divergences 424 Sudipto Guha, Piotr Indyk, and Andrew McGregor

Online and Reinforcement Learning III

Competing with Stationary Prediction Strategies 439 Vladimir Vovk

Improved Rates for the Stochastic Continuum-Armed Bandit Problem 454

Peter Auer, Ronald Ortner, and Csaba Szepesvdri

Learning Permutations with Exponential Weights 469 David P. Helmbold and Manfred K. Warmuth

Online and Reinforcement Learning IV

Multitask Learning with Expert Advice 484 Jacob Abernethy, Peter Bartlett, and Alexander Rakhlin

Online Learning with Prior Knowledge 499 Elad Hazan and Nimrod Megiddo

Dimensionality Reduction

Nonlinear Estimators and Tail Bounds for Dimension Reduction in li Using Cauchy Random Projections 514

Fing Li, Trevor J. Hastie, and Kenneth W. Church

XII Table of Contents

Sparse Density Estimation with i\ Penalties 530 Florentina Bunea, Alexandre B. Tsybakov, and Märten H. Wegkamp

l\ Regularization in Infinite Dimensional Feature Spaces 544 Saharon Rosset, Grzegorz Swirszcz, Nathan Srebro, and Ji Zhu

Prediction by Categorical Features: Generalization Properties and Application to Feature Ranking 559

Sivan Sabato and Shai Shalev-Shwartz

Other Approaches

Observational Learning in Random Networks 574 Julian Lorenz, Martin Marciniszyn, and Angelika Steger

The Loss Rank Principle for Model Selection 589 Marcus Hutter

Robust Reductions from Ranking to Classification 604 Maria-Florina Balcan, Nikhil Bansal, Alina Beygelzimer, Don Coppersmüh, John Langford, and Gregory B. Sorkin

Open Problems

Rademacher Margin Complexity 620 Liwei Wang and Jufu Feng

Open Problems in Efficient Semi-supervised PAC Learning 622 Avrim Blum and Maria-Florina Balcan

Resource-Bounded Information Gathering for Correlation Clustering . . . 625 Pallika Kanani and Andrew McCallum

Are There Local Maxima in the Infinite-Sample Likelihood of Gaussian Mixture Estimation? 628

Nathan Srebro

When Is There a Free Matrix Lunch? 630 Manfred K. Warmuth

Author Index 633