Publications

2024

1. JMLR

   Estimating the Minimizer and the Minimum Value of a Regression Function

   Arya Akhavan, Davit Gogolashvili, and Alexandre B. Tsybakov

   Journal of Machine Learning Research, 2024

   Abs HTML

   We propose a new method for estimating the minimizer \boldsymbolx^* and the minimum value f^* of a smooth and strongly convex regression function f from the observations contaminated by random noise. Our estimator \boldsymbolz_n of the minimizer \boldsymbolx^* is based on a version of the projected gradient descent with the gradient estimated by a regularized local polynomial algorithm. Next, we propose a two-stage procedure for estimation of the minimum value f^* of regression function f. At the first stage, we construct an accurate enough estimator of \boldsymbolx^*, which can be, for example, \boldsymbolz_n. At the second stage, we estimate the function value at the point obtained in the first stage using a rate optimal nonparametric procedure. We derive non-asymptotic upper bounds for the quadratic risk and optimization risk of \boldsymbolz_n, and for the risk of estimating f^*. We establish minimax lower bounds showing that, under certain choice of parameters, the proposed algorithms achieve the minimax optimal rates of convergence on the class of smooth and strongly convex functions.

2023

1. arXiv

   When is Importance Weighting Correction Needed for Covariate Shift Adaptation?

   Davit Gogolashvili, Matteo Zecchin, Motonobu Kanagawa, and 2 more authors

   arXiv preprint arXiv:2303.04020, 2023

   Abs HTML

   This paper investigates when the importance weighting (IW) correction is needed to address covariate shift, a common situation in supervised learning where the input distributions of training and test data differ. Classic results show that the IW correction is needed when the model is parametric and misspecified. In contrast, recent results indicate that the IW correction may not be necessary when the model is nonparametric and well-specified. We examine the missing case in the literature where the model is nonparametric and misspecified, and show that the IW correction is needed for obtaining the best approximation of the true unknown function for the test distribution. We do this by analyzing IW-corrected kernel ridge regression, covering a variety of settings, including parametric and nonparametric models, well-specified and misspecified settings, and arbitrary weighting functions.

2022

1. Locally Smoothed Gaussian Process Regression

   Davit Gogolashvili, Bogdan Kozyrskiy, and Maurizio Filippone

   Procedia Computer Science, 2022

   Abs HTML

   We develop a novel framework to accelerate Gaussian process regression (GPR). In particular, we consider localization kernels at each data point to down-weigh the contributions from other data points that are far away, and we derive the GPR model stemming from the application of such localization operation. Through a set of experiments, we demonstrate the competitive performance of the proposed approach compared to full GPR, other localized models, and deep Gaussian processes. Crucially, these performances are obtained with considerable speedups compared to standard global GPR due to the sparsification effect of the Gram matrix induced by the localization operation.