A nearest neighbor estimate of the residual variance

Devroye, Luc; Györfi, László; Lugosi, Gábor; Walk, Harro

doi:http://dx.doi.org/10.1214/18-EJS1438

A nearest neighbor estimate of the residual variance

License

Citation

Devroye L, Györfi L, Lugosi G, Walk H. A nearest neighbor estimate of the residual variance. Electron J Stat. 2018 Jun 6;12(1):1752-78. DOI: 10.1214/18-EJS1438

Abstract

We study the problem of estimating the smallest achievable mean-squared error in regression function estimation. The problem is equivalent to estimating the second moment of the regression function of Y on X∈Rd. We introduce a nearest-neighbor-based estimate and obtain a normal limit law for the estimate when X has an absolutely continuous distribution, without any condition on the density. We also compute the asymptotic variance explicitly and derive a non-asymptotic bound on the variance that does not depend on the dimension d. The asymptotic variance does not depend on the smoothness of the density of X or of the regression function. A non-asymptotic exponential concentration inequality is also proved. We illustrate the use of the new estimate through testing whether a component of the vector X carries information for predicting Y.