A Data-Dependent Skeleton Estimate and a Scale-Sensitive Dimension for Classification

Abstract

The classical binary classification problem is investigated when it is known in advance that the posterior probability function (or regression function) belongs to some class of functions. We introduce and analyze a method which effectively exploits this knowledge. The method is based on minimizing the empirical risk over a carefully selected ``skeleton'' of the class of regression functions. The skeleton is a covering of the class based on a data--dependent metric, especially fitted for classification. A new scale--sensitive dimension is introduced which is more useful for the studied classification problem than other, previously defined, dimension measures. This fact is demonstrated by performance bounds for the skeleton estimate in terms of the new dimension.