Linxi Liu
University of Pittsburgh
154 Hurley Hall
3:30 pm - 4:30 pm
Bayesian Trees and Forests for Unsupervised Learning and Their Spatial Adaptation Properties
Tree-based methods are popular nonparametric tools for capturing interactive effects and making predictions in multivariate problems. Under the context of unsupervised learning, trees and their ensembles have also been applied to a wide range of statistical inference problems, such as multi-resolution sketching of distributional variations, localization of high-density regions, and design of efficient data compression schemes. In this talk, I will focus on the density estimation problem, a fundamental one in unsupervised learning. We consider the optional Pólya tree (Wong and Ma, 2010) prior and the Dirichlet prior or their variations on individual trees. First we show that Bayesian density trees can achieve minimax (up to a logarithmic term) convergence over the anisotropic Besov class, which implies that tree based methods can adapt to spatially inhomogeneous features of the underlying density function, and can achieve fast convergence as the dimension increases. We will also introduce a novel Bayesian model for forests and show that for a class of anisotropic H ̈older continuous functions, such type of density forests can achieve faster convergence than trees. The convergence rate is adaptive in the sense that to achieve such a rate we do not need any prior knowledge of the smoothness level of the density. The Bayesian framework naturally endows a stochastic search scheme over the tree or forest space. For both Bayesian unsupervised trees and forests, I will provide several numerical results to illustrate their performance in the moderately high-dimensional case.
Full List of Statistics Seminar Speakers
View Poster
Originally published at acms.nd.edu.