SMNS event
Learning with Q-Exponential Processes
Regularization is one of the most fundamental topics in optimization, statistics and machine learning. To get sparsity in estimating a parameter vector, an l_q penalty term is usually added to the objective function. What is the probabilistic distribution corresponding to such penalty? What is the correct stochastic process for similar penalty when dealing with L_q functions? This is important for statistically modeling high-dimensional inhomogeneous objects such as images, with penalty to preserve certain properties, e.g. edges in the image. In this talk, I will introduce a novel stochastic process named q-exponential (Q-EP) process that corresponds to the L_q regularization of functions to facilitate learning in various supervised and un-supervised tasks. The work is closely related to Besov process which is usually defined in terms of series. Q-EP can be regarded as a probabilistic definition of Besov process with direct control on the correlation strength and tractable prediction formula. From the Bayesian perspective, Q-EP provides a flexible prior on functions with sharper penalty (q<2) than the commonly used Gaussian process (GP, corresponding to q=2). Q-EP has numerical superiority comparing with GP and Besov in modeling functional data, reconstructing images and solving inverse problems. I will also talk about the applications of Q-EP in latent representation learning, deep probabilistic models, and potentially diffusion models.