  We introduce a class of quadratic support (QS) functions, many of which play
a crucial role in a variety of applications, including machine learning, robust
statistical inference, sparsity promotion, and Kalman smoothing. Well known
examples include the l2, Huber, l1 and Vapnik losses. We build on a dual
representation for QS functions using convex analysis, revealing the structure
necessary for a QS function to be interpreted as the negative log of a
probability density, and providing the foundation for statistical
interpretation and analysis of QS loss functions. For a subclass of QS
functions called piecewise linear quadratic (PLQ) penalties, we also develop
efficient numerical estimation schemes. These components form a flexible
statistical modeling framework for a variety of learning applications, together
with a toolbox of efficient numerical methods for inference. In particular, for
PLQ densities, interior point (IP) methods can be used. IP methods solve
nonsmooth optimization problems by working directly with smooth systems of
equations characterizing their optimality. The efficiency of the IP approach
depends on the structure of particular applications. We consider the class of
dynamic inverse problems using Kalman smoothing, where the aim is to
reconstruct the state of a dynamical system with known process and measurement
models starting from noisy output samples. In the classical case, Gaussian
errors are assumed in the process and measurement models. The extended
framework allows arbitrary PLQ densities to be used, and the proposed IP
approach solves the generalized Kalman smoothing problem while maintaining the
linear complexity in the size of the time series, just as in the Gaussian case.
This extends the computational efficiency of classic algorithms to a much
broader nonsmooth setting, and includes many recently proposed robust and
sparse smoothers as special cases.
