  In this work, we study the positive definiteness (PDness) problem in
covariance matrix estimation. For high dimensional data, many regularized
estimators are proposed under structural assumptions on the true covariance
matrix including sparsity. They are shown to be asymptotically consistent and
rate-optimal in estimating the true covariance matrix and its structure.
However, many of them do not take into account the PDness of the estimator and
produce a non-PD estimate. To achieve the PDness, researchers consider
additional regularizations (or constraints) on eigenvalues, which make both the
asymptotic analysis and computation much harder. In this paper, we propose a
simple modification of the regularized covariance matrix estimator to make it
PD while preserving the support. We revisit the idea of linear shrinkage and
propose to take a convex combination between the first-stage estimator (the
regularized covariance matrix without PDness) and a given form of diagonal
matrix. The proposed modification, which we denote as FSPD (Fixed Support and
Positive Definiteness) estimator, is shown to preserve the asymptotic
properties of the first-stage estimator, if the shrinkage parameters are
carefully selected. It has a closed form expression and its computation is
optimization-free, unlike existing PD sparse estimators. In addition, the FSPD
is generic in the sense that it can be applied to any non-PD matrix including
the precision matrix. The FSPD estimator is numerically compared with other
sparse PD estimators to understand its finite sample properties as well as its
computational gain. It is also applied to two multivariate procedures relying
on the covariance matrix estimator -- the linear minimax classification problem
and the Markowitz portfolio optimization problem -- and is shown to
substantially improve the performance of both procedures.
