  In this paper, we propose a new framework to construct confidence sets for a
$d$-dimensional unknown sparse parameter $\theta$ under the normal mean model
$X\sim N(\theta,\sigma^2I)$. A key feature of the proposed confidence set is
its capability to account for the sparsity of $\theta$, thus named as {\em
sparse} confidence set. This is in sharp contrast with the classical methods,
such as Bonferroni confidence intervals and other resampling based procedures,
where the sparsity of $\theta$ is often ignored. Specifically, we require the
desired sparse confidence set to satisfy the following two conditions: (i)
uniformly over the parameter space, the coverage probability for $\theta$ is
above a pre-specified level; (ii) there exists a random subset $S$ of
$\{1,...,d\}$ such that $S$ guarantees the pre-specified true negative rate
(TNR) for detecting nonzero $\theta_j$'s. To exploit the sparsity of $\theta$,
we define that the confidence interval for $\theta_j$ degenerates to a single
point 0 for any $j\notin S$. Under this new framework, we first consider
whether there exist sparse confidence sets that satisfy the above two
conditions. To address this question, we establish a non-asymptotic minimax
lower bound for the non-coverage probability over a suitable class of sparse
confidence sets. The lower bound deciphers the role of sparsity and minimum
signal-to-noise ratio (SNR) in the construction of sparse confidence sets.
Furthermore, under suitable conditions on the SNR, a two-stage procedure is
proposed to construct a sparse confidence set. To evaluate the optimality, the
proposed sparse confidence set is shown to attain a minimax lower bound of some
properly defined risk function up to a constant factor. Finally, we develop an
adaptive procedure to the unknown sparsity and SNR. Numerical studies are
conducted to verify the theoretical results.
