text: Recently, penalties promoting signals that are sparse within and across groups have been proposed. In this letter, we propose a generalization that allows to encode more intricate dependencies within groups. However, this complicates the realization of the threshold function associated with the penalty, which hinders the use of the penalty in energy minimization. We discuss how to sidestep this problem, and demonstrate the use of the modified penalty in an energy minimization formulation for an inverse problem.
  @cite_18: The proximity operator of a convex function is a natural extension of the notion of a projection operator onto a convex set. This tool, which plays a central role in the analysis and the numerical solution of convex optimization problems, has recently been introduced in the arena of inverse problems and, especially, in signal processing, where it has become increasingly important. In this paper, we review the basic properties of proximity operators which are relevant to signal processing and present optimization methods based on these operators. These proximal splitting methods are shown to capture and extend several well-known algorithms in a unifying framework. Applications of proximal methods in signal recovery and synthesis are discussed.
  @cite_9: This paper introduces an expectation-maximization (EM) algorithm for image restoration (deconvolution) based on a penalized likelihood formulated in the wavelet domain. Regularization is achieved by promoting a reconstruction with low-complexity, expressed in the wavelet coefficients, taking advantage of the well known sparsity of wavelet representations. Previous works have investigated wavelet-based restoration but, except for certain special cases, the resulting criteria are solved approximately or require demanding optimization methods. The EM algorithm herein proposed combines the efficient image representation offered by the discrete wavelet transform (DWT) with the diagonalization of the convolution operator obtained in the Fourier domain. Thus, it is a general-purpose approach to wavelet-based image restoration with computational complexity comparable to that of standard wavelet denoising schemes or of frequency domain deconvolution methods. The algorithm alternates between an E-step based on the fast Fourier transform (FFT) and a DWT-based M-step, resulting in an efficient iterative process requiring O(NlogN) operations per iteration. The convergence behavior of the algorithm is investigated, and it is shown that under mild conditions the algorithm converges to a globally optimal restoration. Moreover, our new approach performs competitively with, in some cases better than, the best existing methods in benchmark tests.
  @cite_21: We consider the iterative shrinkage thresholding algorithm (ISTA) applied to a cost function composed of a data fidelity term and a penalty term. The penalty is nonconvex but the concavity of the penalty is accounted for by the data fidelity term so that the overall cost function is convex. We provide a generalization of the convergence result for ISTA viewed as a forward-backward splitting algorithm. We also demonstrate experimentally that for the current setup, using large stepsizes in ISTA can accelerate convergence more than existing schemes proposed for the convex case, like TwIST or FISTA.
  @cite_24: We introduce a new penalty function that promotes signals composed of a small number of active groups, where within each group, only a few high magnitude coefficients are nonzero. We derive the threshold function associated with the proposed penalty and study its properties. We discuss how the proposed penalty threshold function can be useful for signals with isolated nonzeros, such as audio with isolated harmonics along the frequency axis, or reflection functions in exploration seismology where the nonzeros occur on the boundaries of subsoil layers. We demonstrate the use of the proposed penalty threshold functions in a convex denoising and a nonconvex deconvolution formulation. We provide convergent algorithms for both formulations and compare the performance with state-of-the-art methods.
  @cite_15: We show that various inverse problems in signal recovery can be formulated as the generic problem of minimizing the sum of two convex functions with certain regularity properties. This formulation makes it possible to derive existence, uniqueness, characterization, and stability results in a unified and standardized fashion for a large class of apparently disparate problems. Recent results on monotone operator splitting methods are applied to establish the convergence of a forward-backward algorithm to solve the generic problem. In turn, we recover, extend, and provide a simplified analysis for a variety of existing iterative methods. Applications to geometry texture image decomposition schemes are also discussed. A novelty of our framework is to use extensively the notion of a proximity operator, which was introduced by Moreau in the 1960s.
summary: The proposed modification to the SWAG penalty aims to introduce further flexibility in forming the groups. First, groups are allowed to overlap. Second, while the original SWAG penalty in @cite_24 uses constant weights within each group, the modified penalty allows the weights within a group to vary. These in turn allow to achieve a more localized and translation-invariant behavior, which is of interest for processing time-domain signals. However, these modifications come at an expense. While it is possible to realize the SWAG threshold function with a finite terminating procedure @cite_24 , such a procedure is not available for the proposed penalty. Therefore, forward-backward splitting type algorithms that might utilize @math @cite_18 @cite_15 @cite_9 @cite_21 are not readily applicable for the proposed penalty. We propose instead a descent algorithm for a generic formulation that employs the proposed penalty. This algorithm is specific to the proposed penalty, and makes use of the quadratic nature of the penalty. Therefore, it has not appeared elsewhere in the literature as far as we are aware.