document: variable selection methods based on penalty theory have received great attention in high - dimensional data analysis . 
   a principled approach is due to the lasso of @xcite , which uses the @xmath0-norm penalty . 
   @xcite also pointed out that the lasso estimate can be viewed as the mode of the posterior distribution . 
   indeed , the @xmath1 penalty can be transformed into the laplace prior . moreover , this prior can be expressed as a gaussian scale mixture . 
   this has thus led to bayesian developments of the lasso and its variants  @xcite . 
   there has also been work on nonconvex penalization under a parametric bayesian framework . 
   @xcite derived their local linear approximation ( lla ) algorithm by combining the expectation maximization ( em ) algorithm with an inverse laplace transform . 
   in particular , they showed that the @xmath2 penalty with @xmath3 can be obtained by mixing the laplace distribution with a stable density . 
   other authors have shown that the prior induced from a penalty , called the nonconvex log penalty and defined in equation ( [ eqn : logp ] ) below , has an interpretation as a scale mixture of laplace distributions with an inverse gamma mixing distribution  @xcite . 
   recently , @xcite extended this class of laplace variance mixtures by using a generalized inverse gaussian mixing distribution . 
   related methods include the bayesian hyper - lasso  @xcite , the horseshoe model  @xcite and the dirichlet laplace prior  @xcite .    in parallel , 
   nonparametric bayesian approaches have been applied to variable selection  @xcite . 
   for example , in the infinite gamma poisson model  @xcite negative binomial processes are used to describe non - negative integer valued matrices , yielding a nonparametric bayesian feature selection approach under an unsupervised learning setting . 
   the beta - bernoulli process provides a nonparametric bayesian tool in sparsity modeling  @xcite . additionally , @xcite proposed a nonparametric approach for normal variance mixtures and showed that the approach is closely related to lvy processes . 
   later on , @xcite constructed sparse priors using increments of subordinators , which embeds finite dimensional normal variance mixtures in infinite ones . 
   thus , this provides a new framework for the construction of sparsity - inducing priors . 
   specifically , @xcite discussed the use of @xmath4-stable subordinators and inverted - beta subordinators for modeling joint priors of regression coefficients . @xcite 
   established the connection of two nonconvex penalty functions , which are referred to as log and exp and defined in equations ( [ eqn : logp ] ) and ( [ eqn : exp ] ) below , with the laplace transforms of the gamma and poisson subordinators . 
   a subordinator is a one - dimensional lvy process that is almost surely non - decreasing  @xcite .    in this paper 
   we further study the application of subordinators in bayesian nonconvex penalization problems under supervised learning scenarios . 
   differing from the previous treatments , we model latent shrinkage parameters using subordinators which are defined as stochastic processes of regularization parameters . 
   in particular , we consider two families of compound poisson subordinators : continuous compound poisson subordinators based on a gamma random variable  @xcite and discrete compound poisson subordinators based on a logarithmic random variable  @xcite . 
   the corresponding lvy measures are generalized gamma  @xcite and poisson measures , respectively . 
   we show that both the gamma and poisson subordinators are limiting cases of these two families of the compound poisson subordinators . 
   since the laplace exponent of a subordinator is a bernstein function , we have two families of nonconvex penalty functions , whose limiting cases are the nonconvex log and exp . additionally , these two families of nonconvex penalty functions can be defined via composition of log and exp , while the continuous and discrete compound poisson subordinators are mixtures of gamma and poisson processes . 
   recall that the latent shrinkage parameter is a stochastic process of the regularization parameter . 
   we formulate a hierarchical model with multiple regularization parameters , giving rise to a bayesian approach for nonconvex penalization . to reduce computational expenses 
   , we devise an ecme ( for  expectation / conditional maximization either " ) algorithm @xcite which can adaptively adjust the local regularization parameters in finding the sparse solution simultaneously . 
   the remainder of the paper is organized as follows . 
   section  [ sec : levy ] reviews the use of lvy processes in bayesian sparse learning problems . in section  [ sec : gps ] we study two families of compound poisson processes . in section  [ sec : blrm ] we apply the lvy processes to bayesian linear regression and devise an ecme algorithm for finding the sparse solution . 
   we conduct empirical evaluations using simulated data in section  [ sec : experiment ] , and conclude our work in section  [ sec : conclusion ] . 
   our work is based on the notion of bernstein and completely monotone functions as well as subordinators . 
   let @xmath5 with @xmath6 . 
   the function @xmath7 is said to be completely monotone if @xmath8 for all @xmath9 and bernstein if @xmath10 for all @xmath9 .    roughly speaking , a _ 
   subordinator _ is a one - dimensional lvy process that is non - decreasing almost surely . 
   our work is mainly motivated by the property of subordinators given in lemma  [ lem : subord ]  @xcite . 
   [ lem : subord ] if @xmath11 is a subordinator , then the laplace transform of its density takes the form @xmath12 where @xmath13 is the density of @xmath14 and @xmath15 , defined on @xmath16 , is referred to as the _ laplace exponent _ of the subordinator and has the following representation @xmath17 \nu ( d u).\ ] ] here @xmath18 and @xmath19 is the lvy measure such that @xmath20 . 
   conversely , if @xmath15 is an arbitrary mapping from @xmath21 given by expression ( [ eqn : psi ] ) , then @xmath22 is the laplace transform of the density of a subordinator . 
   it is well known that the laplace exponent @xmath15 is bernstein and the corresponding laplace transform @xmath23 is completely monotone for any @xmath24  @xcite . moreover , 
   any function @xmath25 , with @xmath26 , is a bernstein function if and only if it has the representation as in expression ( [ eqn : psi ] ) . 
   clearly , @xmath15 as defined in expression ( [ eqn : psi ] ) satisfies @xmath27 . as a result , @xmath15 is nonnegative , nondecreasing and concave on @xmath16 . 
   we are given a set of training data @xmath28 , where the @xmath29 are the input vectors and the @xmath30 are the corresponding outputs . 
   we now discuss the following linear regression model : @xmath31 where @xmath32 , @xmath33^t$ ] , and @xmath34 is a gaussian error vector @xmath35 . 
   we aim at finding a sparse estimate of the vector of regression coefficients @xmath36 by using a bayesian nonconvex approach .    in particular , we consider the following hierarchical model for the regression coefficients 
   @xmath37 s : @xmath38 & \stackrel{iid}{\sim } p(\eta_j ) , \\ 
   \sigma & \sim\iga(\alpha_{\sigma}/2 , \beta_{\sigma}/2),\end{aligned}\ ] ] where the @xmath39 s are referred to as latent shrinkage parameters , and the inverse gamma prior has the following parametrization : @xmath40 furthermore , we regard @xmath39 as @xmath41 , that is , @xmath42 . here 
   @xmath43 is defined as a subordinator . 
   let @xmath44 , defined on @xmath16 , be the laplace exponent of the subordinator . 
   taking @xmath45 , it can be shown that @xmath46 defines a nonconvex penalty function of @xmath47 on @xmath48 . 
   moreover , @xmath46 is nondifferentiable at the origin because @xmath49 and @xmath50 . 
   thus , it is able to induce sparsity . in this regard 
   , @xmath51 forms a prior for @xmath47 . from lemma  [ lem : subord ] 
   it follows that the prior can be defined via the laplace transform . in summary 
   , we have the following theorem . 
   [ thm : lapexp00 ] let @xmath15 be a nonzero bernstein function on @xmath16 . 
   if @xmath52 , then @xmath46 is a nondifferentiable and nonconvex function of @xmath47 on @xmath53 . 
   furthermore , @xmath54 where @xmath43 is some subordinator . 
   recall that @xmath14 is defined as the latent shrinkage parameter @xmath55 and in section  [ sec : blrm ] we will see that @xmath56 plays the same role as the regularization parameter ( or tuning hyperparameter ) . thus , there is an important connection between the latent shrinkage parameter and the corresponding regularization parameter ; that is , @xmath57 . because @xmath58 , each latent shrinkage parameter @xmath39 corresponds to a local regularization parameter @xmath59 . 
   therefore we have a nonparametric bayesian formulation for the latent shrinkage parameters @xmath39 s . 
   it is also worth pointing out that @xmath60 where @xmath61 denotes a laplace distribution with density given by @xmath62 , then @xmath63 defines the proper density of some random variable ( denoted @xmath64 ) . 
   subsequently , we obtain a proper prior @xmath65 for @xmath47 
   . moreover , this prior can be regarded as a laplace scale mixture , i.e. , the mixture of @xmath66 with mixing distribution @xmath67 . if @xmath68 
   , then @xmath69 is not a proper density . 
   thus , @xmath70 is also improper as a prior of @xmath47 . 
   however , we still treat @xmath70 as the mixture of @xmath66 with mixing distribution @xmath67 . in this case 
   , we employ the terminology of pseudo - priors for the density , which is also used by  @xcite .      obviously , @xmath71 is bernstein . 
   it is an extreme case , because we have that @xmath72 , @xmath73 and that @xmath74 , where @xmath75 denotes the dirac delta measure at @xmath56 , which corresponds to the deterministic process @xmath76 . 
   we can exclude this case by assuming @xmath77 in expression ( [ eqn : psi ] ) to obtain a strictly concave bernstein function . 
   in fact , we can impose the condition @xmath78 . 
   this in turn leads to @xmath77 due to @xmath79 . in this paper 
   we exploit laplace exponents in nonconvex penalization problems . 
   for this purpose , we will only consider a subordinator without drift , i.e. , @xmath77 . 
   equivalently , we always assume that @xmath80 . 
   we here take the nonconvex log and exp penalties as two concrete examples  ( also see * ? ? ? 
   the log penalty is defined by @xmath81 while the exp penalty is given by @xmath82 clearly , these two functions are bernstein on @xmath16 . moreover , they satisfy @xmath27 and @xmath83 . 
   it is also directly verified that @xmath84 \nu(du ) } , \ ] ] where the lvy measure @xmath19 is given by @xmath85 the corresponding subordinator @xmath86 is a gamma subordinator , because each @xmath14 follows a gamma distribution with parameters @xmath87 , with density given by @xmath88    we also note that the corresponding pseudo - prior is given by @xmath89 furthermore , if @xmath90 , the pseudo - prior is a proper distribution , which is the mixture of @xmath91 with mixing distribution @xmath92 .    as for the exp penalty , 
   the lvy measure is @xmath93 . since @xmath94 d b } 
   = \infty,\ ] ] then @xmath95 $ ] is an improper prior of @xmath47 . additionally , @xmath96 is a poisson subordinator . 
   specifically , @xmath14 is a poisson distribution with intensity @xmath97 taking values on the set @xmath98 . 
   that is , @xmath99 which we denote by @xmath100 . 
   in this section we explore the application of compound poisson subordinators in constructing nonconvex penalty functions . 
   let @xmath101 be a sequence of independent and identically distributed ( i.i.d . ) 
   real valued random variables with common law @xmath102 , and let @xmath103 be a poisson process with intensity @xmath104 that is independent of all the @xmath105 
   . then @xmath106 , for @xmath24 , follows a compound poisson distribution with density @xmath107 ( denoted @xmath108 ) , and hence @xmath43 is called a compound poisson process . 
   a compound poisson process is a subordinator if and only if the @xmath105 are nonnegative random variables  @xcite . 
   it is worth pointing out that if @xmath109 is the poisson subordinator given in expression ( [ eqn : possion ] ) , it is equivalent to saying that @xmath14 follows @xmath110 . 
   we particularly study two families of nonnegative random variables @xmath111 : nonnegative continuous random variables and nonnegative discrete random variables . 
   accordingly , we have continuous and discrete compound poisson subordinators @xmath109 . 
   we will show that both the gamma and poisson subordinators are limiting cases of the compound poisson subordinators .      in the first family @xmath111 
   is a gamma random variable . 
   in particular , let @xmath112 and the @xmath111 be i.i.d .  from the @xmath113 distribution , where @xmath114 , @xmath115 and @xmath116 . 
   the compound poisson subordinator can be written as follows @xmath117 the density of the subordinator is then given by @xmath118 we denote it by @xmath119 . 
   the mean and variance are @xmath120 respectively . 
   the laplace transform is given by @xmath121 where @xmath122 is a bernstein function of the form @xmath123.\ ] ] the corresponding lvy measure is given by @xmath124 notice that @xmath125 is a gamma measure for the random variable @xmath126 . 
   thus , the lvy measure @xmath127 is referred to as a generalized gamma measure  @xcite . 
   the bernstein function @xmath128 was studied by @xcite for survival analysis . however 
   , we consider its application in sparsity modeling . 
   it is clear that @xmath128 for @xmath114 and @xmath116 satisfies the conditions @xmath129 and @xmath130 . 
   also , @xmath131 is a nonnegative and nonconvex function of @xmath47 on @xmath48 , and it is an increasing function of @xmath132 on @xmath133 . 
   moreover , @xmath131 is continuous w.r.t . 
   @xmath47 but nondifferentiable at the origin . 
   this implies that @xmath131 can be treated as a sparsity - inducing penalty . 
   we are interested in the limiting cases that @xmath134 and @xmath135 . 
   [ pro : first ] let @xmath136 , @xmath128 and @xmath137 be defined by expressions ( [ eqn : first_tt ] ) , ( [ eqn : first ] ) and ( [ eqn : first_nu ] ) , respectively 
   . then    1 . 
   @xmath138 and @xmath139 ; 2 . 
   @xmath140 and @xmath141 ; 3 . 
   @xmath142 and @xmath143 . 
   this proposition can be obtained by using direct algebraic computations . 
   proposition  [ pro : first ] tells us that the limiting cases yield the nonconvex log and exp functions . 
   moreover , we see that @xmath14 converges in distribution to a gamma random variable with shape @xmath144 and scale @xmath145 , as @xmath146 , and to a poisson random variable with mean @xmath144 , as @xmath147 . 
   it is well known that @xmath148 degenerates to the log function  @xcite . 
   here we have shown that @xmath122 approaches to exp as @xmath147 . 
   we list another special example in table  [ tab : exam ] when @xmath149 . 
   we refer to the corresponding penalty as a _ linear - fractional _ ( lfr ) function . for notational simplicity , 
   we respectively replace @xmath150 and @xmath151 by @xmath145 and @xmath152 in the lfr function . 
   the density of the subordinator for the lfr function is given by @xmath153 we thus say each @xmath14 follows a squared bessel process without drift @xcite , which is a mixture of a dirac delta measure and a randomized gamma distribution  @xcite . 
   we denote the density of @xmath14 by @xmath154 . 
   lllll & bernstein functions & lvy measures @xmath137 & subordinators @xmath14 & priors + log & @xmath155 & @xmath156 & @xmath157 & proper@xmath158 + exp & @xmath159 $ ] & @xmath160 & @xmath161 & improper + lfr & @xmath162 & @xmath163 & @xmath164 & improper + cel & @xmath165 $ ] & @xmath166 & @xmath167 & improper +   +   +      in the second case , we consider a family of discrete compound poisson subordinators . 
   particularly , @xmath111 is discrete and takes values on @xmath168 . and it is defined as logarithmic distribution @xmath169 , where @xmath170 and @xmath171 , with probability mass function given by @xmath172 moreover , we let @xmath173 have a poisson distribution with intensity @xmath174 , where @xmath114 . then @xmath14 is distributed according to a negative binomial ( nb ) distribution  @xcite . 
   the probability mass function of @xmath14 is given by @xmath175 which is denoted as @xmath176 . 
   we thus say that @xmath14 follows an nb subordinator . 
   let @xmath177 and @xmath178 . 
   it can be verified that @xmath179 has the same mean and variance as the @xmath119 distribution . 
   the corresponding laplace transform then gives rise to a new family of bernstein functions , which is given by @xmath180.\ ] ] we refer to this family of bernstein functions as _ compound exp - log _ ( cel ) functions . 
   the first - order derivative of @xmath181 w.r.t . 
   @xmath182 is given by @xmath183    the lvy measure for @xmath181 is given by @xmath184 the proof is given in appendix  1 . 
   we call this lvy measure a 
   _ generalized poisson measure _ relative to the generalized gamma measure .    like @xmath128 
   , @xmath181 can define a family of sparsity - inducing nonconvex penalties . also , @xmath181 for @xmath114 , @xmath185 and @xmath116 satisfies the conditions @xmath186 , @xmath187 and @xmath188 . 
   we present a special cel function @xmath189 as well as the corresponding @xmath14 and @xmath137 in table  [ tab : exam ] , where we replace @xmath190 and @xmath150 by @xmath152 and @xmath145 for notational simplicity . 
   we now consider the limiting cases . 
   [ pro:8 ] assume @xmath137 is defined by expression ( [ eqn : second_nu ] ) for fixed @xmath185 and @xmath116 . 
   then we have that    1 . 
   @xmath191 and @xmath192 . 
   @xmath193 and @xmath194 . 
   3 .   @xmath195 and @xmath142 . 
   4 .   @xmath196 and @xmath197    notice that @xmath198 this shows that @xmath127 converges to @xmath199 , as @xmath200 . 
   analogously , we obtain the second part of proposition  [ pro:8]-(d ) , which implies that as @xmath200 , @xmath14 converges in distribution to a gamma random variable with shape parameter @xmath144 and scale parameter  @xmath145 . an alternative proof is given in appendix  2 . 
   proposition  [ pro:8 ] shows that @xmath181 degenerates to exp as @xmath147 , while to log as @xmath200 . 
   this shows an interesting connection between @xmath128 in expression ( [ eqn : first ] ) and @xmath181 in expression ( [ eqn : second ] ) ; that is , they have the same limiting behaviors . 
   we note that for @xmath201 , @xmath202\ ] ] which is a composition of the log and exp functions , and that @xmath203\ ] ] which is a composition of the exp and log functions . 
   in fact , the composition of any two bernstein functions is still bernstein . 
   thus , the composition is also the laplace exponent of some subordinator , which is then a mixture of the subordinators corresponding to the original two bernstein functions  @xcite . 
   this leads us to an alternative derivation for the subordinators corresponding to @xmath122 and @xmath204 . 
   that is , we have the following theorem whose proof is given in appendix  3 . 
   [ thm : poigam ] the subordinator @xmath14 associated with @xmath128 is distributed according to the mixture of @xmath205 distributions with @xmath206 mixing , while @xmath14 associated with @xmath181 is distributed according to the mixture of @xmath207 distributions with @xmath208 mixing .    additionally , the following theorem illustrates a limiting property of the subordinators as @xmath145 approaches 0 . 
   [ thm : limit ] let @xmath209 be a fixed constant on @xmath210 $ ] .    1 . 
   if @xmath211 where @xmath212 $ ] or @xmath213 , then @xmath14 converges in probability to @xmath56 , as @xmath214 . 
   2 .   if @xmath215 where @xmath216\ ] ] or @xmath213 , then @xmath14 converges in probability to @xmath56 , as @xmath214 . 
   the proof is given in appendix  4 . since 
   @xmath14 converges in probability to @xmath56 " implies  @xmath14 converges in distribution to @xmath56 , " we have that @xmath217    finally , consider the four nonconvex penalty function given in table  [ tab : exam ] . 
   we present the following property . 
   that is , when @xmath213 and for any fixed @xmath116 , we have @xmath218 \leq\frac{s}{\gamma s { + } 1 } \leq\frac{1}{\gamma } [ 1 { - } \exp ( { - } \gamma s ) ] \leq\frac { 1}{\gamma } \log\big({\gamma } s { + } 1 \big ) \leq s,\ ] ] with equality only when @xmath219 . 
   the proof is given in appendix  5 . 
   this property is also illustrated in figure  [ fig : penalty ] . 
   in table  [ tab : exam ] with @xmath220 and @xmath71 . ] 
   we apply the compound poisson subordinators to the bayesian sparse learning problem given in section  [ sec : levy ] . defining @xmath221 , we rewrite the hierarchical representation for the joint prior of the @xmath37 under the regression framework . that is , we assume that @xmath222 & \stackrel{ind}{\sim } & l(b_j|0 , \sigma ( 2\eta_j)^{-1 } ) , \\ 
   f_{t^{*}(t_j)}(\eta_j ) & { \propto } & \eta_j^{-1 } f_{t(t_j)}(\eta_j),\end{aligned}\ ] ] which implies that @xmath223 the joint marginal pseudo - prior of the @xmath37 s is given by @xmath224 we will see in theorem  [ thm : poster ] that the full conditional distribution @xmath225 is proper . 
   thus , the maximum _ a posteriori _ ( map ) estimate of @xmath226 is based on the following optimization problem : @xmath227 clearly , the @xmath59 s are local regularization parameters and the @xmath228 s are latent shrinkage parameters . moreover , 
   it is interesting that @xmath43 ( or @xmath55 ) is defined as a subordinator w.r.t . 
   @xmath56 . 
   the full conditional distribution @xmath229 is conjugate w.r.t . 
   the prior , which is @xmath230 . 
   specifically , it is an inverse gamma distribution of the form @xmath231.\ ] ] in the following experiment , we use an improper prior of the form @xmath232 ( i.e. , @xmath233 ) . 
   clearly , @xmath229 is still an inverse gamma distribution in this setting . additionally , based on @xmath234 \prod_{j=1}^p \exp(-\frac { \eta _ j}{\sigma } |b_j|)\vadjust{\eject}\ ] ] and 
   the proof of theorem  [ thm : poster ] ( see appendix  6 ) , we have that the conditional distribution @xmath235 is proper . 
   however , the absolute terms @xmath236 make the form of @xmath237 unfamiliar . 
   thus , a gibbs sampling algorithm is not readily available and we resort to an em algorithm to estimate the model . 
   notice that if @xmath238 is proper , the corresponding normalizing constant is given by @xmath239 d |b_j|= 2 \int_{0}^{\infty } \exp\big [ -t_j \psi\big(\frac{|b_j| } { \sigma } \big ) \big ] d ( |b_j|/\sigma),\ ] ] which is independent of @xmath240 . also , the conditional distribution @xmath241 is independent of the normalizing term . 
   specifically , we always have that @xmath242 which is proper . 
   as shown in table  [ tab : exam ] , except for log with @xmath243 which can be transformed into a proper prior , the remaining bernstein functions can not be transformed into proper priors . in any case , 
   our posterior computation is directly based on the marginal pseudo - prior @xmath244 . 
   we ignore the involved normalizing term , because it is infinite if @xmath244 is improper and it is independent of @xmath240 if @xmath244 is proper . 
   given the @xmath245th estimates @xmath246 of @xmath247 in the e - step of the em algorithm , we compute @xmath248 p(\eta_j|b_j^{(k ) } , \sigma ^{(k ) } , t_j ) } d \eta_j + \log p(\sigma ) \\ & \propto-\frac{n+\alpha_{\sigma}}{2 } \log\sigma{- } \frac{\|{\bf y } { -}{\bf x}{\bf b}\|_2 ^ 2 + \beta_{\sigma}}{2 \sigma } - ( p+1 ) \log \sigma \\ & \quad- \frac{1 } { \sigma } \sum_{j=1}^p    here we omit some terms that are independent of parameters @xmath240 and @xmath226 . 
   in fact , we only need to compute @xmath249 in the e - step . considering that @xmath250 and taking the derivative w.r.t . 
   @xmath236 on both sides of the above equation , we have that @xmath251    the m - step maximizes @xmath252 w.r.t.@xmath253 . in particular , it is obtained that : @xmath254    the above em algorithm is related to the linear local approximation ( lla ) procedure  @xcite . 
   moreover , it shares the same convergence property given in @xcite and @xcite . 
   subordinators help us to establish a direct connection between the local regularization parameters @xmath59 s and the latent shrinkage parameters @xmath39 s ( or @xmath41 ) . 
   however , when we implement the map estimation , it is challenging how to select these local regularization parameters . 
   we employ an ecme ( for  expectation / conditional maximization either " ) algorithm  @xcite for learning about the @xmath37 s and @xmath59 s simultaneously . for this purpose , 
   we suggest assigning @xmath59 gamma prior @xmath255 , namely , @xmath256 because the full conditional distribution is also gamma and given by @xmath257 \sim\ga\big(\alpha_{t } , 1/[\psi(|b_j|/\sigma ) + \beta_{t}]\big).\ ] ] recall that we here compute the full conditional distribution directly using the marginal pseudo - prior @xmath238 , because our used bernstein functions in table  [ tab : exam ] can not induce proper priors . however , 
   if @xmath238 is proper , the corresponding normalizing term would rely on @xmath59 . as a result , 
   the full conditional distribution of @xmath59 is possibly no longer gamma or even not analytically available .    figure  [ fig : graphal0]-(a ) depicts the hierarchical model for the bayesian penalized linear regression , and table  [ tab : alg ] gives the ecme procedure where the e - step and cm - step are respectively identical to the e - step and the m - step of the em algorithm , with @xmath258 . 
   the cme - step updates the @xmath59 s with @xmath259 in order to make sure that @xmath260 , it is necessary to assume that @xmath261 . in the following experiments , we set @xmath262 . 
   we conduct experiments with the prior @xmath263 for comparison . 
   this prior is induced from the @xmath264-norm penalty , so it is a proper specification . 
   moreover , the full conditional distribution of @xmath59 w.r.t . 
   its gamma prior @xmath265 is still gamma ; that is , @xmath257 \sim\ga\big({\alpha_t}{+}2 , \ ; 1/({\beta_t } { + } \sqrt{|b_j|/\sigma})\big).\ ] ] thus , the cme - step for updating the @xmath59 s is given by @xmath266    the convergence analysis of the ecme algorithm was presented by @xcite , who proved that the ecme algorithm retains the monotonicity property from the standard em . 
   moreover , the ecme algorithm based on pseudo - priors was also used by @xcite .    . 
   the basic procedure of the ecme algorithm [ cols= " < , < " , ]     our analysis is based on a set of simulated data , which are generated according to @xcite . 
   in particular , we consider the following three data models   small , "  medium " and  large . 
   "    data s : : :    @xmath267 , @xmath268 ,    @xmath269 , and @xmath270 is a    @xmath271 matrix with @xmath272 on the diagonal    and @xmath273 on the off - diagonal . 
   data m : : :    @xmath274 , @xmath275 ,    @xmath276 has @xmath277 non - zeros such that    @xmath278 and @xmath279 , and    @xmath280 . 
   data l : : :    @xmath281 , @xmath282 ,    @xmath283 , and    @xmath284 ( five    blocks ) .    for each data model , 
   we generate @xmath285 data matrices @xmath286 such that each row of @xmath286 is generated from a multivariate gaussian distribution with mean @xmath287 and covariance matrix @xmath270 , @xmath288 , or @xmath289 . 
   we assume a linear model @xmath290 with multivariate gaussian predictors @xmath286 and gaussian errors . 
   we choose @xmath240 such that the signal - to - noise ratio ( snr ) is a specified value . following the setting in @xcite , we use @xmath291 in all the experiments . 
   we employ a standardized prediction error ( spe ) to evaluate the model prediction ability . 
   the minimal achievable value for spe is @xmath272 . 
   variable selection accuracy is measured by the correctly predicted zeros and incorrectly predicted zeros in @xmath292 . 
   the snr and spe are defined as @xmath293    for each data model , we generate training data of size @xmath294 , very large validation data and test data , each of size @xmath295 . for each algorithm , 
   the optimal global tuning parameters are chosen by cross validation based on minimizing the average prediction errors . with the model @xmath292 computed on the training data , we compute spe on the test data . 
   this procedure is repeated @xmath296 times , and we report the average and standard deviation of spe and the average of zero - nonzero error . 
   we use `` '' to denote the proportion of correctly predicted zero entries in @xmath226 , that is , @xmath297 ; if all the nonzero entries are correctly predicted , this score should be @xmath298 . 
   we report the results in table  [ tab : toy2 ] . 
   it is seen that our setting in figure  [ fig : graphal0]-(a ) is better than the other two settings in figures  [ fig : graphal0]-(b ) and ( c ) in both model prediction accuracy and variable selection ability . 
   especially , when the size of the dataset takes large values , the prediction performance of the second setting becomes worse . 
   the several nonconvex penalties are competitive , but they outperform the lasso . 
   moreover , we see that log , exp , lfr and cel slightly outperform @xmath264 . 
   the @xmath264 penalty indeed suffers from the problem of numerical instability during the em computations . as we know 
   , the priors induced from lfr , cel and exp as well as log with @xmath299 are improper , but the prior induced from @xmath264 is proper . 
   the experimental results show that these improper priors work well , even better than the proper case .     vs. @xmath300 on  data s " and  data m " where @xmath301 is the permutation of @xmath302 such that @xmath303 . ] 
   recall that in our approach each regression variable @xmath37 corresponds to a distinct local tuning parameter @xmath59 . 
   thus , it is interesting to empirically investigate the inherent relationship between @xmath37 and @xmath59 . 
   let @xmath304 be the estimate of @xmath59 obtained from our ecme algorithm (  alg 1 " ) , and @xmath305 be the permutation of @xmath306 such that @xmath307 . figure  [ fig : tb1 ] depicts the change of @xmath308 vs.@xmath300 with log , exp , lfr and cel on  data s " and  data m. " we see that @xmath308 is decreasing w.r.t . 
   moreover , @xmath308 becomes 0 when @xmath300 takes some large value . 
   a similar phenomenon is also observed for  data l. " this thus shows that the subordinator is a powerful bayesian approach for variable selection . 
   in this paper we have introduced subordinators into the definition of nonconvex penalty functions . 
   this leads us to a bayesian approach for constructing sparsity - inducing pseudo - priors . 
   in particular , we have illustrated the use of two compound poisson subordinators : the compound poisson gamma subordinator and the negative binomial subordinator . 
   in addition , we have established the relationship between the two families of compound poisson subordinators . 
   that is , we have proved that the two families of compound poisson subordinators share the same limiting behaviors . 
   moreover , their densities at each time have the same mean and variance . 
   we have developed the ecme algorithms for solving sparse learning problems based on the nonconvex log , exp , lfr and cel penalties . 
   we have conducted the experimental comparison with the state - of - the - art approach . 
   the results have shown that our nonconvex penalization approach is potentially useful in high - dimensional bayesian modeling . 
   our approach can be cast into a point estimation framework . 
   it is also interesting to fit a fully bayesian framework based on the mcmc estimation . 
   we would like to address this issue in future work . 
   consider that @xmath310 & = \log\big[1-\frac{1}{1{+}\rho } \exp(-\frac{\rho}{1{+}\rho } \gamma s)\big ] - \log\big[1-\frac{1}{1{+}\rho}\big ] \\ & = \sum_{k=1}^{\infty } \frac{1}{k ( 1{+}\rho)^k } \big[1- \exp\big ( { -}\frac{\rho}{1{+}\rho } k \gamma s\big)\big ] \\ & = \sum_{k=1}^{\infty } \frac{1}{k ( 1{+}\rho)^k } \int_{0}^{\infty } ( 1- \exp(- u s ) ) \delta_{\frac{\rho k \gamma}{1{+}\rho}}(u ) d u.\end{aligned}\ ] ] we thus have that @xmath311 . 
   we here give an alternative proof of proposition  [ pro:8]-(d ) , which is immediately obtained from the following lemma . 
   let @xmath312 take discrete value on @xmath313 and follow negative binomial distribution @xmath314 . if @xmath315 converges to a positive constant as @xmath316 
   , @xmath317 converges in distribution to a gamma random variable with shape @xmath315 and scale @xmath272 . 
   since @xmath318 we have that @xmath319 notice that @xmath320 and @xmath321 this leads us to @xmath322    similarly , we have that @xmath323 
   consider a mixture of @xmath324 with @xmath325 mixing . 
   that is , @xmath326 letting @xmath327 , @xmath328 and @xmath329 , we have that @xmath330    we now consider a mixture of @xmath331 with @xmath332 which is @xmath333 . 
   let @xmath334 , @xmath335 , @xmath336 and @xmath337 . 
   thus , @xmath338 
   since @xmath339=1 $ ] , we only need to consider the case that @xmath213 . recall that @xmath119 , whose mean and variance are @xmath340 whenever @xmath213 . by chebyshev s inequality 
   , we have that @xmath341 hence , we have that @xmath342 similarly , we have part ( b ) . 
   we first note that @xmath343 which implies that @xmath344 for @xmath345 . 
   subsequently , we have that @xmath346 \leq0 $ ] . as a result , 
   @xmath347 for @xmath345 . as for @xmath348 
   , it is directly obtained from that @xmath349 since @xmath350 = \frac{\gamma}{\exp(\gamma s ) } - \frac{\gamma}{1+\gamma s}<0 $ ] for @xmath345 , we have that @xmath351 for @xmath345 . 
   first consider that @xmath352 \prod_{j=1}^p \sigma^{-1 } \exp\big(-t_j \psi\big(\frac{|b_j| } { \sigma } \big ) \big).\ ] ] to prove that @xmath353 is proper , it suffices to obtain that @xmath354 \prod_{j=1}^p \sigma^{-1 } \exp \big(-t_j \psi\big(\frac{|b_j| } { \sigma } \big ) \big ) d { \bf b } < \infty}.\ ] ] it is directly computed that @xmath355 \nonumber \\ & = \exp\big [ { - } \frac{1}{2 \sigma } ( { \bf b}{- } { \bf z})^t { \bf x}^t { \bf x } ( { \bf b}- { \bf z } ) \big ] \times\exp\big[- \frac{1}{2 \sigma } { \bf y}^t ( { \bf i}_n - { \bf x } ( { \bf x}^t { \bf x})^{+ } { \bf x}^t ) { \bf y}\big],\end{aligned}\ ] ] where @xmath356 and @xmath357 is the moore - penrose pseudo inverse of matrix @xmath358  @xcite . here 
   we use the well - established properties that @xmath359 and @xmath360 . notice that if @xmath358 is nonsingular , then @xmath361 . in this case 
   , we consider a conventional multivariate normal distribution @xmath362 . 
   otherwise , we consider a singular multivariate normal distribution @xmath363  @xcite , the density of which is given by @xmath364.\ ] ] here @xmath365 , and @xmath366 , @xmath367 , are the positive eigenvalues of @xmath358 . in any case 
   , we always write @xmath368 . 
   thus , @xmath369 d{\bf b } < \infty}$ ] . 
   it then follows the propriety of @xmath370 because @xmath371 \prod _ { j=1}^p \exp\big ( { - } t_j \psi\big(\frac{|b_j| } { \sigma } 
   \big ) \big)\leq\exp\big [ { - } \frac{1}{2 \sigma } \|{\bf y}- { \bf x}{\bf b}\|_2 ^ 2 \big].\ ] ]    we now consider that @xmath372 \prod_{j=1}^p \exp\big(-t_j \psi \big(\frac{|b_j| } { \sigma } \big ) \big).\ ] ] let @xmath373 { \bf y}$ ] . 
   since the matrix @xmath374 is positive semidefinite , we obtain @xmath375 . 
   based on expression ( [ eqn : pf01 ] ) , we can write @xmath376 \varpropto n({\bf b}|{\bf z } , \sigma({\bf x}^t { \bf x})^{+ } ) { \iga}(\sigma|\frac{\alpha_{\sigma } { + } n{+}2p{-}q}{2 } , \nu{+ } \beta_{\sigma}).\ ] ] subsequently , we have that @xmath377 d { \bf b } d \sigma } < \infty,\ ] ] and hence , @xmath377 \prod_{j=1}^p 
   \exp\big(-t_j \psi\big(\frac{|b_j| } { \sigma } \big ) \big ) d { \bf b}d \sigma } < \infty.\ ] ] therefore @xmath378 is proper . 
   thirdly , we take @xmath379 } { \sigma ^{\frac{n+\alpha_{\sigma}+2p}{2 } + 1 } } \prod_{j=1}^p \big\{\exp \big({-}t_j \psi\big(\frac{|b_j| } { \sigma } \big ) \big ) \frac { t_j^{{\alpha_t}{- } 1 } \exp({- } { \beta_t } t_j)}{\gamma({\alpha_t } ) } \big\ } \\ & \triangleq f({\bf b } , \sigma , { \bf t}).\end{aligned}\ ] ] in this case , we compute @xmath380 } { \sigma^{\frac{n+\alpha_{\sigma}+2p}{2 } + 1 } } \prod _ { j=1}^p \frac{1 } { \big({\beta_t } { + } \psi\big(\frac{|b_j| } { \sigma } \big ) \big)^{{\alpha_t } } } d { \bf b}d \sigma}.\ ] ] similar to the previous proof , we also have that @xmath381 because @xmath382 . as a result , @xmath383 is proper . 
   finally , consider the setting that @xmath384 . 
   that is , @xmath385 and @xmath386 . in this case , if @xmath387 , we obtain @xmath388 and @xmath389 . as a result 
   , we use the inverse gamma distribution @xmath390 . 
   thus , the results still hold . 
   polson , n.  g. and scott , j.  g. ( 2010 ) . 
   `` shrink globally , act locally : sparse bayesian regularization and prediction . '' in bernardo , j.  m. , bayarri , m.  j. , berger , j.  o. , dawid , a.  p. , heckerman , d. , smith , a.  f.  m. , and west , m. ( eds . ) , _ bayesian statistics 9_. oxford university press . 
   the authors would like to thank the editors and two anonymous referees for their constructive comments and suggestions on the original version of this paper . 
   the authors would especially like to thank the associate editor for giving extremely detailed comments on earlier drafts . 
   this work has been supported in part by the natural science foundation of china ( no . 61070239 ) .
summary: in this paper we discuss bayesian nonconvex penalization for sparse learning problems . 
   we explore a nonparametric formulation for latent shrinkage parameters using subordinators which are one - dimensional lvy processes . 
   we particularly study a family of continuous compound poisson subordinators and a family of discrete compound poisson subordinators . 
   we exemplify four specific subordinators : gamma , poisson , negative binomial and squared bessel subordinators . 
   the laplace exponents of the subordinators are bernstein functions , so they can be used as sparsity - inducing nonconvex penalty functions . 
   we exploit these subordinators in regression problems , yielding a hierarchical model with multiple regularization parameters . 
   we devise ecme ( expectation / conditional maximization either ) algorithms to simultaneously estimate regression coefficients and regularization parameters . 
   the empirical evaluation of simulated data shows that our approach is feasible and effective in high - dimensional data analysis .