  Parameter estimation in logistic regression is a well-studied problem with
the Newton-Raphson method being one of the most prominent optimization
techniques used in practice. A number of monotone optimization methods
including minorization-maximization (MM) algorithms, expectation-maximization
(EM) algorithms and related variational Bayes approaches offer a family of
useful alternatives guaranteed to increase the logistic regression likelihood
at every iteration. In this article, we propose a modified version of a
logistic regression EM algorithm which can substantially improve
computationally efficiency while preserving the monotonicity of EM and the
simplicity of the EM parameter updates. By introducing an additional latent
parameter and selecting this parameter to maximize the penalized observed-data
log-likelihood at every iteration, our iterative algorithm can be interpreted
as a parameter-expanded expectation-condition maximization either (ECME)
algorithm, and we demonstrate how to use the parameter-expanded ECME with an
arbitrary choice of weights and penalty function. In addition, we describe a
generalized version of our parameter-expanded ECME algorithm that can be
tailored to the challenges encountered in specific high-dimensional problems,
and we study several interesting connections between this generalized algorithm
and other well-known methods. Performance comparisons between our method, the
EM algorithm, and several other optimization methods are presented using a
series of simulation studies based upon both real and synthetic datasets.
