From maechler at stat.math.ethz.ch  Mon Oct 10 11:54:54 2016
From: maechler at stat.math.ethz.ch (Martin Maechler)
Date: Mon, 10 Oct 2016 11:54:54 +0200
Subject: [Rd]
 =?utf-8?b?b3B0aW0o4oCmPz0sID0/dXRmLTg/UT9tZXRob2Q94oCYTC1C?=
 =?utf-8?q?FGS-B=E2=80=99=29_stops_with_an_error_message_while_violating_t?=
 =?utf-8?q?he_lower_bound?=
In-Reply-To: <f29714b9-98d6-3a55-d798-40decfa13461@prodsyse.com>
References: <a4515835-87f7-302d-5420-4cbf10b44fa1@prodsyse.com>
	<CAHz+bWZExjt5YGu0SaAx4bK7hvFo0QsT0NE=d-4Wuc+aFRwPMg@mail.gmail.com>
	<f29714b9-98d6-3a55-d798-40decfa13461@prodsyse.com>
Message-ID: <22523.25966.567603.28049@stat.math.ethz.ch>

>>>>> Spencer Graves <spencer.graves at prodsyse.com>
>>>>>     on Sat, 8 Oct 2016 18:03:43 -0500 writes:

[.............]

    >              2.  It would be interesting to know if the
    > current algorithm behind optim and optimx with
    > method='L-BFGS-B' incorporates Morales and Nocedal (2011)
    > 'Remark on ?Algorithm 778: L-BFGS-B: Fortran Subroutines
    > for Large-Scale Bound Constrained Optimization?'.  I
    > created this vignette and started this threat hoping that
    > someone on the R Core team might decide it's worth
    > checking things like that.

well I hope you mean "thread" rather "threat"  ;-)

I've now looked at the reference above, which is indeed quite
interesting.
	doi 10.1145/2049662.2049669
	--> http://dl.acm.org/citation.cfm?doid=2049662.2049669
A "free" (pre-publication I assume) version of the manuscript is
  http://www.eecs.northwestern.edu/~morales/PSfiles/acm-remark.pdf

The authors, Morales and Nocedal, the 2nd one being one of the
original L-BFGS-B(1997) paper, make two remarks, the 2nd one
about the "machine epsilon" used, and I can assure you that R's
optim() version never suffered from that; we've always been
using a C translation of the fortran code, and then used DBL_EPSILON.
R's (main) source file for that is in .../src/appl/lbfgsb.c, e.g., here
https://svn.r-project.org/R/trunk/src/appl/lbfgsb.c

OTOH, their remark 1 is very relevant and promising faster /
more reliable convergence. 
I'd be "happy" if optim() could gain a new option, say, "L-BFGS-B-2011"
which would incorporate what they call "modified L-BFGS-B".

However, I did not find published code to go together with their
remark.
Ideally, some of you interested in this, would provide a patch
against the above  lbfgsb.c  file

Martin Maechler,
ETH Zurich



[Rd] =?utf-8?b?b3B0aW0o4oCmPz0sID0/dXRmLTg/UT9tZXRob2Q94oCYTC1C?= =?utf-8?q?FGS-B=E2=80=99=29_stops_with_an_error_message_while_violating_t?= =?utf-8?q?he_lower_bound?=
Martin Maechler