From therneau at mayo.edu  Fri Jul 21 14:04:06 2017
From: therneau at mayo.edu (Therneau, Terry M., Ph.D.)
Date: Fri, 21 Jul 2017 07:04:06 -0500
Subject: [Rd] Wrongly converging glm()
Message-ID: <9153c6$7hh050@ironport10.mayo.edu>

I'm chiming in late since I read the news in digest form, and I won't copy the entire 
conversation to date.

The issue raised comes up quite often in Cox models, so often that the Therneau and 
Grambsch book has a section on the issue (3.5, p 58).  After a few initial iterations the 
offending coefficient will increase by a constant at each iteration while the 
log-likelihood approaches an asymptote (essentially once the other coefficients "settle 
down").

The coxph routine tries to detect this case and print a warning, and this turns out to be 
very hard to do accurately.  I worked hard at tuning the threshold(s) for the message 
several years ago and finally gave up; I am guessing that the warning misses > 5% of the 
cases when the issue is true, and that 5% of the warnings that do print are incorrect.  
(And these estimates may be too optimistic.)   Highly correlated predictors tend to trip 
it up, e.g., the truncated power spline basis used by the rcs function in Hmisc.

All in all, I am not completely sure whether the message does more harm than good.  I'd be 
quite reluctant to go down the same path again with the glm function.

Terry Therneau



[Rd] Wrongly converging glm()
Therneau, Terry M., Ph.D.