From therneau at mayo.edu  Thu Sep 14 18:07:27 2017
From: therneau at mayo.edu (Therneau, Terry M., Ph.D.)
Date: Thu, 14 Sep 2017 11:07:27 -0500
Subject: [Rd] vcov and survival
In-Reply-To: <ACD1644AA6C67E4FBD0C350625508EC8366C359A@FHSDB4H16-2.csu.mcmaster.ca>
References: <24682_1505341169_v8DMJSsK025685_9153c6$7uhrg4@ironport10.mayo.edu>
 <ACD1644AA6C67E4FBD0C350625508EC8366C32DC@FHSDB4H16-2.csu.mcmaster.ca>
 <22970.14862.518011.162206@stat.math.ethz.ch>
 <22970.15465.494390.45636@stat.math.ethz.ch>
 <ACD1644AA6C67E4FBD0C350625508EC8366C359A@FHSDB4H16-2.csu.mcmaster.ca>
Message-ID: <9153c6$7umii9@ironport10.mayo.edu>



On 09/14/2017 08:46 AM, Fox, John wrote:
> Dear Martin,
> 
> I made three points which likely got lost because of the way I presented them:
> 
> (1) Singularity is an unusual situation and should be made more prominent. It typically reflects a problem with the data or the specification of the model. That's not to say that it*never*  makes sense to allow singular fits (as in the situations you mentions).

In my medical work singularity is far from unusual.  It often results from imbalance in a 
covariate, e.g., there are 4 pathological stages but one of them turns out to be rare.

> 
> I'd favour setting singular.ok=FALSE as the default, but in the absence of that a warning or at least a note. A compromise would be to have a singular.ok option() that would be FALSE out of the box.

Originally the lm() default was singular.ok=FALSE.  It was a major pain in the ass and 
there was widespread unhappiness.  Enough so that Splus changed it.  (And they were not 
always very responsive to the users, so it took a lot of unrest.)   Another early default 
was na.fail, based on similar logic that "missings are unusual and should require an 
explicit response".  Don't repeat these mistakes.

Terry T.

> 
> Any changes would have to be made very carefully so as not to create chaos. That goes for the points below as well.
> 
> (2) coef() and vcov() behave inconsistently, which can be problematic because one often uses them together in code.
> 
> (3) As you noticed in your second message, lm() has a singular.ok argument and glm() doesn't.
> 
> I'll take a look at the code for glm() with an eye towards creating a patch, but I'm a bit reluctant to mess with the code for something as important as glm().
> 
> Best,
>   John



[Rd] vcov and survival
Therneau, Terry M., Ph.D.