From Hallgeir.Grinde at elkem.no  Wed Oct  5 15:53:53 2005
From: Hallgeir.Grinde at elkem.no (Hallgeir.Grinde@elkem.no)
Date: Wed,  5 Oct 2005 15:53:53 +0200 (CEST)
Subject: [Rd] Ad: Re: Ad: Re: R crashes for large formulas in lm() (PR#8180)
Message-ID: <20051005135353.11EBB1DE48@slim.kubism.ku.dk>

Dette er en melding med flere deler i MIME-format.
--=_alternative 004C4E4A00257091_=
Content-Type: text/plain; charset="US-ASCII"

Yes.
so (x1*x2*x3*x4*x5*x6*x7*x8)^2 = (x1+x2+x3+x4+x5+x6+x7+x8)^8 ?
and there is a difference in
(x1*x2*x3*x4*x5*x6*x7*x8)^2
and
(x1*x2*x3*x4*x5*x6*x7*x8)
althoug the resulting formulas are the same, or?

This fikses my problem, but R still crashes for the large formula. It may 
be due to stack owerflow, but i guess this can be altered maually?







Prof Brian Ripley <ripley at stats.ox.ac.uk>
05.10.2005 12:50
 
        Til:    Hallgeir.Grinde at elkem.no
        cc:     Uwe Ligges <ligges at statistik.uni-dortmund.de>, 
R-bugs at biostat.ku.dk
        Emne:   Re: Ad: Re: [Rd] R crashes for large formulas in lm() 
(PR#8180)


On Wed, 5 Oct 2005 Hallgeir.Grinde at elkem.no wrote:

> And some more informastion I forgot.
> R does not crash if I write out the formula:
>
> set.seed(123)
> x1 <- runif(1000)
> x2 <- runif(1000)
> x3 <- runif(1000)
> x4 <- runif(1000)
> x5 <- runif(1000)
> x6 <- runif(1000)
> x7 <- runif(1000)
> x8 <- runif(1000)
> y <- rnorm(1000)
> fit <- lm(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2)
> -> R crashes
>
> fit <- lm(y~x1+x2+x3+x4+x5+x6+x7+x8
>                +x1:x2+x1:x3+x1:x4+x1:x5+x1:x6+x1:x7+x1:x8
>                +x2:x3++x2:x4+x2:x5+x2:x6+x2:x7+x2:x8
>                +x3:x4+x3:x5+x3:x6+x3:x7+x3:x8
>                +x4:x5+x4:x6+x4:x7+x4:x8
>                +x5:x6+x5:x7+x5:x8
>                +x6:x7+x6:x8
>                +x7:x8)
> -> R does not crash
> This is the same formula, at least it should be.

It is not the same formula at all.  Try

> terms(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2, simplify=TRUE)
y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x1:x2 + x1:x3 + x1:x4 +
     x1:x5 + x1:x6 + x1:x7 + x1:x8 + x2:x3 + x2:x4 + x2:x5 + x2:x6 +
     x2:x7 + x2:x8 + x3:x4 + x3:x5 + x3:x6 + x3:x7 + x3:x8 + x4:x5 +
     x4:x6 + x4:x7 + x4:x8 + x5:x6 + x5:x7 + x5:x8 + x6:x7 + x6:x8 +
     x7:x8 + x1:x2:x3 + x1:x2:x4 + x1:x3:x4 + x1:x2:x5 + x1:x3:x5 +
...
     x1:x3:x4:x5:x6:x7:x8 + x2:x3:x4:x5:x6:x7:x8 + x1:x2:x3:x4:x5:x6:x7:x8

Did you actually want lm(y~(x1+x2+x3+x4+x5+x6+x7+x8)^2) ?

>
>
>
>
>
> Uwe Ligges <ligges at statistik.uni-dortmund.de>
> 05.10.2005 12:13
>
>        Til:    Prof Brian Ripley <ripley at stats.ox.ac.uk>
>        cc:     hallgeir.grinde at elkem.no, R-bugs at biostat.ku.dk
>        Emne:   Re: [Rd] R crashes for large formulas in lm() (PR#8180)
>
>
> Prof Brian Ripley wrote:
>
>> On Wed, 5 Oct 2005 hallgeir.grinde at elkem.no wrote:
>>
>>
>>> Full_Name: Hallgeir Grinde
>>> Version: 2.1.1
>>> OS: Windows XP
>>> Submission from: (NULL) (144.127.1.1)
>>>
>>>
>>> While using lm(y~(x*z*c*...*v)^2) R crashes/closes if the numbers of
> variables
>>> are at least 8.
>>
>>
>> OK, let's try to reproduce that:
>>
>>
>>> x1 <- runif(1000)
>>> x2 <- runif(1000)
>>> x3 <- runif(1000)
>>> x4 <- runif(1000)
>>> x5 <- runif(1000)
>>> x6 <- runif(1000)
>>> x7 <- runif(1000)
>>> x8 <- runif(1000)
>>> y <- rnorm(1000)
>>> fit <- lm(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2)
>>
>>
>> No crash, a quite reasonable fit.
>>
>> Can we please have a reproducible example, as we do ask?
>>
>
> Hmm, crashes for me as well with R-2.1.1 and R-2.2.0 beta (2005-09-27
> r35682M) on WinNT 4.0, SP6.
>
>
> Let's make it reproducible:
>
> set.seed(123)
> x1 <- runif(1000)
> x2 <- runif(1000)
> x3 <- runif(1000)
> x4 <- runif(1000)
> x5 <- runif(1000)
> x6 <- runif(1000)
> x7 <- runif(1000)
> x8 <- runif(1000)
> y <- rnorm(1000)
> fit <- lm(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2)
>
>
> Uwe Ligges
>
>
>
>                                NOTICE
>               Please immediately e-mail back to sender
>               if you are not the intended recipient.
>
>               Thereafter delete the e-mail along with
>               any attachments without making copies.
>
>               Elkem reserves all rights of privilege,
>               confidentiality and copyright.
>
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595


--=_alternative 004C4E4A00257091_=
Content-Type: text/html; charset="US-ASCII"


<br><font size=2 face="sans-serif">Yes.</font>
<br><font size=2 face="sans-serif">so </font><font size=2><tt>(x1*x2*x3*x4*x5*x6*x7*x8)^2</tt></font><font size=2 face="sans-serif">
= </font><font size=2><tt>(x1+x2+x3+x4+x5+x6+x7+x8)^8 ?</tt></font>
<br><font size=2><tt>and there is a difference in</tt></font>
<br><font size=2><tt>(x1*x2*x3*x4*x5*x6*x7*x8)^2</tt></font>
<br><font size=2><tt>and</tt></font>
<br><font size=2><tt>(x1*x2*x3*x4*x5*x6*x7*x8)</tt></font>
<br><font size=2><tt>althoug the resulting formulas are the same, or?</tt></font>
<br>
<br><font size=2><tt>This fikses my problem, but R still crashes for the
large formula. It may be due to stack owerflow, but i guess this can be
altered maually?</tt></font>
<br>
<br>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>Prof Brian Ripley &lt;ripley at stats.ox.ac.uk&gt;</b></font>
<p><font size=1 face="sans-serif">05.10.2005 12:50</font>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Til:
&nbsp; &nbsp; &nbsp; &nbsp;Hallgeir.Grinde at elkem.no</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc:
&nbsp; &nbsp; &nbsp; &nbsp;Uwe Ligges &lt;ligges at statistik.uni-dortmund.de&gt;,
R-bugs at biostat.ku.dk</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Emne:
&nbsp; &nbsp; &nbsp; &nbsp;Re: Ad: Re: [Rd] R crashes for large
formulas in lm() (PR#8180)</font></table>
<br>
<br>
<br><font size=2><tt>On Wed, 5 Oct 2005 Hallgeir.Grinde at elkem.no wrote:<br>
<br>
&gt; And some more informastion I forgot.<br>
&gt; R does not crash if I write out the formula:<br>
&gt;<br>
&gt; set.seed(123)<br>
&gt; x1 &lt;- runif(1000)<br>
&gt; x2 &lt;- runif(1000)<br>
&gt; x3 &lt;- runif(1000)<br>
&gt; x4 &lt;- runif(1000)<br>
&gt; x5 &lt;- runif(1000)<br>
&gt; x6 &lt;- runif(1000)<br>
&gt; x7 &lt;- runif(1000)<br>
&gt; x8 &lt;- runif(1000)<br>
&gt; y &lt;- rnorm(1000)<br>
&gt; fit &lt;- lm(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2)<br>
&gt; -&gt; R crashes<br>
&gt;<br>
&gt; fit &lt;- lm(y~x1+x2+x3+x4+x5+x6+x7+x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x1:x2+x1:x3+x1:x4+x1:x5+x1:x6+x1:x7+x1:x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x2:x3++x2:x4+x2:x5+x2:x6+x2:x7+x2:x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x3:x4+x3:x5+x3:x6+x3:x7+x3:x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x4:x5+x4:x6+x4:x7+x4:x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x5:x6+x5:x7+x5:x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x6:x7+x6:x8<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+x7:x8)<br>
&gt; -&gt; R does not crash<br>
&gt; This is the same formula, at least it should be.<br>
<br>
It is not the same formula at all. &nbsp;Try<br>
<br>
&gt; terms(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2, simplify=TRUE)<br>
y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x1:x2 + x1:x3 + x1:x4 +<br>
 &nbsp; &nbsp; x1:x5 + x1:x6 + x1:x7 + x1:x8 + x2:x3 + x2:x4 + x2:x5 +
x2:x6 +<br>
 &nbsp; &nbsp; x2:x7 + x2:x8 + x3:x4 + x3:x5 + x3:x6 + x3:x7 + x3:x8 +
x4:x5 +<br>
 &nbsp; &nbsp; x4:x6 + x4:x7 + x4:x8 + x5:x6 + x5:x7 + x5:x8 + x6:x7 +
x6:x8 +<br>
 &nbsp; &nbsp; x7:x8 + x1:x2:x3 + x1:x2:x4 + x1:x3:x4 + x1:x2:x5 + x1:x3:x5
+<br>
...<br>
 &nbsp; &nbsp; x1:x3:x4:x5:x6:x7:x8 + x2:x3:x4:x5:x6:x7:x8 + x1:x2:x3:x4:x5:x6:x7:x8<br>
<br>
Did you actually want lm(y~(x1+x2+x3+x4+x5+x6+x7+x8)^2) ?<br>
<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; Uwe Ligges &lt;ligges at statistik.uni-dortmund.de&gt;<br>
&gt; 05.10.2005 12:13<br>
&gt;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp;Til: &nbsp; &nbsp;Prof Brian Ripley &lt;ripley at stats.ox.ac.uk&gt;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp;cc: &nbsp; &nbsp; hallgeir.grinde at elkem.no,
R-bugs at biostat.ku.dk<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp;Emne: &nbsp; Re: [Rd] R crashes for large
formulas in lm() (PR#8180)<br>
&gt;<br>
&gt;<br>
&gt; Prof Brian Ripley wrote:<br>
&gt;<br>
&gt;&gt; On Wed, 5 Oct 2005 hallgeir.grinde at elkem.no wrote:<br>
&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt;&gt; Full_Name: Hallgeir Grinde<br>
&gt;&gt;&gt; Version: 2.1.1<br>
&gt;&gt;&gt; OS: Windows XP<br>
&gt;&gt;&gt; Submission from: (NULL) (144.127.1.1)<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; While using lm(y~(x*z*c*...*v)^2) R crashes/closes if the
numbers of<br>
&gt; variables<br>
&gt;&gt;&gt; are at least 8.<br>
&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt; OK, let's try to reproduce that:<br>
&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt;&gt; x1 &lt;- runif(1000)<br>
&gt;&gt;&gt; x2 &lt;- runif(1000)<br>
&gt;&gt;&gt; x3 &lt;- runif(1000)<br>
&gt;&gt;&gt; x4 &lt;- runif(1000)<br>
&gt;&gt;&gt; x5 &lt;- runif(1000)<br>
&gt;&gt;&gt; x6 &lt;- runif(1000)<br>
&gt;&gt;&gt; x7 &lt;- runif(1000)<br>
&gt;&gt;&gt; x8 &lt;- runif(1000)<br>
&gt;&gt;&gt; y &lt;- rnorm(1000)<br>
&gt;&gt;&gt; fit &lt;- lm(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2)<br>
&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt; No crash, a quite reasonable fit.<br>
&gt;&gt;<br>
&gt;&gt; Can we please have a reproducible example, as we do ask?<br>
&gt;&gt;<br>
&gt;<br>
&gt; Hmm, crashes for me as well with R-2.1.1 and R-2.2.0 beta (2005-09-27<br>
&gt; r35682M) on WinNT 4.0, SP6.<br>
&gt;<br>
&gt;<br>
&gt; Let's make it reproducible:<br>
&gt;<br>
&gt; set.seed(123)<br>
&gt; x1 &lt;- runif(1000)<br>
&gt; x2 &lt;- runif(1000)<br>
&gt; x3 &lt;- runif(1000)<br>
&gt; x4 &lt;- runif(1000)<br>
&gt; x5 &lt;- runif(1000)<br>
&gt; x6 &lt;- runif(1000)<br>
&gt; x7 &lt;- runif(1000)<br>
&gt; x8 &lt;- runif(1000)<br>
&gt; y &lt;- rnorm(1000)<br>
&gt; fit &lt;- lm(y~(x1*x2*x3*x4*x5*x6*x7*x8)^2)<br>
&gt;<br>
&gt;<br>
&gt; Uwe Ligges<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;NOTICE<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Please immediately
e-mail back to sender<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if you are not the
intended recipient.<br>
&gt;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Thereafter delete
the e-mail along with<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; any attachments without
making copies.<br>
&gt;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Elkem reserves all
rights of privilege,<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; confidentiality and
copyright.<br>
&gt;<br>
&gt;<br>
<br>
-- <br>
Brian D. Ripley, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp;ripley at stats.ox.ac.uk<br>
Professor of Applied Statistics, &nbsp;http://www.stats.ox.ac.uk/~ripley/<br>
University of Oxford, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Tel: &nbsp;+44
1865 272861 (self)<br>
1 South Parks Road, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; +44 1865 272866 (PA)<br>
Oxford OX1 3TG, UK &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Fax:
&nbsp;+44 1865 272595<br>
</tt></font>
<br>
--=_alternative 004C4E4A00257091_=--



[Rd] Ad: Re: Ad: Re: R crashes for large formulas in lm() (PR#8180)
Hallgeir.Grinde@elkem.no