From maechler at stat.math.ethz.ch  Fri Aug 29 17:45:09 2014
From: maechler at stat.math.ethz.ch (Martin Maechler)
Date: Fri, 29 Aug 2014 17:45:09 +0200
Subject: [Rd] Unfixed bugs in latest R-patched
In-Reply-To: <21417.30777.209017.331344@stat.math.ethz.ch>
References: <20140623152435.GA22211@cs.toronto.edu>
	<21417.30777.209017.331344@stat.math.ethz.ch>
Message-ID: <21504.40965.94098.976528@stat.math.ethz.ch>

Getting back to 2 months old considerations  [more inline]:

>>>>> Martin Maechler <maechler at stat.math.ethz.ch>
>>>>>     on Tue, 24 Jun 2014 15:08:09 +0200 writes:

>>>>> Radford Neal <radford at cs.toronto.edu>
>>>>>     on Mon, 23 Jun 2014 11:24:35 -0400 writes:

    >> A new version of pqR is now available at pqR-project.org, which fixes
    >> several bugs that are also present in the latest R Core patch release
    >> (r66002).  A number of bugs found previously during pqR development
    >> are also unfixed in the latest R Core release.  Here is the list of
    >> these bugs that are unfixed in r66002 (including documentation
    >> deficiencies), taken from the pqR bug fix and documentation update
    >> sections of the pqR NEWS file (pqR-project.org/NEWS.txt):

    > Duncan Murdoch has already mentioned that "unfixed bugs" is a
    > strong wording...

    > I'm commenting on one subtopic only:

    > [.........]

    >> o Fixed the following bug (also present in R-2.15.0 and R-3.0.2):
      
    >> x <- t(5)
    >> print (x %*% c(3,4))
    >> print (crossprod(5,c(3,4)))

    >> The call of crossprod produced an error, whereas the
    >> corresponding use of %*% does not.

    >> In pqR-2013-12-29, this bug also affected the expression t(5) %*%
    >> c(3,4), since it is converted to the equivalent of
    >> crossprod(5,c(3,4)).
    > (I'm not getting this last para... but that may be unimportant)

    > [........]

    >> o Fixed the following bug (also present in R-2.15.0 and R-3.0.2):
      
    >> v <- c(1,2)
    >> m <- matrix(c(3,4),1,2)
    >> print(t(m)%*%v)
    >> print(crossprod(m,v))

    >> in which crossprod gave an error rather than produce the answer
    >> for the corresponding use of %*%.

    > Both cases above concern  %*%, crossprod() {and also
    > tcrossprod()}, and always when one at least one operand is a
    > vector... 

    > As S and R's terminology has always been mathematically correct and
    > "knows" that a vector is not a 1-row matrix or 1-column matrix 
    > (which some people call row-vector and column-vector), R has
    > indeed much more freedom to be lenient in how to promote vectors
    > to matrices, and it has always done so tolerantly for some cases
    > of "%*%", but even there is room for discussion, see below.

    > Since about January, I have a corresponding entry in the TODO
    > file of the Matrix package (which I maintain), and I (at least
    > partly) agree with you, that we should *consider* changing the
    > current behavior.  
    > The following is from my corresponding R script :

 > ### NOTA BENE:  vector %*% Matrix  _and_  Matrix %*% vector :
 > ### ---------   -----------------         -----------------
 > ## The k-vector is treated as  (1,k)-matrix *or* (k,1)-matrix
 > ## on both sides  whenever it "helps fit" the matrix dimensions:

    > and one kind of reasoning would go to say that 
    > crossprod() and tcrossprod()  should do the same.   This would
    > even make more cases work than (I think) your proposal.

Yes, more cases, indeed:
I've just (svn rev 66495) committed changes to R-devel (aka "the trunk"),
with the following regression tests:

  ## Radford (R-devel, June 24, 2014); M.Maechler
  m <- matrix(1:2, 1,2); v <- 1:3
  stopifnot(identical(crossprod(2, v), t(2) %*% v),
	    identical(crossprod(m, v), t(m) %*% v),
	    identical(5 %*% v, 5 %*% t(v)),
	    identical(tcrossprod(m, 1:2), m %*% 1:2) )
  ## gave error "non-conformable arguments" in R <= 3.2.0

Note that the first two correspond to your "bugs" and the other two
follow the principle of promoting a vector to that (column or
row) matrix which "makes it work".

I did not label these as bug fixes, but new features, and for
this reason -- and for a longer review period and RFC --
these are *not* planned to be part of the next R release  
(R 3.1.2, no release date planned yet), but only of the next
yearly ".0" release next spring, (probably) version 3.2.0.


Comments are welcome, (particularly praises ;-)

Martin

    > However, one can argue that the current behavior is quite
    > consistent {there is more than one kind of consistency here; in
    > this case it's the consistency of underlying C code IIRC}, and
    > not strictly a bug.
    > It is something which I am grateful you bring up for discussion,
    > and I'm happy to settle here.
    > Note that the Matrix package and many other packages that
    > implement matrix multiplication methods, should then also follow
    > the new semantic, and so this is typically rather for R-devel
    > than R-patched.

    > Note that from your first case example, one could also argue
    > -- at least from my 'NOTA BENE' rule above --
    > that    5 %*% 3:4
    > should work, since  
    > 5 %*% rbind(3:4)
    > does work, but
    > 5 %*% cbind(3:4)
    > does not.  So you see, there are quite a few cases one could
    > discuss, and have reasons pro and contra some of the changes.

    > I will be too busy till the end of 'useR! 2014' and would want
    > quite a bit of further thoughts before making a quick decision,
    > notably about the cases I have in the appended  products.Rout

    > Martin Maechler,
    > (ETH Zurich and R Core team)

    > ---------------
    > ### NOTA BENE:  vector %*% Matrix  _and_  Matrix %*% vector :
    > ### ---------   -----------------         -----------------
    > ## The k-vector is treated as  (1,k)-matrix *or* (k,1)-matrix
    > ## on both sides  whenever it "helps fit" the matrix dimensions:

    > ## --------------- %*% ---------------------------------------------

    > ## --------- v . v -----------------------------
    > ## We *could* consider allowing more even here: could argue
    > ## from the 'NOTA BENE' above that
    >> 5 %*% 3:4        ## should work, since
    >> 5 %*% rbind(3:4) ## does work, but
    >> 5 %*% cbind(3:4) ## does not.

    > ## --------- v . M -----------------------------
    >> 1:3 %*% matrix(1:12, 3,4) ## (1, n)  if n == nrow(.)
    > [,1] [,2] [,3] [,4]
    > [1,]   14   32   50   68
    >> 1:4 %*% matrix(1:12, 3,4)
    > Error in 1:4 %*% matrix(1:12, 3, 4) : non-conformable arguments
    > No suitable frames for recover()
    >> 1:4 %*% matrix(1:5, 1,5)  ##  (n, 1) if nrow(.) == 1
    > [,1] [,2] [,3] [,4] [,5]
    > [1,]    1    2    3    4    5
    > [2,]    2    4    6    8   10
    > [3,]    3    6    9   12   15
    > [4,]    4    8   12   16   20
    > ## --------- M . v -----------------------------
    >> matrix(1:4, 3,4) %*% 1:4  ##  (n, 1) if  n == ncol(.)
    > [,1]
    > [1,]   26
    > [2,]   28
    > [3,]   26
    >> matrix(1:4, 3,4) %*% 1:3
    > Error in matrix(1:4, 3, 4) %*% 1:3 : non-conformable arguments
    > No suitable frames for recover()
    >> matrix(1:3, 3,1) %*% 1:3  ##  (1, n) if  ncol(.) == 1
    > [,1] [,2] [,3]
    > [1,]    1    2    3
    > [2,]    2    4    6
    > [3,]    3    6    9
    >> 

    > ## --------------- crossprod -------------------------------------

    > ## --------- M . v -----------------------------
    >> crossprod(matrix(1:4, 3,4), 1:3)  ## (n, 1) if  n == nrow(.)
    > [,1]
    > [1,]   14
    > [2,]   12
    > [3,]   14
    > [4,]   20
    >> crossprod(matrix(1:4, 3,4), 1:4)
    > Error in crossprod(matrix(1:4, 3, 4), 1:4) : non-conformable arguments
    >> m <- matrix(1:2, 1,2); v <- 1:3
    >> crossprod(m,v)	 ##>>> *could* be nice  and use (1,n) as nrow(.)==1 <<<
    > Error in crossprod(m, v) : non-conformable arguments
    > ##>> after all, crossprod(m,v) "should"== t(m) %*% v  and that does work:
    >> t(m) %*% v
    > [,1] [,2] [,3]
    > [1,]    1    2    3
    > [2,]    2    4    6
    >> 

    > ## --------- v . M -----------------------------
    >> crossprod(1:3, matrix(1:4, 3,4))  ## (1, n)  if  n == nrow(.)
    > [,1] [,2] [,3] [,4]
    > [1,]   14   12   14   20
    >> crossprod(1:4, matrix(1:4, 3,4))
    > Error in crossprod(1:4, matrix(1:4, 3, 4)) : non-conformable arguments
    >> crossprod(1:3, matrix(1:3, 1,3)) ##>>> *could* be nice  and use (n,1) as nrow(.)==1 <<<
    > Error in crossprod(1:3, matrix(1:3, 1, 3)) : non-conformable arguments
    >> crossprod(v,m)
    > Error in crossprod(v, m) : non-conformable arguments
    > ##>> "as"   v %*% m   does work  and v could have been  A[1,] instead of A[1, , drop=TRUE]


    >> ## --------------- tcrossprod -------------------------------------

    > ## --------- M . v -----------------------------
    >> tcrossprod(matrix(1:4, 3,4), 1:3)
    > Error in tcrossprod(matrix(1:4, 3, 4), 1:3) : non-conformable arguments
    >> tcrossprod(matrix(1:4, 3,4), 1:4)
    > Error in tcrossprod(matrix(1:4, 3, 4), 1:4) : non-conformable arguments
    >> tcrossprod(matrix(1:4, 1,4), 1:4)##>>> *could* be nice and use (n,1) as ncol(.) == 1 <<<
    > Error in tcrossprod(matrix(1:4, 1, 4), 1:4) : non-conformable arguments
    > ##>> "as"   m %*% v   does work  and v could have been  A[1,] instead of  A[1, , drop=TRUE]
    >> matrix(1:4, 1,4) %*%  1:4
    > [,1]
    > [1,]   30
    >> tcrossprod(matrix(1:2, 2,1), 1:5)  # v -> c(1, n)     as ncol(.) == 1
    > [,1] [,2] [,3] [,4] [,5]
    > [1,]    1    2    3    4    5
    > [2,]    2    4    6    8   10
    >> 

    > ## --------- v . M -----------------------------
    >> tcrossprod(1:4, matrix(1:4, 3,4)) # v -> c(1, n)     as n == ncol(.)
    > [,1] [,2] [,3]
    > [1,]   26   28   26
    >> tcrossprod(1:3, matrix(1:4, 3,4))
    > Error in tcrossprod(1:3, matrix(1:4, 3, 4)) : non-conformable arguments
    >> tcrossprod(1:3, matrix(1:3, 3,1)) # v -> c(n, 1)     as ncol(.) == 1
    > [,1] [,2] [,3]
    > [1,]    1    2    3
    > [2,]    2    4    6
    > [3,]    3    6    9
    >> 
    > ##--------------------------- vector x vector -----------------

    >> 1:3 %*% 1:3         ## must have same n
    > [,1]
    > [1,]   14

    >> crossprod(1:3, 1:3) ## must have same n
    > [,1]
    > [1,]   14


    >> tcrossprod(1:3, 1:4) ## can have different length
    > [,1] [,2] [,3] [,4]
    > [1,]    1    2    3    4
    > [2,]    2    4    6    8
    > [3,]    3    6    9   12

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel



[Rd] Unfixed bugs in latest R-patched
Martin Maechler