From savicky at cs.cas.cz  Thu Mar  5 17:25:04 2009
From: savicky at cs.cas.cz (savicky at cs.cas.cz)
Date: Thu,  5 Mar 2009 17:25:04 +0100 (CET)
Subject: [Rd] Spearman's rank correlation test (PR#13574)
Message-ID: <20090305162504.70C6A2832191@mail.pubhealth.ku.dk>

Full_Name: Petr Savicky
Version: 2.7.2, 2.8.1, 2.9.0
OS: Linux
Submission from: (NULL) (147.231.6.9)


The p-value of Spearman's rank correlation test is calculated in
  cor.test(x, y, method="spearman")
using algorithm AS 89. However, the way how AS 89 is used incures error,
which may be an order of magnitude larger than the error of the original
algorithm.

The paper, which introduced AS 89, provides error bounds, which are larger
than the actual error. The reason may be, that current implementation of
AS 89 uses more acurate estimate of the distribution function of the normal
distribution. The error of the R implementation of cor.test() for positive
correlation and n = 11 is larger than this pessimistic upper bound.

The problem is discussed in more detail in R-devel postings
  https://stat.ethz.ch/pipermail/r-devel/2009-January/051936.html
  https://stat.ethz.ch/pipermail/r-devel/2009-February/052112.html

A patch correcting the problem in the current development version
  R version 2.9.0 Under development (unstable) (2009-03-03 r48046)
is as follows.

--- R-devel/src/library/stats/R/cor.test.R  2008-12-14 17:51:56.000000000 +0100
+++ R-cor.test/src/library/stats/R/cor.test.R   2009-03-05 10:39:07.383841736
+0100
@@ -151,9 +151,9 @@
                 pspearman <- function(q, n, lower.tail = TRUE) {
                     if(n <= 1290 && exact) # n*(n^2 - 1) does not overflow
                         .C("prho",
                            as.integer(n),
-                           as.double(round(q) + lower.tail),
+                           as.double(round(q) + 2*lower.tail),
                            p = double(1L),
                            integer(1L),
                            as.logical(lower.tail),
                            PACKAGE = "stats")$p



[Rd] Spearman's rank correlation test (PR#13574)
savicky at cs.cas.cz