From kevin.hendricks at sympatico.ca  Mon Jul 31 23:19:52 2006
From: kevin.hendricks at sympatico.ca (Kevin B. Hendricks)
Date: Mon, 31 Jul 2006 17:19:52 -0400
Subject: [Rd] Any interest in "merge" and "by" implementations
	specifically for sorted data?
In-Reply-To: <Pine.LNX.4.64.0607310716250.32031@homer23.u.washington.edu>
References: <Pine.GSO.4.31.0607280805550.24504-100000@markov.stats>
	<FFDE23C5-EA11-4E16-B372-012DB00BD589@sympatico.ca>
	<971536df0607281112j43c4ffc0s96d3b4743b93b690@mail.gmail.com>
	<D5EA9DD0-BFC7-45A6-A541-65E8A9E8192C@sympatico.ca>
	<Pine.GSO.4.56.0607281229290.28145@durian.statsci.com>
	<5A40DD7D-3BF4-4451-AC91-9B121BE9901F@sympatico.ca>
	<Pine.GSO.4.56.0607281725120.28145@durian.statsci.com>
	<AAC5E1FB-71BB-46C8-9D51-A72DD25C735E@sympatico.ca>
	<Pine.LNX.4.64.0607310716250.32031@homer23.u.washington.edu>
Message-ID: <83B4A6AC-520E-45C1-AE0B-C875DDDE1F1C@sympatico.ca>

Hi Thomas,

Here is a comparison of performance times from my own igroupSums  
versus using split and rowsum:

 > x <- rnorm(2e6)
 > i <- rep(1:1e6,2)
 >
 > unix.time(suma <- unlist(lapply(split(x,i),sum)))
[1] 8.188 0.076 8.263 0.000 0.000
 >
 > names(suma)<- NULL
 >
 > unix.time(sumb <- igroupSums(x,i))
[1] 0.036 0.000 0.035 0.000 0.000
 >
 > all.equal(suma, sumb)
[1] TRUE
 >
 > unix.time(sumc <- rowsum(x,i))
[1] 0.744 0.000 0.742 0.000 0.000
 >
 > sumc <- sumc[,1]
 > names(sumc)<-NULL
 > all.equal(suma,sumc)
[1] TRUE


So my implementation of igroupSums is faster and already handles NA.   
I also have implemented igroupMins, igroupMaxs, igroupAnys,  
igroupAlls, igroupCounts, igroupMeans, and igroupRanges.

The igroup functions I implemented do not handle weights yet but do  
handle NAs properly.

Assuming I clean them up, is anyone in the R developer group interested?

Or would you rather I instead extend the rowsum appropach to create  
rowcount, rowmax, rowmin, rowcount, etc using a hash function approach.

All of these approaches simply use differently ways to map group  
codes to integers and then do the functions the same.

Thanks,

Kevin



[Rd] Any interest in "merge" and "by" implementations specifically for sorted data?
Kevin B. Hendricks