From h.wickham at gmail.com  Wed Aug 28 20:24:53 2013
From: h.wickham at gmail.com (Hadley Wickham)
Date: Wed, 28 Aug 2013 13:24:53 -0500
Subject: [Rd] Memory allocation in read.table
In-Reply-To: <0CA3536B-D70E-4B6E-A888-A6992B17DEC3@r-project.org>
References: <CABdHhvEtVhBQS7x+S86my3ec26-iCQx+NpVdzM4nh5ca=pGfqg@mail.gmail.com>
	<6603B703-48C9-417E-9495-F68B4015F381@r-project.org>
	<CABdHhvE9LrcseP93KTG4AtkrgffDF9FgkQH-kBnP_fgJRt+i8w@mail.gmail.com>
	<0CA3536B-D70E-4B6E-A888-A6992B17DEC3@r-project.org>
Message-ID: <CABdHhvFWjUT+9W9nGxp-JcV-w03J69rffiZi3JdWsgoNXASNgg@mail.gmail.com>

> Yup - parsing is the most expensive part. That's why for high-throughput data you don't want to use ASCII representation. It's amazing that the disk speeds are now so high that CPUs are the bottlenecks now, not vice versa.

Do you have any recommendations for binary formats? For R, is there
anything obviously better than Rdata?

Hadley

-- 
Chief Scientist, RStudio
http://had.co.nz/



[Rd] Memory allocation in read.table
Hadley Wickham