From csardi at rmki.kfki.hu  Thu Feb 14 17:02:40 2008
From: csardi at rmki.kfki.hu (Gabor Csardi)
Date: Thu, 14 Feb 2008 17:02:40 +0100
Subject: [Rd] Vector binding on harddisk
In-Reply-To: <59d7961d0802140754x216c9753la5ebcfa415439a39@mail.gmail.com>
References: <47B426BE.9010606@gmx.net> <47B45ABD.4010304@bank-banque-canada.ca>
	<47B45CDC.2060803@vanderbilt.edu>
	<59d7961d0802140754x216c9753la5ebcfa415439a39@mail.gmail.com>
Message-ID: <20080214160240.GK10523@localdomain>

On Thu, Feb 14, 2008 at 07:54:38AM -0800, Henrik Bengtsson wrote:
> On Thu, Feb 14, 2008 at 7:23 AM, Jeffrey Horner
> <jeff.horner at vanderbilt.edu> wrote:
> > Paul Gilbert wrote on 02/14/2008 09:14 AM:
> >
> > >
> >  > _ wrote:
> >  >> Hi all,
> >  >> Using big vectors (more than 4GB) is unfortunately not possible under
> >  >> Windows or other OS's if not enough RAM exists.
> >  >
> >  > This is NOT true. It is not limited by RAM, but rather by RAM and swap
> >  > space.  With 500G hard disks at about $100, the more serious limitation
> >  > is a 32bit OS.  Speed is a different consideration, but I doubt that
> >  > taking over what the OS is suppose to do will be the real answer.
> >
> >  Umm... swap... yeah... My experience with any code that I write that
> >  needs to use swap is to reboot the machine and rewrite my code NOT to
> >  use swap.
> 
> ...and the OS can never predict how you are going to access the data -

Actually that is not necessarily true. It might be worth looking 
for patterns in data (say memory pages) access and then create a 
smart OS which predicts the needed pages. Different programs would 
have different patterns, i guess.

Ok, i might be dreaming.

Gabor

> as a developer you will always be able to write a much faster
> file-based "database" compared with a generic swap that you have
> little control over.
> 
> /Henrik
> 

-- 
Csardi Gabor <csardi at rmki.kfki.hu>    UNIL DGM



[Rd] Vector binding on harddisk
Gabor Csardi