From pdalgd at gmail.com  Tue Nov 16 14:04:16 2010
From: pdalgd at gmail.com (peter dalgaard)
Date: Tue, 16 Nov 2010 14:04:16 +0100
Subject: [Rd] Bug in read.table?
In-Reply-To: <loom.20101116T025448-997@post.gmane.org>
References: <4CD1674E.6070807@yorku.ca> <002701cb7b65$5a516c30$0ef44490$@ca>
	<4CD1A6BE.3020801@yorku.ca>
	<1941.78.147.225.236.1288992147.squirrel@paleo.ija.csic.es>
	<4CD6D121.1020402@acm.org>
	<2145.78.147.236.204.1289166988.squirrel@paleo.ija.csic.es>
	<loom.20101107T225942-16@post.gmane.org>
	<loom.20101108T003515-462@post.gmane.org>
	<loom.20101109T021750-294@post.gmane.org>
	<loom.20101116T025448-997@post.gmane.org>
Message-ID: <95D9C788-FDEA-438C-BA19-19E8625389F0@gmail.com>


On Nov 16, 2010, at 02:59 , Ben Bolker wrote:

> Ben Bolker <bbolker <at> gmail.com> writes:
> 
>> 
>> Ben Bolker <bbolker <at> gmail.com> writes:
>> 
>>> 
>>> 
>> 
>>   Can simplify this still farther:
>> 
>> a b'c
>> d e'f
>> g h'i
> 
>  This example file leads to duplicate lines.
> Arguably it should have behavior analogous to:
> 
>> scan(what="")
> 1: a b'c
> 3: d e'f
> 5: g h'i
> 7: Read 6 items
> [1] "a"   "b'c" "d"   "e'f" "g"   "h'i"
> 
> 
>> 
>>> One of the first things that happens in read.table is that
>>> the first few lines are read with readTableHead:
>>> 
>>>  lines <- .Internal(readTableHead(file, nlines, comment.char, 
>>>       blank.lines.skip, quote, sep))
>>> 
>>  in this case, this reads the first two lines as one line;
>> the single quote at pos. 4 of the first line closes on pos.
>> 4 of the second line, preventing the first new line from
>> ending a line.
>> 
>>  R then pushes back two copies of the lines that have
>> been read (this is normal behavior; I don't quite follow the
>> logic).
>> 
>>  The rest of the file is read with scan(), 1 line at a time.
>> However, there is the discrepancy between the way
>> that readTableHead interprets new lines in the middle of
>> quoted strings (it ignores them) and the way that scan()
>> interprets them (it takes them as the end of the quoted string).
> 
> 
>  Ping?
>  I think this counts as a small, but real, bug. Should I go ahead
> and report it as such, or would someone explain why it's not a bug?
> 

I think it can be defended to file as a bug, but it is tricky to pinpoint exactly what the issue is. E.g., notice that adding a few spaces changes the behaviour of scan() considerably:

> scan(what="")
1:  a b 'c
1: d e' f
5: g h' i
8: 
Read 7 items
[1] "a"      "b"      "c\nd e" "f"      "g"      "h'"     "i"     

(I'm confused... What is it that we really want here?)

Also, as you noted originally, beware the "Well don't do that then" aspect...

-- 
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com



[Rd] Bug in read.table?
peter dalgaard