From i.ucar86 at gmail.com  Sun Sep  3 01:18:51 2017
From: i.ucar86 at gmail.com (=?UTF-8?B?ScOxYWtpIMOaY2Fy?=)
Date: Sun, 3 Sep 2017 01:18:51 +0200
Subject: [Rd] readLines() segfaults on large file & question on how to
 work around
In-Reply-To: <CAKstpn6YTRRRjr=GVUv3o9u9PzftHPn9AVSTfXAss50YwBjBYA@mail.gmail.com>
References: <CAKstpn6YTRRRjr=GVUv3o9u9PzftHPn9AVSTfXAss50YwBjBYA@mail.gmail.com>
Message-ID: <CALEXWq150zrG0Ao7c4OBW-Ju=v7zER2RaT6A=Rahq+b1=nLEtQ@mail.gmail.com>

2017-09-02 20:58 GMT+02:00 Jennifer Lyon <jennifer.s.lyon at gmail.com>:
> Hi:
>
> I have a 2.1GB JSON file. Typically I use readLines() and
> jsonlite:fromJSON() to extract data from a JSON file.
>
> When I try and read in this file using readLines() R segfaults.
>
> I believe the two salient issues with this file are
> 1). Its size
> 2). It is a single line (no line breaks)

As a workaround you can pipe something like "sed s/,/,\\n/g" before
your R script to insert line breaks.

I?aki



[Rd] readLines() segfaults on large file & question on how to work around
=?UTF-8?B?ScOxYWtpIMOaY2Fy?=