From murdoch.duncan at gmail.com  Thu Jul  7 21:23:59 2016
From: murdoch.duncan at gmail.com (Duncan Murdoch)
Date: Thu, 7 Jul 2016 15:23:59 -0400
Subject: [Rd] String encoding problem
In-Reply-To: <BC9624BC-91FF-406E-BC3F-12A3A123528F@gmail.com>
References: <CABdHhvHwyZEEK+pjzgjvxgExeRuoSbEgTX+K10hibU6cKVL_TA@mail.gmail.com>
	<a9e15129-684b-0a14-2790-5856aca9eb50@gmail.com>
	<CABdHhvHreFpv9QGNkhrZOM9sD-30J5+N3xkD02fqyVHH6Q4WxA@mail.gmail.com>
	<9D8A3548-910A-46AB-9213-CEE27EBAFC71@r-project.org>
	<CABdHhvFoxnA1UeT_zQWGsJEydkc6YvVtyyymhOkMtrs9mzKFyg@mail.gmail.com>
	<BC9624BC-91FF-406E-BC3F-12A3A123528F@gmail.com>
Message-ID: <1790fde0-b597-760a-18a4-319fe9fd75c6@gmail.com>

On 07/07/2016 12:51 PM, peter dalgaard wrote:
> > On 07 Jul 2016, at 18:15 , Hadley Wickham <h.wickham at gmail.com> wrote:
> >
> > Right - I'm aware of that.  But to me, it doesn't seem correct to
> > print a string that is not a valid R string. Why is an unknown
> > encoding printed like UTF-8?
> >
>
> It isn't -- no UTF-8 would have the \xbf. I may be flogging a dead horse, but it seems to me that there are three alternatives:
>
> - refuse the input (x <- "\xc9\x82\xbf" gives "sorry, not a UTF-8 string" or so)
> - refuse to print it (print(x) gives "cannot print non-UTF-8 string")
> - what happens now
>
> and a fourth one might be to actually allow mixing of \u0007 and \x07 and \007, but I suspect that there are demons down the line which is why it is not happening now. (Does it ring a bell with anyone?)

A fifth option would be to use only hex escapes when invalid UTF-8 was 
found.  That would echo back the input in this case.  No idea if it 
would cause other problems.

Duncan Murdoch



[Rd] String encoding problem
Duncan Murdoch