From h.wickham at gmail.com  Thu Jul  7 17:40:41 2016
From: h.wickham at gmail.com (Hadley Wickham)
Date: Thu, 7 Jul 2016 10:40:41 -0500
Subject: [Rd] String encoding problem
In-Reply-To: <a9e15129-684b-0a14-2790-5856aca9eb50@gmail.com>
References: <CABdHhvHwyZEEK+pjzgjvxgExeRuoSbEgTX+K10hibU6cKVL_TA@mail.gmail.com>
	<a9e15129-684b-0a14-2790-5856aca9eb50@gmail.com>
Message-ID: <CABdHhvHreFpv9QGNkhrZOM9sD-30J5+N3xkD02fqyVHH6Q4WxA@mail.gmail.com>

On Thu, Jul 7, 2016 at 10:11 AM, Duncan Murdoch
<murdoch.duncan at gmail.com> wrote:
> On 07/07/2016 10:57 AM, Hadley Wickham wrote:
>>
>> If you print:
>>
>> "\xc9\x82\xbf"
>>
>> you get
>>
>>  "\u0242\xbf"
>>
>> But if you try and evaluate that string you get:
>>
>>>  "\u0242\xbf"
>>
>> Error: mixing Unicode and octal/hex escapes in a string is not allowed
>>
>> (Probably will only happen on mac/linux with default utf-8 encoding)
>
>
> I'm not sure what should happen here, but that's not a legal string in a
> UTF-8 locale, so it's not too surprising that things go wonky.

Here's bit more context on how I got that sequence of bytes:

x <- "?????"
y <- iconv(x, to = "Shift-JIS")
Encoding(y)
y

I did this to create an example to demonstrate how to handle encoding
problems, and it's bit frustrating that I have to manually mangle the
string in order to be able to re-use it in another session.  Maybe
strings with unknown encoding shouldn't use unicode escapes?

Hadley

-- 
http://hadley.nz



[Rd] String encoding problem
Hadley Wickham