From phgrosjean at sciviews.org  Sun Nov 10 22:00:16 2013
From: phgrosjean at sciviews.org (Philippe Grosjean)
Date: Sun, 10 Nov 2013 22:00:16 +0100
Subject: [Rd] Support writing UTF-8 output in Windows
In-Reply-To: <527F80D2.5030504@gmail.com>
References: <CACtaF7xhmkyuALWyB3LZqEwBAwDwO6J5z5YM6ThQQ=0MuNZY6A@mail.gmail.com>	<527E9533.4070609@gmail.com>
	<CACtaF7wX8JcAhovaxnyvJVqJFBEeOcS6THzN7AEfP7_kv4U7eQ@mail.gmail.com>
	<527F80D2.5030504@gmail.com>
Message-ID: <A8F729B6-AB7F-4826-B01B-FDFC35F61DB2@sciviews.org>

Here is a quick (and largely untested, apart from the toy example) hack that should read and write UTF-8 encoded CSV files on Window. It is *not* the best solution, which should use proper C-level code. But it could help a bit in your case.
Best,

Philippe Grosjean


read.tableUTF8 <- function (file, ...)
{
	if (l10n_info()$`UTF-8`) {
		read.table(file = file, fileEncoding = "UTF-8", ...)
	} else {
		res <- read.table(file = file, ...) # Read in default encoding
		## For each character variable, change encoding to 'UTF-8'
		## For each factor, change encoding to 'UTF-8'
		as.data.frame(lapply(res, function (x) switch(data.class(x),
			character = {Encoding(x) <- "UTF-8"; x},
			factor = {Encoding(levels(x)) <- "UTF-8"; x},
			x))
		)
	}
}

write.tableUTF8 <- function (x, file = "", ...)
{
	if (l10n_info()$`UTF-8`) {
		write.table(x = x, file = file, fileEncoding = "UTF-8", ...)
	} else {
		## Change encoding to "bytes"  and save it like that
		x <- lapply(x, function (x) {
			if (is.character(x)) {
				Encoding(x) <- "bytes"
			} else if (is.factor(x)) {
				Encoding(levels(x)) <- "bytes"
			}
			x
		})
		write.table(x = x, file = file, ...)
	}
}

fact <- factor(c("\u0444", "\u220F", "\u2030"))
char <- c("\u2202x", "\u2202y", "\u2202z")
dfr <- data.frame(x = 1:3, f = fact, s = I(char))
dfr

write.tableUTF8(dfr, file = "testUTF8.txt")
dfr2 <- read.tableUTF8("testUTF8.txt")
dfr2$s <- I(as.character(dfr2$s))
dfr2

identical(dfr$f, dfr2$f)
identical(dfr$s, dfr2$s)



[Rd] Support writing UTF-8 output in Windows
Philippe Grosjean