From ripley at stats.ox.ac.uk  Mon Jun 25 11:24:45 2007
From: ripley at stats.ox.ac.uk (ripley at stats.ox.ac.uk)
Date: Mon, 25 Jun 2007 11:24:45 +0200 (CEST)
Subject: [Rd] problem gsub in the locale of CP932 and SJIS (PR#9751)
Message-ID: <20070625092445.D8EBD65979@slim.kubism.ku.dk>

Thanks for this.

I don't think the patch is quite right.  As I understand it, mbstate_t 
should be initialized at the start of the string, not before each 
character, and that is what is done in the rest of R.

Also, do you have an example I can use to test the patch, please?

R 2.5.0 is now in code freeze and I don't think this is vital for that.


On Sun, 24 Jun 2007, nakama at ki.rim.or.jp wrote:

> Full_Name: Ei-ji Nakama
> Version: R-2.5.0
> OS: any
> Submission from: (NULL) (219.117.236.5)
>
>
> problem by operation of gsub in the locale of CP932 and SJIS.
> The inconvenient character code which used 0x5c after the first byte.
>
> --- R-2.5.0.orig/src/main/character.c   2007-04-03 11:05:05.000000000 +0900
> +++ R-2.5.0/src/main/character.c        2007-06-24 22:31:06.000000000 +0900
> @@ -986,6 +986,17 @@
>     char *p = repl;
>     n = strlen(repl) - (regmatch[0].rm_eo - regmatch[0].rm_so);
>     while (*p) {
> +#ifdef  SUPPORT_MBCS
> +       if(mbcslocale){
> +           int clen;
> +           mbstate_t mb_st;
> +           mbs_init(&mb_st);
> +           if((clen = Mbrtowc(NULL, p, MB_CUR_MAX, &mb_st)) > 1){
> +               p+=clen;
> +               continue;
> +           }
> +       }
> +#endif
>        if (*p == '\\') {
>            if ('1' <= p[1] && p[1] <= '9') {
>                k = p[1] - '0';
> @@ -1014,6 +1025,18 @@
>     int i, k;
>     char *p = repl, *t = target;
>     while (*p) {
> +#ifdef  SUPPORT_MBCS
> +       if(mbcslocale){
> +           int clen;
> +           mbstate_t mb_st;
> +           mbs_init(&mb_st);
> +           if((clen = Mbrtowc(NULL, p, MB_CUR_MAX, &mb_st)) > 1){
> +               for ( i=0; i<clen; i++)
> +                   *t++ = *p++;
> +               continue;
> +           }
> +       }
> +#endif
>        if (*p == '\\') {
>            if ('1' <= p[1] && p[1] <= '9') {
>                k = p[1] - '0';
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595



[Rd] problem gsub in the locale of CP932 and SJIS (PR#9751)
ripley at stats.ox.ac.uk