Skip to content

Commit 506b373

Browse files
authored
fix: Make lstripChars and rStripChars work with multibyte characters (google#663)
We're currently working with the length of the string itself, which is the number of bytes. That's wrong in these functions which operate at the level of characters. When applied to a string with a multibyte character they can return the wrong result. For example, evaluating `std.rstripChars('• foo\n', '\n')` results in `"• f"`. What we need to do instead is get the number of runes, and remove one rune at a time.
1 parent 1233966 commit 506b373

7 files changed

+8
-2
lines changed

builtins.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -517,7 +517,8 @@ func builtinLstripChars(i *interpreter, str, chars value) (value, error) {
517517
return nil, err
518518
}
519519
if member {
520-
s := strType.getGoString()[1:]
520+
runes := strType.getRunes()
521+
s := string(runes[1:])
521522
return builtinLstripChars(i, makeValueString(s), chars)
522523
} else {
523524
return str, nil
@@ -542,7 +543,8 @@ func builtinRstripChars(i *interpreter, str, chars value) (value, error) {
542543
return nil, err
543544
}
544545
if member {
545-
s := strType.getGoString()[:strType.length()-1]
546+
runes := strType.getRunes()
547+
s := string(runes[:len(runes)-1])
546548
return builtinRstripChars(i, makeValueString(s), chars)
547549
} else {
548550
return str, nil
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"foo"
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
std.lstripChars('• foo', '• ')

testdata/std.lstripChars.multibyte.linter.golden

Whitespace-only changes.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"• foo"
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
std.rstripChars('• foo\n', '\n')

testdata/std.rstripChars.multibyte.linter.golden

Whitespace-only changes.

0 commit comments

Comments
 (0)