[SR-3448] Crash in String.replacingOccurrences(of:with:) with emoji on Linux

[marcelofabri]

Previous ID	SR-3448
Radar	None
Original Reporter	@marcelofabri
Type	Bug
Status	Resolved
Resolution	Done

Environment

Swift version 3.0.2 (swift-3.0.2-RELEASE)
Target: x86_64-unknown-linux-gnu

Additional Detail from JIRA

Votes	0
Component/s	Foundation
Labels	Bug, Linux, RunTimeCrash
Assignee	None
Priority	Medium

md5: 8df26be5a0425f2be5e8c1d240db4cce

relates to:

• SR-3449 Crash in String.replacingCharacters(in:, with:) on Linux

Issue Description:

When using String.replacingOccurrences(of:with:) with emoji on Linux, I get a crash. This does not happen on macOS with the Xcode 8.2 Swift build.

The crash only happens when there're occurrences of the searched substring.

Sample code:

import Foundation
let str = "\u{1F468}\u{200D}\u{1F469}\u{200D}\u{1F467}\u{200D}\u{1F466} marcelo "
let replaced = str.replacingOccurrences(of: "ce", with: "")
print(replaced)

Stacktrace:

swift SwiftLint/file3.swift 
0  swift            0x00000000035c37a8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  swift            0x00000000035c3f07
2  libpthread.so.0  0x00007f17a0e7b3e0
3  libFoundation.so 0x00007f179ac812d9 _TTSf4s_gs_n___TFC10Foundation15NSMutableString17replaceCharactersfT2inVS_8_NSRange4withSS_T_ + 1161
4  libFoundation.so 0x00007f179ac8159b _TTSf4gs_gs_n_s_n___TFC10Foundation15NSMutableString18replaceOccurrencesfT2ofSS4withSS7optionsVCS_8NSString14CompareOptions5rangeVS_8_NSRange_Si + 619
5  libFoundation.so 0x00007f179ac81b3f _TTSf4gs_gs_n_s_n___TFC10Foundation8NSString20replacingOccurrencesfT2ofSS4withSS7optionsVS0_14CompareOptions5rangeVS_8_NSRange_SS + 527
6  libFoundation.so 0x00007f179ac73fa8 _TFC10Foundation8NSString20replacingOccurrencesfT2ofSS4withSS_SS + 408
7  libFoundation.so 0x00007f179ac869ff _TFE10FoundationSS20replacingOccurrencesfT2ofSS4withSS7optionsVCS_8NSString14CompareOptions5rangeGSqGVs5RangeVVSS13CharacterView5Index___SS + 255
8  libFoundation.so 0x00007f17a11e318f _TFE10FoundationSS20replacingOccurrencesfT2ofSS4withSS7optionsVCS_8NSString14CompareOptions5rangeGSqGVs5RangeVVSS13CharacterView5Index___SS + 106285199
9  swift            0x0000000000c59d58 llvm::MCJIT::runFunction(llvm::Function*, llvm::ArrayRef<llvm::GenericValue>) + 808
10 swift            0x0000000000c5d782 llvm::ExecutionEngine::runFunctionAsMain(llvm::Function*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, char const* const*) + 1282
11 swift            0x00000000007f208b swift::RunImmediately(swift::CompilerInstance&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, swift::IRGenOptions&, swift::SILOptions const&) + 2555
12 swift            0x00000000007ea727
13 swift            0x00000000007e59a1 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 2737
14 swift            0x00000000007ab0c3 main + 2787
15 libc.so.6        0x00007f179f7ae830 __libc_start_main + 240
16 swift            0x00000000007a8769 _start + 41
Stack dump:
0.  Program arguments: /usr/bin/swift -frontend -interpret SwiftLint/file3.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -color-diagnostics -module-name file3 
Illegal instruction

[swift-ci]

Comment by Alexander Lash (JIRA)

Oddly, adding options: .regularExpression causes this to work as expected. Also, as long as the replaced occurrences are before any multibyte characters there's no crash.

At a glance, I'm guessing it's the range calculation in Foundation/NSString.swift:1267 (which helpfully points out that it's incorrect for "grapheme clusters that have a size greater than a single unichar" 🙂

[swift-ci]

Comment by Alexander Lash (JIRA)

Credit to Tony Allevato for sorting this one out.

The implementation of Foundation.NSMutableString.replaceCharacters is indexing by characters, but taking a range in UTF-16 code points, causing range calculation to fail.

We put together a simple tweak that recalculates indexes from the UTF-16 view and converts them back to characters. With respect to https://github.com/apple/swift-corelibs-foundation/blob/master/Foundation/NSString.swift#L1276 -

{{
let start = _storage.utf16.startIndex
let min = _storage.utf16.index(start, offsetBy: range.location).samePosition(in: _storage)!
let max = _storage.utf16.index(start, offsetBy: range.location + range.length).samePosition(in: _storage)!
}}

I'm working on testing this and putting together a PR. Not sure if there's a more performant way to do this.

[jpsim]

This was fixed in #796 which was first included in the official 3.1 release.