[SR-7620] String(contentsOf: url) not working on Linux

[swift-ci]

Previous ID	SR-7620
Radar	None
Original Reporter	sdidanov (JIRA User)
Type	Bug
Status	Resolved
Resolution	Done

Environment

Ubuntu 16.04(with last updates), Swift 4.1.1

Additional Detail from JIRA

Votes	0
Component/s	Foundation
Labels	Bug
Assignee	@spevans
Priority	Medium

md5: 3d20f9724259d3d924f97a21c1420271

Issue Description:

The following code:

import Foundation
do {
 let url = URL(string: "https://www.google.com")!
 let htmlString = try String(contentsOf: url, encoding: .ascii)
 print("Start", htmlString, "Finish")
}
catch {
 print(error)
}

On swift3 returns google page. On Swift 4 returns:

$ swift test.swift 
Start Finish

[belkadan]

@millenomi, @phausler, any idea what would have changed?

[spevans]

I just tested this with swift-DEVELOPMENT-SNAPSHOT-2018-05-26-a on Ubuntu 16.04 and it works fine. Note that https://www.google.com is actually served with charset=ISO-8859-1 and it does contain non-ASCII bytes in the data.

However String(contentsOf: url) doesnt work because it tries to determine the encoding by looking for a BOM (Byte Order Mark) in the first few bytes for UTF-16/UTF-32, falling back to UTF-8.
This page won't decode as UTF-8, which is correct, so I have written a PR that fixes NSString(contentsOf:usedEncoding: )
(the underlying call) to use the Content-Type header if available to find the encoding to use.

#1565

[swift-ci]

Comment by Sergey Didanov (JIRA)

Confirm. It works correct on last development snapshot.

[swift-ci]

Comment by Sergey Didanov (JIRA)

Hi,

In what release this fix will be?

[spevans]

This has been merged into the 4.2 branch in PR #1615