Second time fetched file is broken

[ulitiy]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest versions of Parse Server and the Parse Swift SDK.

Issue Description

If file is fetched two times (only with real URL), the actual file is replaced by a broken one.

That's the case even across multiple app launches, it puts some URL string inside the actual file. Probably some cache issue.

Steps to reproduce

    func testFetchFile() throws {
        guard let parseFileURL = URL(string: "https://artworkout-parse.nyc3.cdn.digitaloceanspaces.com/116112879a51dfe3ce5e34105a26aae9_autumn.pumpkin.svg") else {
            XCTFail("Should create URL")
            return
        }
        var parseFile = ParseFile(name: "d3a37aed0672a024595b766f97133615_logo.svg", cloudURL: parseFileURL)
        parseFile.url = parseFileURL

        let firstFile = try parseFile.fetch()
        let firstStr = try! String(contentsOf: firstFile.localURL!)
        let secondFile = try parseFile.fetch()
        let secondStr = try! String(contentsOf: secondFile.localURL!)
        XCTAssertEqual(firstStr, secondStr)
    }

Actual Outcome

</g>
</svg>
") is not equal to (""file:\/\/\/Users\/xxx\/Library\/Developer\/CoreSimulator\/Devices\/XXXX\/data\/Library\/Private%20Documents\/parse\/Downloads\/d3a37aed0672a024595b766f97133615_logo.svg"")
Test Case '-[ParseSwiftTests.ParseFileTests testFetchFile]' failed (1.831 seconds).
Test Suite 'ParseFileTests' failed at 2022-09-13 21:33:10.256.

Expected Outcome

Test passing

Environment

Version 4.13.0

Logs

n/a

Workaround

Force fetch without cache

let secondFile = try parseFile.fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)])

[parse-github-assistant]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[cbaker6]

I've tested this in Swift Playgrounds and cannot replicate your issue on version 4.13.0.

A potential issue I see with your test not passing is you are running a block of code that is fetching the same file with same file name back to back. The ParseFile fetch stores the file to a temporary location then moves it to the Parse download directory. The structure of your code is most likely causing the file to collide with itself which is why you are getting garbage output. You can test this theory by running the first fetch, add some delay, then call the second fetch. This is how I run it Playgrounds, run first fetch, then second fetch. You can also fetch two files back to back with two different file names, then fetch those same files again and there's no issue.

Lastly, I recommend using async/await or completion block calls as you are making networking calls. Synchronous calls are not recommended and I don't even add new synchronous calls to the SDK anymore.

[ulitiy]

Of course I'm using async/await in my app and I still have the issue, I made this test just to reproduce it.

It's not a race condition issue, it's a cache issue.

If I fetch one file once in each of the 2 consecutive runs of the app it still happens. First time it's good, second - it's broken (unless I clear cache). Even on the ios device.

.fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)]) fixes it too, so

    func testFetchFile() throws {
        guard let parseFileURL = URL(string: "https://artworkout-parse.nyc3.cdn.digitaloceanspaces.com/116112879a51dfe3ce5e34105a26aae9_autumn.pumpkin.svg") else {
            XCTFail("Should create URL")
            return
        }
        var parseFile = ParseFile(name: "d3a37aed0672a024595b766f97133615_logo.svg", cloudURL: parseFileURL)
        parseFile.url = parseFileURL

        let firstFile = try parseFile.fetch()
        let firstStr = try! String(contentsOf: firstFile.localURL!)
        let secondFile = try parseFile.fetch(options: [.cachePolicy(.reloadIgnoringLocalCacheData)])
        let secondStr = try! String(contentsOf: secondFile.localURL!)
        XCTAssertEqual(firstStr, secondStr)
    }

this test passes

[cbaker6]

All of the fetching calls usually ignore cache like your workaround, but it looks like I missed adding in ParseFile, maybe because the cached worked in tests:

Parse-Swift/Sources/ParseSwift/Types/ParseFile.swift

Lines 508 to 707 in baf70f6

	// MARK: Fetching
	extension ParseFile {
	/**
	Fetches a file with given url *synchronously*.
	- parameter options: A set of header options sent to the server. Defaults to an empty set.
	- parameter stream: An input file stream.
	- parameter callbackQueue: The queue to return to after synchronous completion.
	Default value of .main.
	- returns: A saved `ParseFile`.
	*/
	public func fetch(options: API.Options = [],
	stream: InputStream,
	callbackQueue: DispatchQueue = .main) throws {
	var options = options
	if let mimeType = mimeType {
	options.insert(.mimeType(mimeType))
	} else {
	options.insert(.removeMimeType)
	}
	if let metadata = metadata {
	options.insert(.metadata(metadata))
	}
	if let tags = tags {
	options.insert(.tags(tags))
	}
	options = options.union(self.options)
	return try downloadFileCommand()
	.executeStream(options: options,
	callbackQueue: callbackQueue,
	stream: stream)
	}
	/**
	Fetches a file with given url *synchronously*.
	- parameter includeKeys: Currently not used for `ParseFile`.
	- parameter options: A set of header options sent to the server. Defaults to an empty set.
	- parameter callbackQueue: The queue to return to after synchronous completion.
	- returns: A saved `ParseFile`.
	*/
	public func fetch(includeKeys: [String]? = nil,
	options: API.Options = [],
	callbackQueue: DispatchQueue) throws -> ParseFile {
	var options = options
	if let mimeType = mimeType {
	options.insert(.mimeType(mimeType))
	} else {
	options.insert(.removeMimeType)
	}
	if let metadata = metadata {
	options.insert(.metadata(metadata))
	}
	if let tags = tags {
	options.insert(.tags(tags))
	}
	options = options.union(self.options)
	return try downloadFileCommand()
	.execute(options: options)
	}
	/**
	Fetches a file with given url *synchronously*.
	- parameter includeKeys: Currently not used for `ParseFile`.
	- parameter options: A set of header options sent to the server. Defaults to an empty set.
	- returns: A saved `ParseFile`.
	*/
	public func fetch(includeKeys: [String]? = nil,
	options: API.Options = []) throws -> ParseFile {
	try fetch(includeKeys: includeKeys,
	options: options,
	callbackQueue: .main)
	}
	/**
	Fetches a file with given url *synchronously*.
	**Checking progress**
	guard let parseFileURL = URL(string: "https://parseplatform.org/img/logo.svg") else {
	return
	}
	let parseFile = ParseFile(name: "logo.svg", cloudURL: parseFileURL)
	let fetchedFile = try parseFile.fetch { (_, _, totalDownloaded, totalExpected) in
	let currentProgess = Double(totalWritten)/Double(totalExpected) * 100
	print(currentProgess)
	}
	**Cancelling**
	guard let parseFileURL = URL(string: "https://parseplatform.org/img/logo.svg") else {
	return
	}
	let parseFile = ParseFile(name: "logo.svg", cloudURL: parseFileURL)
	let fetchedFile = try parseFile.fetch { (task, _, totalDownloaded, totalExpected) in
	let currentProgess = Double(totalDownloaded)/Double(totalExpected) * 100
	//Cancel when data exceeds 10%
	if currentProgess > 10 {
	task.cancel()
	print("task has been cancelled")
	}
	print(currentProgess)
	}
	- parameter options: A set of header options sent to the server. Defaults to an empty set.
	- parameter callbackQueue: The queue to return to after synchronous completion.
	Defaults to .main.
	- parameter progress: A block that will be called when file updates it is progress.
	It should have the following argument signature: `(task: URLSessionDownloadTask,
	bytesWritten: Int64, totalBytesWritten: Int64, totalBytesExpectedToWrite: Int64)`.
	- returns: A saved `ParseFile`.
	*/
	public func fetch(options: API.Options = [],
	callbackQueue: DispatchQueue = .main,
	progress: @escaping ((URLSessionDownloadTask, Int64, Int64, Int64) -> Void)) throws -> ParseFile {
	var options = options
	if let mimeType = mimeType {
	options.insert(.mimeType(mimeType))
	} else {
	options.insert(.removeMimeType)
	}
	if let metadata = metadata {
	options.insert(.metadata(metadata))
	}
	if let tags = tags {
	options.insert(.tags(tags))
	}
	options = options.union(self.options)
	return try downloadFileCommand()
	.execute(options: options,
	notificationQueue: callbackQueue,
	downloadProgress: progress)
	}
	/**
	Fetches a file with given url *asynchronously*.
	**Checking progress**
	guard let parseFileURL = URL(string: "https://parseplatform.org/img/logo.svg") else {
	return
	}
	let parseFile = ParseFile(name: "logo.svg", cloudURL: parseFileURL)
	let fetchedFile = try parseFile.fetch { (_, _, totalDownloaded, totalExpected) in
	let currentProgess = Double(totalDownloaded)/Double(totalExpected) * 100
	print(currentProgess)
	}
	**Cancelling**
	guard let parseFileURL = URL(string: "https://parseplatform.org/img/logo.svg") else {
	return
	}
	let parseFile = ParseFile(name: "logo.svg", cloudURL: parseFileURL)
	let fetchedFile = try parseFile.fetch(progress: {(task, _, totalDownloaded, totalExpected)-> Void in
	let currentProgess = Double(totalDownloaded)/Double(totalExpected) * 100
	//Cancel when data exceeds 10%
	if currentProgess > 10 {
	task.cancel()
	print("task has been cancelled")
	}
	print(currentProgess)
	}) { result in
	...
	}
	- parameter options: A set of header options sent to the server. Defaults to an empty set.
	- parameter callbackQueue: The queue to return to after completion. Default value of .main.
	- parameter progress: A block that will be called when file updates it is progress.
	It should have the following argument signature: `(task: URLSessionDownloadTask,
	bytesWritten: Int64, totalBytesWritten: Int64, totalBytesExpectedToWrite: Int64)`.
	- parameter completion: A block that will be called when file fetches or fails.
	It should have the following argument signature: `(Result<Self, ParseError>)`.
	*/
	public func fetch(options: API.Options = [],
	callbackQueue: DispatchQueue = .main,
	progress: ((URLSessionDownloadTask, Int64, Int64, Int64) -> Void)? = nil,
	completion: @escaping (Result<Self, ParseError>) -> Void) {
	var options = options
	if let mimeType = mimeType {
	options.insert(.mimeType(mimeType))
	} else {
	options.insert(.removeMimeType)
	}
	if let metadata = metadata {
	options.insert(.metadata(metadata))
	}
	if let tags = tags {
	options.insert(.tags(tags))
	}
	options = options.union(self.options)
	downloadFileCommand()
	.executeAsync(options: options,
	callbackQueue: callbackQueue,
	downloadProgress: progress) { result in
	completion(result)
	}
	}

Can you open a PR and add your test to the test suite? You will need to remove the force unwrapping for the code to be accepted.

Async tests

Parse-Swift/Tests/ParseSwiftTests/ParseFileAsyncTests.swift

Lines 139 to 167 in baf70f6

	@MainActor
	func testSave() async throws {
	guard let sampleData = "Hello World".data(using: .utf8) else {
	throw ParseError(code: .unknownError, message: "Should have converted to data")
	}
	let parseFile = ParseFile(name: "sampleData.txt", data: sampleData)
	// swiftlint:disable:next line_length
	guard let url = URL(string: "http://localhost:1337/1/files/applicationId/89d74fcfa4faa5561799e5076593f67c_sampleData.txt") else {
	XCTFail("Should create URL")
	return
	}
	let response = FileUploadResponse(name: "89d74fcfa4faa5561799e5076593f67c_\(parseFile.name)", url: url)
	let encoded: Data!
	do {
	encoded = try ParseCoding.jsonEncoder().encode(response)
	} catch {
	XCTFail("Should encode/decode. Error \(error)")
	return
	}
	MockURLProtocol.mockRequests { _ in
	return MockURLResponse(data: encoded, statusCode: 200, delay: 0.0)
	}
	let fetched = try await parseFile.save()
	XCTAssertEqual(fetched.name, response.name)
	XCTAssertEqual(fetched.url, response.url)
	}

Another example

Parse-Swift/Tests/ParseSwiftTests/ParseFileTests.swift

Lines 896 to 934 in baf70f6

	func testFetchFileAysnc() throws {
	// swiftlint:disable:next line_length
	guard let parseFileURL = URL(string: "http://localhost:1337/1/files/applicationId/7793939a2e59b98138c1bbf2412a060c_logo.svg") else {
	XCTFail("Should create URL")
	return
	}
	var parseFile = ParseFile(name: "7793939a2e59b98138c1bbf2412a060c_logo.svg", cloudURL: parseFileURL)
	parseFile.url = parseFileURL
	let response = FileUploadResponse(name: "7793939a2e59b98138c1bbf2412a060c_logo.svg",
	url: parseFileURL)
	let encoded: Data!
	do {
	encoded = try ParseCoding.jsonEncoder().encode(response)
	} catch {
	XCTFail("Should encode/decode. Error \(error)")
	return
	}
	MockURLProtocol.mockRequests { _ in
	return MockURLResponse(data: encoded, statusCode: 200, delay: 0.0)
	}
	let expectation1 = XCTestExpectation(description: "ParseFile async")
	parseFile.fetch { result in
	switch result {
	case .success(let fetched):
	XCTAssertEqual(fetched.name, response.name)
	XCTAssertEqual(fetched.url, response.url)
	XCTAssertNotNil(fetched.localURL)
	case .failure(let error):
	XCTFail(error.localizedDescription)
	}
	expectation1.fulfill()
	}
	wait(for: [expectation1], timeout: 20.0)
	}

Testing file cache

Parse-Swift/Tests/ParseSwiftTests/ParseFileTests.swift

Lines 1099 to 1209 in baf70f6

	func testFetchFile() throws {
	// swiftlint:disable:next line_length
	guard let parseFileURL = URL(string: "http://localhost:1337/1/files/applicationId/d3a37aed0672a024595b766f97133615_logo.svg") else {
	XCTFail("Should create URL")
	return
	}
	var parseFile = ParseFile(name: "d3a37aed0672a024595b766f97133615_logo.svg", cloudURL: parseFileURL)
	parseFile.url = parseFileURL
	let response = FileUploadResponse(name: "d3a37aed0672a024595b766f97133615_logo.svg",
	url: parseFileURL)
	let encoded: Data!
	do {
	encoded = try ParseCoding.jsonEncoder().encode(response)
	} catch {
	XCTFail("Should encode/decode. Error \(error)")
	return
	}
	MockURLProtocol.mockRequests { _ in
	return MockURLResponse(data: encoded, statusCode: 200, delay: 0.0)
	}
	let fetchedFile = try parseFile.fetch()
	XCTAssertEqual(fetchedFile.name, response.name)
	XCTAssertEqual(fetchedFile.url, response.url)
	XCTAssertNotNil(fetchedFile.localURL)
	// Cache policy flakey on older Swift versions
	#if compiler(>=5.5.0)
	// Remove URL mocker so we can check cache
	MockURLProtocol.removeAll()
	let fetchedFile2 = try parseFile.fetch(options: [.cachePolicy(.returnCacheDataDontLoad)])
	XCTAssertEqual(fetchedFile2.name, fetchedFile.name)
	XCTAssertEqual(fetchedFile2.url, fetchedFile.url)
	XCTAssertNotNil(fetchedFile2.localURL)
	// More cache tests
	guard let currentMemoryUsage = URLSession.parse.configuration.urlCache?.currentMemoryUsage,
	let currentDiskUsage = URLSession.parse.configuration.urlCache?.currentDiskUsage else {
	XCTFail("Should have unwrapped")
	return
	}
	XCTAssertGreaterThan(currentMemoryUsage, 0)
	XCTAssertGreaterThan(currentDiskUsage, 0)
	ParseSwift.clearCache()
	guard let updatedMemoryUsage = URLSession.parse.configuration.urlCache?.currentMemoryUsage else {
	XCTFail("Should have unwrapped")
	return
	}
	XCTAssertLessThan(updatedMemoryUsage, currentMemoryUsage)
	#endif
	}
	func testFetchFileWithDirInName() throws {
	// swiftlint:disable:next line_length
	guard let parseFileURL = URL(string: "http://localhost:1337/1/files/applicationId/d3a37aed0672a024595b766f97133615_logo.svg") else {
	XCTFail("Should create URL")
	return
	}
	var parseFile = ParseFile(name: "myFolder/d3a37aed0672a024595b766f97133615_logo.svg", cloudURL: parseFileURL)
	parseFile.url = parseFileURL
	let response = FileUploadResponse(name: "myFolder/d3a37aed0672a024595b766f97133615_logo.svg",
	url: parseFileURL)
	let encoded: Data!
	do {
	encoded = try ParseCoding.jsonEncoder().encode(response)
	} catch {
	XCTFail("Should encode/decode. Error \(error)")
	return
	}
	MockURLProtocol.mockRequests { _ in
	return MockURLResponse(data: encoded, statusCode: 200, delay: 0.0)
	}
	let fetchedFile = try parseFile.fetch()
	XCTAssertEqual(fetchedFile.name, response.name)
	XCTAssertEqual(fetchedFile.url, response.url)
	guard let localURL = fetchedFile.localURL else {
	XCTFail("Should have unwrapped")
	return
	}
	XCTAssertFalse(localURL.pathComponents.contains("myFolder"))
	// Cache policy flakey on older Swift versions
	#if compiler(>=5.5.0)
	// Remove URL mocker so we can check cache
	MockURLProtocol.removeAll()
	let fetchedFile2 = try parseFile.fetch(options: [.cachePolicy(.returnCacheDataDontLoad)])
	XCTAssertEqual(fetchedFile2.name, fetchedFile.name)
	XCTAssertEqual(fetchedFile2.url, fetchedFile.url)
	XCTAssertNotNil(fetchedFile2.localURL)
	// More cache tests
	guard let currentMemoryUsage = URLSession.parse.configuration.urlCache?.currentMemoryUsage,
	let currentDiskUsage = URLSession.parse.configuration.urlCache?.currentDiskUsage else {
	XCTFail("Should have unwrapped")
	return
	}
	XCTAssertGreaterThan(currentMemoryUsage, 0)
	XCTAssertGreaterThan(currentDiskUsage, 0)
	ParseSwift.clearCache()
	guard let updatedMemoryUsage = URLSession.parse.configuration.urlCache?.currentMemoryUsage else {
	XCTFail("Should have unwrapped")
	return
	}
	XCTAssertLessThan(updatedMemoryUsage, currentMemoryUsage)
	#endif
	}

Possible fix if something is up with the cache

The fix is adding the following option and documentation to all of the aforementioned fetching methods (you can try your tests without the fix to see if your test still fails):

Parse-Swift/Sources/ParseSwift/Objects/ParseObject.swift

Lines 979 to 985 in baf70f6

	- note: The default cache policy for this method is `.reloadIgnoringLocalCacheData`. If a developer
	desires a different policy, it should be inserted in `options`.
	*/
	public func fetch(includeKeys: [String]? = nil,
	options: API.Options = []) throws -> Self {
	var options = options
	options.insert(.cachePolicy(.reloadIgnoringLocalCacheData))

Contributing

See the environment setup necessary to make contributions: https://github.com/parse-community/Parse-Swift/blob/main/CONTRIBUTING.md#environment-setup

[cbaker6]

Please add the workaround you posted to the original issue for others to see. In addition, please edit your original issue with the specific version you are using. The main branch is updated periodically and there’s no way to tell which commit you are linked to, SPM doesn’t update your packages automatically, so this can easily be an old commit

[cbaker6]

This was most likely part of the problem with #226

[ulitiy]

I thought there's a ParseSwift internal cache, so you say that it's the OS that replaces the internals of the file with some random URL?

Maybe it should be a symlink and it breaks when copied by ParseSwift? I mean, why is there some random URL inside? It looks like a symlink.

[cbaker6]

#414 removed file caching and leaves this to the developer to handle if they want it. I suspect you won't see your issue in 4.13.1. The link in the file was probably related to how I attempted to cache the file. In your example it points to the location the file was saved. You can see the old code in the PR