-
Notifications
You must be signed in to change notification settings - Fork 216
[Explicit Module Builds] Serialize and re-use the inter-module dependency graph as incremental build state #1230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| @_spi(Testing) public extension Driver { | ||
| /// Scan the current module's input source-files to compute its direct and transitive | ||
| /// module dependencies. | ||
| mutating func gatherModuleDependencies() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code just moved from Planning.swift it is unchanged.
72fbcc7 to
c4029ab
Compare
| /// | ||
| /// FIXME: This is a little ridiculous. We could probably just replace the | ||
| /// build record outright with a serialized format. | ||
| var driverDependencyGraphPath: VirtualPath? { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was duplicated in BuildRecordInfo and this instance was not used anywhere.
5016cf4 to
aeacb87
Compare
|
|
||
| /// A build-record-relative path to the location of a serialized copy of the | ||
| /// driver's inter-module dependency graph. | ||
| var interModuleDependencyGraphPath: VirtualPath { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alternatively we can also make this a part of the .swiftdeps file, but for now it seemed simpler to keep it separate.
|
@swift-ci test |
| Thread.sleep(forTimeInterval: 1) | ||
| print("*** touching \(name) ***", to: &stderrStream); stderrStream.flush() | ||
| let (path, contents) = try! XCTUnwrap(inputPathsAndContents.filter {$0.0.pathString.contains(name)}.first) | ||
| try! localFileSystem.writeFileContents(path) { $0 <<< contents } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed this to actually just re-write the file's contents with what they already are, so it actually just updates the timestamp. Previously touch was overwriting file contents with a pre-determined initial state for that input, instead of what the test may have modified it to be at that point.
nkcsgexi
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussed with Artem about this offline. LGTM as the first step for handling incremental explicit module build!
Sources/SwiftDriver/IncrementalCompilation/BuildRecordInfo.swift
Outdated
Show resolved
Hide resolved
Sources/SwiftDriver/IncrementalCompilation/IncrementalCompilationState+Extensions.swift
Outdated
Show resolved
Hide resolved
Sources/SwiftDriver/IncrementalCompilation/IncrementalDependencyAndInputSetup.swift
Outdated
Show resolved
Hide resolved
Sources/SwiftDriver/ExplicitModuleBuilds/ModuleDependencyScanning.swift
Outdated
Show resolved
Hide resolved
Sources/SwiftDriver/IncrementalCompilation/IncrementalCompilationState+Extensions.swift
Outdated
Show resolved
Hide resolved
…as incremental build state On an incremental build, load a prior dependency graph, check if it is still valid: 1. import set of the current module has not changed 2. input files of every module in the prior graph are older than the output and if it is, re-use it, skipping invoking the dependency scanner again. Partially resolves rdar://66801475
aeacb87 to
9c6e137
Compare
|
@swift-ci test |
|
Thanks @tshortli |
Building on top of #621, on an incremental build, load a prior dependency graph, check if it is still valid:
More concretely, computation of the initial incremental state in
computeIncrementalStateForPlanningwill attempt to read out and validate a prior serialized inter-module dependency graph. If validated, such graph will become a part of the initial incremental state itself.During build planning, at the stage of computing inter-module dependencies, the dependency scanning action will be skipped if the initial incremental state contains a still-valid prior inter-module dependency graph.
In both cases, if a prior graph is being re-used, or if a new graph was computed via a dependency scanning action, the graph will become a component of the incremental build state to be serialized after the build is complete.
This change does not yet result in skipping the actual module build jobs in the planning output, which can be done with this, but this is left to a followup PR.
There is also some notable overlap between this and what is already done for detecting changes in input files to the current compilation:
Once explicit builds become the default, we can consolidate the two approaches into one.
Partially resolves rdar://66801475