Skip to content

Add support for '-explicit-dependency-graph-format=' flag #1181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 14, 2022
Merged

Add support for '-explicit-dependency-graph-format=' flag #1181

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b6762cb
Update 'Options.swift'
artemcm Sep 12, 2022
64ec062
Add support for '-explicit-dependency-graph-format=' flag
artemcm Sep 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions Sources/SwiftDriver/Driver/Driver.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,7 @@ public struct Driver {
fileSystem: fileSystem,
workingDirectory: workingDirectory,
diagnosticEngine: diagnosticEngine)
Self.validateEmitDependencyGraphArgs(&parsedOptions, diagnosticEngine: diagnosticEngine)
Self.validateParseableOutputArgs(&parsedOptions, diagnosticEngine: diagnosticEngine)
Self.validateCompilationConditionArgs(&parsedOptions, diagnosticEngine: diagnosticEngine)
Self.validateFrameworkSearchPathArgs(&parsedOptions, diagnosticEngine: diagnosticEngine)
Expand Down Expand Up @@ -2646,6 +2647,28 @@ extension Driver {
}
}

static func validateEmitDependencyGraphArgs(_ parsedOptions: inout ParsedOptions,
diagnosticEngine: DiagnosticsEngine) {
// '-print-explicit-dependency-graph' requires '-explicit-module-build'
if parsedOptions.hasArgument(.printExplicitDependencyGraph) &&
!parsedOptions.hasArgument(.driverExplicitModuleBuild) {
diagnosticEngine.emit(Error.optionRequiresAnother(Option.printExplicitDependencyGraph.spelling,
Option.driverExplicitModuleBuild.spelling))
}
// '-explicit-dependency-graph-format=' requires '-print-explicit-dependency-graph'
Copy link
Contributor

@bnbarham bnbarham Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could -print-explicit-dependency-graph just take a format, or would that break all the things? (ie. do we need to be backwards compatible?)

Copy link
Contributor Author

@artemcm artemcm Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need it to be backwards compatible, I don't know of anyone who uses this other than me but the ergonomics of this seemed much more obvious by not conflating two things: emit graph, and emit format.

Copy link
Contributor

@bnbarham bnbarham Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it conflating two things 🤔? If we're emitting something we're emitting it in some format, so that seems relevant to me. I don't particularly mind here, I just personally prefer the one.

Copy link
Contributor Author

@artemcm artemcm Sep 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gah, sorry I missed this comment.
That's fair, let me add to my previous comment: my intent was to separate the common case of wanting to emit this graph (just dump the complete JSON) from having to specify a more-specific output format. And I don't know of a way to have a separate or joined flag that optionally takes a parameter is there one?

Copy link
Contributor

@bnbarham bnbarham Sep 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is. My point was more I think the format is always relevant and thus I'd be fine with specifying json all the time for the "default" case. But I can see the pros and cons, so it doesn't really matter :).

if parsedOptions.hasArgument(.explicitDependencyGraphFormat) &&
!parsedOptions.hasArgument(.printExplicitDependencyGraph) {
diagnosticEngine.emit(Error.optionRequiresAnother(Option.explicitDependencyGraphFormat.spelling,
Option.printExplicitDependencyGraph.spelling))
}
// '-explicit-dependency-graph-format=' only supports values 'json' and 'dot'
if let formatArg = parsedOptions.getLastArgument(.explicitDependencyGraphFormat)?.asSingle {
if formatArg != "json" && formatArg != "dot" {
diagnosticEngine.emit(.error_unsupported_argument(argument: formatArg,
option: .explicitDependencyGraphFormat))
}
}
}

static func validateProfilingArgs(_ parsedOptions: inout ParsedOptions,
fileSystem: FileSystem,
Expand Down
7 changes: 6 additions & 1 deletion Sources/SwiftDriver/Jobs/Planning.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -660,7 +660,12 @@ extension Driver {
try resolveDependencyModulePaths(dependencyGraph: &dependencyGraph)

if parsedOptions.hasArgument(.printExplicitDependencyGraph) {
try stdoutStream <<< dependencyGraph.toJSONString()
let outputFormat = parsedOptions.getLastArgument(.explicitDependencyGraphFormat)?.asSingle
if outputFormat == nil || outputFormat == "json" {
try stdoutStream <<< dependencyGraph.toJSONString()
} else if outputFormat == "dot" {
DOTModuleDependencyGraphSerializer(dependencyGraph).writeDOT(to: &stdoutStream)
}
stdoutStream.flush()
}

Expand Down
12 changes: 10 additions & 2 deletions Sources/SwiftOptions/Options.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ extension Option {
public static let embedTbdForModule: Option = Option("-embed-tbd-for-module", .separate, attributes: [.frontend], helpText: "Embed symbols from the module in the emitted tbd file")
public static let emitAbiDescriptorPath: Option = Option("-emit-abi-descriptor-path", .separate, attributes: [.frontend, .noDriver], metaVar: "<path>", helpText: "Output the ABI descriptor of current module to <path>")
public static let emitAssembly: Option = Option("-emit-assembly", .flag, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild], helpText: "Emit assembly file(s) (-S)", group: .modes)
public static let emitAst: Option = Option("-emit-ast", .flag, alias: Option.dumpAst, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild])
Copy link
Contributor

@bnbarham bnbarham Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crazy how many options we add. Didn't I only just run this? This also means that these were only added for the old driver :(

Copy link
Contributor Author

@artemcm artemcm Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My observation is that most of these are .frontend flags, so at least they do not need the logic modified here.

public static let emitBc: Option = Option("-emit-bc", .flag, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild], helpText: "Emit LLVM BC file(s)", group: .modes)
public static let emitClangHeaderPath: Option = Option("-emit-clang-header-path", .separate, alias: Option.emitObjcHeaderPath, attributes: [.frontend, .noDriver, .noInteractive, .argumentIsPath, .supplementaryOutput], helpText: "Emit an Objective-C and C++ header file to <path>")
public static let emitConstValuesPath: Option = Option("-emit-const-values-path", .separate, attributes: [.frontend, .noInteractive, .argumentIsPath, .supplementaryOutput], metaVar: "<path>", helpText: "Emit the extracted compile-time known values to <path>")
Expand Down Expand Up @@ -275,6 +276,7 @@ extension Option {
public static let emitObject: Option = Option("-emit-object", .flag, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild], helpText: "Emit object file(s) (-c)", group: .modes)
public static let emitParseableModuleInterfacePath: Option = Option("-emit-parseable-module-interface-path", .separate, alias: Option.emitModuleInterfacePath, attributes: [.helpHidden, .frontend, .noInteractive, .argumentIsPath, .supplementaryOutput])
public static let emitParseableModuleInterface: Option = Option("-emit-parseable-module-interface", .flag, alias: Option.emitModuleInterface, attributes: [.helpHidden, .noInteractive, .supplementaryOutput])
public static let emitParse: Option = Option("-emit-parse", .flag, alias: Option.dumpParse, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild])
public static let emitPch: Option = Option("-emit-pch", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Emit PCH for imported Objective-C header file", group: .modes)
public static let emitPcm: Option = Option("-emit-pcm", .flag, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild], helpText: "Emit a precompiled Clang module from a module map", group: .modes)
public static let emitPrivateModuleInterfacePath: Option = Option("-emit-private-module-interface-path", .separate, attributes: [.helpHidden, .frontend, .noInteractive, .argumentIsPath, .supplementaryOutput], metaVar: "<path>", helpText: "Output private module interface file to <path>")
Expand Down Expand Up @@ -319,7 +321,7 @@ extension Option {
public static let enableExperimentalAsyncTopLevel: Option = Option("-enable-experimental-async-top-level", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable experimental concurrency in top-level code")
public static let enableExperimentalConcisePoundFile: Option = Option("-enable-experimental-concise-pound-file", .flag, attributes: [.frontend, .moduleInterface], helpText: "Enable experimental concise '#file' identifier")
public static let enableExperimentalConcurrency: Option = Option("-enable-experimental-concurrency", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable experimental concurrency model")
public static let enableExperimentalCxxInterop: Option = Option("-enable-experimental-cxx-interop", .flag, attributes: [.helpHidden, .frontend], helpText: "Enable experimental C++ interop code generation and config directives")
public static let enableExperimentalCxxInterop: Option = Option("-enable-experimental-cxx-interop", .flag, attributes: [.helpHidden, .frontend, .moduleInterface], helpText: "Enable experimental C++ interop code generation and config directives")
public static let enableExperimentalDistributed: Option = Option("-enable-experimental-distributed", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable experimental 'distributed' actors and functions")
public static let enableExperimentalEagerClangModuleDiagnostics: Option = Option("-enable-experimental-eager-clang-module-diagnostics", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable experimental eager diagnostics reporting on the importability of all referenced C, C++, and Objective-C libraries")
public static let enableExperimentalFeature: Option = Option("-enable-experimental-feature", .separate, attributes: [.frontend], helpText: "Enable an experimental feature")
Expand Down Expand Up @@ -386,6 +388,8 @@ extension Option {
public static let experimentalSkipNonInlinableFunctionBodiesWithoutTypes: Option = Option("-experimental-skip-non-inlinable-function-bodies-without-types", .flag, attributes: [.helpHidden, .frontend], helpText: "Skip work on non-inlinable function bodies that do not declare nested types")
public static let experimentalSkipNonInlinableFunctionBodies: Option = Option("-experimental-skip-non-inlinable-function-bodies", .flag, attributes: [.helpHidden, .frontend], helpText: "Skip type-checking and SIL generation for non-inlinable function bodies")
public static let experimentalSpiImports: Option = Option("-experimental-spi-imports", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Enable experimental support for SPI imports")
public static let experimentalSpiOnlyImports: Option = Option("-experimental-spi-only-imports", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Enable use of @_spiOnly imports")
public static let explicitDependencyGraphFormat: Option = Option("-explicit-dependency-graph-format=", .joined, attributes: [.helpHidden, .doesNotAffectIncrementalBuild], helpText: "Specify the explicit dependency graph output format to either 'json' or 'dot'")
public static let explicitInterfaceModuleBuild: Option = Option("-explicit-interface-module-build", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Use the specified command-line to build the module from interface, instead of flags specified in the interface")
public static let driverExplicitModuleBuild: Option = Option("-explicit-module-build", .flag, attributes: [.helpHidden], helpText: "Prebuild module dependencies to make them explicit")
public static let explicitSwiftModuleMap: Option = Option("-explicit-swift-module-map-file", .separate, attributes: [.frontend, .noDriver], metaVar: "<path>", helpText: "Specify a JSON file containing information of explicit Swift modules")
Expand Down Expand Up @@ -545,7 +549,7 @@ extension Option {
public static let printAst: Option = Option("-print-ast", .flag, attributes: [.frontend, .noInteractive, .doesNotAffectIncrementalBuild], helpText: "Parse and type-check input file(s) and pretty print AST(s)", group: .modes)
public static let printClangStats: Option = Option("-print-clang-stats", .flag, attributes: [.frontend, .noDriver], helpText: "Print Clang importer statistics")
public static let printEducationalNotes: Option = Option("-print-educational-notes", .flag, attributes: [.frontend, .doesNotAffectIncrementalBuild], helpText: "Include educational notes in printed diagnostic output, if available")
public static let printExplicitDependencyGraph: Option = Option("-print-explicit-dependency-graph", .flag, attributes: [.helpHidden], helpText: "Print the result of module dependency scanning after external module resolution to output")
public static let printExplicitDependencyGraph: Option = Option("-print-explicit-dependency-graph", .flag, attributes: [.helpHidden, .doesNotAffectIncrementalBuild], helpText: "Print the result of module dependency scanning after external module resolution to output")
public static let printInstCounts: Option = Option("-print-inst-counts", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Before IRGen, count all the various SIL instructions. Must be used in conjunction with -print-stats.")
public static let printLlvmInlineTree: Option = Option("-print-llvm-inline-tree", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Print the LLVM inline tree.")
public static let printModule: Option = Option("-print-module", .flag, attributes: [.noDriver], helpText: "Print module names in diagnostics")
Expand Down Expand Up @@ -948,6 +952,7 @@ extension Option {
Option.embedTbdForModule,
Option.emitAbiDescriptorPath,
Option.emitAssembly,
Option.emitAst,
Option.emitBc,
Option.emitClangHeaderPath,
Option.emitConstValuesPath,
Expand Down Expand Up @@ -987,6 +992,7 @@ extension Option {
Option.emitObject,
Option.emitParseableModuleInterfacePath,
Option.emitParseableModuleInterface,
Option.emitParse,
Option.emitPch,
Option.emitPcm,
Option.emitPrivateModuleInterfacePath,
Expand Down Expand Up @@ -1098,6 +1104,8 @@ extension Option {
Option.experimentalSkipNonInlinableFunctionBodiesWithoutTypes,
Option.experimentalSkipNonInlinableFunctionBodies,
Option.experimentalSpiImports,
Option.experimentalSpiOnlyImports,
Option.explicitDependencyGraphFormat,
Option.explicitInterfaceModuleBuild,
Option.driverExplicitModuleBuild,
Option.explicitSwiftModuleMap,
Expand Down
95 changes: 93 additions & 2 deletions Tests/SwiftDriverTests/ExplicitModuleBuildTests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1076,8 +1076,7 @@ final class ExplicitModuleBuildTests: XCTestCase {
return
}
guard try dependencyOracle.supportsScannerDiagnostics() else {
XCTSkip("libSwiftScan does not support diagnostics query.")
return
throw XCTSkip("libSwiftScan does not support diagnostics query.")
}

try withTemporaryDirectory { path in
Expand Down Expand Up @@ -1240,6 +1239,98 @@ final class ExplicitModuleBuildTests: XCTestCase {
}
}

func testPrintingExplicitDependencyGraph() throws {
try withTemporaryDirectory { path in
let main = path.appending(component: "testPrintingExplicitDependencyGraph.swift")
try localFileSystem.writeFileContents(main) {
$0 <<< "import C;"
$0 <<< "import E;"
$0 <<< "import G;"
}
let cHeadersPath: AbsolutePath = testInputsPath.appending(component: "ExplicitModuleBuilds").appending(component: "CHeaders")
let swiftModuleInterfacesPath: AbsolutePath = testInputsPath.appending(component: "ExplicitModuleBuilds").appending(component: "Swift")
let sdkArgumentsForTesting = (try? Driver.sdkArgumentsForTesting()) ?? []

let baseCommandLine = ["swiftc",
Copy link
Contributor

@bnbarham bnbarham Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not particularly relevant for this review, but IMO it'd be really nice if we could go from typed argument -> string. Ie. instead of -explicit-module-build just .explicitModuleBuild.str (or something similar). Though I suppose there's an argument to be made for making renaming arguments fairly difficult (ie. would have to rename in tests) so that it's not done accidentally 🤷

"-target", "x86_64-apple-macosx11.0",
"-I", cHeadersPath.nativePathString(escaped: true),
"-I", swiftModuleInterfacesPath.nativePathString(escaped: true),
main.nativePathString(escaped: true)] + sdkArgumentsForTesting
do {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-print-explicit-dependency-graph"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertTrue(diagnosticEngine.hasErrors)
XCTAssertEqual(diagnosticEngine.diagnostics.first?.message.data.description,
"'-print-explicit-dependency-graph' cannot be specified if '-explicit-module-build' is not present")
}
do {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-explicit-module-build",
"-explicit-dependency-graph-format=json"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertTrue(diagnosticEngine.hasErrors)
XCTAssertEqual(diagnosticEngine.diagnostics.first?.message.data.description,
"'-explicit-dependency-graph-format=' cannot be specified if '-print-explicit-dependency-graph' is not present")
}
do {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-explicit-module-build",
"-print-explicit-dependency-graph",
"-explicit-dependency-graph-format=watercolor"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertTrue(diagnosticEngine.hasErrors)
XCTAssertEqual(diagnosticEngine.diagnostics.first?.message.data.description,
"unsupported argument \'watercolor\' to option \'-explicit-dependency-graph-format=\'")
}

let _ = try withHijackedOutputStream {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-explicit-module-build",
"-print-explicit-dependency-graph",
"-explicit-dependency-graph-format=json"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertFalse(diagnosticEngine.hasErrors)
}

let output = try withHijackedOutputStream {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-explicit-module-build",
"-print-explicit-dependency-graph",
"-explicit-dependency-graph-format=json"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertFalse(diagnosticEngine.hasErrors)
}
XCTAssertTrue(output.contains("\"mainModuleName\" : \"testPrintingExplicitDependencyGraph\","))

let output2 = try withHijackedOutputStream {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-explicit-module-build",
"-print-explicit-dependency-graph",
"-explicit-dependency-graph-format=dot"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertFalse(diagnosticEngine.hasErrors)
}
XCTAssertTrue(output2.contains("\"testPrintingExplicitDependencyGraph\" [shape=box, style=bold, color=navy"))

let output3 = try withHijackedOutputStream {
let diagnosticEngine = DiagnosticsEngine()
var driver = try Driver(args: baseCommandLine + ["-explicit-module-build",
"-print-explicit-dependency-graph"],
diagnosticsEngine: diagnosticEngine)
let _ = try driver.planBuild()
XCTAssertFalse(diagnosticEngine.hasErrors)
}
XCTAssertTrue(output3.contains("\"mainModuleName\" : \"testPrintingExplicitDependencyGraph\","))
}
}

func testDependencyGraphDotSerialization() throws {
let (stdlibPath, shimsPath, toolchain, hostTriple) = try getDriverArtifactsForScanning()
let dependencyOracle = InterModuleDependencyOracle()
Expand Down
Loading