Skip to content

[Dependency Scanning] On failure to locate a module, attempt to diagnose if binary dependencies contain search paths with this module. #81919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 12, 2025
Merged

[Dependency Scanning] On failure to locate a module, attempt to diagnose if binary dependencies contain search paths with this module. #81919

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
90f2fba
[Dependency Scanning] On failure to locate a module, attempt to diagn…
artemcm Jun 3, 2025
aedc2fa
[Dependency Scanning] Serialized Swift binary module serialized searc…
artemcm Jun 3, 2025
d4abba1
Add a new diagnostic group and documentation for the module-not-found…
artemcm Jun 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions include/swift/AST/DiagnosticGroups.def
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ GROUP(ExistentialAny, "existential-any")
GROUP(ExistentialMemberAccess, "existential-member-access-limitations")
GROUP(IsolatedConformances, "isolated-conformances")
GROUP(MemberImportVisibility, "member-import-visibility")
GROUP(MissingModuleOnKnownPaths, "missing-module-on-known-paths")
GROUP(MultipleInheritance, "multiple-inheritance")
GROUP(MutableGlobalVariable, "mutable-global-variable")
GROUP(NominalTypes, "nominal-types")
Expand Down
6 changes: 6 additions & 0 deletions include/swift/AST/DiagnosticsSema.def
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2372,12 +2372,18 @@ ERROR(alignment_not_power_of_two,none,

// Dependency Scanning
ERROR(dependency_scan_module_not_found, none, "Unable to find module dependency: '%0'", (StringRef))
GROUPED_ERROR(dependency_scan_module_not_found_on_specified_search_paths, MissingModuleOnKnownPaths, none,
"Compilation search paths unable to resolve module dependency: '%0'", (StringRef))
NOTE(unresolved_import_location,none,
"also imported here", ())
NOTE(dependency_as_imported_by_main_module,none,
"a dependency of main module '%0'", (StringRef))
NOTE(dependency_as_imported_by, none,
"a dependency of %select{Swift|Clang}2 module '%0': '%1'", (StringRef, StringRef, bool))
NOTE(inherited_search_path_resolves_module,none,
"'%0' can be found using a search path that was specified when building module '%1' ('%2'). "
"This search path was not explicitly specified on the current compilation.", (StringRef, StringRef, StringRef))

ERROR(clang_dependency_scan_error, none, "Clang dependency scanner failure: %0", (StringRef))
ERROR(clang_header_dependency_scan_error, none, "Bridging header dependency scan failure: %0", (StringRef))

Expand Down
27 changes: 18 additions & 9 deletions include/swift/AST/ModuleDependencies.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include "swift/AST/LinkLibrary.h"
#include "swift/Basic/CXXStdlibKind.h"
#include "swift/Basic/LLVM.h"
#include "swift/Serialization/Validation.h"
#include "clang/CAS/CASOptions.h"
#include "clang/Tooling/DependencyScanning/DependencyScanningService.h"
#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h"
Expand Down Expand Up @@ -406,15 +407,18 @@ class SwiftBinaryModuleDependencyStorage
StringRef sourceInfoPath,
ArrayRef<ScannerImportStatementInfo> moduleImports,
ArrayRef<ScannerImportStatementInfo> optionalModuleImports,
ArrayRef<LinkLibrary> linkLibraries, StringRef headerImport,
StringRef definingModuleInterface, bool isFramework, bool isStatic,
StringRef moduleCacheKey, StringRef userModuleVersion)
ArrayRef<LinkLibrary> linkLibraries,
ArrayRef<serialization::SearchPath> serializedSearchPaths,
StringRef headerImport, StringRef definingModuleInterface,
bool isFramework, bool isStatic, StringRef moduleCacheKey,
StringRef userModuleVersion)
: ModuleDependencyInfoStorageBase(ModuleDependencyKind::SwiftBinary,
moduleImports, optionalModuleImports,
linkLibraries, moduleCacheKey),
compiledModulePath(compiledModulePath), moduleDocPath(moduleDocPath),
sourceInfoPath(sourceInfoPath), headerImport(headerImport),
definingModuleInterfacePath(definingModuleInterface),
serializedSearchPaths(serializedSearchPaths),
isFramework(isFramework), isStatic(isStatic),
userModuleVersion(userModuleVersion) {}

Expand All @@ -441,6 +445,9 @@ class SwiftBinaryModuleDependencyStorage
/// Source files on which the header inputs depend.
std::vector<std::string> headerSourceFiles;

/// Search paths this module was built with which got serialized
std::vector<serialization::SearchPath> serializedSearchPaths;

/// (Clang) modules on which the header inputs depend.
std::vector<ModuleDependencyID> headerModuleDependencies;

Expand Down Expand Up @@ -613,15 +620,17 @@ class ModuleDependencyInfo {
StringRef sourceInfoPath,
ArrayRef<ScannerImportStatementInfo> moduleImports,
ArrayRef<ScannerImportStatementInfo> optionalModuleImports,
ArrayRef<LinkLibrary> linkLibraries, StringRef headerImport,
StringRef definingModuleInterface, bool isFramework,
bool isStatic, StringRef moduleCacheKey, StringRef userModuleVer) {
ArrayRef<LinkLibrary> linkLibraries,
ArrayRef<serialization::SearchPath> serializedSearchPaths,
StringRef headerImport, StringRef definingModuleInterface,
bool isFramework, bool isStatic, StringRef moduleCacheKey,
StringRef userModuleVer) {
return ModuleDependencyInfo(
std::make_unique<SwiftBinaryModuleDependencyStorage>(
compiledModulePath, moduleDocPath, sourceInfoPath, moduleImports,
optionalModuleImports, linkLibraries, headerImport,
definingModuleInterface,isFramework, isStatic, moduleCacheKey,
userModuleVer));
optionalModuleImports, linkLibraries, serializedSearchPaths,
headerImport, definingModuleInterface,isFramework, isStatic,
moduleCacheKey, userModuleVer));
}

/// Describe the main Swift module.
Expand Down
18 changes: 18 additions & 0 deletions include/swift/DependencyScan/ModuleDependencyScanner.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,26 @@ class ModuleDependencyScanner {
template <typename Function, typename... Args>
auto withDependencyScanningWorker(Function &&F, Args &&...ArgList);

/// Use the scanner's ASTContext to construct an `Identifier`
/// for a given module name.
Identifier getModuleImportIdentifier(StringRef moduleName);

/// Diagnose scanner failure and attempt to reconstruct the dependency
/// path from the main module to the missing dependency.
void diagnoseScannerFailure(const ScannerImportStatementInfo &moduleImport,
const ModuleDependenciesCache &cache,
std::optional<ModuleDependencyID> dependencyOf);

/// Assuming the \c `moduleImport` failed to resolve,
/// iterate over all binary Swift module dependencies with serialized
/// search paths and attempt to diagnose if the failed-to-resolve module
/// can be found on any of them. Returns the path containing
/// the module, if one is found.
std::optional<std::pair<ModuleDependencyID, std::string>>
attemptToFindResolvingSerializedSearchPath(
const ScannerImportStatementInfo &moduleImport,
const ModuleDependenciesCache &cache, const SourceLoc &importLoc);

private:
const CompilerInvocation &ScanCompilerInvocation;
ASTContext &ScanASTContext;
Expand Down
17 changes: 16 additions & 1 deletion include/swift/DependencyScan/SerializedModuleDependencyCacheFormat.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ using llvm::BCVBR;
const unsigned char MODULE_DEPENDENCY_CACHE_FORMAT_SIGNATURE[] = {'I', 'M', 'D','C'};
const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MAJOR = 9;
/// Increment this on every change.
const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MINOR = 2;
const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MINOR = 3;

/// Various identifiers in this format will rely on having their strings mapped
/// using this ID.
Expand Down Expand Up @@ -78,6 +78,7 @@ using ModuleCacheKeyIDField = IdentifierIDField;
using ImportArrayIDField = IdentifierIDField;
using LinkLibrariesArrayIDField = IdentifierIDField;
using MacroDependenciesArrayIDField = IdentifierIDField;
using SearchPathArrayIDField = IdentifierIDField;
using FlagIDArrayIDField = IdentifierIDField;
using DependencyIDArrayIDField = IdentifierIDField;
using SourceLocationIDArrayIDField = IdentifierIDField;
Expand All @@ -101,6 +102,8 @@ enum {
LINK_LIBRARY_ARRAY_NODE,
MACRO_DEPENDENCY_NODE,
MACRO_DEPENDENCY_ARRAY_NODE,
SEARCH_PATH_NODE,
SEARCH_PATH_ARRAY_NODE,
IMPORT_STATEMENT_NODE,
IMPORT_STATEMENT_ARRAY_NODE,
OPTIONAL_IMPORT_STATEMENT_ARRAY_NODE,
Expand Down Expand Up @@ -169,6 +172,17 @@ using MacroDependencyLayout =
using MacroDependencyArrayLayout =
BCRecordLayout<MACRO_DEPENDENCY_ARRAY_NODE, IdentifierIDArryField>;

// A record for a serialized search pathof a given dependency
// node (Swift binary module dependency only).
using SearchPathLayout =
BCRecordLayout<SEARCH_PATH_NODE, // ID
IdentifierIDField, // path
IsFrameworkField, // isFramework
IsSystemField // isSystem
>;
using SearchPathArrayLayout =
BCRecordLayout<SEARCH_PATH_ARRAY_NODE, IdentifierIDArryField>;
Copy link
Contributor

@cachemeifyoucan cachemeifyoucan Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to serialize this? We only serialize successful scanning output right? If the search path changed, I assume the hash will change, thus the entire graph is invalidated.

Copy link
Contributor Author

@artemcm artemcm Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer that at least with small things like these the serialized scanner cache be relatively self-contained. And however contrived, the added cached-missing-module-found-in-serialized-paths.swift test shows one example where we may be re-using prior scan results which are legitimately not invalidated and still end up emitting this diagnostic because a module which was previously on the search paths got removed and can now only be resolved using binary dependency serialized search paths which also got captured in the serialized data.

Copy link
Contributor Author

@artemcm artemcm Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also still want to add a tool that dumps the contents of these files in a human-readable format and that would make a nice diagnostic tool for compiler folk to be able to examine prior scan results and have them capture these details.


// A record capturing information about a given 'import' statement
// captured in a dependency node, including its source location.
using ImportStatementLayout =
Expand Down Expand Up @@ -248,6 +262,7 @@ using SwiftBinaryModuleDetailsLayout =
FileIDField, // definingInterfacePath
IdentifierIDField, // headerModuleDependencies
FileIDArrayIDField, // headerSourceFiles
SearchPathArrayIDField, // serializedSearchPaths
IsFrameworkField, // isFramework
IsStaticField, // isStatic
IdentifierIDField, // moduleCacheKey
Expand Down
Loading