Rebase main with @stackotter's same package product dependencies #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It seems like for reasons we have a fallback scratch directory even if we couldn't determine a package root, but it doesn't seem as if we should lock that since it is likely not the scratch directory we want.
SwiftPM uses ProgressAnimation from TSCUtilities in a variety of inconsistent manners across various swift-something commands. This commit replaces the uses of ProgressAnimation throughout SwiftPM with common entrypoints and lays the ground work for creating multi-line parallel progress animations as seen in tools like `bazel build` and `docker build`. --------- Co-authored-by: Max Desiatov <[email protected]>
This is obviously dangerous, but arguably has been the default behavior of SwiftPM for many years, so offering an escape hatch seems appropriate.
We use the "-latest" signifier to avoid confusion here, but basically this means we'll use any "-prerelease" tags and also the final release once it is out.
Seems like a good time to ship this since I am wrapping up my last few PRs and this way I don't get added as reviewer to new PRs which will set the correct expectation that I won't be doing any future code reviews.
Addresses swiftlang#7170: this adds an option to the `package-registry` `set` and `publish` commands to allow using an HTTP registry endpoint, bypassing the existing requirement that package registries use HTTPS. ### Motivation: In short, the motivation is to allow publishing to a registry that is hosted on a local machine. It is difficult to host a fully TLS-compliant HTTPS endpoint on developer machines at scale. See more detail in swiftlang#7170 ### Modifications: * Adds a new `allowInsecureHTTP` flag to the Registry model. * Adds a new `--allow-insecure-http` flag to the `package-registry` `set` and `publish` commands. * Adds tests that validate the behavior of the model and command additions. * Adds tests that validate the requested constraints in swiftlang#7170 regarding HTTP endpoints, such as not being able to publish to an HTTP endpoint if there is authentication configured for it, and `login` command requiring HTTPS. ### Result: We will be able to use the `swift` CLI to publish to our HTTP registries locally hosted on our development machines. As a result, we will be able to migrate to the official tooling instead of needing to continue using home-rolled publish tooling.
### Motivation: The test runner build was failing on WASI after the introduction of the experimental summary output. ### Modifications: This PR fixes several minor issues to pass `--build-tests` build on WASI: * Missing `WASILibc` import * The use of `flock` which is not available on WASI. * Signature incompatibility of `XCTMain` on WASI. ### Result: `swift build --build-tests --triple wasm32-unknown-wasi` will pass when compiler supports the target.
This should allow satisfying a package dependency with a package in the SDK (or toolchain) based on metadata. If the given prebuilt library is version-compatible, we essentially elide the dependency from the graph. If it is not, we will fetch and build the dependency as normal. If a library with associated metadata is linked without a corresponding package dependency, we'll emit a warning.
### Motivation: Not supporting macros in cross-compilation is a major limitation, especially for libraries like https://github.com/apple/swift-mmio. ### Modifications: Added `enum BuildTriple { case tools, destination }` and `var buildTriple: BuildTriple` on `ResolvedTarget` and `ResolvedProduct`. We're not using "host" and "target" triple terminology in this enum, as that clashes with build system "targets" and can lead to confusion in this context. Corresponding value is assigned to this property depending on target and product type: `tools` for macros, plugins, and their dependencies, `destination` for everything else (the default). Based on this property we can choose between `buildParameters.hostTriple` and `buildParameters.targetTriple` during build plan construction. Additionally, the resolved products and resolved targets graph is now constructed in a way that allows certain targets and products to be built twice: once for host triple, once for target triple if needed. This required modifying build description and build manifest generation to distinguish these products and targets from each other that are built twice. Artifacts built for the host now have `-tools` suffix appended to their names. This cascaded into making changes throughout the code base for build tool plugins and package plugins handling, which constructed their paths in an ad-hoc manner without accounting for possible changes to these names. Also added `CrossCompilationPackageGraphTests` and `CrossCompilationBuildPlanTests` to verify the changes made are applied correctly. ### Result: Resolves swiftlang#6950 Resolves rdar://105991372
swiftlang#7337 is breaking toolchain builds, so we should pass `SKIP_RESOURCE_SUPPORT` by default to avoid that.
`assertMacroExpansion` requires macros to be compiled for target platform and evaluated in-process. We can't handle the case of indirect macro dependencies cleanly, those will be still compiled for the host. Supersedes swiftlang#7349. This doesn't contain tests to expedite the fix and unblock broken CI jobs, but I've verified locally that test products of the `Examples` package in `apple/swift-syntax` repo can be built successfully and these tests all pass.
### Motivation: Without passing this flag, Swift Driver links such products as if they're not built for embedded platforms at all, passing incorrect flags to the linker. This is not reproducible when using plain `swiftc`, as that combines both compile and link jobs by default, but SwiftPM prefers to run those separately, which requires this special handling. ### Modifications: Directly checking in `ProductBuildDescription/linkArguments` whether all of the product's targets are built in the embedded mode. If so, we're passing the embedded mode flag to Swift Driver. Unfortunately, `BuildSettings.AssignmentTable` is formed too early in the build process right in `PackageModel`, while we still need to run checks specific build settings quite late in the build stage. Because of that, there's no clean way to check if `Embedded` flag is passed other than directly rendering `BuildSettings.Scope` to a string and running a substring check on that. In the future we should move `BuildSettings` out of `PackageModel`, ensuring that SwiftPM clients don't rely on this behavior anymore. ### Result: Products that have targets using Embedded Swift can be built with SwiftPM, assuming that Swift Driver handles other linker flags correctly.
…#7355) Absence of this extension on produced binaries makes it inconvenient to work with triple, as you have to add that extension manually after the fact. For examples, HTTP servers frequently don't infer correct mime-type for these binaries, which confuses browsers that try to load these binaries.
Enable `--gc-sections` back for WebAssembly targets ### Motivation: We disabled `--gc-sections` for Wasm targets due to a lack of features in the object file format. However, we recently added the missing piece in wasm object file format, so we no longer have any reason to disable it. ### Modifications: This effectively reverts 3a366cc. After swiftlang/swift#71768, we don't have any reason to disable `--gc-sections` for WebAssembly targets. Note that the new wasm segment flags are only supported by the latest LLVM and wasm-ld, but we can assume that toolchains or Swift SDKs for WebAssembly have wasm-ld built from the latest LLVM. ### Result: Allow dead code stripping at link time for WebAssembly targets.
…ang#7324) We're seeing a large amount of issues created that are unrelated to this repository. We need to add a clear warning to our issue template that such issues will be closed.
786c513 added support for symlinks, but didn't handle `executableBlob` in the added `switch`. `executableBlob` is the returned type for a file with the executable bit set, so needs to be handled in the same way as a regular `blob`.
XCBuildSupport is not available on non-Darwin platforms, especially Windows, where we have to have full CMake support at the moment. Maintaining XCBuildSupport in these unsupported configurations adds unnecessary overhead, especially in cases like swiftlang#7258, where we have to add new dependencies only when `XCBuildSupport` is available. We should exclude from CMake builds to reduce this maintenance overhead.
This is a stripped down version of swiftlang#6624 that modifies only the test suite and doesn't make any substantial to `ManifestLoader.swift` other than providing an `async` overload in `extension ManifestLoaderProtocol` that uses `withCheckedThrowingContinuation` to bridge into the existing implementation.
These libraries are not usually available for embedded platforms. A workaround of passing this flag via `unsafeFlags` in `Package.swift` still remains after this change.
This is a follow up to a corresponding toolchain PR swiftlang/swift#71707.
We have to operate on two different graphs in the `PackageGraph` module: 1. A graph of packages in PubGrub package resolution code; 2. A graph of resolved products and modules that belong to those products. Currently the second graph is misleadingly called `PackageGraph`, although that name is much more suitable for the first graph. This naming is confusing and unfortunate, and can be especially misleading for first-time contributors. We should better document the SwiftPM resolution and build pipeline in the future, but cleaning up naming is the first step. I'm keeping old names as deprecated overloads or typealiases to make migration easier for SwiftPM clients. This renaming has no impact on any public stable modules like `PackageDescription`. In the short term we should also split the `PackageGraph` module into `PubGrub` and `ModulesGraph` modules, but for now I'm renaming a single type to keep the PR manageable.
`package` access modifier was previously not supported in `swift build --build-system xcode`. This causes build issues when attempting to produce Universal binaries for SwiftPM, for example in this job https://ci.swift.org/job/swift-PR-source-compat-suite-debug-macos/1297/consoleFull. The reason was that `-package-name` option was not added to `OTHER_SWIFT_FLAGS` in `PIFBuilder.swift`. Additionally, in llbuild support code we were shelling out to Swift Driver for every target to check whether `-package-name` is supported. Now with `-package-name` options calculation generalized across build systems, Swift Driver checks are done once per `BuildParameters` initialization, which reduces excessive shelling for llbuild. Resolves rdar://120925895.
This removes an existing point of confusion, where `SwiftBuildTool` that handles `swift build` CLI invocations could be misunderstood as something responsible for build tools in plugins. It becomes especially problematic when working on the plugins codebase. Additionally, `SwiftTool` is a grab bag reference type for storing state injected into a CLI command, so we should rename it to `SwiftCommandState` for clarity.
Due to a bug in Swift 5.9, we can't have multiple executables with `async` entrypoints linked into test products. This was the reason why we weren't able to make `swift-build` entrypoint `async`. Since we have a single `swift-package-manager` executable anyway, we can link that one into tests instead, and pass "simulated" executable name via an environment variable. This allows us to work around the bug and start using structured concurrency from `swift-build` entrypoint. This change is a prerequisite for swiftlang#7367.
This adds a dependency on the [`package-benchmark`](https://github.com/ordo-one/package-benchmark) library, isolated to the new `Benchmarks` folder. Thus it doesn't touch CMake or self-hosted builds, and one can only pull the library if they run `swift package benchmark` in the `Benchmarks` directory. A single benchmark is added that loads SwiftPM's `Package.swift` and builds a package graph for it.
This function always returns `true` result right now, which is discarded. We simply shouldn't return anything at all.
…lsp API (swiftlang#7492) This is used so we don’t search for tests in targets of package dependencies. Companion of swiftlang/sourcekit-lsp#1201 rdar://126965614
…kages While inserting updated `.tools` targets to `modulesToPackages` the logic should use the package where the target resides instead of the package where it's referenced. Resolves: rdar://127369576
SDK macro plugins are now provided within `Platforms/<Platform>.platform/Developer/usr/lib/swift/host/plugins` now, which is always added by the driver.
…iftlang#7525) The exportability rules around `package` access level are updated in a way that an `@_spi public` can't be bound to a `package` var, similar to how `@_spi public` can't be bound to a `public` var. For reference, see swiftlang/swift#73161. This PR corrects such use case.
I fixed warning appears when building with Swift 6.
Tools and scripts may rely on command output to remain parseable, so printing these warnings on `stdout` can break such workflows. We should output these warnings on `stderr` instead. As these warnings are emitted before any logging is available, we're using `fputs` for output on `stderr`
…lang#7481) Introduce a command-line argument `--testing-library` to the add-target command to specify which test library to generate the test for. This can be 'xctest' (the prior XCTest behavior), 'swift-testing' (to use the new swift-testing library), or 'none' (for no test library at all). For the new swift-testing generation, also add the appropriate package and test target dependency, along with a stub testsuite to start from. Fixes swiftlang#7478
…st` (swiftlang#7534) This PR adds two undocumented/unsupported experimental options to `swift test` to allow passing through JSON files/streams for input and output. These streams are still under development in swift-testing, so these options are not yet supported, but this change will allow interested coders to experiment with them. --------- Co-authored-by: Max Desiatov <[email protected]>
… introduce a cycle in a build graph (swiftlang#7530) ### Motivation: It should be possible for packages to depend on each other if such dependence doesn't introduce cycles in the build graph. ### Modifications: - Introduced a new DFS method to walk over graphs that breaks cycles. - Replaces use of `findCycle` + `topologicalSort` with `DFS` while building manifest and package graphs. This allows cycles in dependencies to be modeled correctly. - Removes some of the redundant data transformations from modules graph. - Modifies `ResolvedPackage` to carry identities of its dependencies instead of resolved dependencies themselves. This helps to simplify logic in `createResolvedPackages`. - Adds detection of target cycles across packages. ### Result: Makes it possible for package A to depend on package B and B to depend on A if their targets don't form a cycle.
…ang#7535) Per [the proposal text](https://github.com/apple/swift-evolution/blob/main/proposals/0387-cross-compilation-destinations.md#swift-sdk-installation-and-configuration): > `swift sdk configure <identifier> <target-triple>`, which allows users to provide additional search paths and toolsets to be used subsequently when building with a given Swift SDK. Specifically, multiple `--swift-resources-path`, `--include-search-path`, `--library-search-path`, and `--toolset` options with corresponding paths can be provided, which then will be stored as configuration for this Swift SDK. `swift sdk configure <identifier> --show-configuration` will print currently set paths, while `swift sdk configure <identifier> --reset` will reset all of those at once.
…ng#7542) rdar://127782783
- Add an API to get the name of a target from SourceKit-LSP - We will need this API to prepare a target for indexing. - Add an API to get all the targets in a build description / modules graph in a topological order - We need this for background preparation for indexing so that we can prepare and index lower-level targets first.
We missed updating this when adding 6.0. The change itself is extremely simple, with the bulk being updates to our tests so they don't fail on old compiler versions (where language version "6" is unknown). Also removes disabling the concurrency and string processing imports from manifest loading, as this was causing the `package` global to not be main actor isolated by default and thus error in Swift 6. They seemed to have been disabled to avoid erroring in SDKs that didn't have them, but that is definitely not an issue any longer.
…sion` The value of this field is computed based on the tools version of the manifest and is intended to be used as a fallback if there is no swift language version specified in the build settings for a swift target.
… into `toolsSwiftVersion`
…wiftlang#7551) Follow-on to swiftlang#7534. swift-testing has an additional argument specifying the schema version of the JSON being used that we also need to pass through SwiftPM. See: swiftlang/swift-testing#383
…SION` declaration Instead of putting it into `OTHER_SWIFT_FLAGS`, let's introduce a dedicated declaration scope - `SWIFT_VERSION` to ease handling and discovery. Resolves: rdar://127883018
…ift 6 mode (swiftlang#7553) - Mark `FileRuleDescription` as `Sendable` - Make `Platform` sendable and make `Platform.current` a constant
This shouldn’t be needed anymore with swiftlang#7504.
…tlang#7555) If we have a package like this ```swift let package = Package( name: "MyLibrary", targets: [ .target(name: "Lib"), .executableTarget(name: "MyExec", dependencies: ["Lib"]), .plugin( name: "MyPlugin", capability: .command( intent: .sourceCodeFormatting(), permissions: [] ), dependencies: ["MyExec"] ) ] ) ``` Then the package graph contains two targets for `Lib`. We need to be able to differentiate them in SourceKit-LSP. For that, we need to expose the build triple of a `BuildTarget`.
…#7552) As a convenience for when we want to be able to add new target dependencies, introduce a manifest editing API that adds a new target dependency to an existing target in the manifest.
…top-level code owners
This setting is going to be used if there are no non-default assignments that match build environment conditions.
…of a build setting Start using "default" assignments to represent tools version based swift language version setting that is going to be produced by `BuildSettings.Scope` if no other assignments match build environment conditions. This removes ambiguity from `-swift-version` selection for downstream clients.
Since `toolSwiftVersion` is gone and swift language version is now handled by the build settings we need to adjust `PIFBuilder` to handle that properly. PIF build settings can handle per-target platform specific settings which are now expressible in SwiftPM via `.swiftLanguageVersion` build setting, handling of such settings is implemented in this commit.
…s (no cycle checks or tests)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.