Merge branch 'main' into uts18_testcases

Author: natecook1000

Documentation/Evolution/RegexLiteralPitch.md

@@ -1,4 +1,3 @@
-
 # Regex Type and Overview
 
 - Authors: [Michael Ilseman](https://github.com/milseman)
@@ -225,7 +224,7 @@ func processEntry(_ line: String) -> Transaction? {
 
 The result builder allows for inline failable value construction, which participates in the overall string processing algorithm: returning `nil` signals a local failure and the engine backtracks to try an alternative. This not only relieves the use site from post-processing, it enables new kinds of processing algorithms, allows for search-space pruning, and enhances debuggability.
 
-Swift regexes describe an unambiguous algorithm, were choice is ordered and effects can be reliably observed. For example, a `print()` statement inside the `TryCapture`'s transform function will run whenever the overall algorithm naturally dictates an attempt should be made. Optimizations can only elide such calls if they can prove it is behavior-preserving (e.g. "pure").
+Swift regexes describe an unambiguous algorithm, where choice is ordered and effects can be reliably observed. For example, a `print()` statement inside the `TryCapture`'s transform function will run whenever the overall algorithm naturally dictates an attempt should be made. Optimizations can only elide such calls if they can prove it is behavior-preserving (e.g. "pure").
 
 `CustomMatchingRegexComponent`, discussed in [String Processing Algorithms][pitches], allows industrial-strength parsers to be used a regex components. This allows us to drop the overly-permissive pre-parsing step:
 
@@ -278,14 +277,14 @@ func processEntry(_ line: String) -> Transaction? {
 *Note*: Details on how references work is discussed in [Regex Builders][pitches]. `Regex.Match` supports referring to _all_ captures by position (`match.1`, etc.) whether named or referenced or neither. Due to compiler limitations, result builders do not support forming labeled tuples for named captures.
 
 
-### Algorithms, algorithms everywhere
+### Regex-powered algorithms
 
 Regexes can be used right out of the box with a variety of powerful and convenient algorithms, including trimming, splitting, and finding/replacing all matches within a string.
 
 These algorithms are discussed in [String Processing Algorithms][pitches].
 
 
-### Onward Unicode
+### Unicode handling
 
 A regex describes an algorithm to be ran over some model of string, and Swift's `String` has a rather unique Unicode-forward model. `Character` is an [extended grapheme cluster](https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries) and equality is determined under [canonical equivalence](https://www.unicode.org/reports/tr15/#Canon_Compat_Equivalence).
 
@@ -310,12 +309,12 @@ public struct Regex<Output> {
   /// Match a string in its entirety.
   ///
   /// Returns `nil` if no match and throws on abort
-  public func matchWhole(_ s: String) throws -> Regex<Output>.Match?
+  public func wholeMatch(in s: String) throws -> Regex<Output>.Match?
 
   /// Match part of the string, starting at the beginning.
   ///
   /// Returns `nil` if no match and throws on abort
-  public func matchPrefix(_ s: String) throws -> Regex<Output>.Match?
+  public func prefixMatch(in s: String) throws -> Regex<Output>.Match?
 
   /// Find the first match in a string
   ///
@@ -325,17 +324,17 @@ public struct Regex<Output> {
   /// Match a substring in its entirety.
   ///
   /// Returns `nil` if no match and throws on abort
-  public func matchWhole(_ s: Substring) throws -> Regex<Output>.Match?
+  public func wholeMatch(in s: Substring) throws -> Regex<Output>.Match?
 
   /// Match part of the string, starting at the beginning.
   ///
   /// Returns `nil` if no match and throws on abort
-  public func matchPrefix(_ s: Substring) throws -> Regex<Output>.Match?
+  public func prefixMatch(in s: Substring) throws -> Regex<Output>.Match?
 
   /// Find the first match in a substring
   ///
   /// Returns `nil` if no match is found and throws on abort
-  public func firstMatch(_ s: Substring) throws -> Regex<Output>.Match?
+  public func firstMatch(in s: Substring) throws -> Regex<Output>.Match?
 
   /// The result of matching a regex against a string.
   ///
@@ -344,19 +343,19 @@ public struct Regex<Output> {
   @dynamicMemberLookup
   public struct Match {
     /// The range of the overall match
-    public let range: Range<String.Index>
+    public var range: Range<String.Index> { get }
 
     /// The produced output from the match operation
-    public var output: Output
+    public var output: Output { get }
 
     /// Lookup a capture by name or number
-    public subscript<T>(dynamicMember keyPath: KeyPath<Output, T>) -> T
+    public subscript<T>(dynamicMember keyPath: KeyPath<Output, T>) -> T { get }
 
     /// Lookup a capture by number
     @_disfavoredOverload
     public subscript(
       dynamicMember keyPath: KeyPath<(Output, _doNotUse: ()), Output>
-    ) -> Output
+    ) -> Output { get }
     // Note: this allows `.0` when `Match` is not a tuple.
 
   }
@@ -482,6 +481,7 @@ We're also looking for more community discussion on what the default type system
 
 The actual `Match` struct just stores ranges: the `Substrings` are lazily created on demand. This avoids unnecessary ARC traffic and memory usage.
 
+
 ### `Regex<Match, Captures>` instead of `Regex<Output>`
 
 The generic parameter `Output` is proposed to contain both the whole match (the `.0` element if `Output` is a tuple) and captures. One alternative we have considered is separating `Output` into the entire match and the captures, i.e. `Regex<Match, Captures>`, and using `Void` for for `Captures` when there are no captures.

Documentation/Evolution/RegexLiterals.md

@@ -187,7 +187,7 @@ public protocol CustomMatchingRegexComponent : RegexComponent {
         _ input: String,
         startingAt index: String.Index,
         in bounds: Range<String.Index>
-    ) -> (upperBound: String.Index, match: Match)?
+    ) throws -> (upperBound: String.Index, match: Match)?
 }
 ```
 

Documentation/Evolution/RegexTypeOverview.md

@@ -3,6 +3,13 @@
 
 import PackageDescription
 
+let availabilityDefinition = PackageDescription.SwiftSetting.unsafeFlags([
+    "-Xfrontend",
+    "-define-availability",
+    "-Xfrontend",
+    #"SwiftStdlib 5.7:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999"#,
+])
+
 let package = Package(
     name: "swift-experimental-string-processing",
     products: [
@@ -22,7 +29,6 @@ let package = Package(
     ],
     dependencies: [
         .package(url: "https://github.com/apple/swift-argument-parser", from: "1.0.0"),
-        .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
     ],
     targets: [
         // Targets are the basic building blocks of a package. A target can define a module or a test suite.
@@ -31,12 +37,14 @@ let package = Package(
             name: "_RegexParser",
             dependencies: [],
             swiftSettings: [
-                .unsafeFlags(["-enable-library-evolution"])
+                .unsafeFlags(["-enable-library-evolution"]),
+                availabilityDefinition
             ]),
         .testTarget(
             name: "MatchingEngineTests",
             dependencies: [
-              "_RegexParser", "_StringProcessing"]),
+                "_RegexParser", "_StringProcessing"
+            ]),
         .target(
             name: "_CUnicode",
             dependencies: []),
@@ -45,53 +53,64 @@ let package = Package(
             dependencies: ["_RegexParser", "_CUnicode"],
             swiftSettings: [
                 .unsafeFlags(["-enable-library-evolution"]),
+                availabilityDefinition
             ]),
         .target(
             name: "RegexBuilder",
             dependencies: ["_StringProcessing", "_RegexParser"],
             swiftSettings: [
                 .unsafeFlags(["-enable-library-evolution"]),
-                .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"])
+                .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"]),
+                availabilityDefinition
             ]),
         .testTarget(
             name: "RegexTests",
             dependencies: ["_StringProcessing"],
-            swiftSettings: [.unsafeFlags(["-Xfrontend", "-enable-experimental-string-processing"])]
-        ),
+            swiftSettings: [
+                .unsafeFlags(["-Xfrontend", "-enable-experimental-string-processing"]),
+                .unsafeFlags(["-Xfrontend", "-disable-availability-checking"]),
+            ]),
         .testTarget(
             name: "RegexBuilderTests",
             dependencies: ["_StringProcessing", "RegexBuilder"],
             swiftSettings: [
-                .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"])
+                .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"]),
+                .unsafeFlags(["-Xfrontend", "-disable-availability-checking"])
             ]),
-        .target(
+        .testTarget(
             name: "Prototypes",
-            dependencies: ["_RegexParser", "_StringProcessing"]),
+            dependencies: ["_RegexParser", "_StringProcessing"],
+            swiftSettings: [
+                .unsafeFlags(["-Xfrontend", "-disable-availability-checking"])
+            ]),
 
         // MARK: Scripts
         .executableTarget(
             name: "VariadicsGenerator",
             dependencies: [
-              .product(name: "ArgumentParser", package: "swift-argument-parser")
+                .product(name: "ArgumentParser", package: "swift-argument-parser")
             ]),
         .executableTarget(
             name: "PatternConverter",
             dependencies: [
-              .product(name: "ArgumentParser", package: "swift-argument-parser"),
-              "_RegexParser",
-              "_StringProcessing"
+                .product(name: "ArgumentParser", package: "swift-argument-parser"),
+                "_RegexParser",
+                "_StringProcessing"
             ]),
 
         // MARK: Exercises
         .target(
-          name: "Exercises",
-          dependencies: ["_RegexParser", "Prototypes", "_StringProcessing", "RegexBuilder"],
-          swiftSettings: [
-              .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"])
-          ]),
+            name: "Exercises",
+            dependencies: ["_RegexParser", "_StringProcessing", "RegexBuilder"],
+            swiftSettings: [
+                .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"]),
+                .unsafeFlags(["-Xfrontend", "-disable-availability-checking"])
+            ]),
         .testTarget(
-          name: "ExercisesTests",
-          dependencies: ["Exercises"]),
+            name: "ExercisesTests",
+            dependencies: ["Exercises"],
+            swiftSettings: [
+                .unsafeFlags(["-Xfrontend", "-disable-availability-checking"])
+            ])
     ]
 )
-

Documentation/Evolution/StringProcessingAlgorithms.md

@@ -10,6 +10,21 @@ See [Declarative String Processing Overview][decl-string]
 
 - [Swift Trunk Development Snapshot](https://www.swift.org/download/#snapshots) DEVELOPMENT-SNAPSHOT-2022-03-09 or later.
 
+## Trying it out
+
+To try out the functionality provided here, download the latest open source development toolchain. Import `_StringProcessing` in your source file to get access to the API and specify `-Xfrontend -enable-experimental-string-processing` to get access to the literals.
+
+For example, in a `Package.swift` file's target declaration:
+
+```swift
+.target(
+    name: "foo",
+    dependencies: ["depA"],
+    swiftSettings: [.unsafeFlags(["-Xfrontend", "-enable-experimental-string-processing"])]
+ ),
+```
+
+
 ## Integration with Swift
 
 `_RegexParser` and `_StringProcessing` are specially integrated modules that are built as part of apple/swift.

Package.swift

@@ -16,7 +16,6 @@ public enum Exercises {
       HandWrittenParticipant.self,
       RegexDSLParticipant.self,
       RegexLiteralParticipant.self,
-      PEGParticipant.self,
       NSREParticipant.self,
     ]
   }

README.md

@@ -9,6 +9,9 @@
 //
 //===----------------------------------------------------------------------===//
 
+// Disabled because Prototypes is a test target.
+#if false
+
 struct PEGParticipant: Participant {
   static var name: String { "PEG" }
 }
@@ -51,3 +54,4 @@ private func graphemeBreakPropertyData(forLine line: String) -> GraphemeBreakEnt
 
 }
 
+#endif

Sources/Exercises/Exercises.swift

@@ -62,8 +62,8 @@ private func extractFromCaptures(
 private func graphemeBreakPropertyData<RP: RegexComponent>(
   forLine line: String,
   using regex: RP
-) -> GraphemeBreakEntry? where RP.Output == (Substring, Substring, Substring?, Substring) {
-  line.matchWhole(regex).map(\.output).flatMap(extractFromCaptures)
+) -> GraphemeBreakEntry? where RP.RegexOutput == (Substring, Substring, Substring?, Substring) {
+  line.wholeMatch(of: regex).map(\.output).flatMap(extractFromCaptures)
 }
 
 private func graphemeBreakPropertyDataLiteral(
@@ -80,7 +80,7 @@ private func graphemeBreakPropertyDataLiteral(
 private func graphemeBreakPropertyData(
   forLine line: String
 ) -> GraphemeBreakEntry? {
-  line.matchWhole {
+  line.wholeMatch {
     TryCapture(OneOrMore(.hexDigit)) { Unicode.Scalar(hex: $0) }
     Optionally {
       ".."

Sources/Exercises/Participants/PEGParticipant.swift

@@ -12,6 +12,7 @@
 import _RegexParser
 @_spi(RegexBuilder) import _StringProcessing
 
+@available(SwiftStdlib 5.7, *)
 public struct Anchor {
   internal enum Kind {
     case startOfSubject
@@ -28,6 +29,7 @@ public struct Anchor {
   var isInverted: Bool = false
 }
 
+@available(SwiftStdlib 5.7, *)
 extension Anchor: RegexComponent {
   var astAssertion: AST.Atom.AssertionKind {
     if !isInverted {
@@ -62,6 +64,7 @@ extension Anchor: RegexComponent {
 
 // MARK: - Public API
 
+@available(SwiftStdlib 5.7, *)
 extension Anchor {
   public static var startOfSubject: Anchor {
     Anchor(kind: .startOfSubject)
@@ -107,6 +110,7 @@ extension Anchor {
   }
 }
 
+@available(SwiftStdlib 5.7, *)
 public struct Lookahead<Output>: _BuiltinRegexComponent {
   public var regex: Regex<Output>
 
@@ -117,15 +121,15 @@ public struct Lookahead<Output>: _BuiltinRegexComponent {
   public init<R: RegexComponent>(
     _ component: R,
     negative: Bool = false
-  ) where R.Output == Output {
+  ) where R.RegexOutput == Output {
     self.init(node: .nonCapturingGroup(
       negative ? .negativeLookahead : .lookahead, component.regex.root))
   }
 
   public init<R: RegexComponent>(
     negative: Bool = false,
     @RegexComponentBuilder _ component: () -> R
-  ) where R.Output == Output {
+  ) where R.RegexOutput == Output {
     self.init(node: .nonCapturingGroup(
       negative ? .negativeLookahead : .lookahead, component().regex.root))
   }