-
Notifications
You must be signed in to change notification settings - Fork 0
Consolidate _stdlib_random functions
#2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consolidate _stdlib_random functions
#2
Conversation
|
One of the tests will fail, with "Fatal error: Division by zero in remainder operation". RandomTests.test("random floating points in ranges") {
floatingPointRangeTest(Float.self)
floatingPointRangeTest(Double.self)
- floatingPointRangeTest(Float80.self)
}Your previous workaround was also needed, in order to compile and test. - public func random<T: RandomNumberGenerator>(
- using generator: T
+ public func random(
+ using generator: RandomNumberGenerator
) -> Element? { |
|
This looks great, thanks! However, is changing the generator this late in the review process acceptable? I don't really see why this would be controversial, but it does alter that from what the reviewers have seen. Nate's pr moves this failing test into its own test, and the compilation error is fixed in his pr as well. |
2c2b6bd to
f7bb388
Compare
|
I've rebased and updated, to use:
I think it's now close enough to your proposed solution. |
|
@Azoy, should we use "/dev/urandom" instead of |
|
Sorry for being a bit inactive! Yeah, I was thinking about going with that approach, but my concern is that the device file is not present on simulators due to sandboxing. I'm not sure if you can make that distinction and can use |
|
Using "/dev/urandom" seems to work in the iOS 8.1 Simulator (the oldest available for the latest Xcode). If sandboxing is an issue, we could use There's a |
|
I imagine the proposal will be revised to not explicitly state the specific implementations for each platform, but rather describe that platforms will be secure. This means we have more flexibility over how we choose the secure generators for the future. I think if it does get revised to something along those lines, we can look at using |
|
I've removed I thought using |
_stdlib_random with an ENOSYS fallback_stdlib_random functions
e00fdb8 to
6caa082
Compare
1062c68 to
21801da
Compare
6caa082 to
9400a6e
Compare
21801da to
3a684c0
Compare
|
Question, should we cache actual_bytes if it returns -1? Example: if we call |
|
We can do, if you think there will be a measurable difference. #if defined(GRND_RANDOM)
+ static const bool getrandom_available =
+ !(getrandom(nullptr, 0, 0) == -1 && errno == ENOSYS);
+ if (getrandom_available) {
actual_nbytes = WHILE_EINTR(getrandom(buf, nbytes, 0));
+ }
#elif defined(__Fuchsia__) |
|
When opening "/dev/urandom", should we use the
When reading
I've searched for any mention of the iOS sandbox issue. There are some comments in an article from March 2011 (iOS 4). But it's difficult to find anything specifically about iOS 7, 8 or 9. |
|
Yeah I think we ought to go ahead with caching the return value of |
9400a6e to
2f4294b
Compare
3a684c0 to
90fd95a
Compare
* Use the `__has_include` and `GRND_RANDOM` macros * Use `getentropy` instead of `getrandom` * Use `std::min` from the <algorithm> header * Move `#if` out of the `_stdlib_random` function * Use `getrandom` with "/dev/urandom" fallback * Use `#pragma comment` to import "Bcrypt.lib" * <https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp> * <https://clang.llvm.org/docs/UsersManual.html#microsoft-extensions> * Use "/dev/urandom" instead of `SecRandomCopyBytes` * Use `swift::StaticMutex` for shared "/dev/urandom" * Add `getrandom_available`; use `O_CLOEXEC` flag Add platform impl docs Update copyrights Fix docs Add _stdlib_random test Update _stdlib_random test Add missing & Notice about _stdlib_random Fix docs Guard on upperBound = 0 Test full range of 8 bit integers Remove some gyb
* Use the `__has_include` and `GRND_RANDOM` macros * Use `getentropy` instead of `getrandom` * Use `std::min` from the <algorithm> header * Move `#if` out of the `_stdlib_random` function * Use `getrandom` with "/dev/urandom" fallback * Use `#pragma comment` to import "Bcrypt.lib" * <https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp> * <https://clang.llvm.org/docs/UsersManual.html#microsoft-extensions> * Use "/dev/urandom" instead of `SecRandomCopyBytes` * Use `swift::StaticMutex` for shared "/dev/urandom" * Add `getrandom_available`; use `O_CLOEXEC` flag Add platform impl docs Update copyrights Fix docs Add _stdlib_random test Update _stdlib_random test Add missing & Notice about _stdlib_random Fix docs Guard on upperBound = 0 Test full range of 8 bit integers Remove some gyb Clean up integerRangeTest Remove FixedWidthInteger constraint Use arc4random universally Fix randomElement Constrain shuffle to RandomAccessCollection warning instead of error Move Apple's implementation Fix failing test on 32 bit systems
Witness table accessors return a witness table for a given type's conformance to a protocol. They are called directly from IRGen (when we need the witness table instance) and from runtime conformance checking (swift_conformsToProtocol digs the access function out of the protocol conformance record). They have two interesting functions: 1) For witness tables requiring instantiation, they call swift_instantiateWitnessTable directly. 2) For synthesized witness tables that might not be unique, they call swift_getForeignWitnessTable. Extend swift_instantiateWitnessTable() to handle both runtime uniquing (for #2) as well as handling witness tables that don't have a "generic table", i.e., don't need any actual instantiation. Use it as the universal entry point for "get a witness table given a specific conformance descriptor and type", eliminating witness table accessors entirely. Make a few related simplifications: * Drop the "pattern" from the generic witness table. Instead, store the pattern in the main part of the conformance descriptor, always. * Drop the "conformance kind" from the protocol conformance descriptor, since it was only there to distinguish between witness table (pattern) vs. witness table accessor. * Internalize swift_getForeignWitnessTable(); IRGen no longer needs to call it. Reduces the code size of the standard library (+assertions build) by ~149k. Addresses rdar://problem/45489388.
The standard library has two versions of the `abs(_:)` function: ``` func abs<T : SignedNumeric>(_ x: T) -> T where T.Magnitude == T func abs<T : SignedNumeric & Comparable>(_ x: T) -> T ``` The first is more specialized than the second because `T.Magnitude` is known to conform to `Comparable`. Indeed, it’s a more specialized implementation that returns `magnitude`. However, this overload behaves oddly: in the expression `abs(-8)`, the type checker will pick the first overload because it is more specialized. That’s a general guiding principle for overloading: pick the most specialized overload that works. However, to select that overload, it needs to pick a type for the literal “8” for which that overload works, and it chooses `Double`. The “obvious” answer, `Int`, doesn’t work because `Int.Magnitude == UInt`. There is a conflict between the two rules, here: we prefer more-specialized overloads (but we’ll fall back to less-specialized if those don’t work) and we prefer to use `Int` for integer literals (but we’ll fall back to `Double` if it doesn’t work). We have a few options from a type-checker perspective: 1. Consider the more-specialized-function rule to be more important 2. Consider the integer-literals-prefer-`Int` rule to be more important 3. Call the result ambiguous and make the user annotate it The type checker currently does #1, although at some point in the past it did #2. Moving forward, #1 is a better choice because it prunes the number of overloads that need to be considered: if the more-specialized overload succeeds its type-check, the others need not be considered. It’s also easier to reason about than the literal-scoring approach, because there can be a direct definition for “more specialized than” that can be reasoned about. I think we should dodge the issue by removing the more-specialized version of `abs(_:)`. Its use of `magnitude` seems unlikely to provide a significant performance benefit, and the presence of overloading either forces us to consider both overloads always (which is bad for type checker performance) or accept the regression that `abs(-8)` is `Double`. Better to eliminate the overloading and, if needed in the future, find a better way to introduce the more-specialized implementation without it being a separate signature. Fixes rdar://problem/42345366.
The standard library has two versions of the `abs(_:)` function: ``` func abs<T : SignedNumeric>(_ x: T) -> T where T.Magnitude == T func abs<T : SignedNumeric & Comparable>(_ x: T) -> T ``` The first is more specialized than the second because `T.Magnitude` is known to conform to `Comparable`. Indeed, it’s a more specialized implementation that returns `magnitude`. However, this overload behaves oddly: in the expression `abs(-8)`, the type checker will pick the first overload because it is more specialized. That’s a general guiding principle for overloading: pick the most specialized overload that works. However, to select that overload, it needs to pick a type for the literal “8” for which that overload works, and it chooses `Double`. The “obvious” answer, `Int`, doesn’t work because `Int.Magnitude == UInt`. There is a conflict between the two rules, here: we prefer more-specialized overloads (but we’ll fall back to less-specialized if those don’t work) and we prefer to use `Int` for integer literals (but we’ll fall back to `Double` if it doesn’t work). We have a few options from a type-checker perspective: 1. Consider the more-specialized-function rule to be more important 2. Consider the integer-literals-prefer-`Int` rule to be more important 3. Call the result ambiguous and make the user annotate it The type checker currently does #1, although at some point in the past it did #2. Moving forward, #1 is a better choice because it prunes the number of overloads that need to be considered: if the more-specialized overload succeeds its type-check, the others need not be considered. It’s also easier to reason about than the literal-scoring approach, because there can be a direct definition for “more specialized than” that can be reasoned about. I think we should dodge the issue by removing the more-specialized version of `abs(_:)`. Its use of `magnitude` seems unlikely to provide a significant performance benefit, and the presence of overloading either forces us to consider both overloads always (which is bad for type checker performance) or accept the regression that `abs(-8)` is `Double`. Better to eliminate the overloading and, if needed in the future, find a better way to introduce the more-specialized implementation without it being a separate signature. Fixes rdar://problem/42345366. (cherry picked from commit 85d488d)
[test] Add missing runtime availability check around diffing tests
Due to the fact that `matchCallArgument` can't and
doesn't take types into consideration while matching
arguments to parameters, when both arguments are
un-labeled, it's impossible to say which one is missing:
func foo(_: Int, _: String) {}
foo("")
In this case first argument is missing, but we end up with
two fixes - argument mismatch (for #1) and missing argument
(for #2), which is incorrect so it has to be handled specially.
…est2 [CodeCompletion] Migrate some tests to batch completion test #2
* Synchronize both versions of actor_counters.swift test
* Synchronize on Job address
Make sure to synchronize on Job address (AsyncTasks are Jobs, but not
all Jobs are AsyncTasks).
* Add fprintf debug output for TSan acquire/release
* Add tsan_release edge on task creation
without this, we are getting false data races between when a task
is created and immediately scheduled on a different thread.
False positive for `Sanitizers/tsan/actor_counters.swift` test:
```
WARNING: ThreadSanitizer: data race (pid=81452)
Read of size 8 at 0x7b2000000560 by thread T5:
#0 Counter.next() <null>:2 (a.out:x86_64+0x1000047f8)
#1 (1) suspend resume partial function for worker(identity:counters:numIterations:) <null>:2 (a.out:x86_64+0x100005961)
#2 swift::runJobInEstablishedExecutorContext(swift::Job*) <null>:2 (libswift_Concurrency.dylib:x86_64+0x280ef)
Previous write of size 8 at 0x7b2000000560 by main thread:
#0 Counter.init(maxCount:) <null>:2 (a.out:x86_64+0x1000046af)
#1 Counter.__allocating_init(maxCount:) <null>:2 (a.out:x86_64+0x100004619)
#2 runTest(numCounters:numWorkers:numIterations:) <null>:2 (a.out:x86_64+0x100006d2e)
#3 swift::runJobInEstablishedExecutorContext(swift::Job*) <null>:2 (libswift_Concurrency.dylib:x86_64+0x280ef)
swiftlang#4 main <null>:2 (a.out:x86_64+0x10000a175)
```
New edge with this change:
```
[4357150208] allocate task 0x7b3800000000, parent = 0x0
[4357150208] creating task 0x7b3800000000 with parent 0x0
[4357150208] tsan_release on 0x7b3800000000 <<< new release edge
[139088221442048] tsan_acquire on 0x7b3800000000
[139088221442048] trying to switch from executor 0x0 to 0x7ff85e2d9a00
[139088221442048] switch failed, task 0x7b3800000000 enqueued on executor 0x7ff85e2d9a00
[139088221442048] enqueue job 0x7b3800000000 on executor 0x7ff85e2d9a00
[139088221442048] tsan_release on 0x7b3800000000
[139088221442048] tsan_release on 0x7b3800000000
[4357150208] tsan_acquire on 0x7b3800000000
counters: 1, workers: 1, iterations: 1
[4357150208] allocate task 0x7b3c00000000, parent = 0x0
[4357150208] creating task 0x7b3c00000000 with parent 0x0
[4357150208] tsan_release on 0x7b3c00000000 <<< new release edge
[139088221442048] tsan_acquire on 0x7b3c00000000
[4357150208] task 0x7b3800000000 waiting on task 0x7b3c00000000, going to sleep
[4357150208] tsan_release on 0x7b3800000000
[4357150208] tsan_release on 0x7b3800000000
[139088221442048] getting current executor 0x0
[139088221442048] tsan_release on 0x7b3c00000000
...
```
rdar://78932849
* Add static_cast<Job *>()
* Move TSan release edge to swift_task_enqueueGlobal()
Move the TSan release edge from `swift_task_create_commonImpl()` to
`swift_task_enqueueGlobalImpl()`. Task creation itself is not an event
that needs synchronization, but rather that task creation "happens
before" execution of that task on another thread.
This edge is usually added when the task is scheduled via
`swift_task_enqueue()` (which then usually calls
`swift_task_enqueueGlobal()`). However, not all task scheduling goes
through the `swift_task_enqueue()` funnel as some places call the more
specific `swift_task_enqueueGlobal()` directly. So let's annotate this
function (duplicate edges aren't harmful) to ensure we cover all
schedule events, including newly-created tasks (our original problem
here).
rdar://78932849
Co-authored-by: Julian Lettner <[email protected]>
…with reflection metadata, take #2 (swiftlang#39878)
Case #1. Literal zero = natural alignment %1 = integer_literal $Builtin.Int64, 0 %2 = builtin "uncheckedAssertAlignment" (%0 : $Builtin.RawPointer, %1 : $Builtin.Int64) : $Builtin.RawPointer %3 = pointer_to_address %2 : $Builtin.RawPointer to [align=1] $*Int Erases the `pointer_to_address` `[align=]` attribute: Case #2. Literal nonzero = forced alignment. %1 = integer_literal $Builtin.Int64, 16 %2 = builtin "uncheckedAssertAlignment" (%0 : $Builtin.RawPointer, %1 : $Builtin.Int64) : $Builtin.RawPointer %3 = pointer_to_address %2 : $Builtin.RawPointer to [align=1] $*Int Promotes the `pointer_to_address` `[align=]` attribute to a higher value. Case #3. Folded dynamic alignment %1 = builtin "alignof"<T>(%0 : $@thin T.Type) : $Builtin.Word %2 = builtin "uncheckedAssertAlignment" (%0 : $Builtin.RawPointer, %1 : $Builtin.Int64) : $Builtin.RawPointer %3 = pointer_to_address %2 : $Builtin.RawPointer to [align=1] $*T Erases the `pointer_to_address` `[align=]` attribute.
While trying to reuse the liveness-points analysis originally in DI for
injecting actor hops for more general purposes, Pavel and I discovered
that the point at which we are injecting the hops might not have
fully-computed the liveness information.
That appears to be the case because we were computing the fully-initialized
points before having processed destroy/releases of TheMemory. While this
most likely had no influence on the actor hop injection, it does affect
what the outgoing AvailabilitySet contains for a block. In particular, for
this example:
```swift
struct X {
init(cond: Bool) {
var _storage: (name: String, age: Int)
_storage.name = ""
if cond {
_storage.age = 30
} else {
_storage.age = 40
}
}
}
```
But because we are determine the full initialization points before processing
the destroy, the liveness analysis doesn't iterate to correctly determine the
out-availability of block 1 and 3 (corresponding to the then and else blocks
in the example above). Here's the debug output showing that issue:
```
*** Definite Init looking at: %5 = mark_uninitialized [var] %4 : $*(name: String, age: Int) // users: %37, %12, %22, %32
Get liveness 0, #1 at assign %11 to %13 : $*String // id: %14
Get liveness 1, #1 at assign %21 to %23 : $*Int // id: %24
Get liveness for block 1
Iteration 0
Result: (yn)
Get liveness 1, #1 at assign %31 to %33 : $*Int // id: %34
Get liveness for block 3
add block 2 to worklist
Iteration 0
Block 2 out: (yn)
Iteration 1
Block 2 out: (yn)
Result: (yn)
full-init-finder: rejecting bb0 b/c non-Yes OUT avail
full-init-finder: rejecting bb1 b/c non-Yes OUT avail
full-init-finder: rejecting bb2 b/c no non-load uses.
full-init-finder: rejecting bb3 b/c non-Yes OUT avail
full-init-finder: rejecting bb4 b/c no non-load uses.
Get liveness 0, #2 at destroy_addr %5 : $*(name: String, age: Int) // id: %37
Get liveness for block 4
add block 3 to worklist
add block 1 to worklist
Iteration 0
Block 1 out: (yy)
Block 3 out: (yy)
Iteration 1
Block 1 out: (yy)
Block 3 out: (yy)
Result: (yy)
```
So, this patch basically just sinks the computation so it happens after, so that
we force the incremental liveness analysis to also consider the liveness at the
point of the destroy, but before having done any other transformations or modifications
to the CFG to handle a destroy of something partially initialized.
# This is the 1st commit message: utils: update the build-windows-toolchain.bat to extract the toolchain Fetch a prebuilt toolchain to build the toolchain. This is required to enable the macro support on Windows. # The commit message #2 will be skipped: # build: build SwiftSyntax before the toolchain build # # Perform a build of Swift Syntax prior to the build of the toolchain so # that we can enable the early swift syntax parser builds. This is a # prerequisite for enabling macros on Windows. # The commit message #3 will be skipped: # # This is a combination of 5 commits. # # This is the 1st commit message: # # build: wire up the early swift-syntax build to the build # # This enables the early swift syntax build to get us macro support on # Windows. # # # The commit message #2 will be skipped: # # # Update build-windows-toolchain.bat # # # The commit message #3 will be skipped: # # # Update build-windows-toolchain.bat # # # The commit message swiftlang#4 will be skipped: # # # Update build-windows-toolchain.bat # # # The commit message swiftlang#5 will be skipped: # # # Update build-windows-toolchain.bat
Co-authored-by: Karoy Lorentey <[email protected]>
This inserts a suitably named function into the stack trace whenever a dynamic cast failure involves a NULL source or target type. Very often, crash logs include backtraces with function names but no log output; with this change, such a backtrace might look like the following -- note `TARGET_TYPE_NULL` in the function name here to mark the missing type information: ``` frame #0: __pthread_kill + 8 frame #1: pthread_kill + 288 frame #2: abort + 128 frame #3: swift::fatalErrorv() frame swiftlang#4: swift::fatalError() frame swiftlang#5: swift_dynamicCastFailure_TARGET_TYPE_NULL() frame swiftlang#6: swift::swift_dynamicCastFailure() frame swiftlang#7: ::swift_dynamicCast() ``` Resolves rdar://130630157
Two are fixes needed in most of the `RawSpan` and `Span` initializers. For example:
```
let baseAddress = buffer.baseAddress
let span = RawSpan(_unchecked: baseAddress, byteCount: buffer.count)
// As a trivial value, 'baseAddress' does not formally depend on the
// lifetime of 'buffer'. Make the dependence explicit.
self = _overrideLifetime(span, borrowing: buffer)
```
Fix #1. baseAddress needs to be a variable
`span` has a lifetime dependence on `baseAddress` via its
initializer. Therefore, the lifetime of `baseAddress` needs to include the call
to `_overrideLifetime`. The override sets the lifetime dependency of its result,
not its argument. It's argument still needs to be non-escaping when it is passed
in.
Alternatives:
- Make the RawSpan initializer `@_unsafeNonescapableResult`.
Any occurrence of `@_unsafeNonescapableResult` actually signals a bug. We never
want to expose this annotation.
In addition to being gross, it would totally disable enforcement of the
initialized span. But we really don't want to side-step `_overrideLifetime`
where it makes sense. We want the library author to explicitly indicate that
they understand exactly which dependence is unsafe. And we do want to
eventually expose the `_overrideLifetime` API, which needs to be well
understood, supported, and tested.
- Add lifetime annotations to a bunch of `UnsafePointer`-family APIs so the
compiler can see that the resulting pointer is derived from self, where self is
an incoming `Unsafe[Buffer]Pointer`. This would create a massive lifetime
annotation burden on the `UnsafePointer`-family APIs, which don't really have
anything to do with lifetime dependence. It makes more sense for the author of
`Span`-like APIs to reason about pointer lifetimes.
Fix #2. `_overrideLifetime` changes the lifetime dependency of span to be on an
incoming argument rather than a local variable.
This makes it legal to escape the function (by assigning it to self). Remember
that self is implicitly returned, so the `@lifetime(borrow buffer)` tells the
compiler that `self` is valid within `buffer`'s borrow scope.
…e SIL module add `Test`, which is the SIL-equivalent of `FunctionTest`. It's invocation closure gets a `TestContext` instead of a `FunctionContext`. ^ The commit message #2 will be skipped: ^ - test
No description provided.