[Servers] Investigate MissingAuthority_200Status failures

[pr-benchmarks]

These failures should all have logs available, I (@anurse) don't have the specific build numbers handy (sounds like a good feature to add to What's Broken).

Metadata

Affected Tests

Project	Type	Method
InMemory.FunctionalTests	Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http3StreamTests	MissingAuthority_200Status
InMemory.FunctionalTests	Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TimeoutTests	DATA_Sent_TooSlowlyDueToSocketBackPressureOnLargeWrite_AbortsConnectionAfterRateTimeout
InMemory.FunctionalTests	Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.RequestHeaderLimitsTests	ServerAcceptsRequestWithHeaderTotalSizeWithinLimit
InMemory.FunctionalTests	Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TimeoutTests	DATA_Sent_TooSlowlyDueToSocketBackPressureOnSmallWrite_AbortsConnectionAfterGracePeriod
InMemory.FunctionalTests	Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.HttpsTests	DoesNotThrowObjectDisposedExceptionFromWriteAsyncAfterConnectionIsAborted

[BrennanConroy]

https://dev.azure.com/dnceng/public/_build/results?buildId=860738&view=ms.vss-test-web.build-test-results-tab&runId=27566828&resultId=100037&paneView=debug
DATA_Sent_TooSlowlyDueToSocketBackPressureOnLargeWrite_AbortsConnectionAfterRateTimeout:

at Moq.Mock.VerifyCalls(Mock targetMock, InvocationShape expectation, LambdaExpression expression, Times times, String failMessage)
   at Moq.Mock.Verify[T](Mock`1 mock, Expression`1 expression, Times times, String failMessage)
   at Moq.Mock`1.Verify(Expression`1 expression, Times times)
   at Moq.Mock`1.Verify(Expression`1 expression, Func`1 times)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TimeoutTests.DATA_Sent_TooSlowlyDueToSocketBackPressureOnLargeWrite_AbortsConnectionAfterRateTimeout() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TimeoutTests.cs:line 426
--- End of stack trace from previous location ---

Moq.MockException : 
Expected invocation on the mock once, but was 0 times: h => h.OnTimeout(TimeoutReason.WriteDataRate)

Configured setups: 
ITimeoutHandler h => h.OnTimeout(It.IsAny<TimeoutReason>())
No invocations performed.

[0.020s] [TestLifetime] [Information] Starting test DATA_Sent_TooSlowlyDueToSocketBackPressureOnLargeWrite_AbortsConnectionAfterRateTimeout at 2020-10-21T02:32:12
[0.309s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending SETTINGS frame for stream ID 0 with length 18 and flags NONE
[0.389s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending WINDOW_UPDATE frame for stream ID 0 with length 4 and flags 0x0
[3.032s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" received SETTINGS frame for stream ID 0 with length 0 and flags NONE
[3.032s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending SETTINGS frame for stream ID 0 with length 0 and flags ACK
[3.184s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" received HEADERS frame for stream ID 1 with length 318 and flags END_HEADERS
[3.188s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" received DATA frame for stream ID 1 with length 16384 and flags END_STREAM
[3.226s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null", Request id ":00000001": started reading request body.
[3.228s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null", Request id ":00000001": done reading request body.
[3.241s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending HEADERS frame for stream ID 1 with length 32 and flags END_HEADERS
[3.243s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending DATA frame for stream ID 1 with length 16384 and flags NONE
[3.339s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending DATA frame for stream ID 1 with length 0 and flags END_STREAM
[3.342s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null" is closed. The last processed stream ID was 1.
[3.342s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending GOAWAY frame for stream ID 0 with length 8 and flags 0x0
[3.347s] [TestLifetime] [Information] Finished test DATA_Sent_TooSlowlyDueToSocketBackPressureOnLargeWrite_AbortsConnectionAfterRateTimeout in 3.3433323s

[BrennanConroy]

MissingAuthority_200Status:

Assert.True() Failure
Expected: True
Actual:   False
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http3TestBase.Http3RequestStream.ReceiveFrameAsync(UInt32 maxFrameSize)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http3TestBase.Http3RequestStream.ExpectHeadersAsync() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http3/Http3TestBase.cs:line 291
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http3StreamTests.MissingAuthority_200Status() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http3/Http3StreamTests.cs:line 282
--- End of stack trace from previous location ---

[0.001s] [TestLifetime] [Information] Starting test MissingAuthority_200Status at 2020-10-27T00:59:54
[0.035s] [Microsoft.AspNetCore.Server.Kestrel] [Warning] Connection is shutting down.
Microsoft.AspNetCore.Connections.ConnectionAbortedException: Connection is shutting down.
[0.254s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null", Request id "null": started reading request body.
[0.256s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null", Request id "null": done reading request body.
[0.363s] [TestLifetime] [Information] Finished test MissingAuthority_200Status in 0.3622128s

[BrennanConroy]

DATA_Sent_TooSlowlyDueToSocketBackPressureOnSmallWrite_AbortsConnectionAfterGracePeriod:

Moq.MockException : 
Expected invocation on the mock once, but was 0 times: h => h.OnTimeout(TimeoutReason.WriteDataRate)

Configured setups: 
ITimeoutHandler h => h.OnTimeout(It.IsAny<TimeoutReason>())
No invocations performed.

   at Moq.Mock.VerifyCalls(Mock targetMock, InvocationShape expectation, LambdaExpression expression, Times times, String failMessage)
   at Moq.Mock.Verify[T](Mock`1 mock, Expression`1 expression, Times times, String failMessage)
   at Moq.Mock`1.Verify(Expression`1 expression, Times times)
   at Moq.Mock`1.Verify(Expression`1 expression, Func`1 times)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TimeoutTests.DATA_Sent_TooSlowlyDueToSocketBackPressureOnSmallWrite_AbortsConnectionAfterGracePeriod() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TimeoutTests.cs:line 371
--- End of stack trace from previous location ---

[0.001s] [TestLifetime] [Information] Starting test DATA_Sent_TooSlowlyDueToSocketBackPressureOnSmallWrite_AbortsConnectionAfterGracePeriod at 2020-10-26T21:07:45
[0.006s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending SETTINGS frame for stream ID 0 with length 18 and flags NONE
[0.019s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending WINDOW_UPDATE frame for stream ID 0 with length 4 and flags 0x0
[0.760s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" received SETTINGS frame for stream ID 0 with length 0 and flags NONE
[0.760s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending SETTINGS frame for stream ID 0 with length 0 and flags ACK
[0.773s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" received HEADERS frame for stream ID 1 with length 318 and flags END_HEADERS
[0.773s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" received DATA frame for stream ID 1 with length 12 and flags END_STREAM
[0.773s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null", Request id ":00000001": started reading request body.
[0.774s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null", Request id ":00000001": done reading request body.
[0.774s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending HEADERS frame for stream ID 1 with length 32 and flags END_HEADERS
[0.774s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending DATA frame for stream ID 1 with length 12 and flags NONE
[0.799s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "null" is closed. The last processed stream ID was 1.
[0.799s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending GOAWAY frame for stream ID 0 with length 8 and flags 0x0
[0.799s] [Microsoft.AspNetCore.Server.Kestrel] [Verbose] Connection id "null" sending DATA frame for stream ID 1 with length 0 and flags END_STREAM
[0.799s] [TestLifetime] [Information] Finished test DATA_Sent_TooSlowlyDueToSocketBackPressureOnSmallWrite_AbortsConnectionAfterGracePeriod in 0.7985851s