Fix path length 255

[danmoseley]

Fix https://github.com/dotnet/corefx/issues/34359
Tests dotnet/corefx#34389

It's fortunate that the PAL did not need changing (SystemNative_ReadDirR) as well because then we would have a dependency in two directions.

[stephentoub]

It's fortunate that the PAL did not need changing

But maybe it should? Wouldn't it be better to have NameBufferSize match NAME_MAX, which would mean exposing a function from System.Native.so/dylib to retrieve it and storing its result in a NameBufferSize static readonly field rather than const?

Or are we absolutely sure this length is the maximum size on every platform we currently target and plan to target in the foreseeable future?

[danmoseley]

Sure, I can do that. I see we already have SystemNative_GetReadDirRBufferSize. I still need to understand the GetMaxCharCount issue.

[danmoseley]

Hmm, if the buffer size is not a constant anymore, this buffer in FileSystemEntry can no longer be fixed:

        private fixed char _fileNameBuffer[Interop.Sys.DirectoryEntry.NameBufferSize];

I have to change to ArrayPool, or at least fall back to it. This change starts potentially modifying perf. Thoughts?

[danmoseley]

I think I'm inclined to leave it hard coded at 255 + 1 and see whether this assert fires again - as it will if there is a system that has a larger limit and such a file or directory is encountered on it. From this it seems that may be true on ReiserFS, but then this assert may fail also:

    // dirent should be under 2k in size
    assert(sizeof(struct dirent) < 2048);

Edit: and according to Wikipedia/Stackoverflow: "ReiserFS shows 4032 bytes (but it is limited to 255 chars because of Linux VFS)."

Incidentally NTFS is 512 bytes (256 UTF-16 code units).

[stephentoub]

Ok, leave it hardcoded for now so that we can fix the original issue without it becoming a can of worms. It does make me uneasy, though.

[danmoseley]

Unrelated build failures.

1. @adiaaida is this familiar? x86 perf

C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\bin\Microsoft.Common.CurrentVersion.targets(4176,5): error MSB3027: Could not copy "D:\j\w\perf_perflab_---6af0d14a\bin\int\Managed\GC\Regressions\v2.0-beta1\289745\289745\289745.exe" to "D:\j\w\perf_perflab_---6af0d14a\bin\tests\Windows_NT.x86.Release\GC\Regressions\v2.0-beta1\289745\289745\289745.exe". Exceeded retry count of 10. Failed.  [D:\j\w\perf_perflab_---6af0d14a\tests\src\GC\Regressions\v2.0-beta1\289745\289745.csproj]
1

2. @BruceForstall is this ARM issue known? On both ARM64 and ARM Windows cross builds.

13:20:13 D:\j\workspace\arm64_cross_d---ffc60a4b>powershell -NoProfile -Command "Add-Type -Assembly 'System.IO.Compression.FileSystem'; [System.IO.Compression.ZipFile]::CreateFromDirectory('.\bin\tests\Windows_NT.arm64.Debug', '.\bin\tests\tests.zip')" 
13:22:00 Exception calling "CreateFromDirectory" with "2" argument(s): "Operation did not complete successfully because the 
13:22:00 file contains a virus or potentially unwanted software.

3. Also a test failure on arm_cross_checked_ubuntu. Rerun?

14:18:16 #################################################################
14:18:16 Output of failing tests:
14:18:16 
14:18:16 [/ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/GC/Scenarios/Dynamo/dynamo/dynamo.sh]: 
14:18:16 
14:18:16 /n/nReturn code:      1/nRaw output file:      /ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/GC/Scenarios/Reports/GC.Scenarios/Dynamo/dynamo/dynamo.output.txt/nRaw output:/nBEGIN EXECUTION
14:18:16 /ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/Tests/Core_Root/corerun dynamo.exe 1000 40 191919
14:18:16  
14:18:16 Total amount of RandomNode Memory: 
14:18:16 12487500
14:18:16  
14:18:16 Running Finalize Test...
14:18:16  
14:18:16 Deleting Node:
14:18:16 0
14:18:16 Deleting Node:
14:18:16 19
14:18:16 Deleting Node:
14:18:16 10
14:18:16 Deleting Node:
14:18:16 4
14:18:16 The registered number: 50
14:18:16 The analyzed number: 39
14:18:16 The registered number and analyzed number are not equal
14:18:16 
14:18:16 Test Failed with seed: 191919
14:18:16 Expected: 100
14:18:16 Actual: 1
14:18:16 END EXECUTION - FAILED

4. OSX
   14:03:41 java.nio.channels.ClosedChannelException
   @dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test please

[danmoseley]

I see the ARM failures are on other jobs too.
https://ci.dot.net/job/dotnet_coreclr/job/master/job/arm_cross_debug_windows_nt_innerloop_prtest/4900/consoleText

[danmoseley]

@dotnet-bot test Windows_NT x86 min_opt ryujit CoreCLR Perf Tests Correctness please

[BruceForstall]

@danmosemsft That's a well-known issue (https://github.com/dotnet/coreclr/issues/21843). We've started seeing it relatively recently. Hopefully dnceng can fix it soon.

[danmoseley]

Thanks @BruceForstall

[danmoseley]

@BruceForstall should I just rerun then? I don't see a timeline on https://github.com/dotnet/core-eng/issues/4555

[michellemcdaniel]

@danmosemsft looks like other PRs around yours have completed successfully (though one or two have also failed). I'd just re-run.

[danmoseley]

@dotnet-bot test Windows_NT arm64 Cross Debug Innerloop Build please
@dotnet-bot test Windows_NT arm Cross Debug Innerloop Build please
@dotnet-bot test Ubuntu arm Cross Checked Innerloop Build and Test please

[BruceForstall]

I hope rerun helps; I don't know at this point how repeatable the failure is.

[danmoseley]

ARM Ubuntu failure @BruceForstall

16:44:39       GC_Scenarios._Dynamo_dynamo_dynamo_._Dynamo_dynamo_dynamo_sh [FAIL]
16:44:39         /n/nReturn code:      1/nRaw output file:      /ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/GC/Scenarios/Reports/GC.Scenarios/Dynamo/dynamo/dynamo.output.txt/nRaw output:/nBEGIN EXECUTION
16:44:39         /ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/Tests/Core_Root/corerun dynamo.exe 1000 40 191919
16:44:39          
16:44:39         Total amount of RandomNode Memory: 
16:44:39         12487500
16:44:39          
16:44:39         Running Finalize Test...
16:44:39          
16:44:39         Deleting Node:
16:44:39         0
16:44:39         Deleting Node:
16:44:39         19
16:44:39         Deleting Node:
16:44:39         10
16:44:39         The registered number: 50
16:44:39         The analyzed number: 39
16:44:39         The registered number and analyzed number are not equal
16:44:39         
16:44:39         Test Failed with seed: 191919
16:44:39         Expected: 100
16:44:39         Actual: 1
16:44:39         END EXECUTION - FAILED
16:44:39         Test Harness Exitcode is : 1
16:44:39         /nTo run the test:/n> set CORE_ROOT=/ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/Tests/Core_Root/n> /ssd/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_tst_prtest/bin/tests/Linux.arm.Checked/GC/Scenarios/Dynamo/dynamo/dynamo.sh/n
16:44:39         Expected: True
16:44:39         Actual:   False
16:44:39         Stack Trace:
16:44:39           /mnt/j/workspace/dotnet_coreclr/master/arm_cross_checked_ubuntu_innerloop_prtest/bin/tests/Linux.arm.Checked/TestWrappers/GC.Scenarios/GC.Scenarios.XUnitWrapper.cs(1229,0): at GC_Scenarios._Dynamo_dynamo_dynamo_._Dynamo_dynamo_dynamo_sh()

[danmoseley]

1 gc test failure in ARM. Unrelated

[BruceForstall]

FWIW, the dynamo issue is https://github.com/dotnet/coreclr/issues/17129