FileStream file preallocation performance

[dmex]

Edit by carlossanlop: API Proposal can be found here.

---

Description

The FileStream class doesn't currently support Windows file preallocation hints reducing the performance of file operations while also increasing the disk fragmentation of newly created files.

If we use the example for file uploads for aspnetcore here:
https://docs.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-5.0#file-upload-scenarios

using (var stream = System.IO.File.Create(filePath))
{
     await formFile.CopyToAsync(stream);
}

None of the .NET classes support passing the AllocationSize when creating the file even though it's been a feature included with Windows since 2000. When you're creating multiple files with large file sizes such as with servers and file uploads, installers or build servers (we use .netcore for our builds servers compiling native C) passing the file length as AllocationSize when creating the file can significantly reduce fragmentation and improve performance.

An example of passing the allocation size:

using (var stream = System.IO.File.Create(filePath, allocationSize: 1073741824))
{
     await formFile.CopyToAsync(stream);
}

Windows Vista and above support the creation of files with their initial AllocationSize. This hint is passed to the file system driver and we can use the published FAT driver source code to show these optimizations:
https://github.com/microsoft/Windows-driver-samples/blob/master/filesys/fastfat/allocsup.c#L1164
https://github.com/microsoft/Windows-driver-samples/blob/master/filesys/fastfat/allocsup.c#L1233-L1250

The FatTruncateFileAllocation procedure becomes a noop when the AllocationSize is valid:
https://github.com/microsoft/Windows-driver-samples/blob/master/filesys/fastfat/allocsup.c#L1533

You can search for AllocationSize and see the other optimizations when the value is known during file creation:
https://github.com/microsoft/Windows-driver-samples/blob/6c1981b8504329521343ad00f32daa847fa6083a/filesys/fastfat/create.c#L6493-L6506

It would be safe to assume similar optimisations in the NTFS and ReFS drivers when AllocationSize is valid.

Configuration

The AllocationSize must be passed with NtCreateFile when a file is being created, overwritten, or superseded. NtCreateFile is documented here: https://docs.microsoft.com/en-us/windows/win32/api/winternl/nf-winternl-ntcreatefile

NTSTATUS
NTAPI
NtCreateFile(
    _Out_ PHANDLE FileHandle,
    _In_ ACCESS_MASK DesiredAccess,
    _In_ POBJECT_ATTRIBUTES ObjectAttributes,
    _Out_ PIO_STATUS_BLOCK IoStatusBlock,
    _In_opt_ PLARGE_INTEGER AllocationSize, // <---- File size
    _In_ ULONG FileAttributes,
    _In_ ULONG ShareAccess,
    _In_ ULONG CreateDisposition,
    _In_ ULONG CreateOptions,
    _In_reads_bytes_opt_(EaLength) PVOID EaBuffer,
    _In_ ULONG EaLength
    );

FileStream should also support get/set for the file allocation size like it currently does for the file size using GetFileInformationByHandleEx with the FileAllocationInfo class and FILE_ALLOCATION_INFO however this doesn't receive the full benefits of passing the AllocationSize up front with NtCreateFile so both methods should be supported for different use cases.

Analysis

A reduction in IO for our workloads when using C# to create files:

Data

There's also a discussion on stackoverflow about preallocation with more details:
https://stackoverflow.com/questions/53334343/windows-refs-ntfs-file-preallocation-hint

.NET including support for preallocation hints would be very welcome feature for reducing disk fragmentation on servers (file uploads) and desktop applications (installers and build tools). Please consider adding support for this feature into the next version of the runtime.

[iSazonov]

I like it on Windows but what are the benefits on Unix?

[danmoseley]

Thanks for the suggestion. It would be interesting for someone to propose the API changes. (using template but in this issue)

[dmex]

@danmosemsft

I would propose adding an extra parameter to the FileSteam class so optimizations can be made throughout the framework:

- public FileStream(String path, FileMode mode, FileAccess access, FileShare share, int bufferSize)  {
+ public FileStream(String path, FileMode mode, FileAccess access, FileShare share, int bufferSize, long allocationSize) 

For example the FileSystem.CopyFile function:

runtime/src/libraries/System.IO.FileSystem/src/System/IO/FileSystem.Unix.cs

Lines 22 to 24 in 8e913a2

	// Copy the contents of the file from the source to the destination, creating the destination in the process
	using (var src = new FileStream(sourceFullPath, FileMode.Open, FileAccess.Read, FileShare.Read, DefaultBufferSize, FileOptions.None))
	using (var dst = new FileStream(destFullPath, overwrite ? FileMode.Create : FileMode.CreateNew, FileAccess.ReadWrite, FileShare.None, DefaultBufferSize, FileOptions.None))

Can be changed to pass src.Length into dst :

using (var src = new FileStream(sourceFullPath, FileMode.Open, FileAccess.Read, FileShare.Read, DefaultBufferSize, FileOptions.None))
using (var dst = new FileStream(
          destFullPath, 
          overwrite ? FileMode.Create : FileMode.CreateNew, 
          FileAccess.ReadWrite, 
          FileShare.None, 
          DefaultBufferSize, 
          src.Length, // <-- value passed to fallocate and/or NtCreateFile as the AllocationSize
          FileOptions.None))
{
    Interop.CheckIo(Interop.Sys.CopyFile(src.SafeFileHandle, dst.SafeFileHandle));
}

File uploads via ASP.NET could also be optimized: (this uses the sample code from msdn)

using (var stream = System.IO.File.Create(filePath, allocationSize: formFile.Length)) // <-- formFile.Length passed to fallocate and/or NtCreateFile
{
     await formFile.CopyToAsync(stream);
}

@iSazonov

I like it on Windows but what are the benefits on Unix?

Linux/Unix has disk fragmentation just like Windows but most linux/unix tools are using the fallocate syscall so its not really an issue but .NET doesn't use fallocate and so the performance of .NET is at a disadvantage on Linux compared to other tools.

.NET should include support for fallocate on Linux/Unix:
fallocate: https://man7.org/linux/man-pages/man2/fallocate.2.html
posix_fallocate: https://man7.org/linux/man-pages/man3/posix_fallocate.3.html

You can view the fragmentation using the fsck command:
fsck -fnv /dev/sdb

[iSazonov]

@dmex Under Unix I assume not only Linux. We need a consistency behavior on all supported platforms including MacOs, FreeBSD. So we need a workaround for Linux fallocate on the platforms.

long allocationSize

preallocationSize name looks better for me.

[dmex]

We need a consistency behavior on all supported platforms including MacOs, FreeBSD. So we need a workaround for Linux fallocate on the platforms.

BSD has posix_fallocate (BSD documentation) and OSX has fcntl with the F_PREALLOCATE (OSX documentation) flag.

There isn't much documentation on OSX about F_PREALLOCATE but there's an example here: https://lists.apple.com/archives/darwin-dev/2007/Dec/msg00040.html

preallocationSize name looks better for me.

preallocationSize would probably be more consistent and a lot better at conveying the meaning.

[adamsitnik]

@dmex I've added it to #40359 and we are going to try to fix it for .NET 6.0

[adamsitnik]

Background and Motivation

Specifying file allocation size when creating the file can improve:

• performance as the following writes would not need to extend the file, but just write the content
• reliability as copying files could fail as soon as we know there is not enough space on the disk when creating the file, not after we copy a part of the file and find out there is not enough space for the rest of it

Proposed API

namespace System.IO
{
    public class FileStream : Stream
    {
        public FileStream(string path, FileMode mode)
        public FileStream(string path, FileMode mode, FileAccess access)
        public FileStream(string path, FileMode mode, FileAccess access, FileShare share)
        public FileStream(string path, FileMode mode, FileAccess access, FileShare share, int bufferSize)
        public FileStream(string path, FileMode mode, FileAccess access, FileShare share, int bufferSize, bool useAsync)
        public FileStream(string path, FileMode mode, FileAccess access, FileShare share, int bufferSize, FileOptions options)
+       public FileStream(string path, FileMode mode, FileAccess access, FileShare share, int bufferSize, FileOptions options, long allocationSize)        
    }
    
    public partial class StreamWriter : System.IO.TextWriter
    {
        public StreamWriter(Stream stream)
        public StreamWriter(Stream stream, System.Text.Encoding encoding)
        public StreamWriter(Stream stream, System.Text.Encoding encoding, int bufferSize)
        public StreamWriter(Stream stream, System.Text.Encoding? encoding = null, int bufferSize = -1, bool leaveOpen = false)
        public StreamWriter(string path)
        public StreamWriter(string path, bool append)
        public StreamWriter(string path, bool append, System.Text.Encoding encoding)
+       public StreamWriter(string path, bool append = false, System.Text.Encoding? encoding, int bufferSize = -1, FileOptions options = FileOptions.None, long allocationSize = -1)
    }
    
    public static partial class File
    {
        public static FileStream Create(string path)
        public static FileStream Create(string path, int bufferSize) 
        public static FileStream Create(string path, int bufferSize, FileOptions options)
+       public static FileStream Create(string path, int bufferSize, FileOptions options, long allocationSize)
    }
    
    public sealed partial class FileInfo : System.IO.FileSystemInfo
    {
        public FileInfo(string fileName)
        public System.IO.FileStream Create()
+       public System.IO.FileStream Create(FileOptions options, long allocationSize)
    }
 }

Note: Some of the methods above add FileOptions options to the arguments list. This has been done to avoid conflict with #24698 which is going to hopefully add them anyway.

Usage Examples

using (FileStream source = File.Open(sourceFilePath))
using (FileStream destination = File.Create(destinationFilePath, allocationSize: source.Length))
{
     source.CopyTo(destination);
}

Alternative Designs

The proposed design is so simple that it's hard to come up with an alternative.

Risks

The API itself does not introduce breaking changes, but using it in all *File*.Copy* and File.WriteAll* methods can potentially introduce breaking changes if we don't pay attention to what exception was thrown when we were running out of disk space so far.