[API Proposal]: APIs for working with unix file mode

[tmds]

Background and motivation

Add direct support for getting/setting Unix file permissions and special bits.

These APIs allow to restrict/extend access to files and directories from .NET.

There have been some requests for these APIs: #925, #928, #17540.

The tar implementation can make use of them (#65951) without resorting to internal APIs.

API Proposal

namespace System.IO;

[System.FlagsAttribute]
public enum UnixFileMode
{
// From https://github.com/dotnet/runtime/issues/65951.
    None = 0,
    OtherExecute = 1,
    OtherWrite = 2,
    OtherRead = 4,
    GroupExecute = 8,
    GroupWrite = 16,
    GroupRead = 32,
    UserExecute = 64,
    UserWrite = 128,
    UserRead = 256,
    StickyBit = 512,
    SetGroup = 1024, // or, more generic name: GroupSpecial
    SetUser = 2048,  // or, more generic name: UserSpecial

// Maybe:
    AccessPermissionMask = UserRead | UserWrite | UserExecute | GroupRead | GroupWrite | GroupExecute | OtherRead | OtherWrite | OtherExecute,
    DefaultFileOpenPermissions = UserRead | UserWrite | GroupRead | GroupWrite | OtherRead | OtherWrite
}

static partial class Directory
{
    // Set mode when creating file, returns SafeFileHandle.
    public static SafeFileHandle OpenHandle (string path, FileMode mode = FileMode.Open, FileAccess access = FileAccess.Read, FileShare share = FileShare.Read, FileOptions options = FileOptions.None, long preallocationSize = 0, UnixFileMode unixCreateMode = UnixFileMode.DefaultFileOpenPermissions);

    // Set mode when creating directory
    public static DirectoryInfo CreateDirectory(string path, UnixFileMode unixCreateMode);
}

static partial class File
{
    public static SafeFileHandle OpenHandle (string path, System.IO.FileMode mode = System.IO.FileMode.Open, System.IO.FileAccess access = System.IO.FileAccess.Read, System.IO.FileShare share = System.IO.FileShare.Read, System.IO.FileOptions options = System.IO.FileOptions.None, long preallocationSize = 0);
    [EditorBrowsable(EditorBrowsableState.Never)]
    public static SafeFileHandle OpenHandle (string path, FileMode mode, FileAccess access, FileShare share, FileOptions options, long preallocationSize = 0); // Existing API, marked as Never Browsable to add a new default parameter.

    // Get/Set from SafeHandle, (.NET 7) https://github.com/dotnet/runtime/issues/20234
    public static UnixFileMode GetUnixFileMode(SafeFileHandle fileHandle);
    public static void SetUnixFileMode(SafeFileHandle fileHandle, UnixFileMode mode);
}

class FileSystemInfo
{
    // Get/Set from string path
    public UnixFileMode UnixFileMode { get; set; }
}

// Set mode when creating file, returns FileStream (by `File.Open`).
class FileStreamOptions
{
    // Set mode when creating file
    public UnixFileMode UnixCreateMode { get; set; } = UnixFileMode.DefaultFileOpenPermissions;
}

// `UnixFileMode` replaces `TarFileMode` (https://github.com/dotnet/runtime/issues/65951)
namespace System.Formats.Tar
{
-    enum TarFileMode { ... }
}

Notes

On Windows/non-Unix:

• The unixCreateMode arg on Directory.CreateDirectory/File.OpenHandle and FileStreamOptions.UnixCreateMode are ignored. These APIs can be used in in cross-platform code.

• FileSystemInfo.UnixFileMode get returns UnixFileMode.None.

• FileSystemInfo.UnixFileMode set throws PNSE.

APIs that create a file/directory (Directory.CreateDirectory, FileStreamOptions.UnixCreateMode, File.OpenHandle) have 'POSIX behavior':

• If the file/directory exists, the mode argument is ignored.

• The OS kernel filters the provided mode by the process umask. The umask protects users from unintentionally opening up permissions. A regular Linux user has a umask of 0002 which prevents it from giving write permissions to other. Use-cases that require the exact mode, must make an additional call to SetUnixFileMode. See example 3.

• note: creating and setting permissions is atomic to ensure permissions are applied immediately

UnixFileMode can replace TarFileMode introduced in #65951.

AccessPermissionMask, DefaultFileOpenPermissions enum values.

• These are convenient masks. Example usage of these masks:

  runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.Unix.cs

  Lines 45 to 51 in b42adad

  	// Only extract USR, GRP, and OTH file permissions, and ignore
  	// S_ISUID, S_ISGID, and S_ISVTX bits.
  	// It is off by default because it's possible that a file in an archive could have
  	// one of these bits set and, unknown to the person extracting, could allow others to
  	// execute the file as the user or group.
  	const int ExtractPermissionMask = 0x1FF;
  	int permissions = (int)Mode & ExtractPermissionMask;

  and

  runtime/src/libraries/System.Private.CoreLib/src/Microsoft/Win32/SafeHandles/SafeFileHandle.Unix.cs

  Lines 167 to 174 in b42adad

  	// If the file gets created a new, we'll select the permissions for it. Most Unix utilities by default use 666 (read and
  	// write for all), so we do the same (even though this doesn't match Windows, where by default it's possible to write out
  	// a file and then execute it). No matter what we choose, it'll be subject to the umask applied by the system, such that the
  	// actual permissions will typically be less than what we select here.
  	private const Interop.Sys.Permissions DefaultOpenPermissions =
  	Interop.Sys.Permissions.S_IRUSR | Interop.Sys.Permissions.S_IWUSR |
  	Interop.Sys.Permissions.S_IRGRP | Interop.Sys.Permissions.S_IWGRP |
  	Interop.Sys.Permissions.S_IROTH | Interop.Sys.Permissions.S_IWOTH;

  .

API Usage

Example 1: limit a file to be read/written by the owner only:

File.SetUnixFileMode("filename", UnixFileMode.UserRead | UnixFileMode.UserWrite);

Example 2: exclude 'Other' access to entries in a directory:

var di = new DirectoryInfo("somedir");
foreach (var fi in di.GetFileSystemInfos())
{
  fi.UnixFleMode = fi.UnixFileMode & ~(UnixFileMode.OtherExecute | UnixFileMode.OtherRead | UnixFileMode.OtherWrite);
}

Example 3: create new files/directories while extracting a tar file with the exact permissions of the tar file.

if (entry.EntryType == EntryType.Directory)
{
    DirectoryInfo directoryInfo = Directory.CreateDirectory(path, entry.Mode); // filtered by umask
    di.UnixFileMode = entry.Mode;
}
else if (entry.EntryType == EntryType.RegularFile)
{
    using var file = File.OpenHandle(path, entry.Mode); // filtered by umask
    File.SetUnixFileMode(file, entry.Mode);
}

Alternative Designs

No response

Risks

No response

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Add direct support for getting/setting Unix file permissions and special bits.

These APIs allow to restrict/extend access to files and directories from .NET.

There have been some requests for these APIs: #925, #928, #17540.

The tar implementation can make use of them (#65951) without resorting to internal APIs.

API Proposal

namespace System.IO;

[System.FlagsAttribute]
public enum UnixFileMode
{
// From https://github.com/dotnet/runtime/issues/65951.
    None = 0,
    OtherExecute = 1,
    OtherWrite = 2,
    OtherRead = 4,
    GroupExecute = 8,
    GroupWrite = 16,
    GroupRead = 32,
    UserExecute = 64,
    UserWrite = 128,
    UserRead = 256,
    StickyBit = 512,
    GroupSpecial = 1024, // or: SetGroup
    UserSpecial = 2048,  // or: SetUser

// Maybe:
    AccessPermissions = {User,Group,Other}{Execute,Read,Write}
}

static partial class Directory
{
    // Set mode when creating
    public static DirectoryInfo CreateDirectory(string path, UnixFileMode fileMode);

    // Get/Set using path
    public static UnixFileMode GetUnixFileMode(string path);
    public static void SetUnixFileMode(string path, UnixFileMode fileMode);
}

static partial class File
{
    // Set mode when creating
    public static SafeFileHandle OpenHandle(string path UnixFileMode fileMode, System.IO.FileMode mode = System.IO.FileMode.Create, System.IO.FileAccess access = System.IO.FileAccess.Write, System.IO.FileShare share = System.IO.FileShare.None, System.IO.FileOptions options = System.IO.FileOptions.None, long preallocationSize = (long)0);

    // Get/Set using path
    public static UnixFileMode GetUnixFileMode(string path);
    public static void SetUnixFileMode(string path, UnixFileMode fileMode);

    // Get/Set using SafeHandle, (.NET 7) https://github.com/dotnet/runtime/issues/20234
    public static UnixFileMode GetUnixFileMode(SafeFileHandle fileHandle);
    public static void SetUnixFileMode(SafeFileHandle fileHandle, UnixFileMode fileMode);
}

class FileSystemInfo
{
    // Get/Set
    public UnixFileMode UnixFileMode { get; set; }
}

Notes

On Windows/non-Unix:

• The argument for fileMode is ignored on Directory.CreateDirectory and File.OpenHandle to allow these APIs to be used in cross-platform code.

• {Get/Set}UnixFileMode methods throw PNSE.

• FileSystemInfo.UnixFileMode get returns UnixFileMode.None.

• FileSystemInfo.UnixFileMode set throws PNSE.

APIs that create a file/directory (Directory.CreateDirectory, File.OpenHandle) have 'POSIX behavior':

• If the file/directory exists, the mode argument is ignored.

• The mode argument is filtered by the process umask.

File.OpenHandle

• If the FileMode never creates a file (it does not match: FileMode.*Create*), ArgumentException is thrown.

• note: FileMode, FileAccess, FileShare have sensible defaults which are different from existing OpenHandle.

API Usage

Example 1: limit a file to be read/written by the owner only:

File.SetUnixFileMode("filename", UnixFileMode.UserRead | UnixFileMode.UserWrite);

Example 2: exclude 'Other' access to entries in a directory:

var di = new DirectoryInfo("somedir");
foreach (var fi in di.GetFileSystemInfos())
{
  fi.UnixFleMode = fi.UnixFileMode & ~(UnixFileMode.OtherExecute | UnixFileMode.OtherRead | UnixFileMode.OtherWrite);
}

Example 3: create new files/directories while extracting a tar file with the exact permissions of the tar file.

if (entryType == EntryType.Directory)
{
    DirectoryInfo directoryInfo = Directory.CreateDirectory(path, entry.Mode); // restricted by umask
    di.Mode = entry.Mode;
}
else
{
    using var file = File.OpenHandle(path, entry.Mode); // restricted by umask
    File.SetUnixFileMode(file, entry,Mode);
}

Alternative Designs

No response

Risks

No response

Author:	tmds
Assignees:	-
Labels:	api-suggestion, area-System.IO, untriaged
Milestone:	-

[tmds]

@eerhardt @carlossanlop @stephentoub @dotnet/area-system-io ptal.

[KalleOlaviNiemitalo]

File.OpenHandle with UnixFileMode looks useful for fixing the race condition https://github.com/AzureAD/microsoft-authentication-extensions-for-dotnet/issues/174 and could simplify NuGetExtractionFileIO.

[hez2010]

On Windows/non-Unix:

Shouldn't those API be put in a separate TFM net7.0-linux or net7.0-unix since they are not available on Windows platforms? In this way a warning can also be generated during compilation due to TFM mismatch, if users are trying to use unix APIs on Windows.

[eerhardt]

Shouldn't those API be put in a separate TFM net7.0-linux or net7.0-unix since they are not available on Windows platforms?

Those TFMs don't exist. We don't have TFMs for -linux or -unix.

But instead we can use the SupportedOSPlatform and UnsupportedOSPlatform attributes that work with https://docs.microsoft.com/dotnet/standard/analyzers/platform-compat-analyzer. Then you will get CA1416 warnings when you try using APIs that are unsupported in code that could run on Windows.

[eerhardt]

This looks great @tmds. Thanks for writing this up. I have the following questions:

1. Should we put the UnixFileMode on File.Create instead of File.OpenHandle?
2. In Example 3, why are we setting the UnixFileMode twice? Once on File.OpenHandle and then again with File.SetUnixFileMode?

3. The mode argument is filtered by the process umask.

   • Do we need a way to set umask from .NET? Or is this not a typical usage scenario in an app?
4. Do we want convenience Flags values for UnixFileMode? For example, UserExecute | UserWrite | UserRead and OtherExecute | OtherWrite | OtherRead.

[tmds]

Should we put the UnixFileMode on File.Create instead of File.OpenHandle?

I'll add it in addition to the OpenHandle method. They can be considered independent during API review.

In Example 3, why are we setting the UnixFileMode twice? Once on File.OpenHandle and then again with File.SetUnixFileMode?

When you provide a mode to open, the kernel applies the umask so the effective mode for the file may be more restricted.
When you provide a mode to chmod, the umask is not applied.

The .NET example makes two calls. One for open, one for fchmod. The second one is to undo the effect of the umask.

If we don't want that, we can perform an fchmod as part of the first .NET call.

Consequently, if the file already exists (e.g. FileMode.OpenOrCreate), we'd be changing its permissions. That may be unexpected.

Do we need a way to set umask from .NET? Or is this not a typical usage scenario in an app?

The umask is process wide, so it doesn't work well if several threads are setting it to a value they need.

[eerhardt]

Do we need a way to set umask from .NET? Or is this not a typical usage scenario in an app?

The umask is process wide, so it doesn't work well if several threads are setting it to a value they need.

I meant a public API that .NET developers could call to set the umask.

[tmds]

I meant a public API that .NET developers could call to set the umask.

Because .NET is multi-threaded, it would end badly.

One thread doing 'untar', would set it to umask 0.

While another thread doing File.OpenWrite would end up creating a file that is readable and writable by other, because umask became 0 instead of 2.

[eerhardt]

Because .NET is multi-threaded, it would end badly.

I can write multi-threaded C code. Does it end badly there as well?

One thread doing 'untar', would set it to umask 0.

I'm not talking about tar. I'm talking about any .NET application. umask is a system-level API that allows applications to manipulate file modes. Do .NET applications need to control it?

[tmds]

I can write multi-threaded C code. Does it end badly there as well?

Yes.

One thread doing 'untar',

By this I meant, use the API to untar a file: #65951.

Do .NET applications need to control it?

I don't think there are strong use-cases for this. They should use the SetUnixFileMode to avoid multi-threading issues with umask.

[tmds]

I had a look. nodejs and python APIs that create files preserve the default behavior of applying umask.

Both also provide a umask function. Still, I don't think we should add it in .NET because of the multi-threading issues. The other APIs allow to set the permissions in a thread-safe way without using umask.