[Proposal]: Lock statement pattern (VS 17.10, .NET 9)

[stephentoub]

Lock statement pattern

• Proposed
• Prototype: Not Started
• Implementation: Add Lock object feature roslyn#71716
• Specification: https://github.com/dotnet/csharplang/blob/main/proposals/lock-object.md

(This proposal comes from @kouvel. I've populated this issue primarily with text he wrote in a separate document and augmented it with a few more details.)

Summary

Enable types to define custom behaviors for entering and exiting a lock when an instance of the type is used with the C# “lock” keyword.

Motivation

.NET 9 is likely to introduce a new dedicated System.Threading.Lock type. Along with other custom locks, the presence of the lock keyword in C# might lead developers to think they can use it in conjunction with this new type, but doing so won't actually lock according to the semantics of the lock type and would instead treat it as any arbitrary object for use with Monitor.

Detailed design

Example

A type would expose the following to match the proposed pattern:

class Lock : ILockPattern
{
    public Scope EnterLockScope();

    public ref struct Scope
    {
        public void Dispose();
    }
}

public interface ILockPattern { }

EnterLockScope() would enter the lock and Dispose() would exit the lock. The behaviors of entering and exiting the lock are defined by the type.

The ILockPattern interface is a marker interface that indicates that usage of values of this type with the lock keyword would override the normal code generation for arbitrary objects. Instead, the compiler would lower the lock to use the lock pattern, e.g.:

class MyDataStructure
{
    private readonly Lock _lock = new();

    void Foo()
    {
        lock (_lock)
        {
            // do something
        }
    }
}

would be lowered to the equivalent of:

class MyDataStructure
{
    private readonly Lock _lock = new();

    void Foo()
    {
        using (_lock.EnterLockScope())
        {
            // do something
        }
    }
}

Lock pattern and behavior details

Consider a type L (Lock in this example) that may be used with the lock keyword. If L matches the lock pattern, it would meet all of the following criteria:

• L implements interface ILockPattern
• L has an accessible instance method S EnterLockScope() that returns a value of type S (Lock.Scope in this example). The method must be at least as visible as L. Extension methods don't qualify for the pattern.
• A value of type S qualifies for use with the using keyword

A marker interface ILockPattern is used to opt into the behaviors below, including through inheritance, and so that S may be defined by the user (for instance, as a ref struct). For a type L that implements interface ILockPattern:

• If L does not fully match the lock pattern, it would result in an error
• If a value of type S may not be used in the context, it would result in an error
• If a value of type L is implicitly or explicitly casted to another type that does not match the lock pattern (including generic types), it would result in a warning
  • This includes implicit casts of this to a base type when calling an inherited method
  • This is intended to prevent accidental usage of Monitor with values of type L that may be masked under a different type and used with the lock keyword, such as with casts to base types, interfaces, etc.
• If a value of type L is used with the lock keyword, or a value of type S is used with the using keyword, and the block contains an await, it would result in an error
  • The warning would only be issued if in-method analysis can detect it
  • This is intended to prevent the enter and exit from occurring on different threads. Monitor and Lock have thread affinity.
  • SpinLock is optionally thread-affinitized. It can opt into the pattern, but then usage of it with the lock or using keywords would still disallow awaits inside the block statically.

SpinLock example

System.Threading.SpinLock (a struct) could expose such a holder:

struct SpinLock : ILockPattern
{
    [UnscopedRef]
    public Scope EnterLockScope();

    public ref struct Scope
    {
        public void Dispose();
    }
}

and then similarly be usable with lock whereas today as a struct it's not. When a variable of a struct type that matches the lock pattern is used with the lock keyword, it would be used by reference. Note the reference to SpinLock in the previous section.

Drawbacks

• If an existing reference type opted-in to using the pattern, it would change the meaning of lock(objectsOfThatType) in existing code, and in ways that might go unnoticed due to the general unobservability of locks (when things are working). This also means if System.Threading.Lock is introduced before the language support, it likely couldn't later participate.
  • Also if one library L1 that exposes the type is updated to opt into the lock pattern, another library L2 that references L1 and was compiled for the previous version of L1 would have different locking behavior until it is recompiled with the updated version of L1.
• The same thing is achievable today with only a little bit more code, using using instead of lock and writing out the name of the enter method manually (i.e. writing out the lowered code in the first example).
• Having the lock semantics vary based on the static type of the variable is brittle.
• ILockPattern itself doesn't match the lock pattern, though an interface could be created to match the lock pattern.

Alternatives

• Use an attribute instead of a marker interface. A marker interface was used because it works more naturally with inheritence. Although AttributeUsageAttribute.Inherited = true could be used for an attribute, it doesn't seem to work with interfaces.
• Use a new syntax like using lock(x) or other alternatives instead, which would eliminate any possibility of misusing values of the type with Monitor.
• System.Threading.Lock could be special-cased by the compiler rather than having it be a general pattern.
• Various naming options in the pattern itself.
• Instead of having a holder type, instead have the compiler explicitly recognize certain enter/exit methods by naming convention or attribution.
• Potentially allow/require the ref keyword for certain uses, e.g. in the SpinLock example.
• The pattern as suggested doesn't support being hardened against thread aborts (which are themselves obsoleted). It could be extended to now or in the future.

Unresolved questions

Design meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-05-01.md#lock-statement-improvements
https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-10-16.md#lock-statement-pattern
https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-12-04.md#lock-statement-pattern

[tfenise]

1. If a System.Threading.Lock lo becomes typed as Object at compile-time for whatever reason, lock(lo) would break silently. For example:

Dictionary<string, object> dic = new();
Lock lo = new();
lock(dic["my lock"] = lo) {}//this does not work.

2. This pattern match design creates potential problems for codes which implement custom locks and want to take advantage of this new language feature. For example, suppose I implement my own lock:

class MyLock
{
    public LockHolder EntersLockWithHolder() {...}

    public struct LockHolder
    {
        public void Dispose() {...}
    }
}

...

static readonly MyLock lo = new();

static void MyMethod()
{
    lock(lo)
    {
        ...
    }
}

Everything compiles even though the spelling EntersLockWithHolder is wrong and lock(lo) does not do what I intended.

This may also be problematic if MyLock is first written against .NET 8 and is later ported to .NET 7 or .NET Framework for whatever reason. In these cases, the codes would also break silently.

This problem could be solved by introducing a custom attribute only to be found in .NET 8 BCL that must be present on such lock types. The compiler would then detect any misspelling, and backporting to earlier version would result in a compilation error.

3. This new design creates inconsistency between newer lock types that should be used with lock(...) and older lock types like Mutex that should not be used with lock(...).

4. If this new design is implemented in C#, it should also be implemented in VB.NET, as VB.NET has SyncLock which is basically equivalent to lock of C#.

[vladd]

If we consider implementing this feature, shouldn’t we consider its async counterpart which is lowered to await using, too?

[HaloFour]

@vladd

If we consider implementing this feature, shouldn’t we consider its async counterpart which is lowered to await using, too?

I believe that should follow a completely different pattern as locks should have to opt-in to being "async"-friendly.

[HaloFour]

@tfenise

If a System.Threading.Lock lo becomes typed as Object at compile-time for whatever reason, lock(lo) would break silently.

I have to agree, I think there's enough danger of these newer-style locks being accidentally treated as monitor locks by the compiler in any case where the pattern just happens to not match, with no warning to the developer. I think that needs to be added to the list of "Drawbacks" for this proposal.

Is the plan to also extend this to the myriad of other locks or synchronization primitives that exist in the BCL?

[svick]

Would it be feasible to introduce the ability of a type to forbid using Monitor on it into the runtime? Let's say this feature would be enabled by annotating the type as [ForbidMonitor]. Then the new Lock could look like this:

[ForbidMonitor]
class Lock
{
    public LockHolder EnterLockWithHolder();

    public struct LockHolder
    {
        public void Dispose();
    }
}

And used like this:

private readonly Lock _lock = new();

void Foo()
{
    lock (_lock) // lowers to EnterLockWithHolder()
    {
    }

    lock ((object)_lock) // lowers to Monitor.Enter, but throws at runtime
    {
    }
}

[HaloFour]

@svick

How would that be enforced? If you upcast the Lock it's just a normal object and the compiler wouldn't know to not use Monitor.

[alexrp]

@HaloFour As per the comment, it would be runtime-enforced. Presumably, if [ForbidMonitor] is applied to the type, some bit in the method table would be set to indicate that any Monitor methods called on the object should throw.

[svick]

@HaloFour I meant that the compiler would use Monitor, but the implementation of Monitor.Enter would detect that the type of the object is one of the annotated ones and would throw an exception.

But I don't know if that could be implemented without making every existing use of lock slower.

[HaloFour]

@alexrp @svick

Sorry, I see what you meant after another re-reading of the comment. It'd prevent a subtle break, but you could still end up with exceptions at runtime if you're not careful.

I'd love to see more language support around locking types. It's always annoying me that none of the lock types in the BCL even offer support for using, and I've written a bunch of extension methods specifically to enable that. It's nice to see that they're considering it for this new Lock class. I, personally, think that's sufficient. The problem with lock is that it's always going fallback to Monitor, and unless we want to make generic support for these locks a part of the runtime itself it feels a bit too dangerous to me to try to expand on that, at least without some novel guardrails. Honestly, I wish .NET could deprecate Monitor in general.

[timcassell]

@vladd

If we consider implementing this feature, shouldn’t we consider its async counterpart which is lowered to await using, too?

I believe that should follow a completely different pattern as locks should have to opt-in to being "async"-friendly.

Perhaps await lock?

Also, though I have seen await using used for an async lock before, I don't think it makes much sense. The popular AsyncEx library just uses using. So await lock (_lock) would lower to using (await _lock.EnterLockWithHolderAsync()).

[TahirAhmadov]

Hmm, my initial reaction is, not only we shouldn't add to lock() {} in C#, but we should discourage its use, and instead recommend the using(...) {} approach, even for the Monitor-based locking. This seems to be too much of a specific type of feature to be part of the language. And I also don't like the lock suddenly doing different things depending on the type's surface area.
However, I don't think the Monitor locking is bad idea by itself, using something like using(Monitor.Lock(obj)) { ... } is perfectly fine with me and I use lock currently in many places, it's just that it's really not a language-level feature in my opinion.
PS. To clarify, from a clean slate perspective, I wouldn't have made all .NET objects "lockable" with Monitor; it should be a feature offered by a special type, similar to Mutex/etc. But given the legacy, the Monitor just isn't going anywhere any time soon.

[TahirAhmadov]

Having thought about it, I would have liked in a "perfect world" to have lock (and await lock) statement in C#, which would do something similar to what the OP proposes, as syntactic sugar, and then a few different types could be offered in the BCL to address various common usage scenarios. However, given how lock is now tightly related to Monitor, it's not a good idea to expand it. Perhaps, we can introduce a new keyword here? Like synch and await synch? Then lock and Monitor can both be obsoleted over time with suggestions/warnings, auto-fixers, etc.

[jaredpar]

If a System.Threading.Lock lo becomes typed as Object at compile-time for whatever reason, lock(lo) would break silently. For example:

Correct and that is an explicit design choice. This proposal breaks compat for any type which is being locked on today that just happens to fit the new locking pattern. Casting to object in a lock is the mechanism for restoring the existing behavior. Essentially, it's a key part of the proposal that it be allowed and have this behavior.

lock(dic["my lock"] = lo) {}//this does not work.

This particular case is not very realistic to me. In practice, it's very uncommon for the expression inside a lock to be anything other than a variable (local or field) so the chance of this type of problem feels very low. I think the more realistic case is a Lock instance getting converted to object as a part of a more vanilla call and the receiver later decides to lock on it. Even that seems fairly unlikely cause locking on instances you don't own is a dangerous behavior.