Champion "Readonly ref" (C# 7.2)

[MadsTorgersen]

• Proposal added: proposals/csharp-7.2/readonly-ref.md
• Discussed in LDM
• Decision in LDM
• Finalized (done, rejected, inactive)
• Spec'ed

[alrz]

I'm not quite clear why this has to be a separate feature from readonly locals and paramerters?

[jaredpar]

@alrz they are very different features.

• readonly ref: restricts the ability to mutate the target of a ref variable.
• readoonly locals: restricts the ability to retarget a variable to a new location.

These features do have some overlap with st. t values. But overall are different features.

[alrz]

So potentially, we could have readonly in parameters at some point?

[jaredpar]

@alrz

Possibly but that's a bit of a redundant notation. All ref identifiers are unassignable by default. Hence adding readonly is just making it more explicit for no real gain.

[alrz]

I'm under the impression that you are distinguishing something like const char* and char* const, does that analogy work here?

[gafter]

We might support reassignment of ref variables in the future. In order for that to be possible, we will need to distinguish between ref readonly (a ref to something you can't write), readonly ref (a ref that you can't reassign), and readonly ref readonly (a ref that you can't reassign to something you can't write).

[Thaina]

readonly ref readonly (a ref that you can't reassign to something you can't write).

Should just use const. const local and const parameter

for ref readonly and readonly ref. I think there are sealed,fixed and lock. 3 keywords that currently cannot be local or param modifier. We should reuse these

[ufcpp]

Can in parameters be contravariant?

interface ISetter<in T>
{
    void SetValue(in T value);
}

[jaredpar]

@ufcpp

Can in parameters be contravariant?

No. The CLI only truly recognizes ref parameters. The C# notions of in and out are a construct specific to C#. There is no underlying runtime support for enforcing these notions and hence it can't be a part of the core CLI type system.

[Athari]

and readonly ref readonly

And here I thought that C++-style of consts will never be considered for C#... Are ~~~const~~~ readonly functions also something being considered?

[BreyerW]

Could this feature enable ref readonly property with both setter and getter (getter being ref readonly returning while setter being normal method)? Main use case is to be able to reliably call OnValueChanged events and the like while still being able to benefit from ref performance gain on struct

[soroshsabz]

ITNOA

I think #188 that describe about readonly parameters is more general than this issue and that proposal is superset of this proposal.

[jnm2]

ITNOA

What does this mean?

[Ultrahead]

I happened to find this champion as I was searching the repo for proposals of const/fixed refs.

In proposals/readonly-ref.md it is mentioned this example of XNA. Please due note that the proposed solution will not match perf of, in the example used, the ADD operation of XNA, since it's returning a copy of the resulting/new vector.

[Ultrahead]

In terms of C# 7, it should be something like, instead:

static void Add (in Vector3 v1, in Vector3 v2, out var result)
{
    // OK
    result = new Vector3(v1.X +v2.X, v1.Y + v2.Y, v1.Z + v2.Z);
}

[JosephTremoulet]

Would there be a way to elide the ldelema-implied type check when taking the address of an array element as a readonly ref? It would be nice if e.g. Voltile.Read on an array element (as in ThreadPool.TrySteal) didn't require a type test...

[BreyerW]

What about out shorthand for ref readonly return?

[jaredpar]

@JosephTremoulet

Would there be a way to elide the ldelema-implied type check when taking the address of an array element as a readonly ref? It would be nice if e.g. Voltile.Read on an array element (as in ThreadPool.TrySteal) didn't require a type test...

That's so tempting. Within the realm of C# it would be fine. We would never actually write to the location hence it should be fine to avoid the type check. But it's still possible for us to call out to other DLLS that ignore ref readonly (even though we have a mod req).

Hmm, then again, maybe the mod req could be enough to save us here. At that point the code is willfully ignoring the IL.

@VSadov

[VSadov]

@JosephTremoulet @jaredpar - yes, in the world of C# it would be safe to emit .readonly ldelema when fetching an element ref to pass to a ref readonly parameter.

However if the calee misbehaves, we can end up not just with something that is readonly unexpectedly mutated (we are kind of ok with that), but with a type-system hole. That would seem a bigger problem.

Here is the scenario:

string[] sArr = ... get an array of strings
object[] oArr = sArr;    // allowed!!

// bravely suppress type check here via  .readonly
Evil(oArr[2]);

void Evil([IsReadOnly] ref object arg)
{
     // muhahaha
     arg = new Exception();
}

[tannergooding]

For interop scenarios, Is the compiler going to emit a warning for static extern Vector3 Normalize([In, Out] ref readonly Vector3 value)?

That is, when the user attempts to explicitly tell the marshaller that ref readonly has copy-back semantics?

I did log an issue (https://github.com/dotnet/coreclr/issues/11830) requesting that the marshaller be updated to recognize ref readonly as [In] rather than [In, Out] (assuming that no attributes were specified).

[jaredpar]

For interop scenarios, Is the compiler going to emit a warning for static extern Vector3 Normalize([In, Out] ref readonly Vector3 value)?

No. The compiler doesn't impart any semantic meaning to [In] or [Out]. It would be pretty inconsistent to warn on [Out] ref readonly but not warn on [In] out.

[yizhang82]

@jaredpar Can the compiler emit [In] by default with ref readonly? This way runtime don't have to make any special assumptions about in semantics. This also aligns with out keyword being [Out] by default.
If user specify anything other than the default, the runtime would do what exactly user specifies.

[jcouv]

Some LDM notes: https://github.com/dotnet/csharplang/blob/master/meetings/2017/LDM-2017-05-31.md#ref-stuff

[tannergooding]

[Out] ref is CS0662: Cannot specify only Out attribute on a ref parameter. Use both In and Out attributes, or neither

[In] out and [In, Out] out are CS0036: An out parameter cannot have the In attribute (edited)

I would expect we also have [Out] ref readonly and [In, Out] ref readonly output CSxxxx: A ref readonly parameter cannot have the Out attribute

I would also expect that the compiler output ref readonly with an explicit System.Runtime.CompilerServices.InAttribute so that the marshaler behaves correctly and the readonly contract is not broken (https://github.com/dotnet/coreclr/issues/11830).

@VSadov, I believe you indicated the attributes are compiler features, rather than language features. Is this something we can ensure the compiler emits?

[tannergooding]

Also @jaredpar ^

[Thaina]

@Voidzer0 I was proposed #521

There was also #497 and I was mentioned in #497 (comment)

[bbarry]

How does the ability to represent an undefined value with a ref become the ability to state an out variable is a readonly reference?

There is some overlap in this particular use case, but there could be other motivating examples, for example in some interop (this is not a real api, but is inspired by one available in shell32.dll):

[DllImport("shell32.dll")]
public static extern IntPtr GetFileInfo(string path, out readonly FileInfo fi);

[Voidzer0]

Hey @Thaina
similar to @bbarry, I can't follow exactly and do not find this particular suggestion in the linked issues.
We do not need to put an undefined value into a ref, since we can make it point readonly to a default initialized variable/array element, in case no value with given key exists.

I see a similar syntax mentioned in "#497 (comment)" but there it is shortly discussed and then the conversation seems to switch focus to that null problem which we don't have, because we can always make the ref point to something "valid".

But you are right, I should have used "out ref readonly" rather than "out readonly" in my example, since I want to initialize the reference and not the value it is pointing to as out parameter:
if (myLookup.TryGetRef(key, out ref readonly T value)) { /* do something with value */ }

One thing said there seems wrong:

The out keyword already implies ref.

Is not exactly true, because with out the caller provides a reference to a piece of memory on the caller side, where then a value from the callee (Collection) is copied into.
What we would like here instead is to provide just a reference variable (no memory on caller side except for the reference/address itself) as out variable to the callee, which then is made pointing to a piece of memory (value) from the callee (Collection) and thus does not involve copying the value into a different memory location.
Is this suggested in your links?

[bbarry]

@Voidzer0 out does in fact mean ref in C#, specifically it means ref that is not yet assigned to and that the method must assign to before the method completes (in order to allow the definite assignment language contract).

In this particular proposal, a ref return value was permitted, which is similar to a ref parameter, and a readonly attribute was permitted on ref parameters (becoming in parameters) to state that the callee has readonly semantics, and a readonly attribute was permitted on a ref return value to state that the caller has readonly semantics on the return value.

Since this proposal doesn't deal with output parameters, I think you should enter a new issue for out readonly.

[mikedn]

What we would like here instead is to provide just a reference variable (no memory on caller side except for the reference/address itself) as out variable to the callee, which then is made pointing to a piece of memory (value) from the callee (Collection) and thus does not involve copying the value into a different memory location.

This is basically a reference to a reference, that's not something that the runtime supports.

Besides, it's not clear how that would address your problem - out arguments are required to always be assigned a value in the callee. If the collection doesn't contain the element you're looking for, what value are you going to assign to value in TryGetRef?

[Voidzer0]

@bbarry
Yes out is a reference but to memory on the caller side in which a value from the callee side is copied into.
And this is fine for small value types and reference types.
"out ReferenceType variable" gives a reference to the memory address of "variable" to copy a value into, in case of a reference type the value is the memory address of the pointer the callee is assigning to.
Why shouldn't it be possible to do the same with references to value types?
Just think about a large struct "LargeValueType", then "out LargeValueType variable" gives a reference to the memory of "variable" to copy a value of LargeValueType into, but this value is large and thus costly to copy.
Now imagine we could instead use reference variable as out parameter rather than a value variable via "out ref LargeValueType variable" (which is more or less the same as if we'd have a normal reference type variable).
This would give now a reference to the value of the reference variable (which is just a memory address) to the callee.
Then the callee puts a reference (address) to an internal value (eg. Array element) of LargeValueType into that reference.
Now the caller can operate directly on the large struct without any need of copying it, while doing contains check and retrival in one lookup.
In C++ at least this is doable, which is why I hope it could be done in C# as well.

@mikedn

Besides, it's not clear how that would address your problem - out arguments are required to always be assigned a value in the callee. If the collection doesn't contain the element you're looking for, what value are you going to assign to value in TryGetRef?

We assign a reference to a default initialized value or array element.
See @bbarry's code example which does that.
I would have used a default initialized member variable:
[code]
class MyTookup<T> // where T is a large struct
{
T defaultValue = default; // to refer to in case the element doesn't exist
T[] array;
...
public bool TryGetRef(int key, out ref readonly T referenceVariable)
{
if (key < 0 || key >= array.Length)
{
referenceVariable = ref defaultValue; // element not found assign ref to default value
return false;
}
referenceVariable = ref array[key];
return true;
}
}
[/code] (sorry I don't know how to make nicely indented code snipplets using this forum)
If this makes sense?

[HaloFour]

@Voidzer0

(sorry I don't know how to make nicely indented code snipplets using this forum)

You can use fenced code blocks:

https://help.github.com/articles/creating-and-highlighting-code-blocks/

The suffix for C# formatting is cs

[mikedn]

In C++ at least this is doable, which is why I hope it could be done in C# as well.

Not quite. C++ does not allow references to references, pointers to references, null references. It does allow references to pointers which is kind of equivalent to something like ref string x. And while its standard container classes make heavy use of references they handle cases like TryGetRef rather differently.

I would have used a default initialized member variable:

This would work but it has the disadvantage that your class is larger due to that extra defaultValue value member. There are ways to minimize the impact of this default value (e.g. make it static) but it probably cannot be completely eliminated.

It also seems rather risky - it's a form of "null" that's hard to detect. If someone forgets to check the return of TryGetRef you'll end up with some funny silent bugs.

You really should open a separate issue if you want to follow up, it's a very different situation than readonly ref.

[Voidzer0]

@mikedn

Not quite. C++ does not allow references to references, pointers to references, null references. It does allow references to pointers which is kind of equivalent to something like ref string x. And while its standard container classes make heavy use of references they handle cases like TryGetRef rather differently.

While I get that, I wonder if this can not be treated like a normal reference assignment, since an out parameter must be assigned to a valid value anyway.
Together with the new way of declaring a var while being out parameter, the reference does not have to exist before it is assigned (and thus is not pointing to nirvana).
Under these circumstances
ref readonly T r = ref lookup.GetRef(key);
is equivalent to:
lookup.GetRef(key, out ref readonly T r);
because assignment of r on creation is assured.
This makes it a normal reference assignment, rather than needing references to references here.
If we would use an unassigned ref local declared before, then this would be required indeed.

This would work but it has the disadvantage that your class is larger due to that extra defaultValue value member. There are ways to minimize the impact of this default value (e.g. make it static) but it probably cannot be completely eliminated.

I wouldn't do a lookup/dictionary if there wouldn't be many structs, so the one struct extra memory is neglectable.
This practice is quite common when using emplacement new in C++.
If I would use classes for all these structs the total overhead is likely much bigger.

It also seems rather risky - it's a form of "null" that's hard to detect. If someone forgets to check the return of TryGetRef you'll end up with some funny silent bugs.

I would never do that, if I wouldn't hand out a readonly ref of an immutable struct. Everything else would be borderline insane I guess xD
The reference is not supposed to be used if the TryGet-result is false anyway.

You really should open a separate issue if you want to follow up, it's a very different situation than readonly ref.

You are probably right, but given that there is a (less nice looking) work around (as @bbarry pointed out) and that reference semantic is only valid for declaring the out parameter in the call where it is assigned,
I guess the likelyhood that support for this is added is rather low I guess.
Thanks to all your comments I realize how niche the requirements are to use this optimization.
Still I did run into a case where this would have been quite useful, given that it is supposed to be exposed as an API for other devs.
I will see if I can make the workaround look a bit nicer for now.

[mikedn]

This makes it a normal reference assignment, rather than needing references to references here.

That's how it looks in C# but in IL this is still a reference (aka byref aka managed pointer) to a reference. I suppose it may be possible to cheat to an extent - byrefs can only live on the stack so a byref to a byref can actually be passed as an unmanaged pointer. But that's probably a can of worms as you end up with a method signature that's basically lying.

Still I did run into a case where this would have been quite useful, given that it is supposed to be exposed as an API for other devs.

For completeness it may be worth detailing how C++ containers handle this:

• std::map's "indexer" automatically inserts a default value if the key does not exist and returns a reference to the newly added value. This would work in C# if you're ok with this automatic insertion behavior.
• If automatically inserting a value is not desirable std::map offers find that returns an "iterator". If it's not the "end" iterator then it can be dereferenced and that gives you a reference to the stored value. This would too work in C# but probably it would be less efficient than in C++ as the "iterator" would likely need to store multiple values (e.g. an array reference and an index).
• std::vector's "indexer" returns a reference that upon use produces undefined behavior if the index is not valid. In C# you'd probably throw an exception in this case, exactly like arrays do.
• It's probably worth mentioning std::stack as well - its pop returns void. It can't a return a reference because popping destroys the referenced object. .NET's Stack.Pop shot itself in the foot here as it will generate a copy, unless you're lucky and the JIT inlines it and removes the useless copy.

I'm not aware of a case where standard C++ classes return references to some sort of hidden, null/default-like objects.

[Thaina]

I also support reference of reference. It also solve the return of multiple ref or Tuple of ref

[Voidzer0]

I use the new tools now on a daily basis and I got now stuck on a compiler behaviour I do not understand.
Now I tried to use the pattern @bbarry suggested to get a "TryGetValue with return references", but I encountered a case where even this workaround doesn't work.
How is this a compile error: (I simplified it as much as I could)

class CompileError
{
bool _someInternalValidCheck = false;
int _resultIfValid = 7;
int _resultIfInvalid = 0;
ref readonly int TryGetResult(out bool valid)
{
    if (_someInternalValidCheck)
    {
        valid = true;
        return ref _resultIfValid;
    }
    else
    {
        valid = false;
        return ref _resultIfInvalid;
    }
}

int _error = -1;
public ref readonly int Test()
{
    ref readonly int result = ref TryGetResult(out bool valid);
    if (valid)
        return ref result;
    return ref _error;
}
}

It says "result" was initialized in a way it can not be returned by reference, but it is a reference already, so why not?
What ever I try to change to make it work, it shows a different error.
But funnily when I simply get a rid of the "out bool valid" parameter it becomes valid, while this out boolean param should have absolutely nothing to do with the issue.
Now I think I am either borderline blind to not see the obvious or might I've run for real into a little bug with the new compiler?
Any help or insightful answer would be much appreciated.
(Btw. I am using VS 2017 (Version 15.7.5 newest), Console Application, targeting .NET Framework 4.6)

Just another thing:
Also I know return tuples are structs (ValueTuple) and can not have refs as members, but it would be so nice if the compiler could forget that limitation in certain moments and allow us to do:

(ref AnyType name1, ref AnotherType name2, YetAnotherType name3) = FunctionReturningTwoRefsAndAnotherValue(); 
// That would make things simpler, there is nothing unsafe if the result tuple is not stored anywhere, but I know that with current ValueTuple implementation it cannot be supported.

[svick]

@Voidzer0 Your issue is not really about readonly ref, it's a general limitation of ref returns. The best source for this I could find is this blog post:

In a general case, compiler has no knowledge of what is going on inside Callee. Conservatively, compiler must assume that any byref parameter or its field may be returned back by reference, so as long as any of the ref arguments are not safe to return, the result of the call is not safe to return either.
Note that in some cases, knowing the types of the ref parameters and the return type, it could be proven that the return can not possibly be referencing data from one of the parameters. However, it was decided to be conservative here for the sake of simplicity and consistency. (considering structs, interfaces and generics, the additional rules could get really complicated).

Here are the actual “safe to return” rules as enforced by the language:

[…]

6. a ref, returned from another method is safe to return if all refs/outs passed to that method as formal parameters were safe to return.

In your case, the valid variable is not safe to return, which makes the ref returned from TryGetResult not safe to return.

[Voidzer0]

Oh this is interesting, I was totally aware that locals are not safe to return, but this is quite around the corner.
However in this case the programmer would need to be plain evil to make it not safe to return.

1. I mean we are passing an out param not a ref. Shouldn't it rather be prohibitted to return an out param (by reference) rather than preventing this very valid code?
2. We are passing an boolean out parameter to the Callee, which returns an int ref parameter. How could a programmer ever make Callee refer to this boolean with an int ref? Casting would cause an exception and there is no reinterpret_cast in C#. How could that ever be not safe to return?

So I still don't buy that the compiler has to be this conservative (defensive) in such cases. There must be a way to relax at least some of these limitations to support more valid code.
I really hope the future C# versions make the compiler less defensive and more powerful in supporting safe cases.
But in this way I can not make use of @bbarry's workaround to solve my issue and need to do two lookups anyway.

My actual problem looks like this:
I have two hashtables (hashtable1 and hashtable2) containing int-struct-pairs and lookups return readonly refs to the structs.
If a key isn't in hashtable1, it must be in hashtable2 and most of the time keys are present in hashtable1.
Thus I could safe one lookup to hashtable1, if I could have a "TryGetRef" (a by ref equivalent of Dictionary<K,V>.TryGetValue(key, out value)).
Here some code showing the problem:

public ref readonly T GetCorrect(int id) // where T : struct
{ 
  // Compiling fine, but wasteful:
  return ref _hastable1.ContainsKey(id) ? ref _hastable1.GetRef(id) : ref _hastable2.GetRef(id);
  //                     ^  lookup                        ^ lookup                    ^ lookup
  // vs
  // Doesn't compile, but correct and more efficient:
  ref readonly var current = ref _hastable1.TryGetRef(id, out bool exists);
  //                                         ^ lookup
  return ref exists ? ref current : ref _hastable2.GetRef(id);
  //                       ^ NO LOOKUP              ^ lookup
  // => If key exists in _hastable1, only one lookup is needed, but two are required due to compiler limitations
}

I really hoped I could safe this additional lookup in C#, but it seems like I can't.

[svick]

@Voidzer0

Shouldn't it rather be prohibitted to return an out param (by reference) rather than preventing this very valid code?

Maybe, but I think it's too late for that. You can return an out parameter by reference, and changing that now would be a breaking change, so it's very unlikely to happen.

I still don't buy that the compiler has to be this conservative (defensive) in such cases. There must be a way to relax at least some of these limitations to support more valid code.

As I understand it, it doesn't have to. It was a design choice made at that point in time and it's one that could be relaxed now.

If this is something you want to see, I think you should open a new issue on this repo specifically about that. And it would probably help if you also had a set of rules for what exactly should be allowed. (Consider e.g. that a if a method takes a ref ValueTuple<int, int> parameter and returns ref int, its result can't be safe to return if the parameter wasn't.)

I really hoped I could safe this additional lookup in C#, but it seems like I can't.

I think you can, assuming you control the hashtable type. Though it would require more complex design: TryGetRef will return a struct that has two properties: bool Exists and ref readonly T Value. Internally, it stores index into your hashtable and uses that to retrieve the value.

[jaredpar]

@Voidzer0

Shouldn't it rather be prohibitted to return an out param (by reference) rather than preventing this very valid code?

From the perspective of correctness out is no different than ref. This is just a reality we have to deal with as other languages see out as ref. This is definitely unsafe for ref given our safety rules hence it's also unsafe for out.

I still don't buy that the compiler has to be this conservative (defensive) in such cases. There must be a way to relax at least some of these limitations to support more valid code.

Not sure how to convince you of this. But I can assure you that this is how we view out parameters in terms of ref safety.

[VSadov]

It was possible to pass out by reference since version 1.0
As long as it is assigned, it is a normal l-value. Not different from other l-values like ref arguments or fields.
Why would it be illegal to return it by reference?
Especially since you can lose track of out-ness very easily.

ref int M1(out int arg)
{
   arg = 42;
   return ref ReturnsItsArg(ref arg); // <- which part of this can be made illegal and not break things?
}

[Voidzer0]

Thanks a lot for these answers guys.

Don't get me wrong, I am utterly happy to have these features in C# and I am very positive about the direction that has been taken with C# 7+ and what is coming in version 8.
I am raising critics on a high level here, just because I want to maximize the utility of the new language features to get the most out of C# and maybe at some point I can use it just for everything what I am doing.

@jaredpar @VSadov
I am not fimiliar with CLR internals, but when out is just ref from the perspective of other .NET languages then this point is essentially lost.
Still let me elaborate my thoughts.

@VSadov:
Why would it be illegal to return it by reference?

I was just thinking from C# perspective, which could have considered "out" as a higher level construct which is bound to some rules, rules which would prevent it from being referenced in any other way than as "out", which in turn would allow the compiler to make my example code and generally more code a valid and safe option.
Because for me it makes sense, as "out" is something which is meant to be "filled in by the callee" and in this regard there is very limited use of returning out params per reference.
And if returning a reference to it would be prohibitted, then more code could be assured to be safe to return.
At least I hope everyone agrees, that it is far from obvious for a programmer why my code sample wouldn't be valid and this is just because someone could do something which has likely almost no use case, but forces the compiler to assume it to happen eventually.
But I get the technical issue that out is seen as a normal ref and thus makes certain language use and compiler optimizations impossible.

However I still see potential, despite this limitation:

Solution 1)

- If a reference in question to be not safe to return has a different type than (and is not a nested type of) the ref parameter passed to a Callee, then the reference returned is still safe to return (unless other reasons are found).
Agreed? Can a compiler see this as well?
I mentioned nested type to take this problem into account:

@svick:
(Consider e.g. that a if a method takes a ref ValueTuple<int, int> parameter and returns ref int, its result can't be safe to return if the parameter wasn't.)

Even if someone could have bad luck with types matching, in my case I would be sorted by this improvment.

Solution 2)

Furthermore especially now that with C# 7 out variables can be declared right in the place they are initialized (Thank you so much for this devs!), even my original and more elegant suggestion:
bool TryGetRef<T>(int key, out (ref) readonly T outReference)
could be made work, if the compiler would simply enforce that out references are always declared in-place (and thus no dangling reference can exist, since the callee is forced to initialize it).
An out reference could be treated with the same rules as a return reference.
This would avoid the other problem above on its base, because no returned reference need to be returned.
This should be safe and valid either, agreed?

Solution 3)

Well there is yet another idea which seems valid to me and would maybe be the most flexible of all
(bool keyExists, ref readonly var value) = ref _hastable1.TryGetRef(id);
I know this is not possible because ValueTuple<...> is a struct and can not have a ref member, but if it could, it would perfectly solve my problem.
And the only reason why it can't be done, is because ValueTuple is a normal mutable struct, since it was invented before (ReadOnly)Span.
But what would be if ValueTuple would be a ref struct instead?
Then many of the issues could be avoided, since no one can store a value tuple and it could easily hold plain (readonly) references like Span essentially does.
And I would have no need to use an out or ref parameter, because the only reason out parameters were needed, is to compensate the lack of mutiple return values.

Now refs can neither be returned in tuples nor passed by out parameters. Any method returning a ref is thus limited to one ref only and given the limitations for out parameters with respect to return refs, it is basically limited to only return the ref and nothing else and I think this is a rough limitation in quite some cases which would be worth to relax if anyhow possible.

No matter which of the three ideas would be considered, they all should be safe (correct me if I'm wrong) and would fix my current issue.

Which brings me to
@svick
First of all: Thanks a lot for always taking the time to answer questions precisely. It made me always understand quickly.

@svick:
I think you can, assuming you control the hashtable type. Though it would require more complex design: TryGetRef will return a struct that has two properties: bool Exists and ref readonly T Value. Internally, it stores index into your hashtable and uses that to retrieve the value.

I didn't think about this possibility and that would work for me.
But I hope you agree this is quite hacky, has additional overhead and is far from elegant, but a rather smart way to work around the problem indeed.
In the end, if no better solution can be found I might consider doing this, even if it might make my coding hands a bit bloody. ;)

@svick:
As I understand it, it doesn't have to. It was a design choice made at that point in time and it's one that could be relaxed now.
If this is something you want to see, I think you should open a new issue on this repo specifically about that.

If you still think this is the case after reading the other answers and if you think it is likely that things could change in this way, I would be clearly up for that.
My issue is I do not even know what to suggest in a new issue, since as I showed there is an entire spectrum of possible solutions that all would work for me, but unfortunately none of them seems to work currently.
Since I am new to this forum, I would appreciate any help to find the right wording and format for such an issue thread.

Cheers everyone for patiently reading and answering all the questions.

[tomoyo255]

A very long time ago, in C++98, I wrote a template class LazyPtr<T> (a.k.a a smart copy-on-write pointer), that acted somewhat as a shared pointer, but would automatically clone (T* T::clone() had to exist) the pointed dynamically allocated data of type T if a non-const access was done, whereas a const access wouldn't duplicate data. The clone only happens when the counter was strictly greater than 1. The duplication effectively split the read-shared data and thus decremented the original counter, while setting the duplicated data counter to one.

For example, if three instances of the pointer existed pointed to a single shared data:

LazyPtr (count = 3) -> write through my instance of the smart pointer => leaves other smart pointers like LazyPtr (count = 2) and my cloned-then-written copy held by LazyPtr (count = 1)

L3 -> write => L2 and L1 (that was written to)

This is very useful where you've got lots of business data in a cache, but you need times to times to tweak one data (let's say bump a rate curve if you're doing finance) while leaving the original data in place. This usually leads to the dilemma, should I clone or not? How do I manage my data to keep the original safe while not consuming to much process memory. Shared pointer doesn't really help, etc. Here the LazyPtr solves this elegantly in a single move.

But,

Here comes C# that doesn't provide this-constness at method level (or readonly in C# dialect). The question is, how can I provide in C# as elegantly a copy-on-write pointerreference paradigm when I can't differentiate const from non-const method calls. Or to be more accurate, how can I make sure the users of my copy-on-write pointer don't mix it all up as constness/non-constness can't be enforced by the compiler.

So for now, C# 7.2 supports ref readonly which protects against non-const access... on properties only. Indeed, I wrote a test where I called a method modifying the struct contents from a ref readonly and it worked as a charm, whereas I expected it to fail. What's the use of a qualifier that only guards property accesses and not method's.

A summary of my test (I didn't check that actual code, but it does the same).

struct A {
  int X { get; set; }
  void setX(int x) { X = x; }
}

class HoldA {
  A _a;
  public HoldA(int x) { _a = new A() { X = x }; }
  public ref readonly A RefReadonly { get => ref _a; }
}

public class Test { public static void test() {
  var ha = new HoldA(42);
  //ha.RefReadonly.X = 69; // *Illegal, good!!!
  ha.RefReadonly.setX(69); // Works, but really shouldn't.
} }

[HaloFour]

@tomoyo255

C# makes a defensive copy when you invoke setX, so while the syntax is legal it doesn't result of a mutation of the ref. It actually does this for every method because the language doesn't know whether or not a method might mutate the struct.

[tomoyo255]

@HaloFour, hello,

Interesting though, it means ref readonly actually already approximately acts like a copy-on-write reference. If you could put ref readonly into a collection, it means it could by itself somehow play the role of a copy-on-write reference in a business object cache for example.

But ref readonly playing the role of a copy-on-write reference in a collection would be very sensitive and prompt to copy on even the gentlest flick.

But I suspect already that an IDictionary<K, ref readonly T> doesn't work... But maybe you could encapsulate a regular IDictionary<K, T> //where T : struct and provide an API only returning ref readonly T...

Hmm, maybe, just maybe...

[JesOb]

For now it looks like ref readonly return act not only like implicit copy on write, but also like implicit copy on access in case of access not field.

Suggest to add compiler warning or even better compiler switch to generate error instead of creating implicit copy, so programmer needs to create explicit copy to write to.

Another suggestion is to add readonly methods that will have this parameter passed as in parameter so suth methods (and properties) will be save to call on readonly ref returned structs.

[CyrusNajmabadi]

@Jes28 you can do this by writing an analyzer.

[JesOb]

I dont think that writing analyzer myself to work around language or compiler bugs is good way to go.

I am not alone and just realized that C# 8 already have all this :)

[CyrusNajmabadi]

I dont think that writing analyzer myself to work around language or compiler bugs is good way to go.

It's not a compiler bug afaict. It's working as spec'ed. If you want to place additional restrictions on the language, the right way to do that is per analyzers.