Champion "Nullable reference types" (16.3, Core 3) about csharplang

[iam3yal]

In the following case:

var y = b ? "Hello" : null; // string? or error
var z = b ? 7 : null; // Error today, could be int?

I think that in both cases it should be <type>?; more and more people are using var in their codebase and switching to use a type just to evaluate an expression seems odd, however, in the case you really want it to be an error, using a type actually make more sense.

[gafter]

@gulshan Objects exist at runtime. Types exist at compile-time. This is about changing what happens at compile-time, not at runtime.

[Pzixel]

@mlidbom I'm against the ! syntax becuase it provides a lot of mess. Syntax should be consistent thus we need ? for nullables and just a type for common variables, but not ?/type for value types and type/! for reference ones. It's better to have a little breaking change than open this bottle with djinns. All constants must be of not-null types (it's obvious, because otherwise you just cannot have non-null constants), thus var x = some_constant should be of non-null type even if in previous language version it is inferred as nullable.

Another question if non-null type will be a csc part without CLR support or we will have it? I mean readonly is a csc hack and reflection allows to modify readonly field. Are we going to do the same with non-null references when you cannot pass nulls in C# but fully able to do it from reflection?

[jnm2]

@Pzixel

I mean readonly is a csc hack

Not exactly; it translates directly to the CLR's initonly and is taken very seriously in IL verification.
Reflection is the only hack here. But it's a reasonable hack. I mean, on a purely physical level, the memory is mutable if you can get a pointer to it. People would be able to mutate it anyway, so disallowing reflection to mutate initonly memory isn't really protection.

[jnm2]

@Pzixel I wish I even could write that. I can only write this:

public T GetValueOrDefault<T>(...) where T : class => null;
public T? GetValueOrNull<T>(...) where T : struct => null;

[mlidbom]

To my mind the most important issue, by far, to fix is the current need to manually validate that every single reference type parameter is not null at runtime. This latest proposal seems to be de-emphasize that to the point where it is just mentioned as a possible "tweak" at the end. The way I understand it you would get warnings but zero guarantees.

I believe I am far from alone in feeling that without fixing that this feature would be of limited value. Here is a vote for ensuring that runtime not-null guarantees are prioritized. It would probably remove something like 90% of the current parameter validation code that that C# developers write.

I would suggest going a step further than the "!" syntax and supplying a compiler switch that would automatically generate such checks for ALL reference type parameters not explicitly allowed to be null by marking them with "?". This would be a huge time saver and if implemented in the compiler it should be possible to implement optimizations that would make the cost negligible.

I would also suggest extending the "!" syntax to check for default(TValue) in struct parameters. So that you could, as one example, validate that a Guid is not Guid.Empty with just a single "!" character.

[HaloFour]

@Kukkimonsuta

At the boundary those types would still be adorned with attributes. They would likely work similarly to the DynamicAttribute which is used to denote when a normal object parameter should be treated as dynamic.

For example:

public Dictionary<int, (string, dynamic)> M() { ... }

//is really
[Dynamic(new bool[] { false, false, false, false, true })]
public Dictionary<int, ValueTuple<string, object>> M() { ... }

The array of bools counts recursively into the generic types and their generic type arguments as follows:

1. false - Dictionary<,>
2. false - int
3. false - ValueTuple<,>
4. false - string
5. true - object

[gafter]

@SamuelEnglard The compiler will certainly give a warning when a method's override relaxes the nullability guarantee on its returned value. Although these nullability annotations are recorded in assemblies as mere attributes, the compiler pays attention to them.

[HaloFour]

@Pzixel

No, that won't compile because both methods have the same signature: void SomeClass(System.String). You can't overload based on parameter attributes.

Yes, it's a shame that this feature can't have more teeth out of the gate, but I understand why. It's a question of adoption. Sure, there will be some developers who jump on this right out of the gate. They'll immediately opt-in and set those warnings to errors and spend the time and effort to correct their codebase as soon as possible. I'll be in that camp and as I suspect that most people here will be also. But for the majority of developers this represents a massive breaking change and a lot of additional work that they would rather not have to do (at least all at once) and if they are immediately greeted by a wall of error messages they'd be more likely to shy away from updating. Hopefully, over the course of a couple releases as the BCL and common third party assemblies are updated, the safeguards can become not only default but also stricter.

[gulshan]

I really want "Non-nullable reference types" as mentioned in the title, not just "Non-nullable references" actually proposed in the proposal. It seems the proposed backward compatible "soft-break" may bring more liabilities into the ecosystem rather than making the language safer. I think, maintaining binary backward compatibility, breaking source compatibility with an option for opt-out using explicit language version as an argument to the compiler is the way to go. That's just my opinion though.

[mlidbom]

@Pzixel
I don't really care about the syntax as long as I can get a guarantee that at runtime non-null references are not null. Regardless of which language calls the method, or if it is called through reflection I need to know that it will be non-null or an exception will be thrown. That guarantee not being present is the "billion dollar mistake" in my view and any "fix" that does not adress that is fatally flawed in my opinion.

[mattwar]

Because the issues related to having actual guaranteed non-nullable types in C# historically have scuttled any attempt to add such a feature ever, we've opted to take a different approach with this proposal. This proposal is not about making guarantees, its just about allowing you to declare your intent so the compiler can find most of your potential bugs. TypeScript has been doing this for a few years now.

[Pzixel]

@Kukkimonsuta I hope so because I'm tired of writing something like

public T GetValueOrDefault<T>(...) where T : class => null;
public T? GetValueOrDefault<T>(...) where T : struct => null;

I with I have only latter variant because if don't set class constraint, I cannot use null as return value, and if do, then I can't use T? to perform operation over value types. In this case I have to duplicate all APIs to work with classes and structs. If we accept unified syntax with ? we are able to write algorithm like this very consistently.

[iam3yal]

Just to add to @jnm2 by default reflection requires full-trust, specifically, when accessing non-public members.

[mlidbom]

@mattwar
Ah, that is unfortunate indeed. I'm afraid I missed the information about those flaws. Any links to information would be greatly appreciated.

@Pzixel In no way were any of my comments meant as criticism of your thoughts on any of this. From what I can tell we are in pretty much 100% agreement :)

[mattwar]

I have a prototype of an analyzer that uses attributes here: https://github.com/mattwar/nullaby

This proposal is superior to attributes because the null annotations can be encoded in and flow through the type system (in and out of generic type parameters).

In an attribute based system I can declare parameters and return types to have non-null or null intent easily enough, but I can't easily describe a parameter or return type having the type List<string?>.

[Richiban]

@benjamin-hodgson In your first example I would expect (2): the ?s collapse.

If we take the view that, since the value of T is not actually wrapped in any way to make the type T? then we can view T? as a type union with T and null (other languages represent this as T | null, so that's what I'll use from now on).

In these cases the types follow the rules of set unions, and T | null | null is equivalent to T | null.

This works in your example because (I assume) T is assignable to T?, so the line return _value!; is a T? that is asserted to be a T and the result is then assigned to a T?.

[shmuelie]

@Pzixel I wasn't arguing against this, sorry if it came out that way. I was more trying to say how I see the parallels between what C# vNext is trying to do and what TypeScript does with nulls.

[Joe4evr]

@navozenko You may be under the mistaken impression (like some others also were) that this feature introduces new types for nullable and non-nullable references, but that's not the case. A string? would still be the exact same thing to the CLR as a string, it's just that the former has some attributes on it to allow compilers to see if that variable/field/return value/parameter may be null and would warn if you try to dereference it without checking first. As I stated earlier:

A) it's not a runtime breaking change and B) doesn't hamper interop with prior language versions and stuff like generics. The only downsides then are A) it's opt-in behavior and B) it's not a silver bullet (developers can choose to ignore the warnings and the runtime can still allow null to be assigned anyway).

When it comes to the BCL, I presume that it would be updated with the appropriate annotations wherever a null is considered to be a possible/handled value, a Herculean task it may be. (Most of this feature would fall flat on its face if the BCL didn't lead the way, IMO.)

[gulshan]

Will default constructor also generate warnings for all non-initialized reference type members?

[yaakov-h]

Nullability adornments should be represented in metadata as attributes. This means that downlevel compilers will ignore them.

Can we have a serious discussion about using modopts (or even modreqs) to allow for nullability overloads, i.e. two methods who only differ by nullability?

This would be useful for functions where null input = null output, and non-null input = non-null output, or even vice-versa.

[HaloFour]

@yaakov-h

I believe that conversation has happened a couple of times already. modreq is not-CLS. modopt does allow for overloads but requires specific understanding on the part of all consuming compilers since the modifier must be at the very least copied into the call signature. Both break compatibility with existing method signatures. For something that will hopefully proliferate across the entire BCL very quickly using modopt would create a big hurdle to doing that.

[gulshan]

I think this feature is about objects and not types as it does not change anything about the actual type system. When nullability was introduced for value types, it was about the types. But that's not the case here. Quoting the proposal (emphasis mine)-

This feature is intended to:

• Allow developers to express whether a variable, parameter or result of a reference type is intended to be null or not.
• Provide optional warnings when such variables, parameters and results are not used according to their intent.

I think the "variables, parameters and results" (are the members of a class being excluded here?) can be easily replaced by "objects". The proposal is introducing non-nullable objects(even including value types) and putting them to be default with separate notation for nullable objects. So, the proposal can be renamed to-
"Default non-nullable objects" or something like that IMHO.

[gulshan]

@gafter I have two questions-

• Would typeof return same type for nullable and non-nullable references?
• And, as default returns null for classes, will FooClass foo = default(FooClass) generate warnings?

For nullable value types, both the answers are "no". But, I guess for nullable references, the answer is "yes". Because there is nothing changed about types. Only thing changed is, whether a reference should hold null or not. If I am wrong here, please correct me.

BTW, now I propose the title "Default to non-nullable references" or simply "Non-nullable references".

[gafter]

@gulshan Yes, and yes. The first because System.Type objects only exist at runtime. The second because the compiler knows that default(SomeReferenceType) is the constant null.

[Thaina]

Forgot to ask that. Is this feature would allow generic nullable type?

I mean code like this should be able to compile with this feature enabled?

public T? GET<T>(string obj)
{
    try { return HttpGet(url).Convert<T>(); }
    catch { return null; }
}

int n = GET<int>(url0) ?? -1;
var m = GET<MyCustomClass>(url1);

[gafter]

@Thaina Not as currently envisioned, no. There is no way to translate that into something expressible in the CLR (except by duplicating the method for each "nullable" unconstrained type parameter).

[orthoxerox]

@gafter how would duplicating work if C# can't support methods that have mutually exclusive constraint flags (gpReferenceTypeConstraint and gpNotNullableValueTypeConstraint) but have otherwise identical signatures from the viewpoint of overload resolution?

[yaakov-h]

The proposal doesn't take into considerations any of the discussion from the roslyn repo about telling the compiler "I know at this point in time that the value is not null. Do not warn me.", for example using a ! postfix operator.

Can we take that into consideration, or will #pragma warning disable be the only way around false positives?

[Richiban]

@gafter In @Thaina 's example, would it be appropriate to lift the return into a Nullable type at that point? I'm not sure of the feasibility of a compiler feature that sometimes works with Nullable<T> and sometimes works with plain T and hides the difference between the two to the compiler.

I was thinking that the original example:

	public T? GET<T>(string obj)
	{
	    try { return HttpGet(url).Convert<T>(); }
	    catch { return null; }
	}

would like this if you decompiled it:

	public Nullable<T> GET<T>(string obj)
	{
	    try { return HttpGet(url).Convert<T>(); }
	    catch { return null; }
	}

But in C# vNext you wouldn't know. The type just appears to be T? still.

[MikeyBurkman]

I'm still a bit curious how this could possibly be implemented with var without possibly breaking a lot of existing code:

var s = "abc";
...
s = null;

This is perfectly valid currently. However, what type should s be inferred to? If it's inferred to just String, then it'll break existing valid code. If it's inferred to String?, then it'll essentially force devs to make a choice between type inference and strict nullability checks.

[Pzixel]

@MikeyBurkman it's may be inferred in different ways based on project checkbox (like it's done for unsafe)

[mattwar]

@MikeyBurkman the var can infer the type to be string? with a known non-null value. This means the variable will pass all the tests and be able to be used as not-null until it is changed to the null value or a value not known to be not null.

[mattwar]

@Richiban you could only write that if T was either constrained to class or struct.

[MikeyBurkman]

@mattwar The issue with inferring it to the nullable value isn't that it's necessarily wrong, it's that it's extremely inconvenient, to the point that it becomes a nuisance. I don't recall seeing any suggestion about flow typing (where a variable's type can be narrowed by, for instance, a non-null assertion), so in order to pass that variable to a function not expecting a null, I'd either have to assign it some default value (foo(s || "")) which would be quite redundant in most cases, or I'd have to just assign it the non-null type explicitly, making var useless.

[HaloFour]

@MikeyBurkman

There have been multiple conversations on the Roslyn forums about this subject. The "nullability" isn't part of the type of the local, it's how it's used. So the following would produce zero warnings:

string? s = "1234";
int len = s.Length; // the compiler knows that s is not null here

So in most circumstances you wouldn't be forced to do anything special with the variable in order to placate the compiler, as long as the variable was definitely assigned to something that wasn't null.

[orthoxerox]

Another question if non-null type will be a csc part without CLR support or we will have it? I mean readonly is a csc hack and reflection allows to modify readonly field. Are we going to do the same with non-null references when you cannot pass nulls in C# but fully able to do it from reflection?

Yes, like readonly.

[Pzixel]

@mlidbom but I care about both. Of course, it's better to have non-null types with ugly syntax than nothing but it's not a question. And as you can see, people think that it should behave like readonly so you can easily pass null through reflection. And I really don't understand how syntax question makes it "fix" that does not adress that. I said that I dislike !, not the whole idea.

[mattwar]

@mlidbom unfortunately, so far all proposals that attempt to solve the null problem by making guarantees have been the ones to be too flawed to pursue. That's why we have this proposal. It doesn't make a guarantee, but it does help you find your bugs by letting you declare your intent.

[shmuelie]

Figured I throw my 2 cents in:

As the proposal stands it's more like an analyzer than a compiler feature to me. Since in the end it would have to be stored as attributes anyways why not sure them to start? While easy to use is great, easy to use and easy to misunderstand what you get is bad. Using attributes makes it clearer that there is no true guarantee here.

[HaloFour]

@SamuelEnglard

Probably because attributes are very limited as to where they can be applied. That and the amount of flow analysis would be quite a bit beyond what a bog-standard analyzer could be expected to perform.

Yes, it ain't perfect. Yes, you can defeat it by trying hard enough. But that really shouldn't matter. It should still catch the majority of cases, particularly the accidental cases.

I look at it like Java generics. They aren't perfect and it's not hard to intentionally trip up the compiler. But they're still immensely helpful and do what they're expected to do the vast majority of the time, particularly when the developer isn't trying to be "clever".

[shmuelie]

@HaloFour I do see that. I'd rather have this than not 😄

[Pzixel]

Yes, i'd defenitly like it to be a part of type system because otherwise we have no way to express List of non-null references. It complicates everything, but makes this feature much more powerful.

[oxfordemma]

I ended up taking a stab at this as well with attributes. https://github.com/FUR10N/NullContracts

This analyzer is definitely trick-able, but I think I'd prefer it that way. I went super strict with my first attempt, but it ended up being too frustrating to use.

A common scenario for me was checking for null on a field that could be null, and then using that value in some method that required a non-null parameter. If you go really strict, then the analysis probably has to throw away that null-check info since some other method call could change the value after your code checked it (example).
This resulted in far too many false-positive errors to actually be useful, so I had to relax it a bit.

I also hit a wall with the whole List of non-null references idea, but that was one of the assumptions I ended up giving it - iterators and list indexers are assumed not-null.

It's not perfect, but my team's pretty happy with the issues it's caught so far.

[jeffanders]

The current proposal only briefly touches on generics and has some open questions. I have added a proposal the attempts to go into detail regarding how non-nullable reference types would interact with generics and generic type parameters. See #403

[rorymurphy]

I tend to see this as something better handled as an analyzer function better handled with attributes, especially if there are edge cases it does not fully cover.

The comparison was made favorably to Java generics, but those are somewhat acknowledged to be a syntactical sugar kludge (and have never really worked as well as they should). Hoping .NET does not head down a similar path with this.

Would like to see something that provides a guarantee and also solves the runtime aspect. I see that the current proposal nullability being viewed as a usage restriction and therefore does not affect the underlying type of the variable. However, it seems like the issue could be handled by wrapping the reference type in a NonNullable<T> struct. I took a shot in a gist, including implicit conversion back to T. And certainly, it could be assigned a shorthand operator similar to T? for Nullable<T>.

Seems like that would allow the guarantee and address the runtime issues @mlidbom raised. An analyzer could key off references to the struct just as easily as the proposed new syntax for compile-time support. Trying to understand, given that the new syntax does not provide any runtime guarantee, what's the major advantage over a wrapper based approach (similar in spirit to Nullable<T>) that includes a runtime guarantee?

[Pzixel]

@rorymurphy I had proposed NotNullable<T> struct solution some time ago but then we found some drawbacks. You can find it in linked topics. However with CLR and language support they can be workarounded, I guess.

[shmuelie]

@rorymurphy I've spent days trying to get a NotNullable<T> to work. It works in very limited cases making it a bad fit for the language or framework. For an individual development team maybe...

[rorymurphy]

@SamuelEnglard - can you provide any specific challenges you had with it? Tough to respond without any specifics. Offhand, I don't see how the approach is any more limited than Nullable<T>.

[shmuelie]

@rorymurphy the primary issue is that it only works with concrete types. T cannot be interfaces or abstract classes without it starting to crack

[Richiban]

@rorymurphy @SamuelEnglard The big problem for me (as someone who also, some time ago, tried to write a NotNull type) is that you can't stop someone from writing default(NotNull<string>) or new NotNull<string>(). Which results in a NotNull instance that contains null. Due to the very, very deeply ingrained concept in the CLR that every type has a default value I'm not sure there's much you can do about it.

[yaakov-h]

Also, FormatterServices.GetUninitializedObject(typeof(NotNull<string>)).

[rorymurphy]

@Richiban - good catch, hadn't remembered how deeply embedded in the CLR value type initialization to 0 was embedded until I looked it up. Although since all these proposals are talking about compiler changes, couldn't this be overcome by a compile-time check that NonNullable is not used as a type parameter to any class/method that does a default initialization (either as you described, or simply by declaring a variable without explicitly initializing it)? Granted, that still leaves reflection as a gap to be solved.

[Richiban]

I'm sure this is exactly the sort of issue that the C# compiler team is dealing with right now... I'm sure that what they've come up with is some form of advanced null-tracking combined with additions to the type system, rather than any fundamental changes to the CLR re: nullability.

What they should be able to offer you is that any method written in C# vNext (let's say that's the version of the language that supports non-nullable reference types) will, if written in the correct manner, get decorated with a compiler attribute that the return result is guaranteed non-null (obviously not the case if anyone has used default(T)).

I imagine it will also be necessary to have a where T : null or where T : default generic constraint available for those occasions when you do want to be able to work with default values.

[benjamin-hodgson]

I have a question regarding generics, which I haven't seen (or at least couldn't find) addressed explicitly in this proposal or in @jeffanders's discussion in #403.

class Foo<T>
{
    private T? _value;
    public T Value
    {
        get
        {
            if (!ValueWasSet)
            {
                throw new InvalidOperationException();
            }
            return _value!;
        }
        set
        {
            ValueWasSet = true;
            Value = value;
        }
    }
    public bool ValueWasSet { get; private set; } = false;
}

In this example I've typed _value as a nullable T, because in between creating a Foo and setting its Value the _value field will be default(T). In order not to get a warning about _value being uninitialised I have to give it a nullable type. Of course there's no way to observe a null Value from outside this class. (I can see this coming up in classes like List<T>, which maintains a partially-initialised array of values but maintains an invariant that the uninitialised portion is inaccessible from outside.)

But this raises the possibility of some sort of doubly-nullable type. If I instantiate T to be a nullable reference type,

var foo = new Foo<string?>();

then what is the type of foo._value? Is it...

1. string??, allowing the ?s to build up? What does a type like string?? mean, exactly? Can I construct types with hundreds of ?s?
2. string?, collapsing all the ?s down into one? Then the ! in Value's getter would convert from string? to string?, which I find strange. It also doesn't obey the usual rules of type expressions, namely that the type checker doesn't perform any sort of evaluation or reduction of type expressions. (T[][] is not the same as T[]!)
3. Or would the language require some sort of where T is not nullable constraint on the generic parameter? That would restrict the usefulness of the class. (It clearly wouldn't be acceptable for List<T> to reject nullable Ts.)

More concerning is the question of representation. If I instantiate T to a value type,

var foo = new Foo<int>();

then how would _value be typed?

1. Nullable<int>? I find this strange: when T is a reference type then foo._value is represented in memory in the same way as T (a possibly null pointer) but when T is a value type then foo._value is represented differently than T (a struct containing a T and a bool). This would presumably require a change to the JIT, or a crazy system wherein the C# compiler generates two CLR classes for every C# class.
2. int? I find this strange as well: when instantiating T to a non-nullable reference type then _value is typed as a nullable T, but when it's a non-nullable value type then it's a non-nullable T.
3. Some sort of crazy system wherein it's typed as an int? for the purposes of Intellisense and type-checking but it's represented as an int?

Similar (although even more mind-bending) concerns apply when T is a nullable value type,

var foo = new Foo<int?>();

or itself a nullable generic type,

class Bar<T>
{
    private Foo<T?> _foo;
}

and so on...

Anyway the point is that my original Foo seems like perfectly reasonable code but could quite easily lead to surprising behaviour. Keen to hear feedback on my concerns!

[MikeyBurkman]

@mattwar The TypeScript compiler pretty much guarantees (assuming TS 2.x and you have the compiler check on) that anything that is not a Null type is in fact not null. In TS, Null is its own type, just like string and number are.

The only time the TS compiler is not going to be correct in this is in interop with JS and with libraries that were written without the compiler flag. To say that the TS compiler is not making null-safe guarantees is no different than saying that it doesn't guarantee that your string object is not actually a string at runtime. The compiler does I think the best job imaginable for nullability, if your codebase is all strictly TS 2.x code.

[shmuelie]

@MikeyBurkman I think what @mattwar is getting at though is that the runtime for TypeScript has support for null and nothing is stopping a user of your code or something at runtime from creating a null value. Theoretically this ability in C# should also would if ALL your code is C# vNext.

[Pzixel]

@SamuelEnglard Theoretically you have an ability to modify readonly field in C#, but it's not a concern. We always have some tradeoffs between security and possibility to implement. If compiler guarantees that there is no way to pass a null value in not-null function without reflection and using C# vNext it still be very useful. And don't forget, nowadays we have nothing to help us with nulls, and any help would be nice.

[MikeyBurkman]

That's just the thing though -- TS does almost nothing special with nulls as compared to other values. There is some syntactic sugar of course, and null assertions are a little bit different than, for instance, seeing if something is a string or number. But fundamentally, nullability is done through union types, and null is essentially now its own type with no functions/members on it.

The important change in TS 2.x was making null no longer a top-level type that is assignable to anything. Once they did this, then doing let x: string = null; made about as much sense as let x: string = 123;. This is a very fundamental difference from what C# has. As long as null remains a top-level type in C#, the implementation of this proposal will almost certainly differ a lot from what TS has, unfortunately.

Maybe Kotlin is a better choice of languages to copy? I don't know much about it personally, but it supposedly adds nullability checks and it certainly doesn't have a robust union type system like TS or Ceylon.

[jeffanders]

@benjamin-hodgson To answer your first question my proposal states "It is important to note that the nullable modifier is only encoded via an attribute and it therefore does not affect the runtime or assembly representation of T".

Therefore foo._value has type string at runtime. As far as the runtime is concerned string can legally contain nulls.

In my original proposal (dotnet/roslyn#4443) based on T! the very first point I address is runtime types. So the following conditions should always be true.

typeof(string?) == typeof(string) // true
string? s1 = ""; 
string s2 = "";
s1.GetType() == s2.GetType() // true

So for your Foo<int> example foo._value is just an int and your code as written still all works as expected for an int.

For Foo<int?>, foo._value is an int? (or Nullable<int>) and your code as written still all works as expected for an int? (that is to say you can set the Value property to null and retrieve that without an error).

I think it might be clearer to state that under my proposal #403 for a preserving type parameter T that T? should be pronounced as "defaultable T" rather than "nullable T".

Going back to the T? FirstOrDefault<T>(this IEnumerable<T> e) example from my proposal and running through your scenarios again but this time for the return type and what value would be returned for an empty sequence.

• string returns a defaultable string with the default of null.
• int returns a defaultable int with the default of 0.
• int? returns a defaultable int? with the default of null.

This preserves the existing runtime semantics while allowing the compiler to warn us at compile time if we use the return value in a way we should not.

[gulshan]

The latest sprint summery in Roslyn repo dotnet/roslyn#18719 mentioned this feature as "Non-Null References". I'm glad the word "type" was omitted.

[fubar-coder]

@MadsTorgersen @gafter When #52 requires a change to the CLR, then maybe it's an opportunity to add non-nullable reference types (enforced by the CLR) to the CLR?

EDIT: This would avoid the automatic if (ReferenceEquals(x, null)) check generation in most places which should result in a performance boost (and not a penalty) when using non-nullable reference types.

[fubar-coder]

If this feature gets implemented using the NotNullable<T> struct, then only the following parts of the CLR must be modified:

• Forbid initobj used by default(T) and new struct for NotNullable<T>
• Modify ldelem to ensure that the NotNullable<T> element is initialized
• Modify nativeGetUninitializedObject to check whether the requested type is NotNullable<T>

The CLR opcodes must be modified to disallow default initialization and we get full support for non-nullable reference types. This should be much easier than real non-nullable reference types.

It is easier to forbid default(T) (see #146) just for NotNullable<T> because we can easily check for ldelem if the returned element is initialized (or invalid) by checking if it's a null reference pointer.

The performance for a NotNullable<T> struct should increase as soon as dotnet/coreclr#11407 gets implemented.

There should be a project setting to enable automatic wrapping of reference types not annotated with the ? in a NotNullable<T> struct. This should also mean inserting .Value accesses whenever a member of T gets accessed. Compilers without support of NotNullable<T> will continue to use nullable reference types without breaking any compatibility. You could also explicitly use the NotNullable<T> struct without native support by the compiler.

An implicit conversion from nullable T to non-nullable T must emit a warning (or maybe even an error) - except when suppressed with a !. The use of the ! must not result in a null check.

Special consideration must be taken in cases like Dictionary<K,V>.TryGetValue(K, out V) with V being an NotNullable<T>. There are two possible solutions using new functions:

• Dictionary<K,V>.TryGetNullable(K, out V?) which is implemented as a default interface method of IDictionary or IReadOnlyDictionary
• bool TryGetValue<K, V>(this IDictionary<K, NotNullable<V>> dict, out V v) which is implemented as an extension method for IDictionary and IReadOnlyDictionary.

A NotNullable<T>? must be converted to T.

[Kukkimonsuta]

Nullability adornments should be represented in metadata as attributes. This means that downlevel compilers will ignore them.

It was noted before typeof(string?) is equal to typeof(string). However how will Task<string?> be represented? Consider following example:

var stringType = typeof(Task<string>).GetGenericArguments()[0];
var nullableStringType = typeof(Task<string?>).GetGenericArguments()[0];

// as stated before, types should be equal
stringType == nullableStringType;

// but this returns null
stringType.GetTypeInfo().GetCustomAttribute<NotNullAttribute>();

// and this somehow should return `NotNullAttribute`
nullableStringType.GetTypeInfo().GetCustomAttribute<NotNullAttribute>();

[Richiban]

@Kukkimonsuta

stringType.GetTypeInfo().GetCustomAttribute();

I think there will need to be a special provision made for generic methods in general, i.e. the compiler will have to tag methods to say "I guarantee that this method does not return default(T)".

[mattwar]

@Kukkimonsuta the attributes are not on the constructed generic type, they are on the members of types that refer to it, and in the compiler's logic only inside method bodies.

[Kukkimonsuta]

God that's ugly, but clever :) Thanks for explaining.

[fubar-coder]

Uh, isn't using dynamic also very slow?

[Pzixel]

@fubar-coder it's beside the point, but yes.

[HaloFour]

@Kukkimonsuta

I won't argue against that. 😁

@fubar-coder

Not due to the way they are encoded in attributes.

[Joe4evr]

@fubar-coder

If this feature gets implemented using the NotNullable<T> struct [...]

Nothing in your post is any relevant beyond this point because this proposal does not mention anything at all about a wrapper type. The whole thing is just compile-time static analysis, where the developer can declare that a variable/parameter/field/return type can or cannot have null as an intended/handled value.

In fact, as the team had already discussed over two years ago:

Probably the most damning objection to the wrapper structs is probably the degree to which they would hamper interoperation between the different variations of a type. For instance, the conversion from string! to string and on to string? wouldn't be a reference conversion at runtime. Hence, IEnumerable<string!> wouldn't convert to IEnumerable<string>, despite covariance.

And this was when T! was still a proposed syntax and types would have different variations to declare with. Now the proposal is that the types are really the same under the hood and the compiler assumes that null isn't an valid/intended value at declaration by default (which is like 95~99% of the time).

The attribute-based approach has been deemed the best option by the LDM team because A) it's not a runtime breaking change and B) doesn't hamper interop with prior language versions and stuff like generics. The only downsides then are A) it's opt-in behavior* and B) it's not a silver bullet (developers can choose to ignore the warnings and the runtime can still allow null to be assigned anyway).

* Would be pretty nice if a future version of VS would auto opt-in to nullability analysis on File -> New Project as part of the templates. It doesn't have to be the same version of VS that the feature ships with, just something to consider.

[fubar-coder]

My main problems​ with this approach are, that it might hurt performance and that it really doesn't help against NREs. Resharper uses the same approach and it's difficult to get right and you can still get NREs. IOW: A developer gains nothing beside a slight feeling that his code might be a little bit more stable.

[yaakov-h]

@fubar-coder I agree, but there are no good options at this point for enforcing non-nullability at runtime in a backwards-compatible manner.

I'd love to see something like Swift's nullability model, but that's clearly not going to happen.

What a developer does gain, if the analysis is correct or close to correct, is warnings on obvious places where null deferences might occur, and confidence that either the analysis system knows the code is safe

Or if a developer suppressed warnings (with #pragma or postfix-! or whatever), one would hope that the developer in question knows the codebase better than the flow analysis does.

I currently work on several extremely large codebases which employ Code Contracts. The value from that means I can count on my fingers the number of NREs we've had in Contracted areas of the codebase. It isn't perfect, but it's damn good (when it works), and I've love to replace it with this proposal.

[Joe4evr]

Something I've been wondering for a while: Will there be room for better refinement in versions after nullability analysis initially ships? Specifically, I'm thinking additional attributes that API authors can apply to indicate when some property/field can (or even will) be null and thus give more accurate feedback to consumers of the API.

[Pzixel]

@Joe4evr declare type without ! suffix and it's going to be nullable. What extra attributes you want here?

@ALL Do we really want ! syntax here? So we get inconsistent !, ? and no-suffix for nullable and non-nullable reference and value types? I understand that we want to persist a backward compatibility, but maybe we want to break things here like we have changed foreach loop closure in C# 4.0. I guess in virtually 99% of code we don't want nulls, so we'l have to just spam these ! everywhere in our codebases, just like ConfigureAwait(false) today. Why can't we just accept that starting with C# 8.0 string is a not-null type, and not nullable one? It's quite easy to migrate old code to new one, just add question marks everywhere. So we can just break things and write a simple migration tool that performes all required operations. It's much better that introducing an inconsistent syntax. Eric Lippert agree with me here:

Ritchie's wry remark illustrates the lesson. To avoid the cost of fixing a few thousand lines of code on a handful of machines, we ended up with this design error repeated in many successor languages that now have a corpus of who-knows-how-many billion lines of code. If you're going to make a backward-compatibility-breaking change, no time is better than now; things will be worse in the future.

[Kukkimonsuta]

@Pzixel I think ! was in earlier proposal and the latest proposal states that string would be not-null and string? would be nullable. Even though this is breaking change, there is also supposed to be opt in/out mechanism, so you can actually use C# 8.0 without being forced to update your code.

[navozenko]

I have a question: what will happen with the API of standard .NET libraries? Will it remain "as is" or will it be adjusted to a new syntax? For example, if the standard .NET method/property can input or return null, how should I use them?

[Pzixel]

@navozenko I don't see how syntax may change API. It stay the same, except that somewhere compiler will insert attributes like cannot be null. You don't consume source code of .Net libraries, you consume compiled binaries. And they remain the same except having some extra attributes that modern Visual Studio can warn that You check it on null when it cannot be it or Possible null dereferencing. Older VS won't see any changes.

[navozenko]

@Pzixel @Joe4evr
I did not understand, will all parameters in standard .NET libraries be interpreted as nullable? Or will the attributes "nullable" and "non-nullable" be placed in its?

For example, which list will return a LINQ query or ToString(): nullable or non-nullable?
That is, how will I write:

List<Foo> foos = items.Where(...).ToList();
string s = foo.ToString();

or

List<Foo>? foos = items.Where(...).ToList();
string? s = foo.ToString();

[Pzixel]

I did not understand, will all parameters in standard .NET libraries be interpreted as nullable? Or will the attributes "nullable" and "non-nullable" be placed in its?

The latter.

For example, which list will return a LINQ query or ToString(): nullable or non-nullable?
That is, how will I write:

I think it will be

List<Foo> foos = items.Where(...).ToList();
string? s = foo.ToString();

Because ToList cannot return null by design, while ToString can be overloaded and return any string including null.

[Joe4evr]

while ToString can be overloaded overridden and return any string including null.

Yes, but then the override has to be declared like this:

public override string? ToString()
{
    return null; //since null is a returned value, the return type should reflect that
}

[Pzixel]

@Joe4evr override (yes, thank you) cannot change signature. So, it will be stiring? in object. And yes, declaration of overriden method will be this one.

[sharwell]

@Pzixel object.ToString() has a non-null post-condition, so it would not be updated to have the return type string?.

[Pzixel]

@sharwell where?http://referencesource.microsoft.com/#mscorlib/system/object.cs,ff31a6bf27c58f89,references

[shmuelie]

@Joe4evr @Pzixel @sharwell overriding will not be effected by this since it's an attribute on the method. Attributes are not part of the signature.

[shmuelie]

@gafter true My point was that since it's not changing the signature you can change nullablity (whether or not you should is a different story). So even though ToString() on Object says not nullable you can make it nullable in your override.

[Pzixel]

@SamuelEnglard just because C# team doesn't want to make types first-class citiziens. You can assign null to not-null and suppress warning in the same manner. Or change readonly field via reflection. That's out of point, really. If base object say it's not null you cannot change it. You get same warning as setting null to not-null, it's actually an error, which is the best C# team can offer without breaking things.

[shmuelie]

My comment was not to say that what they're doing is wrong. My point was simply that while yes an override cannot change the signature of the method, nullability is not part of the signature (though it looks like it is). I'm not saying whether anything is good or bad.

[Pzixel]

My position that we have to treat these warnings as errors. Because they are warnings due to backward compatibility only. So it's really doesn't differ from other errors such as "not all members of interface are implemented".

I wrote a big post, but I have deleted it and summarize: nullability are part of signature on C# level. Yes, on IL it's the same, but for C# it's an error. This looks really like readonly - on IL level you can modify it, but not in C#. So if we are talking about CLR - yes, you can change nullability. But talking about C# - no way. You get a "warning" which actually is an error, think about it like if C# didn't have type errors at all but only "warnings". Yet another reason for treat warnings as errors.

[fubar-coder]

When this is solved using attributes, then wouldn't this cause problems with existing .NET runtimes? LDM-2017-02-21.md mentions this problem.

[Pzixel]

@fubar-coder it has nothing with generic attributes. Explain your position more verbose.

[fubar-coder]

@Pzixel I misread the title. I thought the problem would be that the .NET runtime might have problems with attributes attached to generic type arguments.

[Joe4evr]

Nah, the CLR has supported attaching attributes to generic type parameters forever (there's even an AttributeTargets value for them). It's attributes that are themselves generic that are talked about there.

[shmuelie]

@Pzixel

I wrote a big post, but I have deleted it and summarize: nullability are part of signature on C# level. Yes, on IL it's the same, but for C# it's an error. This looks really like readonly - on IL level you can modify it, but not in C#. So if we are talking about CLR - yes, you can change nullability. But talking about C# - no way. You get a "warning" which actually is an error, think about it like if C# didn't have type errors at all but only "warnings". Yet another reason for treat warnings as errors.

I get what you're saying, BUT nullability isn't even part of the signature on the C# level. You can't use it as a way to overload, so void A(string str) and void A(string? str) won't compile (unless I've misunderstood the standard).

[Pzixel]

@SamuelEnglard of course it compiles. It just produce a warning. All these changes won't produce any additional errors at all! So you can write literaly everything, messing any nulls and non-nulls and still have successful compilation. I guess you misunderstood the standard, yes. It's all about warnings, not errors. However, I'l do all my best to force these warnings to be errors on my projects when this feature releases.

[shmuelie]

// C# with non-null reference types
class SomeClass
{
    void SomeMethod(string str) { };
    void SomeMethod(string? str) { };
}

Isn't valid C# (let alone IL).

[Pzixel]

@HaloFour sorry, I misread it. I ment that if you have virtual void A(string) and override void A(string?) it will compile with a warning. Of course you cannot declare two methods with same signature in one class.

@gulshan I believe .net team just cannot afford such a breaking change you propose. I'd like this feature too. However, it require unreal amount of work for a small profit - safety at runtime. If you check everything at compile time you just don't need it. The only drawback is that you cannot do some things like have an overload of same type but which is nullable and so on. But I don't believe it's really worth to perform huge amount of work of these rare use cases. I was on your side some time ago, but now I see that .net team way is kinda better for many reasons.

[Joe4evr]

I think, maintaining binary backward compatibility, breaking source compatibility with an option for opt-out using explicit language version as an argument to the compiler is the way to go.

Remember that this conversation has been going on for years. I think both MS and the community discussed pretty much every possible option by this point, and after weighing all the pros and cons of each of those, MS concluded that this will be the one to go with.

Yes, it's opt-in and not the silver bullet that people had hoped for, but after so many discussions, it's damn well better than nothing at all. The knot as been cut, and the odds of changing the decision now is near impossible.

I ment that if you have virtual void A(string) and override void A(string?) it will compile with a warning.

Which is exactly what Gafter said.

[HaloFour]

Yes, it's opt-in and not the silver bullet that people had hoped for, but after so many discussions, it's damn well better than nothing at all.

I believe that it was also mentioned somewhere that the opt-in+warning approach might be the first iteration and that after wider adoption the default behavior from the compiler might evolve towards opt-out+error. I have no cite for that but this was probably during the Codeplex timeframe and I don't feel like digging through all of those issues and comments trying to find it.

[Joe4evr]

after wider adoption the default behavior from the compiler might evolve towards opt-out+error.

While I may or may not recall the same thing (human memory is quite easily tricked), it still won't be the silver bullet until it can be enforced at runtime.

To be clear: I'm perfectly fine with how the proposal stands right now. I'm just stating the argument that the last few skeptics still have.