Support non-nullable types. about sdk

[larssn]

Time to celebrate! 😁 🎉

[hereisderek]

I'd really love to see this get implemented, or better yet, kotlin/swift style optional variable support

[munificent]

I've heard this for 2 years now.

It's a very big change to the language, tools, and ecosystem. Programming languages don't move that fast. Note that in the meantime we have also shipped set literals, spread, if and for in collections, and extension members.

we have @nullable in Java more than 10 years ago

True. I wish we had added non-nullable types in the beginning. The language team at the time did not feel it was a priority. Only in the past couple of years have we been in a position to want to do this change. And doing so well after 1.0, when there are millions of lines of code out there, makes it much harder.

Note also that we are taking a more difficult but ultimately (we hope) more rewarding path than most other languages. Non-nullable types in Java, C#, TypeScript, and Kotlin are not sound. They will catch most of your nullability errors, but don't guarantee that they will catch all of them. That in turn means that compilers for those languages cannot generally rely on non-nullability in the type system as a lever for optimizations.

In Dart, we support both programs that contain a mixture of null-safe and unsafe code in order to enable a smooth, non-breaking migration. Designing that very hard and adds a lot of complexity. But we also will support a fully sound mode. When your program and all of the packages it uses have been migrated to non-nullable types and you run in that mode, you are statically guaranteed to suffer no nullability errors.

It's taking us longer to get there, yes, but we are also taking you farther.

[mit-mit]

This is now in Dart and Flutter beta channels:
https://medium.com/dartlang/announcing-dart-null-safety-beta-87610fee6730

[mit-mit]

This feature is now in tech preview, see this post for more details: https://medium.com/dartlang/announcing-sound-null-safety-defd2216a6f3

[ivnsch]

After almost a decade ;P

[DartBot]

This comment was originally written by [email protected]

---

I don't think it is possible to completely forbid null-references in a language that allows dynamic types.

However, on page 73 of the Dart language specification draft, the question is asked "Should we do something with respect to non-nullable types?" and my answer is YES, PLEASE, I'M BEGGING YOU.

[larssn]

So the latest Google I/O talks about this finally being implemented: Non-Nullable Types.

Is this part of any milestone yet? If so can we have it added to this thread?

It's a feature I'm very excited about, so I'm hoping its not too far off still.

[DartBot]

This comment was originally written by [email protected]

---

One example of changing behavior:

int foo = null; // throws NPE
int? foo = null // ok

If you strip off the types, you change the behavior of the program.

[sethladd]

FYI a relevant DEP is dart-archive/dart_enhancement_proposals#30

[DartBot]

This comment was originally written by [email protected]

---

I would suggest to borrow C# concept which separate value type from object and only object can be null.

If we want value type (like int) be null we can use a constructor and wrap it as a object nullable<int>.

A plain int can be null is usually a disaster for dev..

[eernstg]

With a dev sdk, you can use --enable-experiment=non-nullable. The analyzer will report a number of nnbd errors, but most other things are not yet working. You may or may not consider this to be 'updates', but there is certainly a lot of work being done. ;-)

[DartBot]

This comment was originally written by [email protected]

---

I have to agree. From reading through the tutorial, it sounds like Dart is handling null even worse than most languages. (Even numbers might be null?!)

Given that types are optional, it would make sense to allow null when the type is unspecified (var or dynamic return), but if the user (programmer) is explicitly saying that this variable is an int, why should the type checker assume that null is ok?

Null can be useful, but if I am choosing to explicitly declare a type, please let me also explicitly say whether it is allowed to be null. I personally would prefer option types, but given that this language seems to be trying to be similar to C++/Java/Python, perhaps declaring things nullable would be better. For example, if you write "nullable int n = 42", then "n = null" would be allowed.

Another option would be to explicitly annotate variables as not null, like "not_null int n = 42", but IMHO, the only advantage to that would be that it wouldn't break existing code, and given that this is a new language, that's not a very good reason.

http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake
http://se.ethz.ch/~meyer/publications/hoare/void-safety.pdf
https://secure.wikimedia.org/wikipedia/en/wiki/Nullable

[DartBot]

This comment was originally written by [email protected]

---

There was discussion of allowing int? foo to mean that foo might be nullable, and just int foo would mean it wasn't. However, there were some complications, particularly with the idea of types being optional -- ie, removing a non-nullable type annotation could change the behavior of the program.

[DartBot]

This comment was originally written by [email protected]

---

Agreed. For a given type T, a variable declared to have type "T" should not be allowed to be null, whereas a variable declared to have type "T?" should be allowed to be null.

I don't see why changing the declaration is not allowed to change the behavior of the program, though. Changing a function from allowing null to not allowing null is always a backwards-incompatible change, and I think -- in a lot of the use cases -- the function would not have actually supported null, to begin with. The use of the annotation just makes this lack of support more well documented and exposes the error sooner.

[ollyde]

@munificent I've heard this for 2 years now. I tried to enable it run our apps (11 in total now with Flutter cause Flutter is amazing) but with no success. No one on any of the teams can run with optionals and it's the MAIN reason our app's are crashing ocassionally because we don't have this major feature :-). Tbh just having the analyzer would be a god send, we have @nullable in Java more than 10 years ago, seems Dart is the only modern langauge without it and we've been promised it for more than 2 years.

[DartBot]

This comment was originally written by [email protected]

---

Issue #24 has been merged into this issue.

[DartBot]

This comment was originally written by [email protected]

---

some input from the author of null references:

http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

[DartBot]

This comment was originally written by [email protected]

---

May I ask what these complications look like?
When would removing a non-nullable type change the behavior of the program, and how exactly would the behavior be different?

[DartBot]

This comment was originally written by [email protected]

---

Nullable types should definitely be added. At least in the expected form where checker would warn you while prod env would let you assign nulls anyway.

[DartBot]

This comment was originally written by [email protected]

---

Nullable types need to exist as long as default constraints are identified for the type. Much like C# default(t) with a user pragma like declaration . This would allow for ease of intrinsic types and nullable types based on conditions.

[DartBot]

This comment was originally written by [email protected]

---

@JaT I don't understand.
Why would a non-nullable
int foo = null;
throw an exception? I thought the type checker only produces compile-time warnings.

A type warning would be perfectly adequate in this case, and not change the behavior of the program if the type is removed. Or am I missing something?

[DartBot]

This comment was originally written by [email protected]

---

So what happens when you dereference a null pointer (that was declared to be non-nullable) at runtime? I don't see how you usefully continue -- the program is broken, and you are better off failing at the point you assign the null pointer to it.

You can do things like require such variables to be initialized and forbid assignment of a null literal, but in general you are not going to be able to tell if the execution of the program would lead to assigning a null to a variable -- it may even reference external things, such as a server or the DOM, that the compiler can't know the behavior of. Even if you require that such functions be declared as "Foo?" and require explicit null checks when assigning to a "Foo", the server might still return a null when it was declared not to.

[dgrove]

Removed Type-Defect label.
Added Type-Enhancement, Area-Language labels.

[DartBot]

This comment was originally written by [email protected]

---

Added Triaged label.

[DartBot]

This comment was originally written by [email protected]

---

jat: I thought I read that when running in production mode any type errors were supposed to be ignored, so types don't affect the behavior of the program. When running with checking enabled they do, but that is regardless of whether you allow non-nullable types.

For example:
int foo = 'World'; // throws something? (not in production at least)
var foo = 'World';

The same applies to your argument wrt a server or the DOM. If it can return a null when it was declared not to, couldn't it just as easily return a string when it was declared not to?

If you are really worried that making types non-nullable by default would result in question marks all over the place, could you at least support explicitly marking a variable as non-nullable? That would be completely backwards compatible, but having it sooner would mean library writers would be more likely to use it, which would benefit everyone.

I still think explicitly marking nullable ones is better, because that is when you should be thinking about the fact that it might be null, and the presence of a marker tends to be a better reminder for people than it's absence.

[munificent]

I don't see why changing the declaration is not allowed to change the behavior of the program, though.

Well, that is sort of the fundamental idea behind optional types: they're optional. You're of course free to disagree with that concept, but that probably means no amount of feature changes will make Dart into the kind of language you want.

Changing a function from allowing null to not allowing null is always a backwards-incompatible change, and I think

Changing the body of a function to something that no longer will work if you pass null is definitely a backwards-incompatible change. That's different, though, than changing the type annotation of what the function expects.

The use of the annotation just makes this lack of support more well documented and exposes the error sooner.

Right, that's what's great about type annotations (or annotations in general): they give a nice user-visible distillation of what the API expects. But you can get that benefit without the type annotation enforcing what it communicates. That's the model of optional types: it will try to tell you that it thinks you're doing something wrong, but it won't totally prevent you from doing it. You're presumed to be smarter than the type checker, so in the case of a disagreement, it will submit to your will.

[DartBot]

This comment was originally written by [email protected]

---

Ok, I'm afraid I just don't get it. My understanding of optional types is that you don't need to declare the types, but you can if you want to. However, where does it say that declaring the types has no effects?

If declaration types have no effects, then what the hell is the point in adding them back? Isn't enforcing type-safety (which would include nullability), a motivation for developers to add back types? If types have no purpose other than documentation, what's the point?

[munificent]

However, where does it say that declaring the types has no effects?

Gilad can explain it better than I can: http://www.dartlang.org/articles/optional-types/

then what the hell is the point in adding them back?

They do a few things for you:

1. They document the expected types of a function for other programmers looking at your code.
2. You can run Dart code in "checked" mode. When you do, all type annotations will be dynamically checked and report an error if the types don't match. (By "dynamically" I mean, at runtime, when you assign to a variable with a declared type, it will check then.)
3. Tools (such as the Dart compiler) may also opt to use those type annotations to perform static type checking and report static type errors that they find.

I think those cover most of what you want from types (performance is the other piece, but I think Dart can be plenty fast without that), but note that none of them affect the runtime behavior in unchecked mode. So you can get what you want from types, but you aren't required to satisfy the type system in order to run your code.

[lrhn]

No milestone yet. It's a big feature, with many small parts and necessary side-features, not all of which are designed yet, so all we can say is that we are working on it, and you can look at the progress in the language issue tracker.

[MihaMarkic]

I assume quite a huge effort, at least by looking at C# 8 which is introducing the same feature. Keeping fingers crossed.

[speller]

Hi guys,
I'm very new to Dart and I found one very disappointing thing. Why if a function is declared as returning bool it can not be used in the if statement without workarounds, because the boolean function still can return null and the if statement will fail with the Failed assertion: boolean expression must not be null? Could anyone explain to me the idea of this very disappointing behavior? When using somebody else's code (yeah, it happen sometimes, sorry) I must check every boolean method will it return null or not, which slows down the development process significantly. Instead of just relying on the declared return type I must always keep in mind will some function or method return null someday or not in order to keep my code safe and reliable. Why so?

You know, today, using Dart, I must write this:

if (boolFunc() ?? false)

or that:

if (boolFunc() == true)

but I was sure that's the style of a very beginner programmer who writes the very first lines of code in his/her life.

[larssn]

@speller You should trust the functions in returning actual boolean values and not null. If a function can return true or false, but then returns null, it's a bug that should be fixed in that function.

Please don't convert your entire codebase to contain null checks all over the place. Trust the code, and fix bugs if they appear.

[eernstg]

@speller wrote:

Why if a function is declared as returning bool it can not be used
in the if statement without workarounds

You're basically asking for non-nullable types, and that's the topic of this issue. Support for non-nullable types in Dart is probably the largest ongoing effort in the Dart team at this point, and it has a lot of implications, but it is happening. Still, it will take some months before it is released, and then there is a transition period where code gets adapted to use it.

Right now you can check for null with every expression (of any type, there is nothing special about bool or if — a + b or foo.bar() is just as dangerous). But that's a huge overkill in practice. The approach that developers usually take in languages without non-nullable types is to check in some cases and rely on (more or less well-documented) conventions that certain expressions don't evaluate to null in other cases. That's not as good as non-nullable types, but it has been used in many languages for many decades.

So it's probably not a good idea to use a form like if (boolFunc() ?? false) every single time you write an if-statement. You will quite often be in a situation where boolFunc() is intended to return a non-nullable value, which should then be documented in its DartDoc comment, and should be guaranteed by the actual implementation of that function.

Then you can turn on support for non-nullable types as soon as it is available and get type system support for verifying that this property actually holds, and then you also avoid the churn of writing if (boolFunc() ?? false) .. everywhere, and then changing it back to if (boolFunc()) .. when you turn on non-nullable types.

[speller]

@larssn In my opinion, only the author of a function must care about the return value if he/she declares a specific type. In case when users of functions/methods must care about the result, it is a shift of responsibility from author to users. I have a big experience working with strictly typed language (Delphi) and non-strictly typed ones (PHP, JavaScript) so this issue is really confusing and disappointing me. If a language is called "typed" then it must allow only those values in a typed variable that are expected there. If a type is called "a boolean" then it must allow only two values. If it is called "a nullable boolean" then it must allow null value in addition to true and false.

@eernstg Thank you for the full answer. I really hope to see the possibility to explicitly declare types as nullable or non-nullable asap (bool and ?bool - are the perfect options). I think Dart and Flutter are the mainstream future of most apps development, so I hope it implementing the best features.

[larssn]

If the function is in the realm of undocumented no-mans-land, then sure, a bit of caution and checks here and there might go a long way. But if it claims it will return a boolean, and doesn't: bug. That holds true in Javascript as well. There is little point in type checks everywhere, unless the code is an undocumented mess, and the people who wrote it are long gone.

But it's not that I don't share your sentiment, but if you want the above, you must wait for non-nullable-types. 😀

[eernstg]

@speller wrote:

I really hope to see the possibility to explicitly declare types as nullable or non-nullable asap

There is no discussion about whether we'd have that: It is a huge, ongoing effort, and it will happen. The syntax is bool for the non-nullable type and bool? for the nullable type. You can keep an eye on this set of issues in order to see what's going on.

[ollyde]

Any updates on this? Simple 'optionals' checker warning would be sufficient tbh. (Like TypeScript or Swift) a bit sick of doing this (screen shot) when it could just have a '?' at the end..

[ollyde]

@eernstg Yes I've done that before but there are many core Flutter classes that don't support it and you can't isolate the directories it runs like other plugins. :-)

[munificent]

there are many core Flutter classes that don't support it

Yet. :)

you can't isolate the directories it runs like other plugins. :-)

You can't do that yet, but the analyzer will support that eventually. It's a specified part of the feature — a program can contain a mixture of libraries some of which use non-nullable types and some of which don't. You won't get static errors around null in the latter.

[GeertJohan]

@OllyDixon I would be really happy with that 'fully sound mode'. Having a guarantee to have cought all the nullability errors at compile-time sounds perfect. I'm glad the team takes the long-term approach by aiming for strong type-safety here.

[hayderux]

@i-schuetz I think dart team start working on this only for the last two years