Consider generalizing macros over applications and annotations about language

[jakemac53]

A macro can sort of do this already on a single library... although you end up needing an extra annotation. A macro can run on an entire library, and then introspect to find the annotations it wants to generate code for, share code/work as desired, and alter anything within the library.

There isn't any particular advantage to macro code executing in the context one of application

Within a single library, probably it would be fine. I wouldn't want to generalize this to all libraries in an app though, because then there is the (I would argue significant) advantage that it encourages writing code which is deterministic and not reliant upon the specifics of how a given compiler works, or whole world versus incremental versus modular compiles etc.

As a concrete example, consider this issue which was recently filed. A macro author with access to the entire world at once is likely to want to write macros like this, which require global knowledge, and either just won't work in certain modes (hot reload) or will be slow, non-deterministic, etc.

The current model where each application is treated in isolation helps macro authors stay on the rails and write macros that will work consistently, as well as avoiding anti-patterns such as global queries.

[davidmorgan]

It's easy to offer libraries on top that make it look like execution is in isolation:

https://github.com/dart-lang/language/blob/main/working/macros/dart_model/macro_client/lib/class_generator_macro.dart

And these can then do optimization behind the scenes, leaving the option to do something more complex only for those who understand it / want it.

I think that then combines the advantages of both approaches :)

[rrousselGit]

I do believe #3854 is a very important scenario. I myself have a similar need in many occasions.

For instance, I want to generate DB binding for a model .... which includes a schema migration.
I want to generate a single function that migrates all data at once. Yet there's no easy way to do so yet.

Generating one function per model/library would be unrealistic, because it'd be very error prone (easy to forget to invoke one library's function).

I think that's fine if the generated code is often invalidated and slower to generate. It's important to be able to do it at all.

[davidmorgan]

Thanks Remi.

Are you saying you want one macro application to do all that work--or is it okay to have one macro applications per data model class?

I think the latter can work nicely: each model class application can output a function for that model class, then a whole program application can output code that aggregates--calls them all. That won't be super fast but you only need one in your program so rerunning it all the time should be fast enough.

[rrousselGit]

I don't care too much about the exact details. Personally, I'd have one annotation per model ... and maybe one on the "main" (that would look-up every files in the package). But if a different pattern enables implementing this, I don't mind.

---

On that topic, is there a certain library order when dealing with generation over multiple libraries?

For instance, given:

// lib/main.dart
import 'src/model.dart';

@macro
void main( ){}

// lib/src/model.dart
@anotherMacro
class Foo {}

Would the macro on Foo generate before the one in main? Or maybe they generate in parallel (based on phases)?

Since macros can use IO a bit, I was wondering if /lib/src/foo.dart could output to .dart_tool/lib/src/foo.my_macro.part. Then have the lib/main.dart lookup for all .dart_tool/**/*.my_macro.part and generate something based on it.

[davidmorgan]

Execution order mostly depends on imports: as long as there are no imports from model.dart onto main.dart, model.dart macros can execute and complete before main.dart macro run at all. If there is a cycle of imports (a "library cycle") they have to run "in parallel" in some sense.

[rrousselGit]

Any chance we could have a mechanism that gives macros a fixed order, even when libraries are not related through an import?
I'd like to avoid asking folks to import their entire project by hand. It's super painful that whenever you add a new file dart, you have to list it in your main.

One way could be to order files based on the path depth. Such that /lib/a.dart runs after /lib/src/b.dart (because leaves first).

So lib/main.dart could have access to anything that's inside the lib/src folder safely.

[davidmorgan]

Dart requires everything to be findable via imports from an entrypoint, so that would require a change independent of macros.

[rrousselGit]

It would be findable via imports from the entrypoint.
I'm talking about applying a macro on the main, and have the generated code add extra imports:

@macro
void main() {}

// ...
import 'src/model.dart' as prefix0;
import 'src/model2.dart' as prefix1;

augment main() {
  prefix0.Model.migrate();
  prefix1.Model2.migrate();
  ...
}

[davidmorgan]

I don't think that's possible. But anyway it's unrelated to this issue, which is for a specific discussion about the work in progress.