Relax Uniqueness Constraint for Write-only Variables about marimo

[akshayka]

I started experimenting with relaxing the constraint. There are more edge cases than I originally anticipated. As well as some usability gotchas: imagine redefining a variable (without knowing it) in a very expensive cell. Only to find that you can't read it because it was defined elsewhere. That would make me very sad.

[dmadisetti]

Mock up example:

[akshayka]

Allow multiple definitions of a variable if it is never read before being assigned to in any cell.

This is a really interesting suggestion. It's certainly possible technically. And it also seems correct ...

I need to think through if there are any edge cases or gotchas. Are there any you can think of?

[dmadisetti]

I was thinking about this earlier when I did for i in range(x) for what felt like the millionth time.

More suggestions:

Using a with
e.g.

with mo.local_scope():
    ax, fig

---

Use type hints (which can be statically parsed)

ax # mo.local

---

catching the error and prompting to rename with the _ prefix (that would be my preference, var defaults are just muscle memory)

[PetterS]

I was thinking about this earlier when I did for i in range(x) for what felt like the millionth time.

Yes, exactly! Currently, you can not have for i in range(n): in two cells, even if i is never read outside the for loop scope. Perhaps I should have mentioned this as well since it is very common!

[PetterS]

I need to think through if there are any edge cases or gotchas. Are there any you can think of?

I don't know any. I have been meaning to use Marimo more seriously but I keep running into this when converting existing notebooks. It seems like it would be possible to relax the constraints a bit and still be safe

[dmadisetti]

More on this;

Maybe have a conversion wizard for Jupyter? Prompt the user for variables that are used like ax or fig and automatically replace during the conversion process.
marimo could even suggest variables based on def frequency.

Doing this in conjunction with the "rename variable" option on error would go a long way.

I can only speak for myself, but I do like the fact that I know that all variables are global and not previously defined. I think relaxation starts to introduce the same issues - but I absolutely get the frustration. I think the biggest sticking for me is the actual conversion though, and aside from that- marimo forces me to write better code

[akshayka]

I don't know any. I have been meaning to use Marimo more seriously but I keep running into this when converting existing notebooks. It seems like it would be possible to relax the constraints a bit and still be safe

Yes, that makes sense. I thought about it and don't foresee any correctness issues either. Relaxation does make multiple definition errors more confusing when they do happen, since the relaxed rule is more complicated to explain.

In the following sequence,

1. Create a cell defining x.
2. Read x in another cell.
3. Create another cell defining x.

Cell 3 would not be run, and instead would display the error:

MultipleDefinitionError: Variables defined in multiple cells cannot be read before assignment within a cell.

This error message would link to the cells from steps 1 and 2.

Whereas in the following sequence:

1. Create a cell defining x.
2. Create another cell defining x.
3. Read x in another cell.

The cell in step 3 cannot be run. I suppose it would show the same error, and also link to the cells from steps 1 and 2.

It may be confusing for a read to trigger a MultipleDefinitionError. marimo already has a learning curve, so I am hesitant to do things that make that learning curve steeper.

Maybe have a conversion wizard for Jupyter? ...

If it's just for marimo convert — marimo could uniquify variable names that are never read before being defined in a cell. It wouldn't be pretty, but it would at least make the conversion work in more cases without user intervention. So perhaps this could be a compromise we settle on for now?

[PetterS]

marimo already has a learning curve, so I am hesitant to do things that make that learning curve steeper.

Fair enough, this is important but the advantages are also pretty big in this case since a quite common pattern would "just work" instead of raising errors

[dmadisetti]

MultipleDefinitionError: Variables defined in multiple cells cannot be read before assignment within a cell.

The error could be more verbose than this.

Caution

InconsistentUsageError: Variables may be declared in one cell and referenced in multiple; or declared in multiple cells and used as a referenced nowhere. Cells x, y, z declare variable, and Cells a, b, c reference variable. Remove all references to variable or reduce declarations to one.

[dmadisetti]

But because execution is dynamic I think this is not super straightforward. Back to Akshay's example:

1. Create a cell defining x.
2. Create another cell defining x.
3. Read x in another cell.

Receive error

4 . Delete the definition in cell 2

Cell 1 gets promoted to definition

5. Change cell 3 to definition

Cell 1 gets demoted.

---

I think there's a lot more bookkeeping than currently takes place. All repeated cell-level variables will have to be kept in memory and then swapped out from hidden context to global context.

[akshayka]

Assigned to myself — to either relax the constraint or improve marimo convert.

[akshayka]

MultipleDefinitionError: Variables defined in multiple cells cannot be read before assignment within a cell.

The error could be more verbose than this.

Caution

InconsistentUsageError: Variables may be declared in one cell and referenced in multiple; or declared in multiple cells and used as a referenced nowhere. Cells x, y, z declare variable, and Cells a, b, c reference variable. Remove all references to variable or reduce declarations to one.

Okay, yes, that's much better than my concise error message :)

[akshayka]

All repeated cell-level variables will have to be kept in memory and then swapped out from hidden context to global context.

@dmadisetti Or we could re-run ancestors as needed. If a cell that has been requested for execution has a reference whose definition isn't in globals, run the cell that defines it (as well as its descendants) before running the requested cell. Thoughts?

[dmadisetti]

Certainly a solution, but could also lead to a number of unintentional reruns.

Another suggestion: app = App(local=("ax", "fig", "i")) might be the easiest from an engineering perspective. Also a couple of user benefits:

• outside of runtime context, which is important for initial graph construction
• explicit behavior. You might trigger one of the modes and not even know it otherwise until you try to use a 3rd cell.
• no runtime penalty
• can be tuned for jupyter conversion (cells may utilize "axis" or "figure" instead of "ax" or "fig", and they might be a keyword worth marking as local)

As for implementation:

1. propagate information to visitor.is_local and just use the current local mechanism
2. Raise custom error if one of the special locals is a ref
3. Ensure exporter saves the information
4. (maybe) UI means of editing the values

I think runtime implementation could put notebooks in an ambiguous state, which is not a great user experience. This is a compromise with the downside of slightly hiding the information that some values will treated as private (but partially mitigated with a custom error).

[akshayka]

How would one specify the locals from inside a notebook? Or is the idea that locals would only be specified by editing the notebook file?

I think with some careful bookkeeping, one could minimize unintentional re-runs. Your solution of keeping variables in memory would also work, and would have no unintentional reruns, at the cost of memory consumption.

[dmadisetti]

The variables panel is a great place to mark variables as being private:

Maybe a checkbox? I was only mentioning saving inside the notebook for persistence (same way layout file reference is saved behind the scenes, or show_code state on the cell level).

The book-keeping approach would have to account for additional kernels (thinking the strict execute pr)- but still could work

[akshayka]

I think runtime implementation could put notebooks in an ambiguous state, which is not a great user experience.

Yeah, you're totally right. That's what I concluded too after experimentation

... This is a compromise with the downside of slightly hiding the information that some values will treated as private (but partially mitigated with a custom error).

The hiding could be confusing ... but otherwise I agree with your pros, especially with the explicitness.

Mock up example:

Nice mockup! If we wanted to use the existing mechanism for local variables, "change to private read-only variable" could just prepend an underscore to the variable name.

Another solution that would be explict, without having to use underscores, would be to define a custom type annotation that marimo understood. Something like:

fig: "local"
ax: "local"

fig, ax = plt.subplots()
...

That's something I would be open to. But maybe underscores is fine for now ...

[akshayka]

Closing as not planned for now — in favor of the marimo convert improvement — given how this can get the runtime in an ambiguous state. Still happy to revist relaxing the constraint later, if anyone has further thoughts, and still happy to riff on other ways we can improve this experience

[PetterS]

Thanks for seriously considering it! Maybe it's possible to improve this somehow....