Question about the software architecture of bionic about bionic

[jqmp]

Hi @ajing, thank you for your question!

First, in general I think it's a good idea for objects to be immutable when possible, because it makes them easier to reason about, harder to make mistakes with, and easier to reuse in large programs. In this particular case it was a harder decision, because a mutable API would be much simpler: we wouldn't have needed a separate FlowBuilder object at all. Still, there were enough concrete benefits to immutability that we thought it was worth it. I'll give two examples.

1. Suppose you're using a Flow in a notebook like this:

flow.get('result_plot')

flow.setting('preprocessing_type', 'pca').get('result_plot')

flow.setting('ensemble_size', 200).get('result_plot')

If flow were mutable, then each of these cells would be modifying a shared object and their outputs would depend on what order you ran them in. This would be really confusing and annoying! The immutable API lets you keep them separate.

2. Suppose you want to use a flow as part of another library, like this:

from .prediction_pipeline import prediction_flow

def predict(features, priors=None):
    flow = prediction_flow.setting('features', features)
    if priors is not None:
        flow = flow.setting('priors', priors)
    return flow.get('prediction')

Like before, if the flow object were mutable, then each call to predict might affect the priors value for subsequent calls.

Of course, we could offer both types of API in the same class, or we could allow users to just make a copy of the flow whenever they want to avoid global changes. However, this makes things even more complicated and makes it easier to make mistakes. The Flow/FlowBuilder approach lets you guarantee that no one will modify your flow object, but still lets you switch to a mutable API when necessary. This seemed like a good compromise.

Finally, as you noted, it's also true that allowing mutation would make things like reloading and parallelism more complicated. These weren't the main motivations, but you get a lot of little benefits like this automatically when you make things immutable.

I hope this answered your question! If you have any more questions or feedback, I'd be happy to either talk here or email you directly.

[ajing]

Thanks for the clarification! This is very useful.