Should `transformed_data()` also drop the columns that are not included in the spec? about vegafusion

[jonmmease]

You could do something like this to demonstrate it:

transformed_spec = chart.to_json(format="vega")
print(transformed_spec["data"][0]["values"])

Where you need to figure out the right dataset to reference in advance.

[jonmmease]

I decided to keep all of the columns so that it's possible to test transforms on a chart without needing to define marks beforehand.

This way you can experiment with the filter transform like this alt.Chart(source).transform_filter(...).transformed_data(). With projection pushdown enabled, all of the columns would be removed in this case because nothing is using them.

And if you're iteratively building a transform pipeline, you may want to filter by a column that's not used yet, and so it's not very helpful if transformed_data doesn't return columns that aren't already in use.

That said, it could be a parameter option if there's reason to support both behaviors.

[joelostblom]

Ah I see, yes that experimentation with transform_filter is neat. I mostly wanted to use it to illustrate how VegaFusion works in a lecture by showing that the shape of the data is reduced when it is activated. Since I was working with a really large dataframe, I didn't want to open the vega editor, but just call shape on the transformed data. I don't think this is a very important use case and I can just explain what is going on or show in the VegaEditor with data containing less rows, so probably not worth spending implementation time on.