Deprecate micropackaging about kedro

[yury-fedotov]

+, I think it's a nice initiative.

[arek544]

I have multiple modular pipelines that l like to share between projects, so I thought micropackaging would be the best solution so far

[merelcht]

Hi @arek544, thanks for commenting. Micropackaging does suit your use case well and that's also the use case we built the feature for. However, adoption has been very low and most users have found alternative solutions. Are you on our Slack (http://slack.kedro.org/) by any chance? It would be good to explore a different solution for you for when the time comes that micropackaging is removed from Kedro.

[merelcht]

that's a shame, we started adopting it and with the changes in version 18 the use got smoother. Our use case is pulling preconfigured pipelines into new projects as a starting point for "similar" projects.

Hi @daniel-ressi, thanks for your comment. For the use case you're describing we actually have Kedro Starters: https://docs.kedro.org/en/stable/starters/starters.html, this is a way to create project templates to standardise projects in a way that fits your organisation's setup.

[arek544]

what's an alternative to micropackaging? Should I switch to packaging instead?

[astrojuanlu]

@arek544 Could you detail your use case?

[merelcht]

@astrojuanlu you wrote in the description:

Since kedro.framework.cli.micropkg is public API too, this warning should also be shown when importing anything from that subpackage.

The only public method aside from the CLI commands in kedro.framework.cli.micropkg is safe_extract: https://github.com/kedro-org/kedro/blob/main/kedro/framework/cli/micropkg.py#L393, does that really warrant an import deprecation warning? I'd be very surprised if anyone is using that since it's a pretty generic utility method.

[astrojuanlu]

Good to see most of that module is private API, wrote my comment without looking. I also doubt that anybody is using that specific function, but strictly speaking we should, right?

[daniel-ressi]

that's a shame, we started adopting it and with the changes in version 18 the use got smoother. Our use case is pulling preconfigured pipelines into new projects as a starting point for "similar" projects.

[daniel-ressi]

that's a shame, we started adopting it and with the changes in version 18 the use got smoother. Our use case is pulling preconfigured pipelines into new projects as a starting point for "similar" projects.

Hi @daniel-ressi, thanks for your comment. For the use case you're describing we actually have Kedro Starters: https://docs.kedro.org/en/stable/starters/starters.html, this is a way to create project templates to standardise projects in a way that fits your organisation's setup.

thank you for your response. There were some benefits to micro-packaging for us (especially just pulling individual pipelines or datasets into a project that we already create with our own template which also includes backend/frontend/infrastructure components), but I do agree that kedro starters definetly are an alternative