Exports with different frequencies about bc2adls

[DuttaSoumya]

Thanks for explaining the situation here @thomas-tarp.

Our experience is that pushing the incremental changes into the deltas folder is usually quite quick, even with a daily frequency. We recommend that you consider a high frequency update so that entities that get populated not so often, will not contribute “much” to the export time. In theory, this idea can only be challenged in cases where an entity gets updated in larger volumes and you do not want those changes exported often. The Synapse side of this solution, on the other hand, is more critical in terms of time and cost.

But, it is possible to run exports for tables at different frequencies in the current solution, with slight changes,

• Please consider the API pages created by @Bertverbeek4PS to dynamically change which tables are going to be exported (ADLSETableAPI- allow modifys and enable/ disable them) and then exporting them (ADLSESetupAPI- call StartExport). Thus you may trigger exports from outside BC.
• Alternatively you may directly invoke the codeunit ADLSEExecute with the desired table parameter that you wish to export.
• On the Synapse side, if you wish to consolidate only a few select tables at a different frequency, please consider directly triggering the Consolidation_OneEntity instead of invoking the Consolidation_AllEntities pipeline- you will, of course, have to pass the name of the entity to invoke the consolidation for.

[DuttaSoumya]

Very good points @thomas-tarp. Would you be interested to cover these points on an online meeting? Kindly write directly to [email protected] so we can setup one and discuss your idea further.

[Bertverbeek4PS]

@thomas-tarp that is indeed a very nice feature! More customers want this!

[thomas-tarp]

Thank you as well for the meeting today.

I will test with differentiated frequencies on the Synapse pipelines. It looks like this should fulfill the needs.

I'm closing the issue.