BC Export Errors about bc2adls

[DuttaSoumya]

Thanks Tom for writing to us.

I assume that the code stack you refer to has the same code lines as in the repo. The place pointed in the stack is a function that simply makes the Http GET/ PUT/ DELETE calls- so it does not do anything bc2adls- specific per se. As you say that this is a consistent error, perhaps you should remove the NonDebuggable attribute on the procedure, and then debug it to see where it gets stuck.

You may also check in the error log for more detailed errors that occur on the server. Please enable telemetry for the extension as well.

Finally, have you just tried making other Http calls in your own extension and see how it works?

[DuttaSoumya]

We are still curious @tglink72 about the issue and its reproducibility. Could you delve further into it based on my last message?

[tglink72]

Soumya, Thanks for the reply. We have initiated an automatic retry on failure that has helped to keep the export more reliable, though it does not solve the underlying issue. This customer is unique as they had performance issues when we were using change tracking and the deltas, so we modified the code to send all records each time. My concern is sending this much data daily is causing some of the inconsistent issues we have seen with processing. We are working on implementing the latest version of the code in our Sandbox environments. Per your comment here #55 , this version can pull from replicated data. Are there any configuration changes we have to make specific to this version to use a replicated database? I am not seeing any options in the Azure Export window that identifies whether to use transactional or replicated data. Thanks again for your reply and for all of your work on this project! Tom Link From: Soumya Dutta ***@***.***> Sent: Thursday, January 12, 2023 5:45 AM To: microsoft/bc2adls ***@***.***> Cc: Tom Link ***@***.***>; Mention ***@***.***> Subject: Re: [microsoft/bc2adls] BC Export Errors (Issue #73) We are still curious @tglink72<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftglink72&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yLcqDhf7PUr9EcG3TKcF2h6VRrGDQW5cQRqbptfiQ80%3D&reserved=0> about the issue and its reproducibility. Could you delve further into it based on my last message? - Reply to this email directly, view it on GitHub<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fbc2adls%2Fissues%2F73%23issuecomment-1380206786&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9qekiHY2XTUHE4y1PeG48G7rEn8MFkEdPVvriRJJ5oE%3D&reserved=0>, or unsubscribe<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAZOTVULMZP5PHT24RVTN47LWR7VDXANCNFSM6AAAAAAS5MUCZE&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6RcJSxj6bh5sXzJZQja5FjBtKhAl1Z8xsSsJD5c7JgA%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[tglink72]

Soumya, Hello. One things we are seeing with the current export is that certain tables are reporting success in the export to ADL, but are not fully exporting. For example the Customer-18 table is reporting success in the export window, but only exporting a fraction of the total records from the table. A full Customer export should be several hundred MB, but we are seeing that it is only exporting less than a MB and then reporting success. Have you seen this issue in the past? Thanks Tom ***@***.*** From: Soumya Dutta ***@***.***> Sent: Thursday, January 12, 2023 5:45 AM To: microsoft/bc2adls ***@***.***> Cc: Tom Link ***@***.***>; Mention ***@***.***> Subject: Re: [microsoft/bc2adls] BC Export Errors (Issue #73) We are still curious @tglink72<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftglink72&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yLcqDhf7PUr9EcG3TKcF2h6VRrGDQW5cQRqbptfiQ80%3D&reserved=0> about the issue and its reproducibility. Could you delve further into it based on my last message? - Reply to this email directly, view it on GitHub<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fbc2adls%2Fissues%2F73%23issuecomment-1380206786&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9qekiHY2XTUHE4y1PeG48G7rEn8MFkEdPVvriRJJ5oE%3D&reserved=0>, or unsubscribe<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAZOTVULMZP5PHT24RVTN47LWR7VDXANCNFSM6AAAAAAS5MUCZE&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6RcJSxj6bh5sXzJZQja5FjBtKhAl1Z8xsSsJD5c7JgA%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[tglink72]

Soumya, Hope all is well. The issue below is becoming more consistent. Several tables (Customer, SalesInvoiceLine) report success in the export from BC, yet are only exporting a fraction of the records for the table, rather than the full table after reset. Have you seen this in other instances of the export tool? Thanks for any help. Tom Link From: Tom Link ***@***.***> Sent: Thursday, January 12, 2023 8:14 AM To: microsoft/bc2adls ***@***.***>; microsoft/bc2adls ***@***.***> Cc: Mention ***@***.***> Subject: RE: [microsoft/bc2adls] BC Export Errors (Issue #73) Soumya, Hello. One things we are seeing with the current export is that certain tables are reporting success in the export to ADL, but are not fully exporting. For example the Customer-18 table is reporting success in the export window, but only exporting a fraction of the total records from the table. A full Customer export should be several hundred MB, but we are seeing that it is only exporting less than a MB and then reporting success. Have you seen this issue in the past? Thanks Tom ***@***.*** From: Soumya Dutta ***@***.******@***.***>> Sent: Thursday, January 12, 2023 5:45 AM To: microsoft/bc2adls ***@***.******@***.***>> Cc: Tom Link ***@***.******@***.***>>; Mention ***@***.******@***.***>> Subject: Re: [microsoft/bc2adls] BC Export Errors (Issue #73) We are still curious @tglink72<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftglink72&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yLcqDhf7PUr9EcG3TKcF2h6VRrGDQW5cQRqbptfiQ80%3D&reserved=0> about the issue and its reproducibility. Could you delve further into it based on my last message? - Reply to this email directly, view it on GitHub<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fbc2adls%2Fissues%2F73%23issuecomment-1380206786&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9qekiHY2XTUHE4y1PeG48G7rEn8MFkEdPVvriRJJ5oE%3D&reserved=0>, or unsubscribe<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAZOTVULMZP5PHT24RVTN47LWR7VDXANCNFSM6AAAAAAS5MUCZE&data=05%7C01%7Ctom.link%40stoneridgesoftware.com%7C0edc70f73c4041cfe0b908daf4926fe6%7C2e14a5b1fbf8415bbc7d93e20829e510%7C1%7C0%7C638091207045829757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6RcJSxj6bh5sXzJZQja5FjBtKhAl1Z8xsSsJD5c7JgA%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[DuttaSoumya]

First of all, as you also shared @tglink72, it is not a best practice to repeatedly export all data from all tables. bc2adls is meant to be run with incremental changes being exported.

Note that in order to export data, bc2adls spawns a new session for every table for every company (when relevant) So depending on the data you are exporting this could be too many! After all, BC runs within the bounds of certain operation limits.

if it is absolutely essential to export all data from a table, please identify such tables and only reset those instead of all tables. You may be able to implement that using the API page that has been created.

As the initial issue about the displayed error is not recurring, I am closing the current case. Please open a new one if relevant.