get a timeout for (api) myapp/api/cspusage about peek

[ssugar]

Yeah, I'm also getting what I think are timeouts when the webjob gets the current cspusage records and puts them into the DB. If I remove the following code from the BillingWebJob project's CspDataHelper.cs file, then I'm able to at least get billing and historic CSP usage, but if this is left in, then the web job times out and crashes:

Console.WriteLine("\nFetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file. ");

HttpOperationResponse<IList<CspAzureResourceUsageRecord>> cspUsageRecordsFromApi = AzureAnalyticsApi.CspUsage.GetAllDataWithOperationResponseAsync().Result;
Console.WriteLine("\n" + cspUsageRecordsFromApi.Body.Count + " records fetched from the API.. ");

if (cspUsageRecordsFromApi.Body != null && cspUsageRecordsFromApi.Body.Count > 0)
{
  totalRecordsCount += CspDataHelper.UpdateCurrentUsageRecordsInDb(cspUsageRecordsFromApi.Body);
}

As the comments in the code note, you can increase the timeout in the web job config, but even with that at 30 minutes, I'm still getting a timeout. should I set it even longer than that? I'm querying about 20 different CSP tenants.

I am getting CSP billing and historic data, so it seems like I have everything set up right, just can't get the CSP current usage to work.

[katomel4]

hi. I am checking with microsoft. aperently there is a hard code 4 minute timeout for Azure Webapp and it cant be bypass.

[ssugar]

Maybe it would be possible to alter the BillingDataApi so that we can call the API for CSPUsage on a per customer basis instead of all customers at once? perhaps that would allow the api call to finish faster?

[ssugar]

Ok, at least in my case, this is not a timeout, it's an exception thrown by the BillingDataApi CspUsageController.cs code. I was able to modify the code and have it just log the error to the application log in Azure, and here's the error being encountered (id's altered to protect the innocent):
Reseller CAID XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX does not have a reseller relationship with customer CAID XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.

By removing the exception throw and changing it to just log to a trace, the data collection run completes and the API returns the CSPUsage data within a few minutes. Here's what I replaced the exception throw with:

Trace.TraceError("Error encountered gathering customer CSP data. Error Message: ");
Trace.TraceError(e.Message.ToString());

Also had to add using System.Diagnostics at the top of the CspUsageController.cs to use Trace.

I can send a pull request if you want, but there might be a better solution by filtering the customers pulled from the PartnerCenter to ensure there is a valid reseller agreement.

[ssugar]

Yes, the issue here (in my case at least) is that we have customers in the partnercenter that we only have an "Advisor" relationship with, not a "Reseller" relationship. So probably the best solution is to pull only customers with a Reseller relationship from the partnercenter api, not all customers.

[UtkarshVIT]

We have made the usage API async. It can be viewed at the CSPUsageAsync branch(here). I request you to test the API project on this branch and let us know if it works