Comments (6)
Yeah, I'm also getting what I think are timeouts when the webjob gets the current cspusage records and puts them into the DB. If I remove the following code from the BillingWebJob project's CspDataHelper.cs file, then I'm able to at least get billing and historic CSP usage, but if this is left in, then the web job times out and crashes:
Console.WriteLine("\nFetching records for Current Month's Usage from the API. This may take a while. If this operation is timimg out, consider increasing the TimeOut limit in Configuration file. ");
HttpOperationResponse<IList<CspAzureResourceUsageRecord>> cspUsageRecordsFromApi = AzureAnalyticsApi.CspUsage.GetAllDataWithOperationResponseAsync().Result;
Console.WriteLine("\n" + cspUsageRecordsFromApi.Body.Count + " records fetched from the API.. ");
if (cspUsageRecordsFromApi.Body != null && cspUsageRecordsFromApi.Body.Count > 0)
{
totalRecordsCount += CspDataHelper.UpdateCurrentUsageRecordsInDb(cspUsageRecordsFromApi.Body);
}
As the comments in the code note, you can increase the timeout in the web job config, but even with that at 30 minutes, I'm still getting a timeout. should I set it even longer than that? I'm querying about 20 different CSP tenants.
I am getting CSP billing and historic data, so it seems like I have everything set up right, just can't get the CSP current usage to work.
from peek.
hi. I am checking with microsoft. aperently there is a hard code 4 minute timeout for Azure Webapp and it cant be bypass.
from peek.
Maybe it would be possible to alter the BillingDataApi so that we can call the API for CSPUsage on a per customer basis instead of all customers at once? perhaps that would allow the api call to finish faster?
from peek.
Ok, at least in my case, this is not a timeout, it's an exception thrown by the BillingDataApi CspUsageController.cs code. I was able to modify the code and have it just log the error to the application log in Azure, and here's the error being encountered (id's altered to protect the innocent):
Reseller CAID XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX does not have a reseller relationship with customer CAID XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.
By removing the exception throw and changing it to just log to a trace, the data collection run completes and the API returns the CSPUsage data within a few minutes. Here's what I replaced the exception throw with:
Trace.TraceError("Error encountered gathering customer CSP data. Error Message: ");
Trace.TraceError(e.Message.ToString());
Also had to add using System.Diagnostics at the top of the CspUsageController.cs to use Trace.
I can send a pull request if you want, but there might be a better solution by filtering the customers pulled from the PartnerCenter to ensure there is a valid reseller agreement.
from peek.
Yes, the issue here (in my case at least) is that we have customers in the partnercenter that we only have an "Advisor" relationship with, not a "Reseller" relationship. So probably the best solution is to pull only customers with a Reseller relationship from the partnercenter api, not all customers.
from peek.
We have made the usage API async. It can be viewed at the CSPUsageAsync branch(here). I request you to test the API project on this branch and let us know if it works
from peek.
Related Issues (20)
- Azure peek dashboard - new customers not appearing HOT 3
- Filter data on basis of customer HOT 1
- How to get the daily consumption HOT 12
- Expose LicenseType Flag (to monitor AHUB usage) HOT 2
- Display Tags of ressources HOT 4
- CSP Usage webjob throwing exception Microsoft.Rest.HttpOperationException HOT 14
- EA usage error: The wait operation timed out HOT 2
- "Message": "An error has occurred." for CSP Usage api HOT 2
- Release date of the promised new version? HOT 3
- Timeout for CspUtilization HOT 3
- ResourceUtilizationList.json is missing HOT 1
- support for MFA requirement
- Anyone still using this code ? HOT 2
- Webjob returning exception.Swagger/cspusage is working but cspbilling error HOT 2
- Error running webjob for csp data. database not being populated
- Webjob thrown exception for csp. Unable to populate database HOT 1
- No Issue
- doesn't work with vs 2019
- This repo is missing important files HOT 2
- This repo is missing a LICENSE file
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from peek.