Comments (14)
Hi Sheel
Is this the complete error log? If yes, then can you run the project locally and see where the issue is happening?
Thanks Utkarsh
from peek.
We are also having the same issue . Webjob tries to fetch data from CSP portal but fails and throws a HTTP exception/timeout error and is unable to transfer any information into database.
One way to bypass the error message is to comment the lines 529-537 in CspDataHelper.cs-file, which will skip the CSP usage data retrieval part. This will allow webjob task to continue and gather Historic billing information and insert data into database. Unfortunately only historic data will be available for powerbi-report, not actual usage data.
Issue #36 looks quite similar and multiple users have reported similar error messages.
from peek.
Hi Utkarsh,
Thanks for your response. Yes, that is the complete log of error. I have checked running the project locally and on Azure as well, getting same error on both. Execution is getting timed out at below line of code in CspUsage.cs webjob class file.
HttpResponseMessage httpResponse =
await this.Client.HttpClient.SendAsync(httpRequest, cancellationToken).ConfigureAwait(false);
Thanks,
Sheel
from peek.
Hi @UtkarshVIT , is there any way to modify that http request or pull usage information in another way? Have you managed to identify whats causing the http request to fail ?
from peek.
@alaniemi @sheelgohe
You can verify if the CSP Usage API is timing out by calling it from the swagger interface as given in the deployment steps of the API project. If that is the case then we have implemented the APIs as async in the usageAsync branch. Can you try if that implementation of usage API is not timing out?
from peek.
Hi @UtkarshVIT - CSP Usage API is timing out when called from Swagger interface as well.
I will take a look at usageAsync branch. and let you know.
Thanks,
Sheel
from peek.
Swagger/cspusage does indeed timeout . The request timed out.- The web server failed to respond within the specified time.
Will try to redploy solution from async branch
from peek.
@UtkarshVIT Redeployed solution from async branch and swagger now returns normal response. Webjob also runs without any errors.
Although we can now finally get usage data into database, webjob is only fetching partial data; database only contains usage data for companies that have a primary domain name starting with letter A e.g. aaaa.onmicrosoft.com .
Is there any easy way to log more detailed info about the API operations and write that to webjob console? How does one print full customer list fetched from the API to the console?
from peek.
@alaniemi glad to hear that it worked :)
So we are currently accessing the custom made APIs which are define in the API Project. To access customer list we can utilize the customers namespace of the Partner Center SDK as given in this sample app.
from peek.
@UtkarshVIT Solution deployed from the async branch has been running for multiple days. Webjob finished without any errors, but fetches only 198 records to usage table.
Same 198 records everyday. No changes whatsoever. Was expecting thousands of records and changes everyday. Something clearly isn't working properly.
How does that API request actually work? Does it try to fetch multiple pages of data and is only able to parse one page? Why are we only getting partial data to database?
How does one troubleshoot the issue and paste fetched information to console or storage log file in more detail?
In order to analyze the issue, would probably need to output the object info to the console and check if the actual list is longer than that 198 lines of data or if there is any error/timeout message somewhere
There was also discussion earlier about releasing newer version of the solution with utilization implementation. What's the status of the utilization branch? Could you please provide us an update on the status of the development work ?
from peek.
Hi @UtkarshVIT - Any luck with identifying what's causing below CSPUsage http request to timeout?
CspUsage.cs:
HttpResponseMessage httpResponse =
await this.Client.HttpClient.SendAsync(httpRequest, cancellationToken).ConfigureAwait(false);
Thanks,
from peek.
@sheelgohe if you call the API from swagger UI, you'll see that it returns a timeout after approximately 4 minutes of execution. Any web service hosted on azure has a maximum execution time of 4 minutes after which it return a timeout error.
The Usage API fetches the maximum amount of data and hence takes the maximum amount of time to process the request. Thus, if the partner has large amounts of usage data, the API may take over 4 minutes to respond and hence the timeout error.
@alaniemi
The code for usage controller is here It iterates through each subscription for each customer of the partner. You can put a breakpoint on line 119 of the controller and see what is being returned.
from peek.
@UtkarshVIT has this fix been incorporated into the main branch?
from peek.
@UtkarshVIT I'm using the async branch and I'm still running into this error unfortunately. Are there any other things I should be trying?
Many thanks
from peek.
Related Issues (20)
- Azure peek dashboard - new customers not appearing HOT 3
- Filter data on basis of customer HOT 1
- How to get the daily consumption HOT 12
- Expose LicenseType Flag (to monitor AHUB usage) HOT 2
- Display Tags of ressources HOT 4
- EA usage error: The wait operation timed out HOT 2
- "Message": "An error has occurred." for CSP Usage api HOT 2
- Release date of the promised new version? HOT 3
- Timeout for CspUtilization HOT 3
- ResourceUtilizationList.json is missing HOT 1
- support for MFA requirement
- Anyone still using this code ? HOT 2
- Webjob returning exception.Swagger/cspusage is working but cspbilling error HOT 2
- Error running webjob for csp data. database not being populated
- Webjob thrown exception for csp. Unable to populate database HOT 1
- No Issue
- doesn't work with vs 2019
- This repo is missing important files HOT 2
- This repo is missing a LICENSE file
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from peek.