- Java
- Kotlin
- Swift
- Objective-C
- C/C++
- C#
- Go
- JavaScript
- Ruby
- Python
- Spring Framework / Spring Boot
- Linux
- Android
- iOS
- AWS
- Azure
- Kubernetes
- Biometrics authentication
- Video analytics
- TCP/IP
Google Play and Google Checkout scraper for developers
Can you make STDERR echo of URL optional, or only done at a 'verbose' logging level?
It is causing our tests to print the url for every request. It adds a lot of noise to our tests.
I am happy to fix the problem and open a pull request. The problem is line 113 of lib/googleplay_dev_scraper/scraper.rb
Thanks!
Hi, I cant "scrape" the estimated Sales Report. The Url cant be found.
I'd guess the URL has changed. If I try to access the url directly in the browser I see a Google Error Page saying:
400. That’s an error.
The requested URL was not found on this server. That’s all we know.
Please find the Console Output of the scraper below:
fetch': 400 => Net::HTTPBadRequest for https://play.google.com/apps/publish/salesreport/download?report_date=2013_08&report_type=sales_report&dev_acc=00158313850196121289 -- unhandled response (Mechanize::ResponseCodeError) from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:974:in
response_redirect'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:298:in fetch' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:974:in
response_redirect'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:298:in fetch' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:974:in
response_redirect'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:298:in fetch' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize.rb:1272:in
post_form'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize.rb:539:in submit' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/form.rb:223:in
submit'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/form.rb:229:in click_button' from /var/lib/gems/1.9.1/gems/googleplay_dev_scraper-1.0.2/lib/googleplay_dev_scraper/scraper_base.rb:55:in
try_get'
from /var/lib/gems/1.9.1/gems/googleplay_dev_scraper-1.0.2/lib/googleplay_dev_scraper/scraper.rb:55:in get_estimated_sales_report' from /var/lib/gems/1.9.1/gems/googleplay_dev_scraper-1.0.2/bin/googleplay_dev_scraper:93:in
estimated'
from /var/lib/gems/1.9.1/gems/googleplay_dev_scraper-1.0.2/bin/googleplay_dev_scraper:33:in main' from /var/lib/gems/1.9.1/gems/googleplay_dev_scraper-1.0.2/bin/googleplay_dev_scraper:122:in
<top (required)>'
from /usr/local/bin/googleplay_dev_scraper:23:in load' from /usr/local/bin/googleplay_dev_scraper:23:in
/usr/local/rvm/gems/ruby-2.1.1/gems/googleplay_dev_scraper-1.0.3/lib/googleplay_dev_scraper/scraper_base.rb:54:in try_get': undefined method
value=' for nil:NilClass (NoMethodError)
When I try to run the appstats command, it creates me a corrupted zip file. Do you know what may be the problem?
Some companies will only use gems with a certain license.
The canonical and easy way to check is via the gemspec
via e.g.
spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']
Bundler now generates gems with a default 'MIT' license. There is even a License Finder
to help companies ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec.
Including a license in your gemspec is a good practice, in any case.
If you need help choosing a license, github has created a license picker tool
How did I find you?
I'm using a script to collect stats on gems, originally looking for download data, but decided to collect licenses too,
and make issues for gemspecs not specifying a license as a public service :)
So far it's going pretty well.
I've written a blog post about it
Any idea what would cause this? I've entered the details correctly.
The only thing I can think of is that I have multiple developer profiles tied to my Google login?
Thanks,
Steve
As of yesterday the appstats download is not working anymore. I'd guess the URL has changed. If I try to access the url directly in the browser I see a Google Error Page saying:
400. That’s an error.
The requested URL was not found on this server. That’s all we know.
Please finde the Console Output of the scraper below:
URL = https://play.google.com/apps/publish/v2/statistics/download?package=my.package&sd=20130804&ed=20130804&dim=overall,country,language,os_version,device,app_version,carrier&met=active_device_installs,daily_device_installs,daily_device_uninstalls,daily_device_upgrades,active_user_installs,total_user_installs,daily_user_installs,daily_user_uninstalls,daily_avg_rating,total_avg_rating
/var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:306:in fetch': 400 => Net::HTTPBadRequest for https://play.google.com/apps/publish/statistics/download?package=my.package&sd=20130804&ed=20130804&dim=overall,country,language,os_version,device,app_version,carrier&met=active_device_installs,daily_device_installs,daily_device_uninstalls,daily_device_upgrades,active_user_installs,total_user_installs,daily_user_installs,daily_user_uninstalls,daily_avg_rating,total_avg_rating -- unhandled response (Mechanize::ResponseCodeError) from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:974:in
response_redirect'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:298:in fetch' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:974:in
response_redirect'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:298:in fetch' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:974:in
response_redirect'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/http/agent.rb:298:in fetch' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize.rb:1272:in
post_form'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize.rb:539:in submit' from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/form.rb:223:in
submit'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.1/lib/mechanize/form.rb:229:in click_button' from /var/lib/gems/1.9.1/gems/googleplay_scraper-0.2.2/lib/googleplay_scraper/scraper_base.rb:55:in
try_get'
from /var/lib/gems/1.9.1/gems/googleplay_scraper-0.2.2/lib/googleplay_scraper/scraper.rb:156:in get_appstats' from /var/lib/gems/1.9.1/gems/googleplay_scraper-0.2.2/bin/googleplay_scraper:136:in
appstats'
from /var/lib/gems/1.9.1/gems/googleplay_scraper-0.2.2/bin/googleplay_scraper:41:in main' from /var/lib/gems/1.9.1/gems/googleplay_scraper-0.2.2/bin/googleplay_scraper:146:in
<top (required)>'
from /usr/local/bin/googleplay_scraper:23:in load' from /usr/local/bin/googleplay_scraper:23:in
Thanks for your effort.
Kind regards
Nicow
Hi, I'm a super beginner with ruby and so I have a really silly question. What are the specific steps of creating the configuration file? Thank you!
Is it possible to add CRASHES & ANRS to the scraper? Was looking for a download link in the web interface but couldn't find one. Any ideas?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.