PSNProfiles player scraper

Scrape PSNProfiles player pages using a command-line interface/terminal, then view individual data, export to XML/JSON or compare with another player.

You can view a demo of the app in action over on YouTube: youtube.com/watch?v=l1yA_LfLz-c

Installation

Install Ruby (help), then in a terminal:

gem install bundler
change directory to PSNProfiles-profile-scraper
(e.g. cd "C:\Users\yndaj\Downloads\PSNProfiles-profile-scraper")
bundle install

Usage

In a terminal:

Make sure you're in the PSNProfiles-profile-scraper directory (via cd)
ruby bin/scrape

Features

Retrieve the following collections of data about PlayStation players¹, compare two players and export player data to XML/JSON.

Basics

PSN ID (as capitalised by the player) and comment
current level and progress to next level (% and points to go)
rank by world and country²
basic stats:
- overall completion rate³
- average game completion rate⁴
- average PSNProfiles trophy rarity
- trophies per day (since first trophy)

Totals

trophies, platinums, golds, silvers and bronzes
unearned trophies
games played
games completed

Summaries (number of x by y)

number of trophies by:
- grade
- rarity band⁵
number of games by:
- platform
- completion band⁶

Length of service (inc. first/latest trophy)

first trophy and latest trophy:
- name
- game
- description
- date/time earned
time between first and latest trophies ("length of service")

Collections (recent trophies/games and rarest trophies)

recent trophies (up to 12):
- name
- game
- description
recent games (up to 12):
- title
- platform
- platinum, golds, silvers and bronzes
- earned trophies, available trophies and completion percentage⁷
- date of latest trophy earned
- time to platinum/100%
- rarity of platinum/100% on PSNProfiles
rarest trophies (up to 5):
- name
- game
- PSNProfiles rarity
- type

Notes

¹ the player must have been pre-scraped/updated by PSNProfiles. Once a player's data has been scraped for the first time by PSNProfiles, it will automatically be updated every 6 hours for the website's premium members, every 24 hours for regular members and every week for unregistered (but tracked) players. Data can be manually updated via the PSNProfiles homepage more frequently: once per minute for premium members and once per hour for everyone else. However, if a profile hasn't been manually updated in a while (the past month?), automatic updates will be suspended.

² of profiles tracked by PSNProfiles - see (1) for details about tracking.

³ overall completion rate is a weighted percentage of non-platinum trophies earned across all games with at least one trophy earned. The weighting is by trophy type, as per the points each are worth on the PlayStation Network itself:

platinum: 300 points (platinums are not included in the calculation)
gold: 90 points
silver: 30 points
bronze: 15 points

⁴ this is different to the overall completion rate - this is the average completion percentage of each of the player's games (so a game with two bronzes and a 50% completion percentage would have equal effect on the average as a game with 500 golds and a 50% completion rate, regardless of the trophy composition of any other games).

⁵ rarity bands are as per PSNProfiles:

0 - 4.99% ('Ultra Rare')
5 - 9.99% ('Very Rare')
10 - 19.99% ('Rare')
20 - 49.99% ('Uncommon')
50 - 100% ('Common')

⁶ completion bands as per PSNProfiles:

80 - 100%
60 - 79.99%
40 - 59.99%
20 - 39.99%
0 - 19.99%

⁷ weighted by trophy type, including platinum - see (3) for points by type.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/yndajas/PSNProfiles-player-scraper.

Licence

This gem is available as open source under the terms of the MIT License.

Disclaimer

This app and its creator have no affiliation with PSNProfiles or the PlayStation Network/PlayStation beyond the creator's use of both services.

yndajas / psnprofiles-player-scraper Goto Github PK