djrscally / pubg_reporting Goto Github PK

Code doesn't support multiple shards.

Config file needs to support shards in a comma separated list.

Currently, the sync pulls all matches for all players. We shouldn't bother syncing data for matches that already exist in the matches table

Hi there,

by running data the table player_season_stats will not be updated until i change Last Sync Datetime to a date in the past.

I just matched it to get_player_season_stats; I.E. it syncs for a player where they've played since the last sync. But actually, only certain matchType (Only one actually; "competitive") counts towards changing the ranked season stats, so we should only sync those data for players who've played a competitive match since the last sync

Hey there, its me again ;)

Szenario:
I added player A and run the sync. Everything works perfect.
Then i added player B who had a match with player A yesterday. Sync works, but:

Issue:
The stats of the match player A and B played together is not showing up in player_match_stats for player B.

Ability to link players together as teams or clans.

Table required for 'Teams', then table required for linking players to the teams.

Hey there,

great work. Don't know if its the right place, but i created a dockerized version under https://github.com/NeospinTTV/pubg_reporting_docker. Feel free to use.

Currently, lifetime and current seasons are synced for all players. Players who's last match was played earlier than the last time the sync was performed should not have their current season or lifetime stats synced.

Basic covering indexes are probably enough. Provide an initial level of indexing to help future queries and anyone not savvy about mysql but using a visualisation layer like metabase.

I currently update a MySQL table with 'Active' players, this is then used in the python query to pull the stats data. At present if I remove a player from the list they are not removed from the data pull.

Ranked games modes went live this week and there is a new call in place to pull the data out. This will need incorporating into your solution.

/players/{accountId}/seasons/{seasonId}/ranked

https://documentation.pubg.com/en/seasons-endpoint.html#/Ranked_Stats

It would be great to add players by insert just the player data (or just the name) to table players.

Background: I would like to build a REST API for getting stats. But adding players won't work with config.json.

Title. A new field got added. we wants it precious, we wants it.

I currently pull out match and season data into metabase leader-boards. Yesterday I added a few new players to my included players table and updated the names of a few existing ones. These new players don't seem to be appearing in my leader-boards (season stats based), the updated name ones are.

I also have a leader-board pulling from match data for specific matches, the new and updated name players are not showing on there.

get_player_season_stats is extremely slow due to the sleep calls to avoid traffic management. Replace witha batch call.

I am currently using your process to create analysis that supports my PUBG tournaments. Cross platform custom matches are now a thing and we had our first tournament on the 18th that included both xbox and PS players.

The current setup only pulls data for Xbox players so none of my PS players are appearing within the analysis. Would be great to develop on the existing setup to pull data from both shards.

Kind regards
MajorClagNut

Hey there,

all data seems to be loaded correctily but the player season stats.
I get the following error:

2020-02-26 20:29:58,090 INFO sqlalchemy.engine.base.Engine {'player_id_1': 'account.712916eb68b24d6691437221cf8b7529', 'season_id_1': 'division.bro.official.pc-2018-04'}
2020-02-26 20:29:58,091 INFO sqlalchemy.engine.base.Engine SELECT EXISTS (SELECT 1
FROM player_season_stats
WHERE player_season_stats.player_id = %(player_id_1)s AND player_season_stats.season_id = %(season_id_1)s) AS anon_1
2020-02-26 20:29:58,091 INFO sqlalchemy.engine.base.Engine {'player_id_1': 'account.712916eb68b24d6691437221cf8b7529', 'season_id_1': 'division.bro.official.pc-2018-05'}
Traceback (most recent call last):
  File "sync.py", line 140, in <module>
    sync()
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "sync.py", line 53, in sync
    __sync(api, pubgdb)
  File "sync.py", line 124, in __sync
    api.get_player_season_stats(combo)
  File "/pubg/pubg_reporting/pubg/pubg_api.py", line 154, in get_player_season_stats
    logging.debug("get_player_season_stats: {0}: {1}: {2}".format(combo[0], combo[1], json.dumps(r.json()['data']['attributes']['gameModeStats'], indent=4)))
KeyError: 'data'

Thanks for your work and support!

There is no file called database/create_db.sql.

PUBG seem to have updated the API to throw a 429 error if you call the seasons endpoint too often. TBF their docs do say not to call more often than once a month but they never seemed to enforce it before so i ignored it. That's changed, so the code needs to too.

Enviornment variables are set, software installed locally. Recieve this error

PS D:\pubg_reporting> & D:/Python/python.exe d:/pubg_reporting/sync.py D:\Python\lib\site-packages\sqlalchemy\dialects\mysql\base.py:1308: SAWarning: Additional column names not matching any column keys in table 'players': 'status' util.warn( D:\Python\lib\site-packages\pymysql\cursors.py:170: Warning: (1287, "'VALUES function' is deprecated and will be removed in a future release. Please use an alias (INSERT INTO ... VALUES (...) AS alias) and replace VALUES(col) in the ON DUPLICATE KEY UPDATE clause with alias.col instead") result = self._query(query) Traceback (most recent call last): File "d:/pubg_reporting/sync.py", line 219, in <module> sync() File "D:\Python\lib\site-packages\click\core.py", line 829, in __call__ return self.main(*args, **kwargs) File "D:\Python\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "D:\Python\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "D:\Python\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs) File "d:/pubg_reporting/sync.py", line 59, in sync __sync(api, pubgdb) File "d:/pubg_reporting/sync.py", line 187, in __sync api.get_player_season_stats(combo) File "d:\pubg_reporting\pubg\pubg_api.py", line 200, in get_player_season_stats logging.debug("get_player_season_stats: {0}: {1}: {2}".format(combo[0], combo[1], json.dumps(r.json()['data']['attributes']['gameModeStats'], indent=4))) KeyError: 'data' PS D:\pubg_reporting>

it looks like it created some tables, but was unable to finish the job