kafagy / fifa-fut-data

Web-scraping script that writes the data of all players from FutHead and FutBin to a CSV file or a DB

FIFA FUT Players Data

What is the purpose of this script?

• The purpose of this repository is to have a script that automatically pulls down all players data from Futhead's website for all FIFA versions starting with FIFA 10. And to pull detialed data from Futbin.com

How to use it?

• Just run the fifa.py script: python3 futbin.py to get Futbin data. python3 futhead.py to get FutHead data.

Players data ouput format:

• After running the script the user will have the data saved in a MySQL database and CSV files.

Requirements:

• Python 3
• MySQL
• PIP

You will need to use PIP to install the following libraries:

• pymysql
• bs4
• requests
• pandas

Note:

• You will need to sucessfully run the futhead.sql script before running the futhead.py parser.
• Input database's host, database name, user and password in the futhead.py connection line.
• Make sure you create tables using the DDL in futhead.sql to be able to correctly encode latin literals when writing to the Database.

fifa-fut-data's Issues

futbin.py not working for fifa10 to fifa14 included

here is the error it gives

Nations

Could you add the players nations to their data?

Is there any way to extract IDs & face PNGs with the Futhead.py?

I did some bits and pieces to make the futhead.py write FIFA 21 data. But i get no IDs or face PNGs, which i really quite need. Thanks!

problems with error 403

Adding a proxy, generating a user agent and random delays do not help, after 1 day of work, futbin blocks

goalies missing

super work,
however I think in your datasets e.g. FIFA19 the goalkeepers are missing. Can you fix that please?

SyntaxError when running

Hi,

I just found your amazing script and i wanted to try it. But when i execute "python fifa.py" on my Ubuntu 18.04.1 there is this error message:

File "fifa.py", line 104
    '''.format(value), (*player, *attribute))
                        ^
SyntaxError: invalid syntax

I updated python and the needed libraries.

Many greetings

Add titles for all columns in the fifa18.csv file

Add the title names to the csv file before the loop that writes the data gets started.

Number of players per page is hard-coded as 48.

I need to dynamically be able to determine how many players are in a page.

Wrong stats in the Fifa20.csv

I noticed that the in Fifa20.csv are completly wrong. The Overall stat is fine but the attributes of the players are wrong most of the time. For example i checked the stats of Robert Lewandowski because it was a bit odd that he had only 28 shooting and then i found out that these were actualy the stats of Kalidou Koulibaly.

Doest Work With FIFA 22

I was trying to make it work for FIFA 22 but doesn't seem to be working, really looking forward to having working script for FIFA 22

Only first 10k players taken

The database is about nearly 15k players wide, and the main players page only shows the "first 10 thousand players". A fix for it could be separately dumping bronze, then silver, then gold players. As no quality exceeds 10k players, taking them all should be seamless.

adding value from all platforms

Good day, is it possible to parse not only PS4 prices, but also other platforms?

Regular DB

Hi i was wondering if you could include the regular Fifa DB