Add populations to Wikidata about li

[shaperilio]

Populations on Wikipedia:
https://en.wikipedia.org/wiki/Provinces_of_Panama#Provinces

Alternative source from @ciscorucinski which has it at county level:
https://www.citypopulation.de/en/panama/admin/

[hyperknot]

Yes, sometimes I'm using Wikipedia as a source, but still it needs to be added manually. Citypopulation.de doesn't allow downloading the map data. Also we prefer official data with matching government dataset.

[shaperilio]

Here's population information down to corregimientos level (the granularity at which we get COVID data):
https://github.com/EricLuceroGonzalez/Panama-Political-Division
Population is presumably from the 2010 census, but it would have to be verified.

Which raises the question: how are we validating any of this?

[ciscorucinski]

It seems like an intimidating interface with wikidata for adding information

[hyperknot]

@ciscorucinski you mean how to add population data?

Basically:

1. click "add statement" at the bottom
2. select populations
3. enter the value
4. add qualifier
5. select point in time
6. enter year
7. add reference
8. select URL or type P4656 for wikipedia import
9. paste URL
10. save

If you do this in multiple steps, it's quite easy to get over the 50 required edits to get your account autoconfirmed. For example add - publish - add qualifier - publish - add reference - publish can get you 4 edits. So with 13 regions you are over 50 edits :-)

[shaperilio]

Done for Panama's provinces.

[ciscorucinski]

There are ways of doing this via Google Sheets and a tool called QuickStatements. Since we are only concerned with one type of data import process, we should be able to create a fairly standardized process within a spreadsheet.

Google Sheets + QuickStatements: https://www.youtube.com/watch?v=bUpJN4IklJ8
OpenRefine: https://www.youtube.com/watch?v=wfS1qTKFQoI

[hyperknot]

@ciscorucinski if you can mass import using this tool it'd be great! So far I've done all my edits by hand.

[ciscorucinski]

@hyperknot you can! But I am uncertain how to go about doing it for this data right now

[hyperknot]

Luckily we don't have that many missing populations. If we encounter an other country with a lot, I'll comment here.

[ciscorucinski]

is there an easy way to find what is missing?

[hyperknot]

Ones without population in this JSON: https://raw.githubusercontent.com/hyperknot/country-levels-export/master/iso2.json

[hyperknot]

Portugal seems like a good candidate: https://github.com/hyperknot/country-levels-export/blob/master/docs/iso2_list/PT.md

[hyperknot]

We need to add: Slovenia, Ireland, Poland, and Lithuania.

[hyperknot]

I fixed Ireland and Poland. What is missing in Lithuania?

For Slovenia, it really needs that batch updating effort! @ciscorucinski can you help with that?

[ciscorucinski]

Let's create a Google Sheet, and try out a few records before mass editing. I have never edited a wikidata entry, so consider me a noob here 😅

What info is needed to identify a population point in terms of wikidata? We need Q IDs for a few datapoints, but these can be retrieved through a wikidata Chrome extension in Google Sheets.

Just datapoint names such as Country, State, and county level names should be good enough I guess??? Along with the population data and url reference

[hyperknot]

All the Q-s we need are here: https://github.com/hyperknot/country-levels-export/blob/master/docs/iso2_list/SI.md

Machine readable format is this:
https://raw.githubusercontent.com/hyperknot/country-levels-export/master/iso2.json

The other side of the equation should be some government census CSV listing those populations in a CSV.

[qgolsteyn]

Really not ideal (has some weird character errors) but here is a CSV from the Slovenian Statistical Bureau. Data is from 2019.
https://gist.github.com/qgolsteyn/145d82f984d65c34e778371a69cf5433

[hyperknot]

@qgolsteyn thanks! Do you have the source for this file? Maybe chardetect would tell us what encoding it's in.

[qgolsteyn]

I don't have it immediately, but will get the source to you by this evening. I also update the list with additional countries that need population info

[hyperknot]

Thanks!

[qgolsteyn]

My appologies, here is Slovenia's data: https://pxweb.stat.si/SiStatDb/pxweb/en/10_Dem_soc/10_Dem_soc__05_prebivalstvo__10_stevilo_preb__20_05C40_prebivalstvo_obcine/05C4002S.px/table/tableViewLayout2/

[shaperilio]

Portugal is done, as is Colombia. Working on Slovenia next.

[shaperilio]

I think Slovenia is done, but I got "errors" on their tool, despite there being hundreds of successful edits....

EDIT

Because I tried to add the atomic number of a municipality among other atrocities 😆 Anyway, it's processing now, should be done soon.

[shaperilio]

Lithuania should be done...after much struggle. I'm off for the rest of the night.

[hyperknot]

@shaperilio thanks so much, I've updated the file already but I'll make a new processing for Lithuania as well.

[ciscorucinski]

Korea should be up to date now

[jzohrab]

@hyperknot , is this issue still open? Wondering what the current status is. Cheers, z