communitydatasciencecollective / covid-19_digital_observatory Goto Github PK

The wikiproject article list (currently COVID-19_Digital_Observatory/wikipedia_views/resources/articles.txt) should include provenance data for each article name.

Provenance information should minimally include date collected and source (e.g., "WikiProject Covid-19").

This definitely requires edits to "wikiproject_scraper.py" and may affect all files that assume the existence of the articles.txt list.

It will be especially important whenever we incorporate multiple sources (e.g. Wikidata entities and translations) that append content to the articles.txt file.

May be valuable to also create an archive file to capture historical instances of article names that have been added to the list multiple times so that provenance data can be maintained while we continue to extend this list.