everypolitician / data_pr_change_summarizer

DEPRECATED: Please use everypolitician-pull_request

EveryPolitician

Data about every national legislature in the world, freely available for you to use

• everypolitician.org — data | about
• Report an issue

Repo summary

These are some of the key repos in the EveryPolitician family. There are others.

• everypolitician (this repo): contains no code, but is where issues/tickets for the whole project live

• everypolitician-data: where the data is stored -- but if you want to download it, get it from:

  • human? go via the EveryPolitician website
  • program? use the RawGit CDN, via links in countries.json, which we explain here

• viewer-static: the live website http://everypolitician.org (gh-pages)

• viewer-sinatra: Sinatra app for generating a dynamic version EveryPolitician website

• webhook-manager: sends out EveryPolitician WebHooks: register your URL here!

• everypolitician-docs: documentation at http://docs.everypolitician.org/ (gh-pages)

• rebuilder rebuilds data from source

• libraries for easily manipulating EveryPolitician data (useful for all devs, but we use the Ruby ones ourselves, of course!):

  • Ruby: everypolitician-ruby and everypolitician-popolo.
  • Python: everypolitician and everypolitician-popolo

• handy gems we use when getting the data: wikidata-fetcher, wikisnakker, twitter_username_extractor, facebook_username_extractor, twitter_list, scraped_page_archive

• gender-balance: repo for the Gender Balance website that crowdsources gender data for EveryPolitician

• data_pr_change_summarizer: code used by the bot to review a data PR and leave a helpful summary as a comment

The repos for many of our scrapers are kept separately in github.com/everypolitician-scrapers.

Technical blog

The EveryPolitician bot's own page is a good jumping-off point to lots of semi-technical explanations of what's going on (it has its own blog on Medium). For example:

• how the website is built (spoiler: viewer-sinatra → viewer-static)
• how webhooks are used (you can easily register your app!)
• how the scrapers run (many live on morph.io)

The bot is on twitter as @everypolitician

Contributing

If you have data for us, or know where to get it, please read our page about how to contribute.

Team

EveryPolitician is a mySociety project.

data_pr_change_summarizer's Issues

Warn about added/removed Areas

similarly to People and Parties, warn if any new / vanishing Areas.

Warn about added/removed Memberships of existing people

There's no need to mention Memberships added or removed where the People are also new/gone, but we should warn of other Membership changes.

Report on People who are no longer mapped to Wikidata

If the Wikidata scraper stops picking someone up, we'll lose their data here, and if that's buried in a lot of other changes, we could miss it.

Show the names of people/parties added/removed

Currently we just list the IDs, which isn't the most helpful:

• everypolitician/everypolitician-data#3458
• everypolitician/everypolitician-data#3474

Would be good to show the name as well (probably good to also keep showing the ID)

Use commit statuses to prevent accidentally merging pulls

Use GitHub's repository status API to add an indicator to pull requests, the same as travis ci does, to show if it's safe to merge a pull request or not.

Rename this repository

After a conversation in Slack earlier it was suggested that we should rename this repository as it can be tricky to find.

• "pull_request_summary"
• something with "summar[y|ise|ize]"

Report records that have changed

Currently we just report records that have been added or removed based on their id field. As the first step towards everypolitician/everypolitician#336 we need to report people/organizations that have changed in some way, e.g. having an other_name or links item added or removed.

Report on any Person|Organization that has changed name

everypolitician/everypolitician-data@61054c0 had a fairly nasty name change that could easily have been overlooked amongst the hundreds of sensible changes.

As a precursor to the much more elaborate #11, we should simply report for now on any records that change name.

Report what has changed in a record

Once #10 is done we will have a list of records that have changed in some way. The next logical step towards everypolitician/everypolitician#336 is to report what has changed about the record.

Don't try to post a blank comment

If there are no popolo files in the target pull request we still try to post a comment, which fails because the comment is empty.