Add support for adding key/values directly to Person and Organization about popolo-spec

[tswicegood]

During a discussion at #mozfest about using Popolo to share civic data, an interesting problem came up: how do you query for all members of a legislature that are of a given party. With the current structure of Popolo, you would have to do two queries (or a subquery): "Give me all Persons who are members of {{ political party }} who are also members of {{ legislative body }}". The current specification makes the relationship to an Organization a many-to-many link through Membership which makes for an inefficient query.

The discussion turned to ways to add meta data directly to the Person or Organization. For example, current party affiliation is common enough in the US to nearly be a requirement. Having a mechanism for adding that would be useful.

A few possible solutions:

• Mechanism for adding arbitrary attributes (couldn't resist the illiteration)
• Mechanism for sub-classing Person to extend them
• Tacking attributes on by defining relationships

Arbitrary Attributes

This solution might look like:

{
  "name": "Travis Swicegood",
  "extra": [
    {"currentParty": "Independent"}
  ]
}

In this case, extra is an array of objects that make no guarantee.

Sub-classing

This would advocate adding extra elements directly to the Person. It means you provide a base level of functionality, plus custom functionality. It would be useful to have a collection of patterns for custom sub-classes if this route is explored.

{
  "name": "Travis Swicegood",
  "currentParty": "Independent"
}

Relationships

You can manage this style by re-using something like HAL and looks like:

{
  "name": "Travis Swicegood",
  "_links": [
    {"currentParty": "http://example.org/schema/politicalparties/independent"}
  ]
}

---

I'm not really sure which of these is the best approach. Sub-classing seems like a nice way to go, but I tend to prefer composition over inheritance. Really curious for some other thoughts here.

CC @dwillis

from popolo-spec.

[jpmckinney]

We're talking about caching properties to improve query times, yes? The data model shouldn't be concerned with caching. At the same time, it shouldn't make implementing caching strategies unnecessarily difficult.

If you use the new onBehalfOf property in your Membership model, then you can just do one query like (in MongoDB):

db.memberships.find({
  on_behalf_of_id: 'party-id',
  organization_id: 'legislature-id',
  start_date: {$lt: '2013-10-27'},
  end_date: null
})

You'll still need to do a second query if you want to get the names, etc. of the people. If you don't like doing queries, you can always choose a database that implements materialized views (or use software that adds that feature to whatever database you use). If it's a common query, you can have your software cache/expire its results as appropriate. The list of active people in a legislature changes very infrequently, so you can cache it for a long time.

The options for adding additional properties are described in an appendix. "Subclassing" (or any other method of extending a class, like composition) is generally what I recommend: use Popolo for its standard properties, and add whatever use case-specific properties you need. I prefer using a dictionary (hash) instead of a list (array) for arbitrary properties, but I guess you can do either.

If you want to implement HAL, you can, but it's not relevant to the discussion of the data model - HAL is about designing standardized APIs. There's no functional difference between putting currentParty at the top-level or within _links; it makes a difference to HAL though, because HAL doesn't know how to interpret a top-level currentParty property, but it does know how to interpret its special _links property. Tools like JSON-LD and RDF, on the other hand, are capable of properly interpreting currentParty without requiring that it appear under a special _links property.

from popolo-spec.

[jpmckinney]

In short, the best strategy for reducing the number of queries is to use a cache. Playing with the data model is not going to make anywhere near the same impact as caching will on performance.

[dwillis]

I agree with the subclassing approach, and using a hash for arbitrary properties. I can see the usefulness of HAL-style links, but I suspect using hashes might be simpler and seem more understandable for other users.

[tswicegood]

Also leaning toward the sub-class approach. There are certain types people (Legislator for example) who can't exist without a party, so it makes sense to make them a more specific Person.

Thoughts on mentioning this in the docs somewhere?

[jpmckinney]

Independent (party-less) legislators exist.

The conformance section explains that you can extend the classes in Popolo, but maybe it should be make explicit on the homepage?

[tswicegood]

Doh! That's what I get for making a universally absolute statement. What I meant was a Legislator in Texas can't exist without a party (in Texas, Independent is considered a party).

Re conformance: a section called Extending Popolo might be good with a few use-cases. I'm happy to contribute a few if it would be helpful.

[jpmckinney]

Is "Independent" legally considered a party, or is that just what legislative IT systems have done because it's simplest? (Are they just fitting a square peg into a round hole?) It's not true that all independents belong to the same party (though maybe in Texas?), so I have trouble understanding how "Independent" can be a party the same way "Conservative Party of Canada" is a party.

If you would write up a section on "Extending Popolo" it will give us something concrete to collaborate and improve on :)

[jpmckinney]

Btw, some discussion about how to represent independents is in this appendix and later under the heading "Revisiting the independent candidate example". It's in an appendix because I don't know what the best recommendation is.

[jpmckinney]

My understanding is that this issue does not require any change to Popolo, although we did raise the possibility of authoring some content about extending Popolo, which can appear in an appendix. I'll close the issue for now (we can open a separate issue to collaborate on that new appendix), but if I've misunderstood, please request that I re-open the issue.