Use Entity-Relationship Model as Definitive Reference about transit

[abyrd]

☝️ Yes. Or even better, this: #127 (which could be easily used to generate the above)

Thanks @e-lo for pointing that out. That ticket, and especially the comments on that ticket, are very related to what I'm proposing here. What I'm proposing is a bit stronger and has a different goal though, so I'd like to treat it as a separate issue.

I believe #127 is suggesting that the GTFS spec include, as an additional derived element or resource, a machine readable file for the specific purpose of generating source code to hold in-memory representations of GTFS data. In the present PR, I'm considering source code generation as a nice potential side effect but not the core purpose. I'm also proposing that we focus attention on the entity-relationship model itself, not the file that represents it, and give this model a very central role in GTFS, both technically and organizationally.

Regarding the problem statement that you and @dedavance listed in comments on that other issue in
#127 (comment) and
#127 (comment)
I generally agree, but with the following emphases and differences in the context of the present issue:

A) Data model as definitive or canonical source. In a comment on the other issue you said:
#127 (comment)

I'm also interested in if the community would be amenable to using this type of definition as the canonical GTFS definition such that we can generate Markdown/HTML from the programatic definition in JSON rather than visa-versa.

This is exactly what I'm talking about. The key idea being that the model is the source of truth, not just another derived artifact to be manually maintained in the spec. And to the extent that something like for example protobuf definition files can't be generated or patched directly from the entity-relationship model language, they should be derived by hand using the most mechanical or automatic process possible. Anything that emerges as non-obvious in this derivation should be clarified in the data model as the source of truth, not in the downstream artifacts.

B) The data model is equally central to GTFS, GTFS-RT, GTFS change proposals, and GTFS-RT change proposals.
Proposals and discussions in all these places would primarily or at least initially concern changes to the data model, and which aggregates within that model are serialized as files or messages.

C) I'm suggesting that the governance and amendment process (currently under discussion in #413) be modified to take this into account. For example, an initial change proposal might consist of only a use case description and a patch to the data model. People would discuss and vote on only the data model change first, before anyone invested time in the minutiae of serialization into text files. Everyone would be looking at and discussing a single visual image. The rest of the process would need to be tweaked accordingly, this is just the initial idea.

I think GTFS has reached a size and complexity, both technically and in community structure, where its maintenance and viability may depend on this kind of formalization.

I'm starting to feel like this is almost a precondition for me to even participate in the change process. It is genuinely exhausting to reverse-engineer what personal data model each person has in their heads when reading each comment, and to try to describe fragments of that data model in text over and over, trying to determine whether one's own model matches others. Frequently I get the impression we are proceeding in the absence of even a mental data model, so the long-term coherence and stability of the spec will rely on the sheer chance of someone showing up at the right moment to infer and verify this shifting ghost of an idea.

[e-lo]

☝️ Yes. Or even better, this: #127 (which could be easily used to generate the above)

[abyrd]

By the way, I fully understand and appreciate that GTFS was created as the an antidote to “model entire world with angle brackets before implementing” traditional standardization processes like those of TCIP or NeTEx. People wanted to be able to create a stop without nesting XML 5 layers deep. But "formally defined data model" does not mean complex or unwieldy or excessive (as evidenced by DBML above).

GTFS started out as a CSV dump of a relational database at TriMet. People with some experience working with databases or data modeling tend to sense that and treat it as a relational model when discussing extensions. I think it benefits greatly from acknowledging that underlying structure.

At this point, the core of that specification has existed largely unchanged for a decade, and is used and directly or indirectly by millions of people. It’s no longer a handful of files from which every reader intuitively perceives the source structure.