GeoJSON in GTFS? (Or the future of GTFS serialisation) about transit

[leonardehrenfried]

As someone that maintains OTP's Flex implementation I only have a one mundane comment on this topic: Flex is a significant departure from GTFS static. It's difficult to implement but that difficulty doesn't stem from it being CSV, wkt or geojson - that's the easy part. It's the huge variability and the explosion of possible results that flex adds that is the real complexity not the choice of geographic representation.

That said I also welcome there to be a discussion so that whatever decision we end up making is a deliberate one rather one that everybody thought someone already made.

[tsherlockcraig]

I can provide limited context here, but I think it's most of the context needed here to explain that decision. I project managed applications that used both wkt and geojson, but am not a developer and can't speak directly to the technical limitations of either approach, but I can speak to the business reasons and reports I received from technical parties at the time.

• the comments from the flex repo discussion above represent the final decision making just before the implementation of GTFS-Flex v1 (ie, wkt-based) in OpenTripPlanner. I believe the resulting code should still be more or less intact in OTP v1.4 and 1.5. Geojson came up at the time, but it was still in the middle of its astronomical rise to geographic-representation-dominance, not established as ubiquitous, and we didn't even really consider going outside the csv. Previously, gtfs-flex in it's very original conception had included a custom geographical representation, and using wkt represented a move to adopting another open standard rather than creating our own.
• wkt worked 'fine' in the v1-based trip planner. I think issues came up about parsing it but they weren't the major issues in the development process.
• the idea of using a geojson file specifically was proposed by MobilityData I believe in 2018. I'll admit at first I was skeptical. However, two things convinced me personally over the next year that it was the superior choice: 1) the developers didn't complain about it, and generally reported it was as easy to work with as wkt in a csv, 2) there were huge benefits for human readability and editability of the files, thanks to the ubiquity of free geojson visualizers and editors. That discussion was on a previous Google doc and in virtual meetings.

I think all the above is important and it's why i came to support geojson in GTFS-flex. It's also why I, from a business and community angle, don't see anything wrong with us bringing in other file types besides .csvs, if they're the right technical tool for the job.

But I think @skinkie raises a different valid and important question that we should hear from technical parties on , and @eliasmbd 's call to the conversation at #127 feels particularly relevant to me although it's way above my head. Even if this is right for business reasons, what are the technical implications for existing systems and the future of the technical options available to or required of the spec?

One question we should ask: are there other options we should seriously consider besides geojson and wkt in a csv? If there are no other serious contenders, that at least might simplify this discussion.

[leonardehrenfried]

Allow me to respond to the side comment about Netex: what I like about GTFS is that it's hugely pragmatic rather than a giant standard where every country has their own "profile" because anything else is too large to manage.

So I take a well done GTFS feed over a "more elegant" but terribly implemented Netex feed any day. It's much, much easier to achieve "well done" with GTFS . The majority of producers struggle with GTFS, so what hope is there of them producing good Netex ones?

[bdferris-v2]

Since @eliasmbd has prompted us to give our thoughts on this before the session tomorrow, here are mine:

The quick version is that I'm not immediately opposed to adding GeoJSON to the GTFS spec.

I ultimately come back to the GTFS Guiding Principles, which is make GTFS easy to produce and edit. My intuition here is that there will be many data producers out there who are managing their geographic assets, including service region polygons for Flex, in standard GIS applications. And for many of the most popular GIS applications, GeoJSON is a well-supported export format that could just work off-the-shelf.

I think a similar case can be made for CSV + WKT, but I think the tooling isn't quite as seamless.

Why didn't GTFS consider GeoJSON for something likes shapes.txt originally? If I understand my GeoJSON history correctly, GeoJSON has only really been a thing since 2007 and only an RFC since 2016 (GTFS being born in 2006). Might history have been different if GTFS has come slightly later? I do not know.

What other data formats might we consider? Conceivably, you might look at anything on the GDAL-supported Vector format list but I think there are only a handful of formats that are simple enough, have reasonable governance, and have been around long enough for consideration. I don't think it's an accident that GeoJSON is at the top of that list.

I recognize that producing GTFS (and GTFS-Flex especially) has gotten complex enough that it may not be reasonable to support the simple use-case of a transit operator typing up data in a spreadsheet and we may have expectations that some sort of GTFS export application will be in use, in which case some of these arguments around facility of creation carry less weight. That said, I do think there is something to be said for being able to quickly visualize data in a feed and GeoJSON does have some advantages there.

Anyways, looking to hear from other folks tomorrow. Thanks!

[westontrillium]

I think this discussion needs some historical context as to why the switch was made from expressing polygons using WKT strings in a .csv file to using GeoJSON, given by someone who was involved in the Flex (v2) drafting process.

[e-lo]

Thanks for bringing this up – I think this is a good discussion to have on its own and separate from (but with dependency on) GTFS-Flex. Managing and using another format does indeed bring in a challenge that should be carefully considered.

Some thoughts on introducing geojson as formulated in proposed locations.geojson:

• As I've discussed in other issues, I am not in favor of storing anything other than a foreign key and geometry in locations.geojson.
• I seem to be in the minority here so am happy to let it go iff....we can easily (think: one line of pandas code) translate the information in locations.geojson into a dataframe (or equiv) format. No nested properties.
• geojson does seem to be the easiest format for many transit agencies to create/maintain/review geometry in, given tools like geojson.io and compatibility with other tools.

I do think shapes would likely benefit from having a natively viewable format as well. I'd be curious about the previous discussion points and why this wasn't pursued in the end.

[e-lo]

The only convo I could find on WKT in the GTFS-Flex repo shows a great deal of support for it >> GeoJson. Will try and find where decision was made in other direction.

MobilityData/gtfs-flex#5

...somebody with edit access to the gtfs-flex google doc might be able to search the comments and version history for more.

[skinkie]

The majority of producers struggle with GTFS, so what hope is there of them producing good Netex ones?

A proper free desktop implementation that manages data as a producer and uses NeTEx as its internal model, not conversion on conversion ;-)

[eliasmbd]

📣 We have an event registration page. Please sign up and share to all relevant parties. As expected the meeting will be held on Tuesday August 8th @ 11am EDT. (Sharpen your Miro pencils 😉 )

[eliasmbd]

@westontrillium Is there anyone that you think can provide that historical context? Maybe someone we can invite to a live meeting?

[westontrillium]

Attn: @tsherlockcraig

[eliasmbd]

One question we should ask: are there other options we should seriously consider besides geojson and wkt in a csv? If there are no other serious contenders, that at least might simplify this discussion.

@tsherlockcraig I second this. I think our primary course of action is to evaluate the viable options (Pros/Cons). What we do here will define the future possibilities of GTFS and I applaud @skinkie for bringing this up when he did.

At this point we are working on a timeframe for a meeting. Before then, I would like to invite you all to share this issue to the relevant people in the community, new and old. It is important that everyone that should see this issue does before we engage in a virtual meeting. Internally, we are working on an appropriate stakeholder outreach as well.

[eliasmbd]

✏️ We have a few dates we would like to propose for the virtual meeting. Please fill out this form to find a good time for everyone.

Fill out form

[eliasmbd]

As we prepare for the meeting, we have asked you to share your expectations with us. In order to help us scope this meeting, we will post your expectations here.

Preferably, the vision 'beyond' GeoJSON. What should be done when we change significant parts of the standard. For example CSV to XML, CVS to JSON, Protocol Buffers to CBOR.

I think we should leave this meeting having answered whether there are options other than geojson and wkt that need to be researched. At very least, we identify a qualified group to make that determination and begin research. Important that we have technical stakeholders in this meeting. Needs of business stakeholders (like myself) should be de-prioritized in my opinion.

hopefully a focused discussion, not a general formats flame war as usual on the internet 😬

As you can see, expectations are diverse. From my end, I would propose to maintain the focus on GeoJSON and the alternate solutions out there. I kept the expectations anonymous but invite you all to participate helping us keep the scope focused and precise.

Also, it can be expected that we will host the meeting on Tuesday 8 August at 11am EDT. More details will follow.

[eliasmbd]

🙏 Thank you for joining us for the strategic meeting held yesterday. It was an eye opening and refreshing discussion for many of us.

📝 Here are some takeaways from the meeting:

• Most participant seemed interested in adding a new format to GTFS

  • MobilityData will reach out to stakeholders representing agencies/users with limited technical capacity and resources to solidify the interest and make sure that we are aware of possible implications for them.

• We noticed a consensus was building around the specific geometries that the community wanted to target - zones and shapes.

  • Stops were a more controversial subject

• Many participants showed support for the GeoJSON format but some voiced the options of using GPKG

  • MobilityData will research GPKG and then compare it to GeoJSON

• MobilityData will provide the community with a few options considering the implications of adopting a new format and recommendations.

🗓️ Once the points above have been resolved, MobilityData will announce a follow up meeting - expect the meeting to be held sometime in September.