aquatix / ns-api

Query the Dutch railways about your routes

nsapi

Query the Dutch railways about your routes, getting info on delays and more. See below for the syntax and example output.

Installation

From PyPI

Assuming you already are inside a virtualenv:

pip install nsapi

From Git

Create a new virtualenv (if you are not already in one) and install the necessary packages:

git clone https://github.com/aquatix/ns-api.git
cd ns-api
mkvirtualenv ns-api
pip install -r requirements.txt

Development in Docker

Git clone this repository, use vscode & docker to remote open in a dev container as explained in this howto.

As part of ns-notifications

Alternatively, follow the installation instructions of ns-notifications, which makes extensive use of this library to serve notifications to for example a smartphone. The requirements of both packages can be installed in the same ns-notifications one mentioned in the project; ns-api will be installed through pip from PyPI.

Home Assistant

The very useful Home Assistant home automation software includes an integration for the ns-api. Be sure to check it out.

Also take a look at nsmaps

Bart Römgens created a fascinating contour map called nsmaps based on ns-api. It visualises Dutch railways travel information with OpenLayer 3 contour maps to show how long it takes to get somewhere in the Netherlands by train and bicycle.

Example application

For example, I use the library to push notifications about my route to my phone through Pushbullet. The program I use to do this has its own repository: ns-notifications.

NS API key

To actually be able to query the Nederlandse Spoorwegen API, you need to subscribe. This immediately gives you a primary and secundary key you need for access.

The library uses the reisinformatie API. Here you can try the API for yourself (and look up station names, but the library has the get_stations() function for this too).

The API portal also provides statistics on usage and errors.

ns-api's Issues

AttributeError: 'TripStop' object has no attribute 'planned_platform' in Home Assistant

using Nederlandse Spoorwegen in Home Assistant errors on

2021-04-14 10:04:33 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.tilburg_roosendaal fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 292, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 490, in async_device_update
    raise exc
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 278, in wrapper
    result = method(*args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/nederlandse_spoorwegen/sensor.py", line 235, in update
    self._trips = self._nsapi.get_trips(
  File "/usr/local/lib/python3.8/site-packages/ns_api.py", line 927, in get_trips
    return self.parse_trips(raw_trips, requested_time)
  File "/usr/local/lib/python3.8/site-packages/ns_api.py", line 838, in parse_trips
    newtrip = Trip(trip, requested_time)
  File "/usr/local/lib/python3.8/site-packages/ns_api.py", line 607, in __init__
    trip_part = TripSubpart(part)
  File "/usr/local/lib/python3.8/site-packages/ns_api.py", line 455, in __init__
    stop = TripStop(raw_stop)
  File "/usr/local/lib/python3.8/site-packages/ns_api.py", line 412, in __init__
    self.platform_changed = bool(self.actual_platform != self.planned_platform)
AttributeError: 'TripStop' object has no attribute 'planned_platform'

Discord chat with HA dev's lead to open an issue here in the Library
more info please see: home-assistant/core#50057
thanks for having a look

Minium delay not working for arrival times

I still get messages of "+1 delay" for arrivals.

Disable notifications for arrival delay

Would be nice to turn off messages for arrival delay.

TypeError for specific input due to missing time for TripStop

The following code,

import ns_api

nsapi = ns_api.NSAPI(USERNAME, APIKEY)
timestamp = "04-02-2016 08:00"
start = "Rotterdam Blaak"
via = ""
destination = "Amsterdam Centraal"

trips = nsapi.get_trips(timestamp, start, via, destination)

results in this TypeError exception,

Traceback (most recent call last):
  File "test.py", line 10, in <module>
    trips = nsapi.get_trips(timestamp, start, via, destination)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 858, in get_trips
    return self.parse_trips(raw_trips, requested_time)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 813, in parse_trips
    newtrip = Trip(trip, requested_time)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 550, in __init__
    trip_part = TripSubpart(part)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 443, in __init__
    stop = TripStop(raw_stop)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 384, in __init__
    self.time = load_datetime(part_dict['Tijd'], NS_DATETIME)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 45, in load_datetime
    offset = value[-5:]
TypeError: 'NoneType' object is not subscriptable

This happens for a small number of start and destination combinations (I try about 30000 combinations :) ).
Cause is that part_dict['Tijd'] is None when used in this line. Probably some missing information from NS. The rest of the trip data looks good.

Changing that line to,

if part_dict is None or part_dict['Tijd'] is None:
    return

fixes it for me. But than you lose the full TripStop, which you may not want.

NS Api to change

Yesterday I received an email that NS is to upgrade their API and the current API will be phased out on October 1st. I wanted to make sure you are aware, and see if this implicates that some changes need to be made to this package, which is used in Home Assistant as well.

Text of the email:

Beste NS-API gebruiker,

U ontvangt deze e-mail omdat u met dit e-mailadres geregistreerd staat als gebruiker van de NS-API.

De NS-API zoals u die kent zal uitgefaseerd worden. Van u als gebruiker verwachten wij, als u gebruik wilt blijven maken van onze services, dat u gaat migreren naar onze nieuwe API.

Hoe u dit aan moet pakken is te lezen in de migratiehandleiding die we met deze mail meesturen.

Kort samengevat gaat het volgende voor u veranderen:
· U heeft voor het gebruik van de nieuwe API een API-key nodig. Deze is aan te vragen via het NS API portaal
· URL’s naar de services veranderen
· Het response is in het JSON formaat
· De veldnamen van de parameters en de responses zijn verandert.

U krijgt 6 maanden de tijd om de migratie uit te voeren. De planning is om per 1 oktober 2019 de oude API uit te schakelen en onze services niet meer beschikbaar te stellen.

Bij vragen kunt u ons bereiken via het volgende e-mailadres: [email protected]

Met vriendelijke groet,

NS API team

I have added the attached file to this issue as well.
NS REST API migration guide.pdf

Confusing departure_time_actual behavior

Hey!

Thanks again for updating the library in response to the API changes. I have made a PR to implement it in home assistant (home-assistant/core#30599) and with the reviewer, we stumbled upon a curious behavior and I wanted to ask @aquatix if you saw any benefit in changing it.

departure_time_planned does exactly what it says. It was the originally planned time.

departure_time_actual is a time if there is a delay, otherwise it is None. The name however suggests that it should either be the planned time or the delayed time but never None, since it's the "actual" time. The delayed state could then be determined by compared the planned time and the actual time.

Would love to hear your opinion on this!

Possible improvements for 3.0.2

Thanks for the recent 3.0.1 changes. I was wondering if we could look at a few possible improvements and perhaps release a new minor update before updating home-assistant as we are discussing in home-assistant/core#26622.

1- The actual/planned behavior that happens for the time I think should also happen for the platform.
2- The API portal states the URL is "https://gateway.apiportal.ns.nl/reisinformatie-api/api" while this library uses "https://gateway.apiportal.ns.nl/public-reisinformatie/api". Obviously, both URLs seem to work but perhaps the library should use the one stated in the documentation.

I'm currently reading the code, but could you elaborate a little bit on the current behavior of delay and has_delay? Are they part of the NSAPI response?

Python 2-3 compatibility

I am using ns-api in a small hobby project (https://github.com/bartromgens/ns-maps) that uses Python 3. I needed to make some small changes to get it working for Python 3, see https://github.com/bartromgens/ns-api3/tree/python3-compatibility. I tried to keep it backwards compatible with Python 2. However, I am new to the 2->3 conversion, and am not sure everything still works with Python 2 (because I did not test).

I used this doc as a guideline: http://python-future.org/compatible_idioms.html.
Will do some more tests later and send a pull request if everything looks ok.

TypeError if no trips were found

The following code,

import ns_api

nsapi = ns_api.NSAPI(USERNAME, APIKEY)
trips = nsapi.get_trips('12-01-2016 08:00', 'Utrecht Centraal', '', 'Amsterdam Van der Madeweg')

Results in the following exception,

Traceback (most recent call last):
  ...
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 857, in get_trips
    return self.parse_trips(raw_trips, requested_time)
  File "/home/bart/dev/ns-maps/env/src/ns-api/ns_api.py", line 811, in parse_trips
    for trip in obj['ReisMogelijkheden']['ReisMogelijkheid']:
TypeError: 'NoneType' object is not subscriptable

It is correct that there are no 'ReisMogelijkheden' from 'Amsterdam Van der Madeweg', but this is not correctly handled. I would expect to get an empty list of trips.

Minor issue, which is easy to work around and/or fix. Just for the record.

dependency on deprecated 'future'

Hello,

first, I hope you don't mind I included your component for Home Assistant Gentoo Overlay.

It uses a dependency on 'dev-python/future', and I now have received a deprecation notice.
This library was made for Python 2 compatibility, and is redundant. Many linux distributions work on removal.

It had to be patched to work with Python 3.9+ and is also vulnerable to CVE-2022-40899.

Would it be possible to switch to a different library?
Thanks a lot.

\B.

Train "rijdt niet" needs timestamp

Message now doesn't show what exact train is being talked about.

delays may have strings in them

A delay may be "+ 2 min", which can't be parsed to an integer. Implement better parsing.

Log rotation to prevent big log files

Let the log files rotate (e.g., per day) to prevent huge files.

Add 'train cancelled' recognition

When the connection at a specific time has been cancelled, notify.

Multiple notifications of same disruption

In case of a disruption, the same notification is send every time the application checks the routes.

TypeError: the JSON object must be str, bytes or bytearray, not NoneType

Was requested to create an issue here about the NoneType error i had in HomeAssistant.

The error is starting to shop up more and more. Will look into if we can turn on extra logging

Yeah, it looks like it does not get any data back, and then wants to convert it to json (Python dict), which of course results in a crash (it talks about NoneType, which means data was None). The library can try to catch that, but will likely then throw a custom error of its own, as it is not an answer we expect.

Wed Feb 12 2020 07:30:05 GMT+0100 (Midden-Europese standaardtijd)
Update for sensor.amersfoort_harderwijk fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 279, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 461, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 240, in wrapper
    result = method(*args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/nederlandse_spoorwegen/sensor.py", line 208, in update
    2,
  File "/usr/local/lib/python3.7/site-packages/ns_api.py", line 932, in get_trips
    return self.parse_trips(raw_trips, requested_time)
  File "/usr/local/lib/python3.7/site-packages/ns_api.py", line 834, in parse_trips
    obj = json.loads(data)
  File "/usr/local/lib/python3.7/json/__init__.py", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

`Punctuality` key missing (sometimes)

When running the nederlandse_spoorwegen integration on the home-assistant platform, I got the following error message:

Traceback (most recent call last):
  File "/home/yarmo/gh/home-assistant/homeassistant/helpers/entity_platform.py", line 299, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/home/yarmo/gh/home-assistant/homeassistant/helpers/entity.py", line 461, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/yarmo/gh/home-assistant/homeassistant/util/__init__.py", line 240, in wrapper
    result = method(*args, **kwargs)
  File "/home/yarmo/gh/home-assistant/homeassistant/components/nederlandse_spoorwegen/sensor.py", line 208, in update
    2,
  File "/home/yarmo/gh/home-assistant/venv/lib/python3.7/site-packages/ns_api.py", line 932, in get_trips
    return self.parse_trips(raw_trips, requested_time)
  File "/home/yarmo/gh/home-assistant/venv/lib/python3.7/site-packages/ns_api.py", line 843, in parse_trips
    newtrip = Trip(trip, requested_time)
  File "/home/yarmo/gh/home-assistant/venv/lib/python3.7/site-packages/ns_api.py", line 606, in __init__
    trip_part = TripSubpart(part)
  File "/home/yarmo/gh/home-assistant/venv/lib/python3.7/site-packages/ns_api.py", line 448, in __init__
    if part_dict['punctuality'] != 100.0:
KeyError: 'punctuality'

So it seems it is missing the punctuality key. Interestingly, this happens for only 1 out of 4 routes that I tried. It doesn't appear like the route is invalid, because by the time you reach line 448, NSAPI is already diving deep in the API's response. Also, line 446 has a similar check and it apparently passes.

Could the NS API be so inconsistent it doesn't show some keys on some routes?

Add minimal delay threshold

Add config option per item: minimum delay threshold. This is useful for cases where for example the delay is only a few minutes.

Get_trips result in keyerror

Hi, See also home-assistant/core#78587

I can reproduce the error I get in home-Assistant in a separate Python script:

import ns_api
from datetime import datetime

now = datetime.now()
current_time = now.strftime("%d-%m-%Y %H:%M")


print("Current Time =", current_time)
ns = ns_api.NSAPI('7612f1390fc748dd9a521194d3d84e99')
_trips = ns.get_trips(current_time, 'Gvc', None, 'Ypb',  True, 0, 0)

result = list(_trips)
print (result)

Output:

Current Time = 01-10-2022 15:20
Traceback (most recent call last):
  File "/config/python_scripts/./ns_test.py", line 11, in <module>
    _trips = ns.get_trips(current_time, 'Gvc', None, 'Ypb',  True, 0, 0)
  File "/usr/local/lib/python3.10/site-packages/ns_api.py", line 928, in get_trips
    return self.parse_trips(raw_trips, requested_time)
  File "/usr/local/lib/python3.10/site-packages/ns_api.py", line 839, in parse_trips
    newtrip = Trip(trip, requested_time)
  File "/usr/local/lib/python3.10/site-packages/ns_api.py", line 608, in __init__
    trip_part = TripSubpart(part)
  File "/usr/local/lib/python3.10/site-packages/ns_api.py", line 442, in __init__
    self.transporter = part_dict['product']['operatorName']
KeyError: 'operatorName'

Implement support for the official NS API

The official NS API is an xml-based service, which has a bit more structure than the website html. Implement support for this, based on login.