seemoo-lab / pyshimmer

pyshimmer provides a Python API to work with the wearable sensor devices produced by Shimmer.

pyshimmer: Unofficial Python API for Shimmer Sensor devices

Contents

• General Information
• Contributing
• Installation
  • pyshimmer Package
  • Shimmer Firmware
    • Compatibility Table
    • Notes on Firmware Version 0.15.4
  • Creating udev rules for persistent device filenames
  • Configuring the Bluetooth Interface
• Using the API
  • Using the Bluetooth interface
  • Using the Dock API
  • Using the Reader API

General Information

pyshimmer provides a Python API to work with the wearable sensor devices produced by Shimmer. The API is divided into three major components:

• The Bluetooth API: An interface to communicate with the Shimmer LogAndStream firmware via Bluetooth
• The Dock API: An interface to communicate with the Shimmer while they are placed in a dock
• The Reader API: An interface to read the binary files produced by the Shimmer devices

Please note that the following README does not provide a general introduction to the Shimmer devices. For this, please consult the corresponding documentation page of the vendor and take a closer look at:

• The Shimmer User Manual
• The LogAndStream Manual
• The SDLog Manual

Contributing

All code in this repository was produced as part of my Master thesis. This means that the API is not complete. Especially the Bluetooth and Dock API do not feature all calls supported by the devices. However, the code provides a solid foundation to extend it where necessary. Please feel free to make contributions in case the code is missing required calls.

Please submit pull requests against the develop branch. When a new version is released, the current state of the develop branch is merged into master to produce the new version.

Installation

The targeted plattform for this library is Linux. It has not been tested on other operating systems. In order to use all aspects of the library, you need to install the package itself, set up the Bluetooth interface, and possibly configure udev rules to ensure that the device names are consistent.

pyshimmer Package

In order to install the package itself, clone it and use pip to install it:

git clone https://github.com/seemoo-lab/pyshimmer.git
cd pyshimmer
pip install .

If you want to run the tests, instead install the package with test extras:

pip install .[test]

You can then run the tests from the repository root by simply issuing:

pytest

Shimmer Firmware

As of version v0.15.4 of the Shimmer3 firmware the Python API is fully compatible to the firmware. Older versions of the vanilla firmware exhibit several bugs and are incompatible. If you intend to use a firmware version older than 0.15.4, you will need to compile and run a custom patched version of the firmware. In the following table, the firmware versions and their compatibility are listed.

Compatibility Table

Firmware Type	Version	Compatibility	Issues
LogAndStream	v0.15.4	Compatible to v0.4.0	You will need to use pyshimmer v0.4 or newer
LogAndStream	v0.11.0	Incompatible	Issue 7 Issue 10
SDLog	v0.21.0	Compatible	Untested
SDLog	v0.19.0	Compatible

It is recommended to use the newest LogAndStream firmware that is compatible to the API. If you want to use an older version with the pyshimmer library, you will need to compile and program a patched version of the firmware. We provide a forked repository which features the necessary fixes here. It also contains instructions on how to compile and program the firmware.

Notes on Firmware Version 0.15.4

Starting with Firmware version v0.15.4, the race condition issue in the Bluetooth stack has been fixed. Additionally, the Shimmer3 now supports an additional command to disable the unsolicited status acknowledgment byte (see Issue 10). The pyshimmer Bluetooth API tries to automatically detect if the Shimmer3 runs a firmware newer or equal to v0.15.4 and automatically issues the command to disable the unsolicited status acknowledgment at startup. You can optionally disable this feature in the constructor. With this new command, the state machine in the Bluetooth API of pyshimmer is compatible to the vanilla firmware version.

Creating udev rules for persistent device filenames

When plugging a Shimmer dock into the host, Linux will detect two new serial interfaces and a block device representing the internal SD card of the Shimmer:

• /dev/ttyUSB0 for the serial interface to the bootloader,
• /dev/tty/USB1 for the serial interface to the device itself,
• /dev/sdX for the block device.

When working with multiple docks and devices, keeping track of the names of the serial interfaces can be quite cumbersome, since udev simply names the devices in the order they are plugged in to the system. You can use udev rules to assign persistent names to the device files. Note that the rules do not actually match the Shimmer but the dock that it is located in. This means that you should always place the Shimmer in the same dock.

The following section provides an example of how to handle two Shimmer docks, one of which holds an ECG and the other a PPG device:

Distinguishing the Shimmer booloader and device interfaces based on their udev attributes is somewhat difficult because the distinguishing attributes are spread across multiple devices in the USB device tree. The rules first check the bInterfaceNumber of the tty device that is being processed. If the device is the bootloader device, its bInterfaceNumber is equal to 00. If the device is the interface to the Shimmer itself, bInterfaceNumber is equal to 01. In a second step, the rule set differentiates between the ECG dock and the PPG dock based on the serial number of the device. The entire udev ruleset is shown in the following code snippet:

SUBSYSTEMS=="usb" ATTRS{bInterfaceNumber}!="00" GOTO="is_secondary_interface"
SUBSYSTEM=="tty" ATTRS{idVendor}=="<id_vendor1>" ATTRS{idProduct}=="<id_product1>" ATTRS{serial}=="<id_serial1>" SYMLINK+="ttyPPGbl"
SUBSYSTEM=="tty" ATTRS{idVendor}=="<id_vendor2>" ATTRS{idProduct}=="<id_product2>" ATTRS{serial}=="<id_serial2>" SYMLINK+="ttyECGbl"
GOTO="end"

LABEL="is_secondary_interface"
SUBSYSTEM=="tty" ATTRS{idVendor}=="<id_vendor1>" ATTRS{idProduct}=="<id_product1>" ATTRS{serial}=="<id_serial1>" SYMLINK+="ttyPPGdev"
SUBSYSTEM=="tty" ATTRS{idVendor}=="<id_vendor2>" ATTRS{idProduct}=="<id_product2>" ATTRS{serial}=="<id_serial2>" SYMLINK+="ttyECGdev"
GOTO="end"

LABEL="end"

You can also find the example file in conf/udev/10-shimmer.rules.example.

In order to create a custom ruleset for your devices, create a new udev rule file /etc/udev/rules.d/10-shimmer.rules and add the above contents. In the file, you need to replace the <id_vendor1>, <id_product1>, and <id_serial1> of the first device, and the <id_vendor2>, <id_product2>, and <id_serial2> of the second device. You can find the values by scanning the dmesg command after plugging in a Shimmer device. Here is an example:

[144366.290357] usb 1-4.3: new full-speed USB device number 34 using xhci_hcd
[144366.386661] usb 1-4.3: New USB device found, idVendor=<id_vendor>, idProduct=<id_product>, bcdDevice= 5.00
[144366.386668] usb 1-4.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[144366.386674] usb 1-4.3: Product: SHIMMER DOCK
[144366.386679] usb 1-4.3: Manufacturer: FTDI
[144366.386684] usb 1-4.3: SerialNumber: <id_serial>

Save the file and reload the rules for them to take effect:

udevadm control --reload-rules && udevadm trigger

You should now have two named device files for each Shimmer dock:

• /dev/ttyPPGbl and /dev/ttyPPGdev for the PPG Shimmer bootloader and device interfaces,
• /dev/ttyECGbl and /dev/ttyECGdev for the ECG Shimmer bootloader and device interfaces.

Configuring the Bluetooth Interface

The library uses a tty serial interface to communicate with the Shimmer over Bluetooth. Before you can use the library, you need to set up the serial channel appropriately. This has only been tested under Arch Linux, but other Linux distributions should work as well.

Requirements:

• Functioning Bluetooth stack
• The rfcomm commandline tool. For Arch Linux, use the bluez-rfcomm AUR package
• The hcitool commandline tool. For Arch Linux, use the bluez-hcitool AUR package
• A Shimmer device with LogAndStream firmware

Scan for the device to find out its MAC address:

hcitool scan

The MAC address of the listed Shimmer device should end with the BT Radio ID imprinted on the back of the device. Next, you can try and ping the device:

hcitool name <mac_addr>

The command should complete with the name listed previously during the scan. Now you can pair the device as follows:

rfcomm <bind_id> <mac_address>

where <bind_id> is an arbitrary integer of your choosing. The command will create a new serial interface node with the following name: /dev/rfcomm<bind_id>. The file acts as a regular serial device and allows you to communicate with the Shimmer. The file is also used by the library.

Using the API

Using the Bluetooth interface

If you want to connect to the Bluetooth interface, use the ShimmerBluetooth class. The API only offers blocking calls.

import time

from serial import Serial

from pyshimmer import ShimmerBluetooth, DEFAULT_BAUDRATE, DataPacket, EChannelType


def handler(pkt: DataPacket) -> None:
    cur_value = pkt[EChannelType.INTERNAL_ADC_13]
    print(f'Received new data point: {cur_value}')


if __name__ == '__main__':
    serial = Serial('/dev/rfcomm42', DEFAULT_BAUDRATE)
    shim_dev = ShimmerBluetooth(serial)

    shim_dev.initialize()

    dev_name = shim_dev.get_device_name()
    print(f'My name is: {dev_name}')

    shim_dev.add_stream_callback(handler)

    shim_dev.start_streaming()
    time.sleep(5.0)
    shim_dev.stop_streaming()

    shim_dev.shutdown()

The example shows how to make simple calls and how to use the Bluetooth streaming capabilities of the device.

Using the Dock API

from serial import Serial

from pyshimmer import ShimmerDock, DEFAULT_BAUDRATE, fmt_hex

if __name__ == '__main__':
    serial = Serial('/dev/ttyPPGdev', DEFAULT_BAUDRATE)
    shim_dock = ShimmerDock(serial)

    mac = shim_dock.get_mac_address()
    print(f'Device MAC: {fmt_hex(mac)}')

    shim_dock.close()

Using the Dock API works very similar to the Bluetooth API. However, it does not require a separate initialization call because it does not use a background thread to decode incoming messages.

Using the Reader API

from pyshimmer import ShimmerReader, EChannelType

if __name__ == '__main__':

    with open('test/reader/resources/ecg.bin', 'rb') as f:
        reader = ShimmerReader(f)

        # Read the file contents into memory
        reader.load_file_data()

        print(f'Available data channels: {reader.channels}')
        print(f'Sampling rate: {reader.sample_rate} Hz')
        print()

        ts = reader[EChannelType.TIMESTAMP]
        ecg_ch1 = reader[EChannelType.EXG_ADS1292R_1_CH1_24BIT]
        assert len(ts) == len(ecg_ch1)

        print(f'Timestamp: {ts.shape}')
        print(f'ECG Channel: {ecg_ch1.shape}')
        print()

        exg_reg = reader.exg_reg1
        print(f'ECG Chip Sampling Rate: {exg_reg.data_rate} Hz')
        print(f'ECG Chip Gain: {exg_reg.ch1_gain}')

If the data was recorded using the SDLog firmware and features synchronization information, the API automatically interpolates the data to the common timestamp information of the master.

Note: Please be aware that although you have configured a sampling frequency f for your measurements, it can happen that observations are missing. Usually the observed time difference is a multiple of the sampling period 1 / f. However, this is not the case for the time difference between the first two observations. Please take this caveat into consideration when you design your code.

pyshimmer's Issues

Implement support for a new status ack removal command

See ShimmerResearch/shimmer3#10

Setting which channels are streamed

Dear Lukas,
first thank you for this library, I have been looking for a long time for a simpler way to access the shimmer via python.
One thing is missing is the possibility to set configure the Shimmer before starting to stream.

One should be able to select which channels are active (e.g., AC13, RAW GSR, SRC, etc) and the streaming frequency.
Maybe the features is already there, if that's the case, could you please provide me a snippet on how to use it?

Thank you!
Dario

Realign functionality is incorrect

Hi Lukas,

I think the realign functionality when loading data is incorrect. From my understanding it does the following right now:

1. take the first timestamp as an initial offset for the new timestamps
2. extrapolate from the first timestamp assuming a constant sampling rate while preserving the total number of observations
3. interpolate the sensor data to match the newly generated timestamps

If the original timestamps are assumed to be correct, keeping the total number of observations in the second step will lead to an too early final timestamp. Instead, the correct approach would be to impute missing timestamps and interpolate new sensor values. Otherwise, if the timestamps are assumed to be incorrect, interpolating the sensor values in the third steps is wrong.

Working with the raw data from the Shimmer devices, I have made the following observations:

• the difference between consecutive observations is almost always an integer multiple of the sample time (inverse of the sampling frequency)
• the only exception I have seen so far is the time difference between the first and second sample: often this one is not an integer multiple of the sampling time

Using this observations, my current approach is to disable the realign functionality, drop the first sample and impute missing values by linear interpolation as outlined below.

import pandas as pd
import numpy as np
from pyshimmer import ShimmerReader, EChannelType

with open("ECG.sb3", "rb") as ecg_file:
    ecg_reader = ShimmerReader(ecg_file, realign=False)
    ecg_reader.load_file_data()
    ecg_fs = ecg_reader.sample_rate

    # convert to pandas dataframe and drop the first sample
    ecg_df = pd.DataFrame({
        "ecg1": ecg_reader[EChannelType.EXG_ADS1292R_1_CH1_24BIT][1:],
        "ecg2": ecg_reader[EChannelType.EXG_ADS1292R_1_CH2_24BIT][1:],
    }, index=ecg_reader[EChannelType.TIMESTAMP][1:])

    # add empty rows for the missing timestamps and interpolate the sensor values
    ecg_ts = pd.Index(np.arange(ecg_df.index[0], ecg_df.index[-1], 1/ecg_fs), name="timestamps")
    ecg_df = ecg_df.reindex(ecg_ts).reset_index().interpolate()

Concluding, I think it would be better to remove the realign functionality and let the users decide on how to deal with the presumably missing observations. Though, it may be helpful to raise a warning on detecting such gaps in the recording and, in particular, on detecting those which are are not an integer multiple of the sample time.

Cheers,
Matthias

Calibration Data

Hi, I'm trying to deploying and end to end system, so I don't want that the user interact with Consensys or other Shimmer software.
The data I'm downloading it through the dock station.
Following your examples I could access to the raw data, but I can't access to the calibration one. Can you help me?

After successful flashing, the device returns exception.

After successful firmware flashing, and rfcomm bind,
The test code provides below error.
'device reports readiness to read but returned no data'.
Should I do something between binding and run the testcode provided in repository?

Issue using the Bluetooth API at high frequency

Issue using the Bluetooth API at high frequency

Hello,

I have encountered an issue for the data retrieval. Currently, I am receiving a data packet of one sample each time via a callback function. The issue appears when working with frequency higher than ~30 Hz; the Shimmer buffer fills faster than the API retrieves data, leading to an offset in real-time data. More importantly, when the shimmer sensor reaches its maximum buffer capacity, it flushes, resulting in data loss.

I'm currently working on Ubuntu 22.04.4 but I have the same bug running the code on Windows. The firmware of the shimmer sensor that I'm using is 0.16.0 and my python version is 3.10.12.

I would like to use a frequency of 512 Hz but cannot go over ~30 Hz for my program to work.

Here is a zip containing a file with a main that produce the bug.
showcase_issue.zip
For the usage, you need to connect a shimmer3 sensor and enter the signal type used between 'ECG' and 'EDA' and the port where the shimmer is paired.

I would greatly appreciate any advice you could provide to help me resolve this issue.

Thank you.

Consensys reading logged data failed

Hello lumagi,

I use your Python API for my Shimmer3 EXG Unit and it works well. Thank you for that :)
Now I want to get the data from the Shimmer SD with Consensys. Sometimes it works, but more often an error occur.
Also I use your Reader API, but with the reader API the data have a different shape, like the streaming data. And i would prefer to have the data in the shape of the output from Consensys.
Could it be that the data got damaged or not closed with the Python API? Even if i use the stop() and shutdown() command?

Best StudentofGermany

Storing and Event Marker to the recording

Hi, thanks for the great work with this API.

We are trying to integrate shimmer with a Psychopy https://www.psychopy.org/ experiment where we present some images and want to record the physiological signals using the shimmer kit.

Is there a way to send a "trigger"/ event marker using the API, such that I can synchronise the image presentation with the recording afterwards?

I hope this makes sense

integration with pylsl

Hi thank you for this code. Could you explain how this could be intergrated with PyLSL to set up data streaming?
This would be a great merge with the LSL project to combine GSR with other sensors.

Using the Bluetooth interface

Hey,
I followed the instructions of the readme. I also patched LogAndStream Firmware on my Shimmer3 EXG.
Everything work till the example Program of the BT API. I am working on an Ubuntu 18.04.4 LTS.
I got the Error:
My name is: Shimmer_86A5
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_api.py", line 294, in _run_readloop
self._bluetooth.process_single_input_event()
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_api.py", line 206, in process_single_input_event
self._process_data_packet()
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_api.py", line 148, in _process_data_packet
cb(packet)
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/test_hallo.py", line 26, in handler
cur_value = pkt[EChannelType.INTERNAL_ADC_13]
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_commands.py", line 60, in getitem
return self._values[item]
KeyError: <EChannelType.INTERNAL_ADC_13: 19>

After that i have to keyinterrupt the session:

Traceback (most recent call last):
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/test_hallo.py", line 43, in
shim_dev.stop_streaming()
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_api.py", line 480, in stop_streaming
self._process_and_wait(StopStreamingCommand())
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_api.py", line 301, in _process_and_wait
compl_obj.wait()
File "/home/z-fcc/PycharmProjects/Test/pyshimmer/pyshimmer/bluetooth/bt_api.py", line 50, in wait
self.__event.wait()
File "/usr/lib/python3.8/threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.8/threading.py", line 302, in wait
waiter.acquire()
KeyboardInterrupt

I can't find the mistake by my own. Can you help me pls?

Dear StudentofGermany

Compatibility with new firmware LogAndStream v0.16.000

Hello,

Thank you for your impressive work on Shimmer devices.
I would like to know if you were able to test the compatibility of pyshimmer with the new version of the LogAndStream v0.16.000 firmware.
I await your feedback before updating my IMUs.

Thank you in advance,
Sincerely