christiankuehnel / btlewrap

Bluetooth LowEnergy wrapper for different python backends.

btlewrap

Bluetooth LowEnergy wrapper for different python backends. This gives you a nice API so that you can use different Bluetooth implementations on different platforms.

This library was initially implemented as part of the miflora library, but then refactored out, so that it can be used on other projects as well.

contribution

Backends

As there is unfortunately no universally working Bluetooth Low Energy library for Python, the project currently offers support for three Bluetooth implementations:

• bluepy library (recommended library)
• bluez tools (via a wrapper around gatttool)
• pygatt for Bluegiga BLED112-based devices

bluepy

To use the bluepy library you have to install it on your machine, in most cases this can be done via:

pip3 install bluepy

This is the recommended backend to be used. In comparision to the gatttool wrapper, it is much faster in getting the data and also more stable.

bluez/gatttool wrapper

To use the bluez wrapper, you need to install the bluez tools on your machine. No additional python libraries are required. Some distrubutions moved the gatttool binary to a separate package. Make sure you have this binaray available on your machine.

pygatt

If you have a Blue Giga based device that is supported by pygatt, you have to install the bluepy library on your machine. In most cases this can be done via:

pip3 install pygatt

Usage

See the depending projects below on how to use the library.

Depending projects

These projects are using btlewrap:

• miflora
• mitemp

btlewrap's Issues

Get BrokenPipeError: [Errno 32] Broken pipe

What is a proper solution to deal with such error?
I'm using btlewrap in my fork of miflora-mqtt-daemon and after some time (totally unpredictable) see crash of daemon with:

"Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]: Exception ignored in: <bound method _BackendConnection.__del__ of <btlewrap.base._BackendConnection object
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]: Traceback (most recent call last):
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:   File "/usr/local/lib/python3.5/dist-packages/btlewrap/base.py", line 54, in __del__
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:     self._cleanup()
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:   File "/usr/local/lib/python3.5/dist-packages/btlewrap/base.py", line 58, in _cleanup
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:     self._backend.disconnect()
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:   File "/usr/local/lib/python3.5/dist-packages/btlewrap/bluepy.py", line 26, in _func_wrapper
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:     return func(*args, **kwargs)
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:   File "/usr/local/lib/python3.5/dist-packages/btlewrap/bluepy.py", line 63, in disconnect
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:     self._peripheral.disconnect()
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:   File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 453, in disconnect
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:     self._writeCmd("disc\n")
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:   File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 305, in _writeCmd
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]:     self._helper.stdin.flush()
Feb 16 06:17:45 wb miflora-mqtt-daemon.py[31469]: BrokenPipeError: [Errno 32] Broken pipe"

Also in dmseg there is such error:
[40293.568271] Bluetooth: hci0 ACL packet for unknown connection handle 16

PS: Maybe this is something wrong with bluepy or somewhere we need add ignore

Find a reproducible way to run integration tests with real BLE devices

The goal is to check the library with real hardware and the entire stack.

The challenge is basically to set up a testing service that we can talk to. The BLE libraries we're using right now do not allow running creating a server.

Maybe the BLE Peripheral Simulator can serve this purpose.

This is also a pre-requisite to solve #28

Update all dependencies to latest version

Remove all Python 2 dependencies

Python 2 has been deprecated for a while, so we can safely remove all references to that. In any case users can still use an earlier version of the library

Support for Python 3.9?

Hi,

I'm a user of this library leveraged in this other project https://github.com/mbrentini/homeassistant_parrotflowerpower

Recently it has stopped working (mbrentini/homeassistant_parrotflowerpower#6), and the root cause seems to be related to the bump to Python 3.9.
home-assistant/core#63883

• Could you check and confirm/fix if your library btlewrap is compatible with Python 3.9 and above?

Thanks!

support bleak backend

There is a new backend out, supporting Linux, Windows and MacOS:
https://pypi.org/project/bleak/

It would be nice if we could support that as well.

setup code formatter for project

1. setup automatic formatting
2. setup of checks in tox and CI
3. do format all of the code

BluePy no longer compatible with Python 3.10

I use Home Assistant, which I believe uses this wrapper for Mi Flora integration.
They have moved to Python 3.10 and unfortunately BluePy is no longer compatible.
I don't have the knowledge to fix this myself, so I am just raising this as an issue to be investigated.

setup pytype for the project

1. add it to tox and ci
2. clean up all findings

gatttool timeout handler does not cleanup subprocesses

The timeout handler in connection.wait_for_notification() (and other gatttool functions) does something like:

with Popen(cmd, shell=True, ...) as process:
    try:
        ...
    except TimeoutExpired:
        os.killpg(process.pid, signal.SIGINT)

Because its using shell=True it sends the SIGINT to the /bin/sh process instead of gatttool, which ignores it. The SIGINT needs to be sent directly to the gatttool process. Since the processes are not cleaned up, they build up over time. Often one of them might still have a lock on something, so all subsequent instances of gatttool fail.

This can be easily fixed by removing the shell=True and passing the arguments as a list. e.g.

cmd = ['gatttool', '--device', self._mac, ...]
with Popen(cmd, shell=True, ...) as process:
    ...

Then process.pid will be that of the gatttool process.

NoneType object has not attribute 'disconnect'.

I am using bt-mqtt-gateway that depends on btlewrap and found an issue that seems to be coming from btlewrap.

Trace:

Starting
Added: miflora with 30 seconds interval
Subscribing to: homeassistant/status with command: update_all
Updating all workers
[{'topic': 'miflora/orchid/temperature', 'payload': 23.4}, {'topic': 'miflora/orchid/moisture', 'payload': 80}, {'topic': 'miflora/orchid/light', 'payload': 2449}, {'topic': 'miflora/orchid/conductivity', 'payload': 3488}, {'topic': 'miflora/orchid/battery', 'payload': 96}]
Exception ignored in: <bound method BluetoothInterface.__del__ of <btlewrap.base.BluetoothInterface object at 0x75973490>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/btlewrap/base.py", line 17, in __del__
    self._backend.disconnect()
  File "/usr/local/lib/python3.6/site-packages/btlewrap/bluepy.py", line 26, in _func_wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/btlewrap/bluepy.py", line 60, in disconnect
    self._peripheral.disconnect()
AttributeError: 'NoneType' object has no attribute 'disconnect'
[]

Thread about bt-mqtt-gateway: zewelor/bt-mqtt-gateway#14.

Fix Bluepy BrokenPipeError

Hello everyone 👋

Seems that there is an error in the bluepy Python library used by btlewrap that produces a BrokenPipe error when the execution is being stopped by another thread/execution.

This is the stacktrace (used in home-assistant):

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 261, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 441, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/beewi_smartclim/sensor.py", line 101, in update
    self._poller.update_sensor()
  File "/usr/local/lib/python3.7/site-packages/beewi_smartclim/__init__.py", line 75, in update_sensor
    with bt_interface.connect(self._mac) as connection:
  File "/usr/local/lib/python3.7/site-packages/btlewrap/base.py", line 44, in __enter__
    self._backend.connect(self._mac)
  File "/usr/local/lib/python3.7/site-packages/btlewrap/bluepy.py", line 26, in _func_wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/btlewrap/bluepy.py", line 55, in connect
    self._peripheral = Peripheral(mac, iface=iface)
  File "/usr/local/lib/python3.7/site-packages/bluepy/btle.py", line 391, in __init__
    self._connect(deviceAddr, addrType, iface)
  File "/usr/local/lib/python3.7/site-packages/bluepy/btle.py", line 431, in _connect
    self._writeCmd("conn %s %s %s\n" % (addr, addrType, "hci"+str(iface)))
  File "/usr/local/lib/python3.7/site-packages/bluepy/btle.py", line 305, in _writeCmd
    self._helper.stdin.flush()
BrokenPipeError: [Errno 32] Broken pipe

I suggest to update the bluepy library version from 1.1.4 to 1.3.0.

btlewrap/requirements.txt

Line 1 in 1b7aec9

bluepy==1.1.4

As they commented into an issue, this should be fixed after version 1.2.0.
IanHarvey/bluepy#282 (comment)

Thanks!

RuntimeError: release unlocked lock

Hi @ChristianKuehnel,

This is a follow up on #8. The program flow somehow tries to release an unreleased lock. I'm not sure what the originating cause of this could be, as the lock must have been previously acquired successfully. It may be some kind of race condition or concurrency issue. We could potentially just test if the lock is actually acquired. What do you think?

Stack-trace of the exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/btlewrap/bluepy.py", line 26, in _func_wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/btlewrap/bluepy.py", line 55, in connect
    self._peripheral = Peripheral(mac, iface=iface)
  File "/usr/local/lib/python3.7/site-packages/bluepy/btle.py", line 391, in __init__
    self._connect(deviceAddr, addrType, iface)
  File "/usr/local/lib/python3.7/site-packages/bluepy/btle.py", line 439, in _connect
    raise BTLEDisconnectError("Failed to connect to peripheral %s, addr type: %s" % (addr, addrType), rsp)
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral C4:7C:8D:66:E3:CC, addr type: public

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/btlewrap/base.py", line 44, in __enter__
    self._backend.connect(self._mac)
  File "/usr/local/lib/python3.7/site-packages/btlewrap/bluepy.py", line 32, in _func_wrapper
    raise BluetoothBackendException() from last_error
btlewrap.base.BluetoothBackendException

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "gateway.py", line 57, in <module>
    raise e
  File "gateway.py", line 47, in <module>
    mqtt.publish(_WORKERS_QUEUE.get(timeout=10).execute())
  File "/bt-mqtt-gateway/workers_manager.py", line 32, in execute
    messages = self._callback(*self._args)
  File "/bt-mqtt-gateway/workers/miflora.py", line 30, in status_update
    ret += self.update_device_state(name, poller)
  File "/bt-mqtt-gateway/workers/miflora.py", line 39, in update_device_state
    ret.append(MqttMessage(topic=self.format_topic(name, attr), payload=poller.parameter_value(attr)))
  File "/usr/local/lib/python3.7/site-packages/miflora/miflora_poller.py", line 132, in parameter_value
    self.fill_cache()
  File "/usr/local/lib/python3.7/site-packages/miflora/miflora_poller.py", line 68, in fill_cache
    with self._bt_interface.connect(self._mac) as connection:
  File "/usr/local/lib/python3.7/site-packages/btlewrap/base.py", line 47, in __enter__
    self._lock.release()
RuntimeError: release unlocked lock

Fix CI

Travis looks broken, I guess it's time to move to Github Actions.

Adaption of wait_for_notification() for Blusensor AIQ

Recently I bought the mentioned sensor and wanted to integrate it using BLE (also it support MQTT via WLAN). I found btlewrap a nice lib to use for, because MI temp as a similar sensor used it too. But in the case of Blusensor AIQ wait_for_notificaton() was not working. The write command ended up in an exception. Therefore I added an additional variant named wait_for_notification_no_write() and simply deleted this function call. Then everything works fin. The sensor spams notification after connection. Perhaps on option for wait_for_notification() is another solution?!

In my fork I added the function: https://github.com/Gezzo42/btlewrap/blob/a1c6953e5ed1dd64f027c2d1a757eb72a21fe15b/btlewrap/bluepy.py

It would be nice, if you can add it to btlewrap :-).

Release pinning

The release pinning for bluepy and pygatt is an issue for distribution packages. E.g., Fedora ships pygatt-4.0.5.

Introduce type hints for all public interfaces

To make life for the users of the library easier: add type hints for all public interfaces.
https://docs.python.org/3/library/typing.html

Implement retries for all backends

Right now only the gatttool backend supports it. But is should be added to all backends.

see basnijholt/miflora#106