Specifically with my modem page I've had this issue. Gestures don't work but any up/down/left/right right-click gesture and then moving my mouse from the Firefox window or to the tabs window will automatically make my bookmark disappear. I don't know if this is a bug in FF or your extension. I'm assuming FF since it seems to have a permission beyond what the extension or the API you're using should be capable of or requested with their new webextension model.
This doesn't happen with the extension disabled. It doesn't happen on my router's page (192.168.1.1), only my modem's (192.168.100.1) which I guess rules out Mozilla not allowing the API to access local network pages. I can click anywhere on the modem page, regardless of what he element is below it and it happens. But even hovering over the context menu after gesturing will trigger the deletion of the bookmark.
I recorded it, not sure why but here you go.
Edit: Forgot, I'm on Nightly v57.0a1.
Also, another unrelated thing, when using the Stylus extension and this userstyle, when right-clicking and moving on wikipedia, the whole page is grayed or blackened if I change it to black by editing the stylesheet. Not a big deal and can't think of any use cases off-hand where it'd be a problem. But if you're wanting to keep your extension more user friendly, probably best to make sure it doesn't mess with the user's page display.