Inconsistent long press right click about python-touchscreen-rightclick

[gevasiliou]

Hello Zyell! How are you! I can see you are back from holidays! Hope you had a good time.
With the risk to be sound as a "stuck person", you could check if the behavior is better using pymouse.
I'am also using nautilus and with pymouse everything works fine. In my system the implementation of uinput is very inconsistent.

[Zyell]

Hello! Thanks! My vacation was incredible! I truly enjoyed it. :-)

No worries, I have tried pymouse. I still have the same behavior I described above with one more inconsistent behavior. With pymouse if I go to long press a window that isn't in the main focus, the right click occurs at the edge of that window no matter where I press... But in order to solve the inconsistency in not bringing up the context menu, I tried initiating a left click immediately before bringing it up. This works perfectly. I thought it might have to do with movement of my finger, but no movement was detected, so very strange. Regardless, now it brings up the right click menu consistently using the uinput and pymouse. But, I cannot click on any items in this context menu...

The double tap right click is not affected by this problem. So I looked into what was happening during the long press that is different. When a touch event is started a BTN_TOUCH KEY_EVT is fired with a value of 1. When you remove your finger, the same event is fired with a value of 0. During the double tap this value of 0 is fired in tandem with the right click. During the long press this happens noticeably after the right click so that the menu comes up while your finger is still on the screen. With the grab and ungrab required to keep this menu from closing, this last event isn't fired. It seems then that the system thinks you are still touching the screen, making the right click context menu unclickable. I've tried firing this event manually and I can't seem to make it work yet... I need to test on my other laptop a bit as I've only worked on my dev laptop recently. But, do you see this issue during a long press?

[Zyell]

Well it doesn't look like the BTN_TOUCH event is the culprit after all... Back to the drawing board! lol.

[gevasiliou]

Good morning!
I did some tests yesterday evening, and i can partially confirm some of the bugs.
I had also plans to send you a kind of video , but my kids had a different opinion for overnight activities... :-) I'll do that later.

Well , here i go with my machine (all tests with pymouse, not uinput).

About the bug that right click menus appear on the edge of the window: i was not capable to reproduce this bug. No matter how many windows i have open, right click always steal focus, brings the "touched" window in front and right click is injected.
Actually is not right click who steal focus. Is my first moment of finger touch.

About Nautilus bugs, i can confirm the bugs you describe in my machine.
Items become unclickable after while.
It seems that right click is operating for the first time, but then it looks like nautilus is hanging.
I was impressed to see that upon hanging, nautilus is not obeying commands either by trackpad.

I noticed that sometimes you can right click on an item but not on user spare (free space around folder and files within Nautilus)
What it is interesting is that i could see in the nautilus window a permanent "rectangle" , similar to the one we use to select multiple files by dragging.

By firing a left click before right click, on the very first time nautilus right click works in files/folders but after first click on userspace then i was able to bring context menu in user space but all items (folder/files) became non clickable (either by left or right click).

Further Thoughts:
I'm using XFCE , but i use nautilus since it is nativelly support scrolling with touch screens.
All my DE, all my progs that rightclick.py works ok are GTK2 progs (i also use python2 to run rightclick.py)
On the other hand Nautilus is a GTK3 prog. I don't know if this is somehow related to our bugs.
I could test the operation of the right click in other GTK3 progs to see how it goes.

[gevasiliou]

Good Evening Zyell!
I was wondering what is the behavior in Nautilus with your very first version (context menu appearing after finger release) ... Have you made any tests?

[gevasiliou]

Moreover, Caja of Mate, has the same behavior as Nautilus. It seems that the bug affects GTK.

[Zyell]

Hello! I have been preoccupied lately, sorry, but I did do some additional tests. I found that the original implementation worked just as well as the two finger tap does. The issue is definitely related to the menu coming up while holding your finger to the screen. Also, I have seen the same behavior, that the files and folders work just fine, but the rest of the area within nautilus and the desktop fails to work. I am working on isolating what events are not being fired during the long press that are being fired during the two finger tap. Hopefully I can find an effective way to inject those into the system and fix this issue. You might be right that it is some quirk or bug of GTK... I hope to get back to testing in the next few days. Thank you for investigating this further as well!!

[gevasiliou]

Hello,
I have some fresh info about this bug. I was capable to identify partially the source of the bug.

First of all i have to point out that script behavior is different within Nautilus if you inject a left click before right click.

Classic setup (no left click injected before right click)
In this case the behavior is like this:
You can select and open (with dbl click) all nautilus files/folder.
Long press click raises the item's context menu correctly.
In this setup the bug is that Nautilus user space context menu never pops up.

In this case the "bug" is related to Nautilus itself.
Nautilus does not allow users to call context menu while you are keeping pressed the left key (dragging).
You can see the same behavior with any usb mouse (or even trackpad).
If you keep the trackpad left click pressed inside Nautilus user space and you press a right click without releasing the left click , the context menu is not shown.
But if you make the same test on XFCE Desktop space or Thunar, right click pops up correctly. The left click does not forbid the right click actions.

The same more or less behavior happens on touchscreen touch evens.
The touchscreen long press is interpreted by Nautilus as a kind of left click / kind of dragging.
Thus right click injection is ignored by Nautilus since you are pressing left click with your finger.

The super sensitive touchscreen emits ABS-X and Y change, which is actually considered to be like a "left click drag".

A lot of users have been complaining about this Nautilus behavior, but nautilus teams has classified this bug as "won't fix". See this bug report:
https://bugs.launchpad.net/hundredpapercuts/+bug/387835

It is possible that this behavior applies to all GTK3 apps. I tested Caja and Nemo WM, and both of them behave like Nautilus. You can NOT have right click context menu while you are pressing left click and while you are dragging.

On the other hand, In my XFCE Desktop (tested even using my touchpad), i am able to perform a long left click (like dragging) and as soon as right click is pressed (either in touchpad or by script), desktop context menu pops up , pausing (not cancelling) the left click action.

Alternative setup (left click injected before right click)
Here Nautilus behavior is different than before.
You can always bring the Nautilus user space context menu but items inside Nautilus (files/folder) become completely non clickable (either for left or right click).
It is impressive in this case that Nautilus remains in this non clickable state even after terminating the Right Click script . To restore Nautilus functionality you need to close Nautilus and reopen it.
I can not find what is causing this bug.

Remarks
Since i do not consider my self as good programmer as you, I built a tiny "RightClick-tester" script without classes and complex functions in order to understand what is going on.
i just wanted to force Nautilus to bring context menus during a long press event (without releasing my finger). It proved that this is not operating well.

More precisely , you can see context menu of Nautilus files/folders with long press (no finger release), but on nautilus user space this is not possible since right click and left click together are not allowed.
To bring Nautilus user space context menu you have to "tap" the Nautilus surrounding space or more precisely to release your finger and stop the left click event before right click is injected (before the timer expires).

You can give yourself a try to the right click tester script if you like :
https://github.com/gevasiliou/PythonTests/blob/master/rightclick-tester.py

How To test:
Run the script and then
a). tap in Desktop space: After one second the desktop context menu appears.
b). tap in a desktop item: After a second item's context menu appears
c). tap a Nautilus item: After a second item's context menu appears
d). tap at Nautilus space : After a second nautilus context menu appears correctly.
e). Repeat a,b,c,d with long presses - no finger release: In all cases context menu will pop up like before except case d (nautilus surrounding space)

Test with regular mouse or touchpad:
Try to make a long left click drag and try to fire a right click without releasing left click.
In Thunar & XFCE, context menu appears . In Nautilus , Caja & Nemo not.

Conclusion:
Having GTK3 apps operating like this (ignoring right click when left click is pressed), maybe is better to keep the very first version in which right click context menu appears upon finger removal.

Or we just need to redesign the script, but i can not figure out in which direction...
Could the use of PyGTK3 be a solution..?
Could we fool the system and disable the touch screen for a moment (i.e by calling xinput --disable "touchscreen id" - inject right click - re enable touch screen)...?
Device grab shouldn't be enough to interrupt the transmission of long press event to Nautilus....?
(i used dev.grab but made no difference)

[gevasiliou]

Update:
Based on the classic setup (no left click injection trick before right click), this very ery ugly hack works fine:

	subprocess.check_call(['xinput', '--disable', 'ELAN Touchscreen'])
	m.press(x1, y1, 2)
	m.release(x1, y1, 2)
	subprocess.check_call(['xinput', '--enable', 'ELAN Touchscreen'])

By disabling the screen for some milliseconds i disconnect left click from Nautilus "on the hard way" and thus right click menu appears ok in Nautilus surrounding space. Actually with this hack, right click menu appears everywhere without problem.

For sure this is not the right way to do coding, but it works...
Simplified working script here: https://github.com/gevasiliou/PythonTests/blob/master/rightclick-gv.py

[gevasiliou]

Hello Michael!!
Did you had any time to check this issue....?

[Zyell]

Hello!! Sorry, work has been crazy going into the holidays! I've been investigating a way to inject an end to the left click and drag prior to initiating the right click, but I haven't found a way to do this just yet. Looking at the events fired from evdev, this looks to be the issue. I'm just not sure at this point how to inject that. Your test for disabling the touchscreen is brilliant and confirms the issue is definitely the lack of completion of the touch event (recognized as a left click). You may be onto something with pursuing pyGTK. I'm not familiar with it, but it might be possible to inject an event. I've been trying to go through evdev, but I just haven't gotten very far with it. In the meantime the subprocess call at least works and I can't find any adverse effects. I need to clean up my code and upload here again where we stand on this. I will allow two paths for the long press to work as it once did, and using the workaround you found. I am in the middle of changing jobs right now, plus the upcoming holidays, but I will aim to get some better testing in before the end of the week! If you want to do a pull request with your testing scripts that would be great. We can start a section for testing. :-)

[gevasiliou]

Hello! No worry about delays! We are all busy with our morning jobs and normal family business,..
From my point of view the bug behavior is initially coming from Nautlius and GTK3 apps restriction to emmit a right click event when left click is kept pressed no matter if you use a script or a regular mouse (while GTK2 apps allow this behavior) and following this behaviour we also have the inability of evdev to correctly grab/ungrab the device whent device is used by GTK3 apps. Grabbing the device "on the hard way" it seems to be actually a workaround (a bit ungly though). Maybe PyGTK as you said could provide any better options, or maybe Python3 could do the job. It is something that we can test in the future.

About pull request, is not necessary... You got the idea, you implement it in your project, everything is fine.

The only point that i would finetune in your revised code is on the subprocess call . We call xinput disable/enable only for ELAN. Maybe we should write this as:

                subprocess.call(['xinput', '--disable', device_name])
                subprocess.call(['xinput', '--enable', device_name])

Just to extend this workaround even in ATMEL screen and not just to ELAN.
PS: device_name has to be assigned in initiate_gesture_find also then has to be sent to _long_press function to be used by subprocess call.

Further fine tune of script's code:
We could add something like that to allow users to select various modes of operation according to their needs (just seems more professional to me :-) )

# import sys #this is required to handle args
if __name__ == '__main__':
	validargs=0
	if len(sys.argv)>1: 
		for arg in sys.argv:
			if arg==sys.argv[0]:
				print "script name:",arg
			elif arg=="--pymouse" or arg=="-pm":
				print arg, 'option received: PyMouse will be used '
				pymouseuse = True
				validargs=validargs+1
			elif arg=="--nofingremove" or arg=="-nfr":
				print arg, 'option received: Right Click will be fired with no finger removed'
				longpressworkaround = True
				validargs=validargs+1
			elif arg=="--help" or arg=="-h":
				print "Python RightClick Script. You can use one of the following options:"
				print "--pymouse (or -pm) to use pymouse module for right click injection"
				print "--nofingremove (or -nfr) to enable right click with out finger release. "
				print "If no args are provided default values will be used : Uinput for injection - right click menu upon figure release"
			else:
				print arg," option receved. This is not a valid arg - will be ignored"

	if len(sys.argv)==1 or validargs==0:
		print "No Valid args received at all. Default Script Values will be used"
		print 'Use --help to see available options"
menu will appear upon finger release.'
		pymouseuse = False
		longpressworkaround = False	
	
	initiate_gesture_find(pymouseuse,longpressworkaround)

In this way , users can call the script by command line combining --pymouse (or -pm) and/or --nofingremove (or -nfr) switches (in any position). If no args are given, default values are used.

We can in the future extend the args list to --tray/--notray, --touchtolerance , --calibrate , --debug and so on.

If you like the idea, feel free to use part or all the args code above in your script.

[Zyell]

Hello!

Thank you for pointing out the inclusion of the atmel device name in the workaround. I just have it use the name of the current device now to handle this generically. I have also added the command line options using the argparse library in Python. It makes handling all sorts of argument options really easy! We can easily extend it in the future. Thank you for pointing out the error and recommending the command line options! I am still looking into another option besides disabling/re-enabling the touchscreen for the long press... I'll let you know when I find something. Thanks for your help!

[gevasiliou]

Hello!
I just found this python lib and i thought that we could give it a try just in case it solves the problem. https://pyautogui.readthedocs.io/en/latest/

[Zyell]

Hello! That is a great library btw. I have used it elsewhere, but I hadn't thought of using it here, so thank you for the recommendation. Unfortunately, it didn't work... So I'm still looking! I'm continuing to dig through more detailed output from xinput and evdev.

[gevasiliou]

Ok, It just worthed to be tried :-) . By the way, i don't know if you will be able to find something usefull inside xinput/evdev signals, since this behavior is a kind of Nautilus feature and not xinput/evdev bug, a behavior that can be reproduced with a regular mouse (impossible to inject a right click inside Nautilus user space while draging = while mouse left click remains pressed). If we are lucky we might find a library that will be correctly "steal/grab" the device from Nautilus and that could be possibly a solution. Unfortunatelly the device.grab feature that we tried in the past was not sufficient, and the xinput disable method is a kind of ugly workaround....

I'm facing a different issue now with your script, and i would like your help to find out what is going wrong.
Under normal running (without the use of --pymouse and --long press workaround flags) , i run script from terminal as usuall but after first or second right click the whole script is terminated violently. Actually the whole terminal window is terminated violently , and dissapears from my window list.
Using ps all i can see that both terminal and python rightclick script are not there.

How i could grab the error messages that cause the whole thing to break (terminal and rightclick.py)?

If i run the script with -pymouse flag only i got a bit better behavior but still after some time of normal use rightclick script is interupted (but terminal is not closed) and i see some errors related to x imported libs (obviously imported by submodules). Sounds like X bugs by recent X11 upgrades... I have to sent you the output for this pymouse case but i would like also to find out what is wrong with uinput usage which terminates script and terminal window.