raiman / sikulix4python Goto Github PK

There will be a white dot in the center of the screen when using
java -jar path-to/sikulixapi.jar -p
on Windows 10.
This may cause OCR failure.
Not tested with pattern recognition. But I think it's best to remove it.

@RaiMan
thanks for your work.

I want to highlight another application UI controls from my application based on the mouse position.

example : when mouse moves over the button then I want to highlight the button region(rect)?

How can I do the same with the help of your library?

ref:
https://dsp.stackexchange.com/questions/1471/how-can-i-detect-buttons-in-a-gui-of-an-app-with-pattern-detection-using-python

please advise.

Python 32-bit 3.8.7
Java 15 version 15 vm 15.0.1+9 class 59.0 arch 64
java -jar sikulixapi.jar -p

[error] Python server: py4j not on classpath

Verbose

S D:\Backup\Projects\SikuliX\sikulix4python> java -jar sikulixapi.jar -v
[DEBUG]startUp: API: Running: D:\Backup\Projects\voice recognition toolbox\Caster projects\SikuliX\sikulix4python\sikulixapi.jar
[DEBUG]startUp: API: AppData: C:\Users\Main\AppData\Roaming\Sikulix
[DEBUG]startUp: API: starting with classpath: sikulixapi.jar ...
[DEBUG]startUp: API: no extensions.txt nor valid content
[46 debug] RunTime:temp folder ok: C:\Users\Main\AppData\Local\Temp\Sikulix_1318413536
Options: *** options dump
Options: testing =
Options: OptionsFile = C:\Users\Main\AppData\Roaming\Sikulix\SikulixStore\SikulixOptions.txt
Options: *** options dump end
***** show environment for 2.0.4-2020-03-14_08:01 API
user.home: C:\Users\Main
user.dir (work dir): D:\Backup\Projects\SikuliX\sikulix4python
user.name: Main
java.io.tmpdir: C:\Users\Main\AppData\Local\Temp
running 64Bit(amd64) on w (10.0) from a jar
java 15 version 15 vm 15.0.1+9 class 59.0 arch 64
app data folder: C:\Users\Main\AppData\Roaming\Sikulix
executing jar: D:\Backup\Projects\SikuliX\sikulix4python\sikulixapi.jar
*** classpath dump sikulix
  0: sikulixapi.jar
*** classpath dump end
***** show environment end
[DEBUG]startUp: API: *********************** leaving start
[29 debug] Sikulix: starting API
[47 debug] RunTime:temp folder ok: C:\Users\Main\AppData\Local\Temp\Sikulix_273099905
Options: *** options dump
Options: testing =
Options: OptionsFile = C:\Users\Main\AppData\Roaming\Sikulix\SikulixStore\SikulixOptions.txt
Options: *** options dump end
***** show environment for 2.0.4-2020-03-14_08:01 API
user.home: C:\Users\Main
user.dir (work dir): D:\Backup\Projects\SikuliX\sikulix4python
user.name: Main
java.io.tmpdir: C:\Users\Main\AppData\Local\Temp
running 64Bit(amd64) on w (10.0) from a jar
java 15 version 15 vm 15.0.1+9 class 59.0 arch 64
app data folder: C:\Users\Main\AppData\Roaming\Sikulix
executing jar: D:\Backup\Projects\SikuliX\sikulix4python\sikulixapi.jar
*** classpath dump sikulix
  0: D:\Backup\Projects\SikuliX\sikulix4python
  1: sikulixapi.jar
*** classpath dump end
***** show environment end
[287 debug] RunTime:loadlib: trying opencv_java342
[442 debug] RunTime:addToWindowsSystemPath: added to systempath:
C:\Users\Main\AppData\Roaming\Sikulix\SikulixLibs
[444 debug] RunTime:libsExport: folder exists: C:\Users\Main\AppData\Roaming\Sikulix\SikulixLibs (2.0.4 - 202003140801)
[461 debug] RunTime:loadLib: opencv_java342.dll (success)
[461 debug] OCR: start: Tess4J 4.4.1 using Tesseract 4.1.0
SikuliX API: nothing to do
[478 debug] RunTime:***** final cleanup at System.exit() *****
[491 debug] FileManager: deleteFileOrFolder:
C:\Users\Main\AppData\Local\Temp\Sikulix_273099905
[1083 debug] RunTime:***** final cleanup at System.exit() *****
[1106 debug] FileManager: deleteFileOrFolder:
C:\Users\Main\AppData\Local\Temp\Sikulix_1318413536

i use pycharm in my linux systerm for AutoTest, now i find a question
click(img), can't find the img . the screenshot below:

the same code works very well on MacOS and Win autoText ~

so thx u help
~ tips:poor english don't mind~ 😄

hi

i want to type ALT+A

my code is:

reg = Region()
reg.type("a", Key.ALT())

and i define Key class by add this code to sxclasses.py :

class Key(SXBase):
    SXClass = SXKey

that SXKey = SXPKG.script.Key

but after run :

Traceback (most recent call last):
  File "testSikulix4python.py", line 22, in <module>
    reg.type("a", Key.ALT())
AttributeError: type object 'Key' has no attribute 'ALT'

Tanks 🌹🙏🏻

Hey! At the moment I am writing my mini framework using Sikuli + py4j. But I just can't seem to solve one problem
How can I change the settings? For example, I want to change the search precision, but changing "JavaGW.jvm.org.sikuli.basic.Settings.MinSimilarity" does nothing.

RaiMan I would really appreciate a hint on how to get around this. I think the problem is in Settings.synchronized init () but I'm not sure.

Code example:

from py4j.java_gateway import JavaGateway

JavaGW = JavaGateway()
SXPKG = JavaGW.jvm.org.sikuli
SXPKG.basic.Settings.AutoWaitTimeout = 100.0

Screen = SXPKG.script.Screen()

print(SXPKG.basic.Settings.AutoWaitTimeout) # <py4j.java_gateway.JavaPackage object at 0x7f3a09724a30>
print(Screen.getAutoWaitTimeout()) # 3.0

when I run the example,
scr.getCenter().grow(100).highlight(2)
I get the Trace:
py4j.Py4JException: Method highlight([class java.lang.Integer]) does not exist
and I cannot understand what is the mean of"
one gets method missing, if signatures do not fit
"

Hello! Is is possible to run search within the screenshot?

Best Regards,
Alex

Hi @RaiMan

As you proposed last year in this topic discussion jpype-project/jpype#911, I enabled both Py4J and JPype on my Robot Framework library proposal (https://github.com/adrian-evo/robotframework-sikulixlibrary). From my testing, SikuliX with Java bridge works fine now also on MacOS with Py4J, and on Windows and Linux with both Py4J and Jython.

I think I didn't used any sikulix4python experimental features you described in your project but I guess I used just your enabled Py4J server and standard Java features. Hopefully this approach will work also in the future SikuliX releases.

However what I noticed might not work yet is some get features, e.g. ImagePath.get(), and the known App open issue with Ubuntu. Also tested the library extensively in Windows project for several hours and didn't encountered any relevant issue.

Win10+jdk1.8+sikulix 1.1.4

Try to run the given code, the relative path does not work as expected.
Here is the debug info from sikulixapi:

corresponding to the code:

img = "img.png"
scr.exists(img,3.0)

when I try to run it on linux ,it can not find the image,here is the exception below
Traceback (most recent call last):
File "test.py", line 15, in
click("desktopShow.png")
File "/home/xuman/Desktop/sikulix4python-master/sikulix4python-master/sikulix4python/sxundotted.py", line 32, in click
return SCREEN.click(*args)
File "/home/xuman/Desktop/sikulix4python-master/sikulix4python-master/sikulix4python/sxregion.py", line 35, in click
return self.instance.click(convertArgs(args))
File "/home/xuman/.local/lib/python3.7/site-packages/py4j/java_gateway.py", line 1310, in call
answer, self.gateway_client, self.target_id, self.name)
File "/home/xuman/.local/lib/python3.7/site-packages/py4j/protocol.py", line 328, in get_return_value
format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling o33.click.
: FindFailed: desktopShow.png: (0x0) in R[0,0 1920x1080]@s(0)
Line 2226, in file Region.java

    at org.sikuli.script.Region.wait(Region.java:2226)
    at org.sikuli.script.Region.wait(Region.java:2244)
    at org.sikuli.script.Region.getLocationFromTarget(Region.java:3298)
    at org.sikuli.script.Region.click(Region.java:3916)
    at org.sikuli.script.Region.click(Region.java:3892)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.base/java.lang.Thread.run(Thread.java:834)

Class not prepared for SikuliX in MacOS

Traceback (most recent call last): File "Main.py", line 17, in start from Client.SocketClient import Client File "/Users/vt100i/Desktop/ST/dcs1000_AutoTest/PointW_Mac/Win/Client/SocketClient/Client.py", line 5, in <module> from Client.SocketClient import Exe_keyword File "/Users/vt100i/Desktop/ST/dcs1000_AutoTest/PointW_Mac/Win/Client/SocketClient/Exe_keyword.py", line 10, in <module> from Client.Libraries import WhiteBoardKeywords File "/Users/vt100i/Desktop/ST/dcs1000_AutoTest/PointW_Mac/Win/Client/Libraries/WhiteBoardKeywords.py", line 4, in <module> from Client.Libraries.Recognise.sikulix4python import * File "/Users/vt100i/Desktop/ST/dcs1000_AutoTest/PointW_Mac/Win/Client/Libraries/Recognise/sikulix4python/__init__.py", line 3, in <module> from . sxundotted import * File "/Users/vt100i/Desktop/ST/dcs1000_AutoTest/PointW_Mac/Win/Client/Libraries/Recognise/sikulix4python/sxundotted.py", line 3, in <module> SCREEN = Screen() File "/Users/vt100i/Desktop/ST/dcs1000_AutoTest/PointW_Mac/Win/Client/Libraries/Recognise/sikulix4python/sxbase.py", line 24, in __init__ raise Exception("Class not prepared for SikuliX") Exception: Class not prepared for SikuliX

java: "1.8.0_251"
python: 2.7.16
sikulixapi: 2.0.5

please help me~ @RaiMan