y2z / screenhunter

:hocho: Automatic image-driven cursor positioning and clicking tool for X11

screenhunter

Automated cursor positioning and clicking tool for X11.

build

make clean all

install

sudo make install

uninstall

sudo make uninstall

usage

Usage: screenhunter [options] path/to/target1.png [path/to/target2.png [...]]

  -s          scan only, do not perform any clicks
  -k          do not return the cursor to its original position
  -r          randomize delays and coordinates
  -o          exit after the first match
  -c <count>  set the amount of clicks done per matching area
  -w <ID>     target a specific X11 window by its ID
  -v          output version information and exit
  -h          display this help message and exit

example

win_id=$(wmctrl -l | grep Firefox | awk '/./{line=$0} END{print $1;}')
screenhunter -w $win_id -os button.png
if [ $? -eq 0 ]
then
    wmctrl -a -i $win_id
    screenhunter -w $win_id -or button.png
fi

purpose

screenhunter can be used to add hotkeys to almost any program. It can as well be helpful in acceptance testing, automating web browsing, setting up alert systems, scripting screen capture videos, and many more.

how it works

It takes one or multiple PNG files on the input, looks for areas matching those images on display and clicks each one of them.

notes

Please keep in mind that certain applications (mostly web browsers) have a mechanism to prevent such an automation for security reasons. In such a case the cursor will move and point to the matching area, but the click event will be rejected by the window.

Different programs may render the same image differently (font face, scaling, anti-aliasing, etc).
Basically you want to make sure to take the screenshot of the desired element using the program you want to automate.

Even if one pixel mismatches between what's on the screen and the target's image, the click won't occur.
That being said, it's better to use the smallest (yet still unique) part of the element you wish to make your computer to click on.

If the provided target image source contains transparent pixels, screenhunter will treat them as positvie matches.
This lets the program seek non-rectangular areas on the screen.

screenhunter's Issues

doesn't run on macOS

Needs to have its source code split into macOS (src/mac) and X11 (src/x), with shared functions put into src/shared, accounting for possible support for MS Windows build target added later down the road.

The build should ideally be accomplished using a modified Makefile to accommodate cross-platform code, or Makefile, src/mac/Makefile, and src/x/Makefile to make the code tree more modular and easier to understand.

return cursor to its original position

Add option to make screenhunter return the cursor to its original position after each autoclick

treat non-opaque pixels as matches for RGBA

this would allow to match against non-square elements on the screen

output

fprintf into stdout coordinates of matching spots

options

• -s (don't click, just scan for matches)
• -o (stop scanning and exit after one match)
• -k (hold cursor without returning it back to its original position)
• -r (random click point & random mousedown time)
• -c (doubleclick [2] instead of a single click, or any other amount of clicks; default's one [1])
• -w 5 (window id)
• -a (animate cursor movement; should have a randomized movement pattern if used with -r)
• -p should be for printing coordinates of matching spots;
• -v for version
• -o should be replaced with -l