A snippet of code was posted on stack overflow on how to find one image inside another. The question is here: http://stackoverflow.com/questions/4720168/image-in-image-algorithm
search-image.py implements this and also optionally draws a red rectangle around the location of the found image.
This script can be used as part of an automated regression testing framework to check for browser compatibility issues. For a responsive website you could check that the right version of the controls are being shown for a given horizontal resolution.
Note that the version numbers specified are correct at time of writing. They are frequently updated so you might need to look for newer versions of the time files. Be sure to stick with Python 3.6.* and 32 bit.
-
Install 64-bit Python 3.6.x from https://www.python.org/downloads/
Choose the option to add
python.exe
to the path.I suggest customising the installation to install to C:\Python36 (so there is no space in the path) and to install for all users. Also delete any existing Python installation folders by hand (after uninstalling by Add / Remove Programs) since the left behind libraries will cause many problems.
-
Open command prompt as admin, then ensure pip is up to date
python -m pip install --upgrade pip
-
Download python bindings for OpenCV,
opencv_python-3.3.0-cp36-cp36m-win_amd64.whl
, from http://www.lfd.uci.edu/~gohlke/pythonlibs/#opencv (then change to the directory where it is downloaded to) and installpip install opencv_python-3.3.0-cp36-cp36m-win_amd64.whl
-
Download Pillow,
Pillow-3.4.2-cp36-cp36m-win_amd64.whl
, from http://www.lfd.uci.edu/~gohlke/pythonlibs/#pillow and installpip install Pillow-3.4.2-cp36-cp36m-win_amd64.whl
-
Download numpy (used by OpenCV),
numpy-1.13.3+mkl-cp36-cp36m-win_amd64.whl
, from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy and installpip install numpy-1.13.3+mkl-cp36-cp36m-win_amd64.whl
search-image.py target_image.png image_to_find.png [markfile.png]
A number of image files are included which you can use with the following examples.
search-image.py examples\100-orig.png examples\menu_hamburger.png
copy examples\100-orig.png marked_result.png
search-image.py examples\100-orig.png examples\menu_hamburger.png marked_result.png
search-image.py examples\100-orig.png examples\content_vertical_mobile.png marked_result.png
In this example we are trying to find the hamburger menu control
and also the vertical content
inside the target image (Selenium WebDriver screen shot)
We made a copy of this screen shot and search-image.py marked the found locations:
The Open Source Computer Vision (OpenCV) library returns the co-ordinates where the best match is found. It will always return co-ordinates regardless of whether the target image really exists or not. It also returns some numbers representing how confident it is that it found the match - the "primary confidence" and the "alternate confidence".
If you set the threshold too high (100% match), then even very minor changes in how a browser renders the control will result in non matches.
On the other hand, if you set the threshold too low, then you will find matches that do not in fact exist!
After a lot of experimentation with regression testing over the years I have come up with some numbers that seem to work well for the image found threshold.
You may need to update the threshold based on your own use cases.
How template matching works: http://opencv.itseez.com/doc/tutorials/imgproc/histograms/template_matching/template_matching.html
Fast image comparison with python (server down on last check): See http://aatiis.me/2010/08/12/fast-image-comparison-with-python.html