Searching Icons in a Screenshot using Template Matching
In this article, I will explain how to search for icons in a screenshot using template matching. The system loads the screenshot and the icon image, then locate similar objects using template matching technique. A box will be drawn around these objects and the screenshot image is displayed on the screen.
Download the full source code, search_icons.c. In the next section I will explain the code and show the result I've got.
Writing the code, step by step
First, load the reference and template image.
Listing 1: Load the images
IplImage *tpl;
img = cvLoadImage( "screenshot.jpg", CV_LOAD_IMAGE_COLOR );
tpl = cvLoadImage( "template.jpg", CV_LOAD_IMAGE_COLOR );
Note that I stripped out the error checking lines from the code above to make it simple. Remember that you should always have some error checking in your programs.
The next step is perform template matching operation. But before that, we need to create a new image for the template matching result. If img is WxH and tpl is wxh, then the template matching result must be W-w+1xH-h+1.
Listing 2: Perform template matching
int w = img->width - tpl->width + 1;
int h = img->height - tpl->height + 1;
/* create a new image for the comparison result */
IplImage *res;
res = cvCreateImage( cvSize( w, h ), IPL_DEPTH_32F, 1 );
/* perform template matching */
cvMatchTemplate( img, tpl, res, CV_TM_SQDIFF_NORMED );
Now res contains the comparison result from template matching operation. From this result, we find each element which value is lower than a predefined threshold.
Listing 3: Find objects
int x, y;
CvScalar s;
/* loop the comparison result array */
for( y = 0 ; y < res->height ; y++ ) {
for( x = 0 ; x < res->width ; x++ ) {
/* get an element */
s = cvGet2D( res, y, x );
/* if value below the threshold, similar object is found */
if( s.val[0] <= threshold ) {
/* draw a box to mark the object */
cvRectangle( img,
cvPoint( x, y ),
cvPoint( x + tpl->width, y + tpl->height ),
cvScalar( 0, 0, 0, 0 ), 3, 0, 0 );
}
}
}
We've marked similar objects by drawing a box around it. Now display the screenshot and free the memory.
Listing 4: Display image and free memory
cvNamedWindow( "screenshot", CV_WINDOW_AUTOSIZE );
cvShowImage( "screenshot", img );
cvWaitKey( 0 );
/* free memory */
cvDestroyWindow( "screenshot" );
cvReleaseImage( &img );
cvReleaseImage( &tpl );
cvReleaseImage( &res );
Result
Figure 1 shows the result for searching PDF icons using the template shown in Figure 2.
Fig 1. The screenshot of my desktop with marked PDF icons
Fig 2. PDF icon used as the template
As you can see, the program successfully found the objects that similar with the given template. However, the result vary depends on the threshold given. Generally, with higher threshold the program will be able to find all similar objects, but it also increase positive false. You may want to try different threshold values to get the best result.
Related Articles
The Downloads
5 Comments
Aurelien on Jan 17, 2010:
Is it possible to build the screenshot automatically? (using OpenCV or msdn...)
edmundo on Jan 26, 2010:
gracias por compartirlo...
saludos desde Sonsonate, El Salvador.
Universidad de Sonsonate.
sping on Mar 8, 2010:
I just want to say thank you so much for your examples. I am working on a project to do Optical Music Recognition but am totally clueless... your examples help me a lot!
Would love to see more of them
hieuiph on Jun 14, 2010:
Leave a comment
Tags
Recent Posts
- OpenCV Utility: Reading Image Pixels Value
- OpenCV Circular ROI
- OpenCV 2.0 Installation on Windows XP and Visual Studio 2008
- Runtime ROI Selection using Mouse
- Real Time Eye Tracking and Blink Detection
About the Author
| Tel. | +62 31 8662872 +62 856 338 6017 |
| ICQ | 489571630 |
| Skype | dede_bl4ckheart |
| Yahoo | dede_bl4ckheart |
| nashruddin.amin |
Recommended Sites:
Hacker's HTTP ClientHTML and CSS Tutorials
Stop Dreaming Start Action
Online Quran and Translation
Victor on Nov 10, 2008:
That hardcoded threshold of 0.12 doesn't work in my case ( where's that value from, anyway ?).
I just searched for the minimum SvScalar value in the result array and draw rectangle there.