I have written some game assistants using the button wizard before. There is a function called FindPic, which searches for a given picture within the screen range and returns the found coordinate position.
Now, Java implements similar functionality to this function.
Algorithm description:
Take a screenshot and get picture A (the target picture you are looking for is picture B);
Traverse the pixels of image A, and according to the size of image B, obtain the four corners of image B mapped to the four points on image A;
The four obtained points are compared with the values of the four corner pixels in Figure B. If the four points are the same, go to step 4; otherwise, go back to step 2 to continue;
For further comparison, compare all points within the mapping range with all points in Figure B. If they are all the same, the picture has been found; otherwise, return to step 2 to continue;
Here, the comparison between pixels is done by obtaining the RGB value of each pixel through the BufferedImage object. As follows, convert BufferedImage into an int two-dimensional array:
/** * Get the image RGB array based on BufferedImage * @param bfImage * @return */ public static int[][] getImageGRB(BufferedImage bfImage) { int width = bfImage.getWidth(); int height = bfImage.getHeight(); int[][] result = new int[height][width]; for (int h = 0; h < height; h++) { for (int w = 0; w < width; w++) { //Use getRGB(w, h) to obtain the color value of the point, which is ARGB, but in actual applications, it is RGB, so ARGB needs to be converted into RGB, that is, bufImg.getRGB(w, h) & 0xFFFFFF. result[h][w] = bfImage.getRGB(w, h) & 0xFFFFFF; } } return result; }Comparing whether the RGB values of two pixels are the same is done through the XOR operation (which is said to be more efficient than ==). If the value obtained after the XOR operation is 0, it means that the RGB values of the two pixels are the same, otherwise it is not Same.
The complete java code of the algorithm is attached below:
package com.jebysun.test.imagefind; import java.awt.AWTException; import java.awt.Rectangle; import java.awt.Robot; import java.awt.Toolkit; import java.awt.image.BufferedImage; import java.io .File; import java.io.IOException; import javax.imageio.ImageIO; /** * Find the specified image on the screen* @author Jeby Sun * @date 2014-09-13 * @website http://www.jebysun.com */ public class ImageFindDemo { BufferedImage screenShotImage; //Screenshot BufferedImage keyImage; //Find the target image int scrShotImgWidth; //Screenshot Width int scrShotImgHeight; //Screenshot height int keyImgWidth; //Find target image width int keyImgHeight; //Find the target image height int[][] screenShotImageRGBData; //Screenshot RGB data int[][] keyImageRGBData; //Find the target image RGB data int[][][] findImgData; //Find results, target The coordinate data of the icon located on the screenshot public ImageFindDemo(String keyImagePath) { screenShotImage = this.getFullScreenShot(); keyImage = this.getBfImageFromPath(keyImagePath); screenShotImageRGBData = this.getImageGRB(screenShotImage); keyImageRGBData = this.getImageGRB(keyImage); scrShotImgWidth = screenShotImage.getWidth(); scrShotImgHeight = screenShotImage.getHeight(); keyImgWidth = keyImage.getWidth(); keyImgHeight = keyImage.getHeight(); //Start looking for this.findImage(); } /** * Full screen screenshot* @return Return BufferedImage */ public BufferedImage getFullScreenShot() { BufferedImage bfImage = null ; int width = (int) Toolkit.getDefaultToolkit().getScreenSize().getWidth(); int height = (int) Toolkit.getDefaultToolkit().getScreenSize().getHeight(); try { Robot robot = new Robot(); bfImage = robot.createScreenCapture( new Rectangle(0, 0, width, height)); } catch (AWTException e) { e.printStackTrace(); } return bfImage; } /** * Read the target image from the local file * @param keyImagePath - the absolute path of the image * @return the BufferedImage object of the local image */ public BufferedImage getBfImageFromPath(String keyImagePath) { BufferedImage bfImage = null; try { bfImage = ImageIO.read(new File(keyImagePath)); } catch (IOException e) { e.printStackTrace(); } return bfImage; } /** * Get the image RGB array based on BufferedImage * @param bfImage * @return */ public int[][] getImageGRB (BufferedImage bfImage) { int width = bfImage.getWidth(); int height = bfImage.getHeight(); int[][] result = new int[height][width]; for (int h = 0; h < height; h++) { for (int w = 0; w < width; w++ ) { //Use getRGB(w, h) The color value obtained at this point is ARGB, but in actual applications RGB is used, so ARGB needs to be converted into RGB, that is, bufImg.getRGB(w, h) & 0xFFFFFF. result[h][w] = bfImage.getRGB(w, h) & 0xFFFFFF; } } return result; } /** * Find images*/ public void findImage() { findImgData = new int[keyImgHeight][keyImgWidth][ 2]; //Traverse the screenshot pixel data for(int y=0; y<scrShotImgHeight-keyImgHeight; y++) { for(int x=0; Are the values of the four points and the four corner pixels of Figure B the same? //If they are the same, compare all the points within the mapping range on the screenshot with all the points in the target image. if((keyImageRGBData[0][0]^screenShotImageRGBData[y][x])==0 && (keyImageRGBData[0][keyImgWidth-1]^screenShotImageRGBData[y][x+keyImgWidth-1])==0 && (keyImageRGBData[keyImgHeight-1][keyImgWidth-1]^screenShotImageRGBData[y+keyImgHeight-1][x+keyImgWidth-1])==0 && (keyImageRGBData[keyImgHeight-1][0]^screenShotImageRGBData[y+keyImgHeight- 1][x])==0) { boolean isFinded = isMatchAll(y, x); //If the comparison results are exactly the same, it means the picture is found, and the found position coordinate data is filled into the search result array. if(isFinded) { for(int h=0; h<keyImgHeight; h++) { for(int w=0; w<keyImgWidth; w++) { findImgData[h][w][0] = y+h; findImgData[ h][w][1] = x+w; } } return; } } } } } /** * Determine whether all the points within the mapping range of the target image on the screenshot correspond to the points of the small image one-to-one. * @param y - The y coordinate of the screenshot that wants to match the pixel point in the upper left corner of the target image * @param x - The x coordinate of the screenshot that wants to match the pixel point in the upper left corner of the target image * @return */ public boolean isMatchAll(int y, int x) { int biggerY = 0; int biggerX = 0; int xor = 0; for(int smallerY=0; smallerY<keyImgHeight; smallerY++) { biggerY = y+smallerY; for(int smallerX=0; smallerX<keyImgWidth; smallerX++) { biggerX = x+smallerX; if(biggerY>=scrShotImgHeight || biggerX>=scrShotImgWidth) { return false; } xor = keyImageRGBData[smallerY][smallerX ]^screenShotImageRGBData[biggerY][biggerX]; if(xor!=0) { return false; } } biggerX = x; } return true; } /** * Output the found coordinate data*/ private void printFindData() { for(int y=0; y<keyImgHeight ; y++) { for(int x=0; x<keyImgWidth; x++) { System.out.print("("+this.findImgData[y][x][0]+", "+this.findImgData[y][x][1]+")"); } System.out. println(); } } public static void main(String[] args) { String keyImagePath = "D:/key.png"; ImageFindDemo demo = new ImageFindDemo(keyImagePath); demo.printFindData(); } }This algorithm is an accurate comparison. As long as there is a difference in one pixel, the picture will not be found. Of course, if you want to specify a comparison accuracy, I also have an idea, which is to make statistics when comparing all pixels within the mapping range in step 4 of the algorithm. If 90% of the points are the same, that means the accuracy is 0.9.
In addition, efficiency issues may also be considered, but I don't care too much about efficiency in my application scenario. If any friends read this article and have better ideas on this topic, please leave a message.