Explore: Sikuli - Software Automation Tool

Sikuli

What is Sikuli ?

Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API's support. You can pro-grammatically control a web page, a Windows/Linux/Mac OS X desktop application, or even an iphone or android application running in a simulator or via VNC.

Why ? and How it works?

Sikuli: Seeing Pixels

Sikuli's greatest value is its generality, "If it has pixels that Sikuli can see, and then it's open to automation". The technique is open to any application with a GUI that can display on a Windows, Mac, or Linux desktop. Users have already been apply it to not just desktop applications, but also Web pages, video games, mobile phone apps (running in a simulator or using a remote connection between the desktop and the phone), and applications from other platforms running in a virtual machine.

User Interface for Taking Screenshots

Sikuli Search allows a user to select a region of interest on the screen, submit the image in the region as a query to the search engine, and browse the search results. To specify the region of interest, a user presses a hot-key to switch to Sikuli Search mode and begins to drag out a rubber-band rectangle around it. Users do not need to fit the rectangle perfectly around a GUI element since screenshot representation scheme allows inexact match. After the rectangle is drawn, a search button appears next to it, which submits the image in the rectangle as a query to the search engine and opens a web browser to display the results.
Sikuli Script’s annotation interface allows a user to save screenshots with custom annotations that can be looked up using screenshots. To save a screenshot of a GUI element, the user draws a rectangle around it to capture its screenshot to save in the visual index. The user then enters the annotation to be linked to the screenshot. Optionally, the user can mark a specific part of the GUI element (e.g., a button in a dialog box) to which the annotation is directed.

Example : Taking ScreenShot