In anticipation of readers time, keeping this post short and simple. Just sharing my experiences with image recognition based automation.
My first encounter with SikuliX was in early 2013. That time it was pretty new, and it used to serve the purpose during test automation, where there was no other means to mimic user actions on the application. The best part was, it used to recognise images based on similarity factor. Lot of time has passed since then, there have been significant development of OpenCv, and numerous new tools been to market.
My second encounter took place in 2017. I supposed to perform some visual testing and also need to validate some images within pdf files. I used Java and OpenCv to achieve the test objectives. OpenCv is used by SikuliX internally to recognize images. But one thing I must admit, learning curve of OpenCv with Java is much longer than with Python. For people without image recognition background, it is tough to master this tool in its raw form. I picked it up because I need to run these tests in headless mode and SikuliX didn't satisfy the criteria.
My third encounter with image recognition based automation was when using Test Complete tool in 2018. It was not as efficient as SikuliX, since the images were searched pixel-by-pixel. Given same test environment setup, it used to work fine, but with different screen resolutions the flakiness starts.
Most recently, I supposed automate an application running on Citrix, and I tried UiPath computer vision. They offers many cool features, like anchors, which works as referential identifiers. During script recording once you select an UI field, it will start suggesting anchors if it can't identify the image uniquely. It will automatically store the images in a repository, saves lot of manual efforts, compared to SikuliX and Test Complete. However, there are some drawbacks. Example, selecting a value from a multi-select listbox. You need to capture individual items, and it gets stored with its Region details. So, unlike SikuliX, you cannot identify an item based on text. So data parameterization is not possible in this case.
At last, I would say SikuliX still have the upper hand, in terms of its simplicity and ease of usage. I have been using the latest version, the actions are performed very fast than earlier, and often I need to put waits to slow it down.
Note: I professionally never used EggPlant tool, only used for training purposes long ago.
