Over the last few months in particular, there has been a significant move over to proxy-injection based testing platforms by those who are testing their web interfaces.
From speaking to developers at various meetups and various colleagues who work as UI developers and have been on the scene for a long while, there seems to be a huge stigma around the use of Selenium. One ex-colleague spoke about it with such disdain when in reality he was largely complaining about its use and implementation.
A problem with low level tools is that whilst they give you great flexibility, they can be used in a gazillion ways and for want of not blaming their colleagues for a substandard use of tooling, they impulsively blame the underlying tool itself. This is totally unfair on the many open-source developers who give up their time to build these amazing tools.
Another developer stated that with Selenium you need to add explicit wait steps of arbitrary lengths which cause flaky tests. It's possible he was either once again seeing a fragile implementation or was using a version of it long ago when the DOMContentLoaded event was not supported and eager mode was not available prior to 2017 see https://stackoverflow.com/questions/43734797/page-load-strategy-for-chrome-driver-updated-till-selenium-v3-12-0
The only perceived benefits of these proxy-injection services over selenium-based frameworks is that:
- they are well documented
- they provide dedicated support by their respective companies
- provide a marginal performance improvement as there are slightly less moving parts
- they have web GUIs for recording their steps
- they also tend to follow freemium models
I guess a draw to them is also the brand effect and also knowing that it's in their interest to make sure things are working for you.
However this is all a facade...
- Selenium and the various flavours of WebDriver are very well documented
- Selenium has over 3x the contributors of the leading proxy-injecting solution
- The majority of the time taken with a well architected UI test setup is the time it takes to spin up your environments if you're testing the full stack, and being that many applications are single page apps these days, you can perform hundreds of actions and assertions using selenium in very little time. I believe that the marketing push from these companies has coincided with this latter point and they've really adopted well from this increase in performance (not just from the tool itself)
- GUIs are great for creating quick and dirty tests which may actually be preferable if you're say in the discovery phase of a product that significantly changes week to week. If you have a fairly complex system though that you want to guarantee works, and you want your test harness to scale, then you'll likely want to hand-craft your tests and have reusable pieces. This is one of those "best tool for the job" situation.
- The freemium models are unlikely to last forever and if they do they'll be reducing the free tiers. It's like Toblerone reducing the amount of mountain-shaped pieces in a pack. They've all got their margins.
Lastly the comfort of using an off-the-shelf product is the small amount of time it takes to get going writing the tests to give you the level of confidence in your product you're after rather than having to write loads of code before you can get started.
This is where Courgette comes in...
Assuming you have nodeJS installed, setup is one line in the terminal, and you can just start writing tests straight away.
To run on your CI pipeline, you just need a browser installed and the corresponding driver. Take the travis setup Courgette has to test itself: https://github.com/canvaspixels/courgette/blob/master/.travis.yml As you can see it installs nodeJS, the latest stable Chrome and runs npm run ci ... simples!
Courgette and other well-crafted selenium-based tools FTW!