When you are developing applications in an Agile manner, continuing to rely on manual testing simply doesn’t make the grade anymore. It’s easy enough to test new functions during an agile sprint. Things become intensely problematic, however, when those functions and every element of the application that has been previously developed- needs to be tested over again, otherwise known as regression testing. The challenge most development teams run into is that as the application increases in size, they simply run out of time to run all the regression tests required if they are relying on manual testing processes.
The only effective way to overcome that issue is to automate tests whenever practical. Test automation not only saves lots of time running those tests, it also frees up application testers to focus on more critical tasks.
MENDIX APPLICATION TEST SUITE (ATS)
Given that need to automate an ever-increasing number of tests, it’s not surprising that test automation tools are one of the hottest categories in all of application development. There’s no shortage of testing tools to choose from, but if your organization needs to automate the testing of low-code applications built using the Mendix low code platform, we’re confident you’ll agree that the Mendix Application Test Suite (ATS) sets itself apart in terms of both the quality of the tests that can be run and the ease at which tests can be constructed.
TEST AUTOMATION TOOLS
To better appreciate the unique capabilities of Mendix ATS, let’s first consider the current state of application testing tools. Most of the test automation tools that execute tests at the graphical user interface level fall broadly into two categories.
The first category are tools such as Selenium Webdriver that require coding skills to set up and configure a framework, as well as analyze, debug and maintain test cases. These types of tools are generally made available for free via open source projects. They provide a high degree of freedom in terms of the type of tests that can be constructed. The issue is they are quite labor intensive to set up and maintain. Most organizations need to hire specialists to maintain testing environments built using these tools. Not only does that get expensive, business leaders, end users and other non-technical stakeholders are also not likely to understand test cases developed in code, let alone be able to contribute to creating and running them in any meaningful way.
At the other end of the spectrum, there are testing tools such as Ranorex, Tosca and TestComplete that abstract away most of that complexity. These commercial tools are typically supported by a vendor that has created a testing experience that works right out-of-the-box. Not only do these tools require a minimal amount of skills to set up, they are also more accessible to non-technical users because they rely on high-level functions to run tests. The downside, however, is that most of these tools are not optimized for a specific platform. In fact, the number of test functions supported tends to be limited. Any extensions made to optimize these tools or test specific functions in application specifically for a platform such as Mendix requires either a programmer or tester with coding skills to develop them. The paradox is that despite the limited number of functions supported, some of these tools provide so many options to configure that it’s roughly akin to being presented with a cockpit without ever having flown a plane before!
Ideal testing environment
In our opinion, a testing environment ideally should:- Make it easy to create, set up, and run test cases quickly;
- Be simple enough for everyone on the development team to understand and use
- Keep the time required to maintain test cases to an absolute minimum.
Let’s look at how Mendix ATS meets those requirements. As a cloud-based service, Mendix ATS substantially reduces the amount of time required to set up and configure a testing environment. There’s no need for expensive specialists to spend weeks setting up a test framework. Development teams can literally start creating and running test cases in minutes. Improvements, fixes, and maintenance of the overall framework are automatically delivered via regularly scheduled updates.
A built-in recorder function in Mendix ATS makes creating test cases simple. In sharp contrast to other record and play back tools, those recordings within Mendix ATS result in a script consisting of high-level functions specific to Mendix being created that act on Widgets. Of course, these functions can also be used to manually build a script, if that is what you prefer. Having functions on the Widget level means there is no need to try to create a robust locator for a certain web element, then write some code to act upon that element such as clicking it, or setting a value, and subsequently guess how long to wait before the page is ready for the next test step. Functions such as ‘Click Widget’ or ‘Set Value’ require just a few parameters, either recorded or manually added, and you’re good to go! Better still, all these functions have a built-in smart waiting mechanism, so tests don’t fail right away or, worse yet, wind up taking way too long to execute.
Those high-level functions also result in the development of scripts that everyone on the team can easily understand , including those who are not all that tech-savvy. That capability will go a very long way indeed toward improving collaboration across the development team.
Finally, Mendix ATS greatly reduces the time and effort spent on test case maintenance. Every test automation engineer that has ever worked with Mendix has experienced the frustration of getting a test case up and running, only to discover a certain Widget is technically changed in an upgrade of the Mendix Modeler. In terms of functionally and visual appearance nothing seems to have changed in the application, but under the hood, something has clearly changed. Test cases break down and development needs to start all over again. Analyzing the problem, building new locators, and then repeating that for every instance of this widget is a giant pain! Fortunately, there are two ways Mendix ATS enables developer teams to work around these issues.
First off, functions in Mendix ATS are backward and forward version compatible with the Mendix platform. If you use a function that addresses a certain widget, that same function, with the same set of parameters, will work for that Widget across all versions of the Mendix runtime that are officially supported by Mendix*. Upgrading to a new modeler? Using different modeler versions in different environments? No problem, you have one script that works on every version.
A second way Mendix ATS addresses this is that with a few clicks of a mouse sections of a test case can be modularized. These so-called extracted actions support input and output parameters and can be easily reused across many test cases. If something changes in the way your action should address the application, fix that issue only in the extracted action itself, and every test case that uses it will be fixed as well.
As great as all that sounds, there’s one other thorny issue development teams may encounter even within Mendix ATS. Development teams could easily encounter a situation in which the recorder doesn’t recognize your custom-built Widget, or you need to do something else that Mendix ATS does not yet address at the widget level. To counter this, Mendix ATS is not restricted to the functions on the (standard) Widget level. Mendix ATS also allows development teams to build their own functions. That can be accomplished either out of its standard building blocks, or you can go ‘wild’ with Selenium-level actions and JavaScript execution.
Get in touch
