When we are estimating an automation project, we try to identify the scope, the team velocity, and certain factors so we can have a good estimation, but we do not always have into account some items that are very common and that I have seen happening over and over again on different projects where I was pulled into as a consultant.
Let´s see some of those common mistakes we make when estimating a test automation project:
Contents
Estimating an automation project and starting too early
Sometimes when we try to estimate how long it´s going to take to automate a certain number of features, we don’t take into account in which development stage we are. For example, if we have the user stories definition, we can have a rough estimate but we can not start until we have at least a basic stable version of the application, so always define the preconditions that you need to start coding your scripts before agreeing on delivery date.
Having an unstable application to automate
It´s common to start automating test cases for a website or mobile app and then to start facing constant changes on the app, which requires refactoring and maintenance of existing test cases, redefinition of steps, and even deprecation of test cases. When estimating the test automation of an app, we need to specify the scope and reestimate in case we need to make changes to existing work.
Not thinking about who is going to work on the project
If we are estimating a test automation framework creation, the automation of a set of test cases, of the refactor of existing tests, we need to think about who is going to work in the project. If we are senior test automation engineers, we can not estimate the whole work as everybody will have the same seniority. Team seniority, ramp-up time, and team rotation need to be part of the equation.
Not thinking about test environments to cover
O.K, we know that we can automate N test cases for this particular project, and we know that we have Y test cases in the backlog, so we can just do the math and have a rough estimation. Almost true, but we need to calculate this based on how many environments we need to support in order to have a better estimation, let´s say that we have a test case, it´s not the same if this test will be executed on 1 browser, or in 3-4 browsers, or if it will be executed in mobile devices, or in the cloud.
Test environments availability
This is another common mistake, to think that we will have everything available all the time. Environments will go down, they will break, they will be under maintenance, and that will delay the work. So as part of our preconditions, the availability of the environments must be part of the equation.