Testing is a vital part of software development, since it is about finding the software bugs. Testing can be done through both automated and manual testing, but which is best and for what circumstances? Whether you use one or the other or both, the decision should depend on a cost vs benefit analysis. Read on for an overview on Manual Testing vs Automated Testing.
Automated testing is meant for large projects with many system users. If you start with automated testing, your automated testing tool will run the tests according to predefined requirements. Once the requirements and the outcomes align, you have a bug-free product. If they do not, you have to alter your code and run tests again until that happens.
In contrary to automated testing, manual testing is good for smaller projects or for companies with insufficient budget and usually where testers are the software developers, sharing the testing role to find software defects.
The Value of Automated Testing
Saving Time and Catching Bugs before Release
Manually repeated tests are often considered to be costly and time consuming. Once an automated test is created, it can run repeatedly, continuously at no additional cost. Once you start with automated testing, you do not have to waste manual effort on complex testing such as exploratory testing. Automated tests can be run continuously and at any time. By running tests regularly, you can catch bugs in your tests or code quickly (ideally before release).
Faster Feedback on Changes
Automated testing provides testers with faster feedback on regressions. Faster feedback enables software developers to trace back what went wrong or to make necessary changes without “ruining” the existing work.
Once an automatic test is written, a computer can run it at very low cost. The same manual test would require a person to work on a list of instructions and it would cost much more.
Even the more experienced software developers and testers can and do make mistakes, herein lies the problem; mistakes by developers or in testing by humans are hard to detect. With automated testing if a problem is detected it is recorded automatically or the test is stopped as long as the computer will execute the same testing process, every time. The Failure Mode and Effects Analysis (FMEA) is a great technique for failure analysis. Once a problem occurs, the test reports stops appearing and the result is a false test result. It will not happen if the tester misses a step and ticks the “OK” button. Automated testing often highlights when someone has forgotten to check in code, – that also ensures better test reliability.
Downside of Automated Testing: Increase in up-front investment
The Time You Have to Invest into Automated Testing
As mentioned above, automated testing will save you time once it is done. However, writing automated tests will take approx. 30% more time. Automated testing is an investment that will save you money over time.
Poorly designed test suite and re-testing expenses
Another cost factor for automated testing is the design. If the test suite is poorly designed, adding new tests might cost more than the value provided by the test suite.
Unreliable Tests Come with Additional Expenses
If the test is unreliable, time needs to be invested to find the cause, was it caused by regression or via the test code? It also means that manual tests need to be introduced into the workflow, – that comes with extra costs and the “wasting” the time of developers and testers.
Automated Testing Tools Have Limits
Many automated testing tools cannot test image colour or font size. Their changes can be only detected by manual testing.
How to Select Automated Testing Tools?
If you look for automated testing tool, you should create a list of requirements that are important to you. Your selection criteria should take into account the following items.
- Various applications and platforms
- Different operation systems
- Mobile devices (important if you have to test mobile applications)
- Diverse software testing types (function testing, running unit tests)
- Various test data sources (text, XML files, xls, SQL Server, Oracle and MySQL)
- A log of all testing actions with detailed report
- Sharing test projects with several testers
- Integration with build software
- Opportunity to export test results to external files
- Technical support is provided.
Continuous testing with codeBeamer ALM
For continuous testing codeBeamer ALM utilizes one of the most popular open source automated web testing tools, – Selenium. We use Jenkins (an open source continuous integration tools) for the test run of the test case and for continuous build. The combination of Selenium tests, Jenkins – codeBeamer ALM integration and test management features ensures a seamless integrated environment ideal for continuous testing and reporting back to stakeholders.