-
Farming out quality assurance testing for new applications, and even enterprise transformations, can cut costs by 30 percent; defects even more so.
March 1
Like most banks, SunTrust is tasked with upgrading its technology quickly on multiple fronts to meet myriad IT mandates, with a quick time to market and manageable expenses.
That means improving the testing it does to ensure the hastened development pace don't adversely impact systems that are in place. SunTrust, which is already in the midst of a multi-year reorganization to standardize IT testing, is expanding its tests to include progressive automation. Progressive automation spots the negative impacts of programming or updates on existing systems much earlier in the development cycle. It's a strategy that enhances traditional regressive testing, in which new code is tested near the end of a project to locate errors.
Richard Gilbert, the senior vice president at SunTrust who is responsible for enterprise testing, this week spoke with BTN about the new testing techniques, which grew out of the bank's Test Center of Excellence. The unit was first formed three years ago at the $185 billion-asset Atlanta-based bank to bring maturity and standards to cross-enterprise testing, which had been siloed and owned by individual units. As the center braces for more software updates to adhere to new regulations and new applications to enable customer-facing innovations in mobile banking, payments and other areas, progressive automation and centralized testing standards will take on a much greater role.
BTN: What are you testing right now, and how has that changed over the past couple of years?
Gilbert: We have put together a centralized organization to test our application landscape known as a Test Center of Excellence (TCoE). We focus on ensuring the quality of our applications and changing technologies that impact internal [staff] and external clients, everything from the core banking system to online banking to mobile applications. Over the past three years, we have worked to build a centralized testing practice as opposed to a distributed organization model. The focus has been on transforming our people, processes, and tools to enhance our testing results.
How does centralization improve testing?
We can ensure standards in testing, which drives both consistent and predictable levels of quality for our applications. A lot of banks are moving toward centralizing testing. Under the old [distributed testing model], teams do their own testing for each line of business and the results vary widely based on the skills of the practitioners and the focus on quality. Our centralized model standardizes processes and tools across the enterprise and trains practitioners on the fundamentals of testing software applications. This enables us to resolve issues with inconsistent delivery and changing IT complexity and make our test process more effective.
How is progressive automation different from other methods of testing?
With progressive automation, you write automated test scripts while the development code is being written, for faster testing and identification of problems and quicker fixes. You are essentially writing scripts to test new software. It takes two or three days to run a progressive automation suite, but we can execute hundreds of test cases, where it would have taken weeks to run manual tests. Progressive automation is different from regression automation in that you are writing automated test cases to validate new functionality [in progressive testing], where in regressive automation, you are validating that existing functionality does not break with changes. Progressive automation also allows you to gain high levels of ROI within a project whereas typical regressive automation suites are built at the end of the project with the hopes of using them again in the next project. We do use regressive testing software for testing our release process as a final check before releasing changes into production. Progressive automation allows us to shorten the cycle time for testing and increase testing coverage in terms of requirements.
How does progressive automation differ from unit testing [an alternative to regressive testing in which individual units of source code are tested]?
Progressive automation is performed by our independent testing team to validate both the functional and system requirements of the application in an automated fashion. [Progressive automation is used to replace manual testing in the Functional Verification Test (FVT) and System Verification Test (SVT) phases of the software delivery cycle]. Typically, unit testing is manually performed by the development team as a self-check to ensure code is working as the developer intended before handing over the code to the testing team for independent validation.
How much time can you save?
We can save about two weeks of a typical three-month development cycle. We generally do three major releases a year and we hope to increase our activity and speed.
How deeply is progressive automation being used?
We started with a proof-of-concept and since have run multiple projects with the new progressive automation testing method. With our targeted projects, we have saved over 20,000 hours leveraging progressive automation versus standard testing methods. Our 2012 goal is to enable the adoption of progressive automation across the enterprise. The idea is if you are making changes to mobile banking, or ATMs, or shrinking development time to meet new federal regulatory mandates, we'll be able to respond quickly and maintain a high level of quality for our clients.
Are the tests produced internally, or with a partner?
We leverage a set of tools from HP [Hewlett Packard] to develop the progressive automation test suites and are working with HP to further develop this concept and continually improve that tool set. The progressive automation strategy was developed in house as a way to increase speed to market, drive high levels of quality, and reduce our testing cost. We have also partnered with our technology vendors to use the progressive automation concepts that are part of our TCoE.
What are some of the challenges behind progressive automation?
Progressive automation is plagued by the same two challenges that are faced during manual testing. The first is having a stable environment and the second is having the right data to validate the test scenarios. At SunTrust, we have teams within our TCoE focused on addressing both issues. The data challenge is where we have the most focus and difficulty. We are working a plan to create what we call a 'golden client.' The 'golden client' is a subset of linked data pulled from all of our IT systems, which meet 85 to 90 percent of test scenarios and data conditions. The remaining 10 to 15 percent will be conditioned based on the needs of the project. This way we are not trying to find data for each of our testing projects-the 'golden client' will meet that need.
