High-Volume Enterprise Performance Testing Strategy

Ensuring Application Scalability

The Business Need

The world’s largest grocery retailer’s Digital Customer Program (DCP) end user population was growing rapidly. Accessible via the web and mobile interfaces, it included key features such as loyalty card, customized coupon distribution, recipe distribution, prescription refill, and more. The platform was experiencing performance and stability issues. These issues coupled with a desire to move to a Java / Open Source software stack led the client to fund an application re-build effort. Before embarking on such a large initiative the client wanted to establish performance guidelines and benchmarks along with mechanisms for ongoing measurement of key performance metrics to ensure that new development efforts met clearly established and visible performance goals.

Centric’s High Impact Results

Our client needed an expert with the technical expertise to develop a comprehensive and ongoing performance testing strategy and approach to provide a framework needed to gather key performance metrics as the number of users and features expanded. When executed, the performance strategy would provide confidence that applications were ready and able to handle increasing loads as well as providing an early warning system for developing performance issues allowing for preemptive tuning and enhancements.

Centric’s National Software QA & Testing practice lead performed the following activities:

  • Documented primary business functions and created diagrams detailing what applications and web services were utilized to complete the process. Third party applications such as coupon customization services were also included in the analysis.
  • Documented primary performance goals and metrics (correlated web page load time, web service response time, agent CPU and memory utilization, queue wait times, etc…) thereby establishing the initial baseline.
  • Interviewed application architects and developers, reviewed logs, and monitored the applications to identify what system components where likely to be causing performance issues.
  • Created a performance evaluation decision matrix that allowed application components to be scored thereby identifying components most in need of immediate attention.
  • Advised the development teams on how to incorporate performance testing into their Agile development processes.
  • Designed an application monitoring approach that triggered alerts when appropriately set thresholds were reached.
  • Built a tool that used Omniture data to forecast application end user load. This tool was in turn used to do infrastructure capacity planning for the production environment. Usage increased form 29K page views per day to 39K page views per day. This 34% growth/week was correctly predicted by forecasting tool.

Results

The analysis of the application lead to rapid identification of performance bottlenecks. Many of these issues were able to be fixed with relatively minor effort resulting in better overall application speed and much better application stability.

Since performance testing was implemented, the customer has not had an unplanned production outage.