How we Built a Scaled Automated Testing and Reporting Framework that Extensively Cut Down Triage Time for Leading Chemistry Content Curator
A leading chemistry content curator and re-publisher had embarked on a sweeping program to replace their content publishing platform. With several trillion documents, the client needed a way to ensure documents were cross-referenced, indexed, and searchable.
To verify the new content publishing platform was performing as expected, they also needed an automation and reporting framework that would allow for eight different agile teams to run over 80,000 automated tests against large volumes of data in multiple environments every day.
Enter Centric: An Automation Framework for Large Data Testing
Centric Consulting’s DevOps team created a parallel automation framework built on Ruby and Cucumber to automate and run over 80,000 automated tests in less than 20 minutes. When dealing with tests at such a large scale, traditional test reporting (Pass/Fail status) mechanisms are inadequate for rapid feedback.
We also created a reporting engine that allowed for rapid triage of failed tests through innovative categorization and tracking of test failures. This allowed the reporting engine to identify per-execution common error messages, and group them together as possible similar root causes.
The Results: Hours Saved by Automating 80,000 Tests
What originally took hours to triage can now be done in less than 15 minutes.
The reporting framework also provides information that quantifies the impact of failures on specific types of documents (e.g. Patents), allowing the business to rapidly identify the group of documents ready for release and those that have problems.