More than 25 business and technical consultants gathered in St. Louis on March 6 to compete in Expedition: Data, our Data & Analytics (D&A) event to identify and develop machine learning (ML) and data science talent within the company.
Two data visualization (DV) teams and four ML teams worked with our tech partner, Microsoft, and national digital marketing company RevLocal to take a deep dive into the real-world problems of employee and customer retention.
An audience of more than 50 people cheered the teams on as they built DV dashboards in real-time and displayed the results of their ML models.
Microsoft and RevLocal each provided a judge for the event, as well as one potential Centric client. In addition, audience members submitted their own ballots.
For judges and audience members alike, the event not only identified data-based solutions to RevLocal’s employee and customer retention problems but also demonstrated that employee retention programs can go far beyond traditional classroom training programs.
“Innovation and employee engagement are important parts of our culture,” says Machine Learning Developer, Tanya Kannon, who created the event. “Expedition: Data is a great way to find employees who have the potential to be great at ML and data science, even if they aren’t ‘tech’ people. I think that helps them feel even more excited about working at our company.”
Data Visualization, Live
In the most interactive part of the day, Business Analytics Designer Monica Lopez served as the emcee for the DV portion. Working the room with a cordless mic, she interviewed DV team members as they built data dashboards on the fly. Two teams had 20 minutes to complete their work. One unique challenge: thinking deliberately about color choices and patterning in graphics to meet the needs of one of the judges, who is color blind.
“When working with visualization, you have to think about ways to present it for all audiences,” Lopez says. “But at the end of the day, your DV is only as good as your data, so that’s where it all starts.”
The first DV group, hailing from our Cincinnati team, included Rob Urbanski, Tam Truong, Larry Wildey and Pete Grimes. Members set out to build a dashboard exploring the correlation between employee and customer retention. Specifically, they wanted to see if customer retention rose or fell based on which consultants they had worked with, and those consultants’ retention rate.
“An employee leaving the company affects clients and other employees and is very disruptive,” Urbanski said. “And while RevLocal has a fantastic customer retention rate, they are always looking to improve it. One challenge is getting customers to understand the longer-term view of digital marketing.”
By breaking employees’ and customers’ data into subpopulations based on the amount of time each had been with the company, the team created models to identify a set of conditions that appeared to drive the decision to leave for both groups.
The second group included Jeff Aalto, Anelia Schmitt, Dave Murray and Mary France. They analyzed 22,000 accounts that changed from active to closed in three months using three factors: client tenure, seasonality of client industry, and top clients by revenue.
“We had three questions we wanted to answer,” Schmitt explained. “Could RevLocal onboard new clients differently during the first three months to improve retention? Could they better serve clients by better understanding client industry seasonality? And what can RevLocal learn from its top revenue clients?”
While presenting their dashboards, both teams made recommendations to RevLocal about key issues like training, compensation and job titles. Judges then tallied the votes, and the result was in: a tie between the audience members and the three judges! After re-polling the judges, the second team won the competition — but, by just a hair.
“The quality of work we have seen in these DV teams is amazing, especially considering the relatively small amount of data they had to work with and the short time period,” Kannon said after announcing the results. “It’s hard to believe how much these teams accomplished. They didn’t even have the data until after the first of the year!”
Machine Learning from Around the World
In the ML competition, teams from across our United States locations and Centric India battled for bragging rights and $100 Amazon gift cards as they sought innovative ways to use Microsoft’s Azure suite and other tools, like Databrick and R-Studio, to analyze RevLocal’s business problems.
Here’s how the teams shook out:
Winning Team: ML Team 1
Team Members: Suhail Ali, Jason Caulk, Amos Long and Shawn Wallace
Key Findings:
- Reducing turnover represents a significant savings potential for RevLocal
- All of the team’s ML models showed improvements in predictive ability, but high model accuracy did not necessarily equal good predictions due to false positives (people predicted to stay who did not)
- A model based on employees’ gross pay, number of accounts converted, ratio of client demos resulting in sales versus rejections, and the number of demos that resulted in follow up but not sales were predictive—but so was the month they hired the employee.
Key Recommendations:
- Strive to create a workplace that fits employees rather than hiring employees to fit the workplace
- Review whether the financial impact of salesperson churn should change hiring strategy and workplace and cultural formation
- Determine management’s strategy for handling salespeople likely to leave during the next month
ML Team 2
Team Members: Nate Davis, Tracy Dixon, Chris Martinez and Joseph Ours
Key Findings:
- Among employees classified as client-retention strategists, employees with the following characteristics have lower retention rates:
- Lower commissions
- Lower salaries
- Lower bonuses
Key Recommendations:
- Make sure pay plans properly align to the behavior they train strategists to perform
- Explore the amount of influence strategists have over their pay, bonus and commissions
- Review pay parity among strategists with similar roles
Centric India
Team Members: Shiv Mohan, Seema Bansal, Mehani Hakim, Ruchika Gupta and Akshat Kulshrestha
Key Findings:
- Centric India’s ML model was 86 percent accurate in predicting which employees would leave the company
- Employee attrition at RevLocal is age-related, with the highest rates occurring among the youngest employees and declining as age increased
- Attrition is also cyclical, with the highest rates in January, July and December
- Other key factors driving attrition included the employee’s manager and the amount of training received
Key Recommendations:
- Conduct a more in-depth exploratory study of data for better market analysis
- Explore more key performance indicators (KPIs) like total compensation and bonuses
- Employ an Azure-centric approach for data consistency
- Leverage Azure Data Factory ADF pipelines to streamline data gathering processes
Great Lakes
Team Members: Tim Hoolihan, Nicholas Tinsley and Chris Thompson
Key Findings:
- The team’s model—using factors like industry, location, product selection, contract price and client health score–successfully predicted retention while providing information RevLocal can use to intervene and “save” accounts
- However, they must use the model at the right point in the client onboarding process to provide sufficient time and opportunity to intervene
- Gross pay was an important predictor of employee retention, but commission was not
Key Recommendations:
- Use tools like spending trends and survey feedback to monitor client health
- Monitor support requests to detect client problems, dissatisfaction and disengagement
- Remember that client satisfaction is complex and may involve combinations of these factors
Conclusion: People First
While Expedition: Data successfully identified ML and D&A talent at Centric, helped RevLocal solve its business problems, and proved the utility of the Microsoft Azure suite, winning team member Amos Long was quick to identify another key finding: that ML should be used to drive decision making, not to make decisions.
“Our model gives insight into what’s going on with employees at RevLocal, but only managers can make decisions with that insight,” Long said. “It’s important to remember with any data project that we are talking about real people, not just numbers.”
