Technical Debt: Why is Such a Simple Change Taking So Long? 

Are you paying for decisions made in the past, with increased costs and complications? Learn how to avoid this technical debt in the future.

Part of a blog series.

Technical debt is a cleansed and politically correct term. If you haven’t heard it before, a more accurate description of it in the data world would be “stuff we know is needed but isn’t done yet.” 

Given that a picture is worth a thousand words, let me paint one for you.

The Scenario

Imagine you’re a commercial lines underwriter. You’re looking at expanding an existing custom commercial multi-peril product, which will require a detailed analysis of your existing book of CMP business.  

As part of the analysis, you’re going to want to look at a series of underwriting questions and third-party data that the policy management system tracks. Most importantly, you’re going to want to see a lot of this data available on a regular basis to track the success of the product expansion. You know this data exists, it’s in the core system you’ve been writing this business on and you just want access to it. Simple right? 

Enter Technical Debt

You submit your request to your information technology team and a meeting is scheduled. If this is your first time making this type of request, you probably walk in excited.  

You’re looking forward to the business value of the product expansion and this meeting is going to get you the critical support you need to make that happen. By the time the meeting ends, a lot of your excitement has likely been replaced with concern, frustration and disappointment.  

During that meeting, you likely were exposed to the concept of technical debt in a fundamental, non-specific way. In short, the complexity of the existing data architecture was explained in enough detail to identify a series of hurdles that would have to be cleared to make this request happen, effectively taking this from “simple” to “complicated.” 

Once that was understood, the conversation of available resources and existing prioritized work comes up and “complicated” becomes “complicated technically and politically.” 

What Just Happened

So, after this experience, you probably walked out of the room feeling confused – and that you and IT are at odds. The unfortunate part about this experience is that everyone involved wants this initiative to be successful and as transparent as possible to make sure the right expectation is set. However, what you didn’t realize walking into that meeting, is your product expansion is going to cost a lot more than you thought, and you’re not sure why.  

The reason for this is the especially vague nature of “technical debt.” 

As an underwriter, you’re likely very familiar with project–specific technical debt. For example, you’ve probably run into minor screen tweaks on your policy management system going on a backlog. That became technical debt because while they are considered a defect, their potential impact is low enough that fixing it has been deferred.  

However, there are other forms of technical debt that are far less clear and whose implications are much further reaching. One of those forms of technical debt just created a large headache for you and the only way to avoid it doing so again in the future is to shine some light on it.  

Data Technical Debt

The most common form of technical debt in the world of data is foundational. To understand what that means, here are a couple of core concepts we need to agree upon: 

1. All data is valuable.
2. Not all data is of equal value and analysis is required to determine exactly how much value it has.
3. Data gathered is not automatically available for analysis.

So, to put this into the context of our hypothetical scenario, the answers to underwriting questions have value. How much value they have is subject to interpretation and requires analysis to prove. That data, while it may be gathered at the time the policy is underwritten, is not automatically available for analysis. It is instead buried in the policy management system which shouldn’t be available for direct access to perform analytics.  

Given this and the general drive for projects to focus on value-add deliverables, the effort necessary to make your underwriting question data available for analysis at the time they were implemented was differed and became “technical debt.” In short, this leaves you paying for decisions made potentially years before and, in that time, many factors have complicated the implementation and increased the cost associated with that decision. 

So, Now What

If you’ve run into this example or something similar in the course of your career, hopefully, this shines a little light on what happened and potentially why. If you’re curious about how to avoid this happening in the future, read on! Above we referenced the three core concepts that generally provide context to Data Technical Debt. What we didn’t cover is how those three concepts have fundamentally created a lot of technical debt over the years with P&C insurers. There has been a long-presumed premise in enterprise analytics that “all data we deliver must be governed.” 

That means any data we provide for analysis must have a standardized name, definition, underlying formula, cleansing rules, and must be published into a centralized platform (usually called an Enterprise Data Warehouse). As you can imagine, governing data costs money and if you want to be smart about where you invest, you don’t want to spend that money unless you know there’s value in it. So, let’s think about that for a moment:

I know data is valuable, but some data is far more valuable than other data. And I don’t want to spend money governing all the data, just the data that is of higher value. However, I don’t have a true answer on which data is more valuable, or how much more valuable it is without doing the analysis. Unfortunately, as noted above, I can’t do the analysis because I don’t have access to that data in the first place. That fundamental chicken or the egg situation is at the core of most data technical debt.  

It also clearly disproves the tenant that “all data we deliver must be governed.” So, what should the new tenant be that will help you avoid this problem in the future? 

“All data has value and should be acquired cheaply, quickly, consistently and in a scalable nature. Only data of proven high value should be governed and made available more broadly.”

Defeat Property & Casualty Insurance Challenges with a Modern Analytics Approach – Ebook, Insurance

Final Thoughts

By partnering with IT to formulate a cost-effective approach to acquire all data into a collective “lake” of information that is available for high–powered analysts to evaluate previously undervalued data, you provide an avenue to evaluate and consume data on an as-needed basis. And, by agreeing that this is the going forward standard for any system implementation, you ensure that all data owned by your organization is retained, in the most cost-effective way possible.  

You additionally ensure that any future data need that’s not currently covered by your existing organization’s data strategy can be addressed both in the near term (tactically) and in the long term (strategically). Lastly, it ensures you can raise the bar on the governance of data to ensure that time and dollars spent on governing data is in itself a valuable exercise that is measurable.