You Can't Fix What You Can't Measure
©2020, George J. Irwin. All rights reserved.

I suppose that I could simply use the title of this entry in the notebook as the entire message. But that wouldn’t be fair, would it?

I don’t know of a single Lean Six Sigma Practitioner who has not run into this situation at least once. I got lucky, if you can call it that… I ran smack into it with my very first official Black Belt Project.

The problem, as it was explained to me, is that there were contractual time limits placed on getting certain things done for a client. If they weren’t finished on time, the client was entitled to a partial refund. My Project Sponsor knew there was a problem because those certain things were not getting done in time. That had to mean that we were refunding money, lots of it.

Right?

Well...

It turned out that there was no information available on the amount of refunding being done, even though it was conceded by the Process Participants (the people actually performing the work) that they didn’t always get done in time. It was further admitted that they didn’t know how many times that they officially didn’t get things done on time, since if something wasn’t done by Friday and it was due to the client by Monday, they’d just work over the weekend to finish up. So even though they weren’t done when they were supposed to be, it didn’t look that way to their customer.

Did they have any data points to share about how many times this happened? Well, no, not really. That’s because there was no incentive to measure this—the staff was rated on what percentage of activity was completed “on time.”

But wait, there’s more. There were a few cases where the client deadline still wasn’t met. That’s where the Client Managers stepped in. One of their key responsibilities was keeping their clients happy, and sometimes that meant convincing them that a missed deadline was “due to circumstances beyond our control” and would be just a little bit late. This worked so often that the clients almost never demanded their refund. And when they did demand a refund, the Client Manager had the authority to simply issue them a credit against the next project instead of actually getting a check written. The result of this, which I verified with the Finance People, was that there appeared to be no refunds at all. This worked just fine for the Client Managers, whose bonuses were significantly affected by refunds but were not as deeply affected by a smaller amount of revenue for the next incremental bit of business.

So, this new Black Belt couldn’t measure what was supposed to be the root of the problem, and the project ended right there. The team that did the work became interested in improving their process so they wouldn’t have to work as many nights and weekends, so we were able to spin off a small side project to accomplish that instead—once we could measure how much extra time they were spending.