Confusion Around Leading and Lagging Metrics

 

The confusion with leading and lagging metrics stems from the fact that we cannot truly make a list of metrics that fall into each category. Some people think you can but it depends on your perspective. What is leading Key Performance Indicator (KPI) to one level of the organization can be lagging KPI to another. Thats right, many metrics or KPIs are actually both leading and lagging. As an example, in the operations and maintenance world, preventative maintenance (PM) completion (which measures the percent of scheduled PM task done when scheduled) is a good example of this confusing element. If you are an operations leader you might call it a leading indicator of equipment reliability. In other words, if I am doing my PMs on time then I should expect reliable equipment. ...But if I am a Maintenace Supervisor then it is a lagging metric or KPI that shows that we did the work that was scheduled. 

As you are selecting metrics another way you can look at them is by using "Behavior and Result." Does this metric drive a behavior, or does it measure a result? My opinion is if you want to drive behavioral change you want to measure things that drive that specific change. You may find that you measure them often maybe even daily or weekly whereas results type metrics you measure monthly or less often to make sure you are getting what you want from the process. 

What are your thoughts?

Is The MTTR Metric Killing Your Reliability?

Metrics or Key Performance Indicators (KPI) are a force for good when they are used at the right time and with the right complementary or supporting elements. But, when they are used at the wrong point in a facilities maturity they can have unintentional consequences, or even worse they can drive the wrong behaviors.
An example that I continue to see causing more issues than it is solving, is the use of the KPI known as Mean Time To Repair (MTTR). When this metric is used alone, in immature organizations, or without an understanding of the unintentional consequences it can drive your organization in the opposite direction of world class performance. If your organization is immature from a reliability cultural standpoint and you choose MTTR as your focus then you set yourself up to become very reactive by being very quick to respond to failures. The facts are:

• Reactive response is at least 5 times more expensive than planned and scheduled work . 
• Operations will beat on you to get faster and faster at responding so that you lower MTTR. 
• Rushed repairs are less reliable.
• Reactive response requires more expensive spare parts stock.
• Repetitive failures and repairs increase the chances of the introduction of infant mortality failures. 
• You will find yourself with high skilled maintenance technicians just standing on the manufacturing floor doing nothing while waiting on a failure to occur. 
• Pressure to make the repair as quickly as possible can lead to taking elevated safety risks either intentional or unintentional. 

Many of the sites that choose MTTR as a primary metric early in their reliability journey create a brigade of firefighters on the ready with crash carts and mounds of spare parts. What we really want is to prevent the failures from happening to start with or at least reduce the frequency. For that we might use other metrics like Mean Time Between Failures (MTBF) early on and then deploy MTTR, after we increase the reliability of our systems. This later use of MTTR will allow us to address the issues we can't prevent and to understand any training gaps and other issues that might be affecting repair times.
Picture it this way: If MTTR is all you have then your organization will create tools like crash carts and quick response teams instead of using tools like Root Cause Analysis (RCA) to understand and eliminate the reoccurring problem. From the real world, I have seen bearing quick change carts developed to speed up re-occurring failures repairs where it they had just tensioned the belts properly the failures would have been eliminated.  Five really fast 2 hour bearing replacements is still much worse that bearings that don't need repairs at all. This site needed to understand better not respond quicker.
Are your metrics driving the wrong behaviors? Are you using them at the right time?
Tell us what metrics have not worked for you and why in the comments below. 

Benchmarking and Internal Assesments: Eight Questions That Will Improve Your Studies and Their Results

If you saw last weeks post on the troubles "benchmarkers" deal with here then you may be wondering how do we make it better? If you are addressing the issues mentioned there and you want to take it to the next level here are eight points to consider when assessing your performance and benchmarking with others. These are based on the seventy plus assessments I have been involved in during the last ten years and the struggles I have seen during those and other assessments.
Does your assessment or benchmarking study have the following: 
an element of Volume – Does the assessment have more than just yes or no questions? Does it look into the amount of application of the concept or element or just the existence? For example, does it ask "Do you have planners?" or "What percentage of work is fully planned?" or as a second example "Do you use work orders?" or "What percentage of your craft hours is captured on a work order?" The second question in each provides the element of volume. This demonstrates the penetration of the concept into the way the site does business not just whether they have pockets of excellence in one area.
Detail - Do you have enough data and detail in each of the sections or elements of the assessment? Do you get a complete picture of the components that make up the element being assessed?  For example: Can you address all of the elements of Planning and Scheduling effectiveness with just one question in your study? Not likely. You will need more details and data in order to identify actionable gaps for closure.
Frequency – Are you doing it often enough? If you are only assessing or benchmarking once every ten years that would show little trend data. I would recommend a robust external assessment every three to four years for maximum effect. External does not have to be a consultant driven process you just need fresh eyes from another site or division.
Process Standard – Do you have a process and standards for the benchmark assessment so that the data is of like kind and comparable with other locations?
Personnel Standard – Do your assessors have the knowledge and competency to assess? Are they task qualified in the process standard. Are they standardized? Will the non-task qualified assessors affect the comparability of the data where they are involved? I have seen this over and over. Having a standard for performance and qualifying individuals is crucial to effective benchmarking.
Performance – Is the assessment having the desired effect? Is the assessment providing a view into the gaps that you need to address? Is the study executed at a level that identifies next steps to close the gaps. Is it driving the organization to close those gap? Is the organization better as a whole because they did the study?
Efficiency – Are you doing the study efficiently? Is it a major disruption to the business or can it be accomplished with minimal interruptions. Is your assessment or benchmark study addressing the elements required without being exhaustive? Is it full of manual calculations or does it standardized on base data that is used to generate the various calculations "automagicly"?
Innovation – Are you finding a better way to do it? Have you improved the assessment and benchmarking process each time you have executed it? This could be through data standardization or tool improvements or collected history.
This become the checklist that I review to ensure the highest level of success with benchmarking and assessments. Every time I use this list I am upping the return on effort of the study.

Are you happy with your assessments and benchmarking studies? Tell us about what you like with yours below in the comments section please.

Best Practice Metrics Performance Requires Best Practices...

Over the years I have worked with many sites all over the world specifically to help them with assessments of their "As is" state. These were in preparation for their reliability, safety, and operations improvement initiative that will help them attain their "to be" state and the performance gain that is associated with that improvement.
The assessment process has many different variations depending on the site and the topic but all of them should have three things in common. The three are:

• a metrics or key performance indicator review

• practices and documentation review

• interaction with as many of the affected and involved individuals as possible to understand application, culture, and norms 

If you are lucky then all three will tell a similar story and you can begin to plot a course to the "to be" state and the value of that progression. This does not always happen. Sometimes two will agree but one does not support the others. It is a bit like looking down at the dash of the car and seeing that your oil pressure is low and you are idling at 500 RPMs but your speedometers says you are traveling at 145 MPH. This indicated disconnect is concerning. Let me give you two industry examples. In the first example, let us say you are assessing a facility and you find well documented business processes but mediocre metrics performance and when you go to the plant floor and speak with the crafts, operators, and the supervisors you realize they don't know anything about the processes or the metrics. Or another possibility, maybe you find that the metrics show great performance but there are no or very limited business process and from the discussions with the site personnel you discover the site culture does not support the current level of metric performance or suggested reliability. This is where the improvement process can bog down. In the first example you would need to investigate to truly understand, but it appears on the surface that the business processes may have come from somewhere else and were never embraced or lost support for what ever reason. In the second example, the metrics may have been "optimized" over time which can cause them to get to a point where the math that is used to make them look so good is beyond science. I worked in one facility many years ago that regularly ran with an overall equipment effectiveness or OEE of 115. They ran a 115 with mediocre practices so there was still room for improvement. If they got it all right, there was potential to see 125 on a 100 scale metric but alas it did not happen as far as I know. Maybe they decided to change the math.
My point is that if the three areas of focus do not match you may want to use some root cause analysis tools to find the underlying causes. Once these are found then you can explain the disconnect between the three areas. The second step is to help the site see the mismatch in metrics, practices, application and culture. I like to use our maturity matrices which you can get here to help with this element. The site has to understand that world class performance on metrics has to be supported by world class performance in business process, culture, and application. Once they are comfortable with this then they can begin to embrace the true "as is" and discover the true potential that they can capture in the gap.
Have you experience the mis-match?

Confucius say: Planning and Scheduling Cause Confusion

The topic of planning and scheduling and what each one means has been a big discussion point here in China this week. The Chinese appear to have only one word that means both planning and scheduling but in reality everywhere I teach from Europe to the Middle East and all across the Americas there is confusion  on the difference so today I thought I would share planning and then scheduling as the topic of the week.



Planning is the creation of the work package including:
Job plan and steps and durations
Parts required
Special tools required
Permits required
Personal protective equipment required
Crafts or personnel types required
Etc.
Scheduling is assigning the jobs to a time slot on a schedule.

The reason that this is important is that many of us measure percentage planned work for bench marking purposes and in more cases than not this number is actual the percent poorly scheduled because when you dive in you find that the jobs do not contain the planning elements mentioned above.
Many sites pursue ninety percent planned and scheduled work as a bench mark but you have only reached that number if you have at least the basic elements of a job plan for each job.
One example that I see often would be a work order that says only "remove and replace hot oil pump Time required 8 hours" this is not a plan at best it is a work request that has been scheduled without planning. It does not break down the task or suggest what parts will be needed or what permits might be required. While some might argue that all work is planned and that it is just the level of planning that is in questions I think you can see that it does not meet the intent of the metric and does not provide for the results we are trying to achieve. 

Have Your Corporate Metrics Become a Time Sponge?

The corporate quest to standardize metrics can become a time sponge absorbing countess hours and distracting facilities from the real focus of bottom line performance improvement.  Today's blog is about discovering how to keep KPIs from soaking up all of your precious time and focusing on driving changes in behavior not just the numbers.
There are metrics and Key Performance Indicators (KPI) standardization efforts happening all over the world. They are happening in plants and corporate offices as well as within organizations such as The Society of Maintenance and Reliability Professionals and the European Federation of National Maintenance Societies. Many of these groups and other authors have spent endless pages defining these metrics. While I believe their work is incredibly valuable for individual facilities to understand their current state, what I commonly see is the documents they produce only lead to further internal corporate argument. Some companies have spent years and countless hours on the quest to define metrics like Overall Equipment Effectiveness (OEE) even with the standards that already exist in the market. It becomes a discussion of what time goes into the calculation, what time is out, what speed is the best demonstrated throughput, how do we define quality product and so on. At the end of the day an apples to apples comparison of different sites can be tough if the metric cannot be pulled from a standard data source such as the ERP, EAM, or Profit and Loss (P&L) statements. The EAM data is also suspect unless many of the steps of the business processes have been standardized from site to site to provide consistent data.  I believe one solution is to get it close but don't take the effort to argue through all the details and then compare sites on a delta basis. So instead of reporting just your percentage OEE or any other metric, report and focus on the change in the value from quarter to quarter or month to month. If we all agree that the goal is to improve then let's compare the efforts to improve not just the numbers. If one site is able to increase their OEE by a full point in one quarter then do an RCA to understand why and learn from them. What behaviors have they changed? What processes have they refined? Focus on the Root Cause of Success and spend the time making behavioral change not debating corporate metrics. Wring out the time sponge and get back hours to devote to real results.