In recent years, compliance benchmarking has become very popular in heavily regulated industries such as banking, health and aged-care. But how useful are these schemes, really?
The truth is most organisations, particularly those in the service-sector, achieve compliance in totally different ways, so setting benchmarks based on industry-best-practice makes little sense. Furthermore, compliance benchmarking is subject to a number of serious limitations that if left unaddressed can render its outcomes meaningless, and the exercise a complete waste of time and resources.
In this article, I outline why compliance benchmarking has developed such a poor reputation, and why organisations are better off benchmarking their compliance internally rather than externally.
About Benchmarking
Benchmarking was introduced by the Xerox company way back in the 1970s and became popular in the 1980s as part of the total quality management movement.
An IBM executive once defined benchmarking as;
“the ongoing activity of comparing one’s own processes, product, or service against the best-known similar activity, so that challenging but attainable goals can be set and a realistic course of action implemented to efficiently become and remain best of the best”.
There are two main categories of benchmarking; performance and practice.
- Performance benchmarking involves gathering and comparing “quantitative” metrics or key performance indicators (KPIs). For this type of benchmarking to operate properly standard compliance metrics and measuring methods are required, and a reliable model of collecting and analysing data.
- Practice benchmarking involves gathering and comparing “qualitative” information about how an organisation achieves specified compliance outcomes through people, processes, and technology. For this type of benchmarking to operate properly reliable methods of gathering, assessing and comparing information is required, such as process-mapping.
Benchmarking can also be broken into two further types; internal and external.
- Internal benchmarking involves comparing the compliance performance of one part of an organisation with other parts of the same organisation i.e. departments, branches, groups, facilities, teams, vendors, etc.
- External benchmarking involves comparing the compliance performance of one organisation with others; usually in the same industry, which can include competitors. This type of benchmarking is also commonly referred to as industry-based benchmarking.
About Compliance Benchmarking
Compliance benchmarking is a type of benchmarking that focuses on an organisation’s ability to comply with specified laws, regulations, standards, agreements, etc.
Most industry-based compliance benchmarking schemes are government-operated and participation is mandatory, but there are independent schemes run by industry-groups and software-vendors that organisations can subscribe to voluntarily.
Limitations
For me, there are many limitations that cause compliance benchmarking schemes not to work, especially industry-based schemes, but the biggest limitations I’ve listed below.
- Unreliable Data
For compliance benchmarking to work properly each party must accurately measure and report their compliance performance in exactly the same way; which is difficult enough when assessing the compliance of one organisation, let alone many.
One reason why compliance benchmarking schemes often fail is they rely on compliance metrics that are open to interpretation and variations in measurement methods.
For example, assessing whether a health-provider has delivered an “timely” level of care to its patients is often open to different interpretations of “timely” and different methods of measuring “timeliness”; which can easily lead to misleading benchmarking outcomes.
- Lack of Context
Another major limitation of industry-based compliance benchmarking schemes is the data is normally stripped of its context, meaning it tells you what other organisations have achieved, but not how they achieved it.
Then there’s the problem that some metrics can be skewed by one-off incidents and accounting exceptions, which aren’t made transparent in benchmarking reports, for example operating costs.
Without supporting compliance evidence benchmarking’s ability to help improve an organisation’s compliance performance is severely compromised, which can significantly reduce its value as a management tool; refer to point 4.
- Incompatible Modelling
Another reason compliance benchmarking doesn’t work is that most schemes rely on a standardised assessment model that is often not matched to the unique operating characteristics of each contributing party.
To be effective compliance benchmarking must be capable of comparing apples with apples, not apples with oranges.
It may sound overly simplistic, but this is a common limitation. Overly simplistic models can also lead to policies and processes that are ineffective, exposing organisations to unacceptable compliance risks, including; regulatory fines, penalties, reputation damage, lost revenue, etc.
- High-Cost / Low-Benefit
No doubt the biggest limitation of compliance benchmarking is its high operating costs compared to its low benefits. This in part is due to the requirement for all parties to collect and report on the same standard metrics; regardless of whether they add value, or not.
For example, a health-provider has to allocate the same amount of time and effort collecting and reporting data on the “timeliness” of its services as other providers, even though it has determined in the past that this is not a compliance risk.
On top of this is the added expense of trying to analyse compliance benchmarking reports for actionable improvement insights, that aren’t there, and in some cases paying exorbitant subscription or levee-fees for a service that delivers little to no value.
Despite its limitations, many organisations continue to use industry-based benchmarking solutions to assess their compliance, set -goals and measure improvements.
Little wonder many of these organisations are searching for a better solution. So, what are the alternatives?
A Better Approach
The first critical step when seeking to improve how your organisation assesses and manages its compliance is to view benchmarking as only a part of the solution, not the complete solution.
Wanting to know how your organisation compares with others is understandable. Unfortunately, in my experience, trying to benchmark your organisation’s compliance performance against others is a waste of its limited resources and will do more harm than good in the long run.
Compliance benchmarking against yourself is a much more efficient and effective option than industry-based benchmarking.
Done well, internal benchmarking can help unearth opportunities for improving compliance processes, practices and performance that external benchmarking can’t. Ultimately, the best person to understand where your organisation’s limited benchmarking resources should be directed is you, not an external regulator or vendor.
Be sure that when you invest in or subscribe to a compliance benchmarking service that you limit your scheme to objective compliance metrics that cannot be misinterpreted.
If you have to include subjective compliance metrics, which is often the case with service-based organisations, make sure your solution provides clear instructions on how each compliance metric is to be assessed, and enables assessors to attach improvement recommendations and supporting compliance evidence alongside their results.
Compliance benchmarking can be beneficial in helping organisations monitor and improve their compliance performance, but it does have its limitations. Understanding and managing these limitations is the key to achieving the best value from your compliance benchmarking solution.
For more information on how to optimise your organisation’s compliance benchmarking processes visit www.compliance-master.com.
