As researchers tend to know, “No response is also a response.” This is an important consideration for compliance professionals, who often rely completely on quantifiable data as the basis for their decisions. To understand the full context behind a compliance metric, it is equally important to go a step beyond the visible and measurable and investigate what close-ended questions do not give you the room to investigate.
Qualitative insights can also help with the ‘fine-tuning’ of GCR programs by expanding or contracting the compliance scope using forward-looking metrics. But none of this is possible if a compliance program functions entirely on the ‘here-and-now’ of quantitative insights. Designing and running a GCR framework is useless if it overlooks qualitative considerations.
Why Quantitative Measures Are Not Enough
Quantitative measures only form half the picture when setting or measuring the effectiveness of compliance metrics. Interestingly, they are based on insights originally derived from qualitative forms of primary data collection.
Seen by themselves, they also fail to explain responses like “Maybe,” “Not Applicable,” and “Never”. In purely quantitative data collection, there’s also the risk of stonewalling, especially when the psychological distance between options is inadequate.
Executives have been known to abuse the purely quantitative nature of metrics to adjust response ranges in their favor. The Likert Scale is a frequent victim. For instance, when asking participants, “How helpful is the monthly compliance review in meeting your targets?” Appropriate responses should include measures such as:
· Extremely unhelpful
· Not helpful
· Unhelpful
· Neutral
· Helpful
· Very helpful
· Extremely helpful
With responses coded 1-7, 7 for “Very helpful.” When the results are published, a high score will obviously mean participants need the meeting.
Now let’s see how the range is distorted:
· Unhelpful
· Neutral
· Helpful
· Very helpful
· Extremely helpful
The researcher simply erased the first two options. So even if respondents want to give their ‘worst possible’ response, it will be ‘Unhelpful,’ with average scores skewed rightwards.
Qualitative Metrics: When To Get Them And How
Qualitative metrics are most useful during change management, as the reasoning and justifications accompanying the figures open a window on what needs to be fixed and in what order. In the pharmaceutical context, ignoring the qualitative aspect means missing out both on issues and opportunities with which regulatory compliance can be achieved. Applicable manufacturers must embark on a wide-angled data collection exercise when undergoing the following:
· Strategy Formulation
· Change Management
· Root Cause Analysis
· Forensic Accounting/Remedial Measures
· Internal investigations
· Repositioning
Fortunately, qualitative insights can be collected without much expense or inconvenience, provided that researchers encourage voluntary disclosures with the promise of data anonymity and respecting research ethics. Starting with workshops explaining the purpose of the exercise and assuring employees of their participant rights, compliance researchers can go a step further with exercises like focus groups, employee depth interviews, and even ethnography. Employee perception surveys can be a great mechanism, both for ‘testing waters’ for one’s compliance programs, as well as collecting employee sentiment about the effectiveness of each compliance initiative.
In the next post, we talk about what to measure to collect the most productive qualitative insights.
