Beware of Homegrown Impact Analysis | CLS

Beware of Homegrown Impact Analysis

It happened years ago but has remained firmly implanted in my mind for reasons a qualified neuroscientist could no doubt explain. I was in the middle of a discovery project for a custom leadership training program we were developing for an industry-leading pharmaceutical company. One of the directors (let’s call her Grace) that was on my interview schedule was a leader in the recently formed Pharmaco-Economics Division. The following is my vivid recollection of the introductory segment of our exchange:

Me: Let’s start with this: what exactly is “pharmaco-economics?”
Grace: In essence, we are an internal research entity. We study the comparative value of our therapies.
Me: Compared to “same class” competition I am assuming.
Grace: Yes, primarily. Many of our competitors have introduced “me too” products and try to appeal to prescribers with a price-per-dose strategy. Our studies allow our Commercial Team to prove to our prescribing physicians that our first-line therapies, while frequently more expensive per dose, are also more efficacious. Patients can take less of them to achieve similar or superior results.
Me: What happens when you conduct a study and your comparative value isn’t an advantage?
Grace: We conduct further analysis.

 

I remember two things coming to mind almost immediately: 1) if my organization had invested $200 million to bring an innovative drug to market, I would be highly reticent to release internal research results that provided a copycat competitor with a certifiable advantage; and 2) when it comes to assessing things like value, or return on investment or business impact, it doesn’t make sense to trust a provider to provide unbiased analysis.

Consider this analogy in the context of all that goes into selecting a partner for a leadership development initiative of significance. Of course you’d want to review case studies describing relative experiences and follow up with references to gain further insight. But when the potential partner seeks to establish credibility by way of sharing published results from studies they conducted on their effectiveness of their own solutions, hoist the red flag and start waving it vigorously! It’s like asking the overwhelming majority of grandmothers around the world to objectively assess the skill sets or relative attractiveness of their grandchildren — ain’t going to happen!

So, what do you do if you are looking for proof sources from potential partners with whom you have limited direct experience? We would suggest favoring the following:

  • Internal studies – Many of our customers conduct sophisticated internal analysis of the impact our programs have on things like productivity and retention. They set up control and experimental groups (i.e., separate divisions selling the same products to highly similar customers) then put managers from the experimental division through our Situational Leadership®: Building Leaders offering. Six months or so later, they compare sales numbers and retention statistics
  • Third-party analysis – Costs more (usually the downside) but this is clearly the most reliable method of analyzing the impact of a training initiative. Firms like ADVANTIS Research & Consulting or Brinkerhoff Evaluation Institute (BEI) are examples of organizations in the business of objective examination

Net-net, the more important the undertaking the more you need independent verification of a solution’s potential to respond to your leadership/influence challenges.

 

APPLICATION CHALLENGE

  1. Google “Leadership Training Companies” (or something like that).
  2. Review the proof sources those organizations provide:
    1. How many are “homegrown?”
    2. How many are actual studies run by end users or objective third-party assessments?