The levels model! When you say those three words to anyone that has been in the training space for more than a decade, the image of Donald Kirkpatrick jumps to mind for a reason. The model, which is attributed to him, has been the foundation of training evaluation since the early 1960s. In a manner of speaking, the levels model describes four stages of appraisal that an organization can employ to determine the effectiveness of training:
- Level I: Reaction – The degree to which learners find the training favorable, engaging and relevant to their jobs
- Level II: Learning – The degree to which learners acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training
- Level III: Behavior – The degree to which learners apply what they learned during training when they are back on the job
- Level IV: Results – The degree to which targeted outcomes occur because of the training and the support and accountability package
For close to 40 years, we saw this model and perceived it to flow in this direction:
The implications were subtly profound for those of us that designed and developed training. Conceptually, as you sat down with a blank sheet of paper to draft the blueprint of your learning experience, you started with Levels I and II. The overwhelming amount of the collective effort that went into development was dedicated to the event itself. Would learners learn? Perhaps even more importantly, would they like what they learned?
As for Level III and IV outcomes, they were celebrated, random occurrences. The paucity of success stories and tangible results were openly acknowledged. But, unintentionally, the standard explanation contributed to an escalating gap between the training function and the business it existed to support. For example:
“We can verify beyond reasonable doubt that learners demonstrate skills during training, but, unfortunately, application on the job is well outside the realm of our control.”
During the late 1980s, Training Industry guru, Bob Pike, attempted to address this disconnect by publishing an article which posed the question:
“What if you turned the levels model upside down?”
Donald Kirkpatrick did not initially respond kindly to that suggestion! So, what did his successors, Jim and Wendy Kirkpatrick of Kirkpatrick Partners, eventually do? Exactly that! With Donald’s blessing, they turned the model upside down. The same levels simply needed to be considered from the top-down as opposed to the bottom-up.
In contemporary Kirkpatrick Partners terms, here’s what we know:
- Level I and Level II outcomes are highly correlated. If you truly liked a training program, you probably learned something. If you learned something during training, you probably reviewed the experience favorably
- Level III and Level IV outcomes are highly correlated. If you change your behavior, eventually you will produce different results. If you want different results, that will likely require changing your approach
But, guess what? There is no correlation between Level II and Level III outcomes! You may well have learned something in training, but that is in no way an accurate predictor of pull-through, training transfer or behavior change.
So, now what? Simply stated, if you are a training professional and you aspire to have a seat at the proverbial “big table,” you need to be able to tie in the active participation and pull-through of training to the outcomes the executive management of your organization is accountable to achieve. Your “flow” needs to be top-down, as opposed to bottom-up. The implications of that directional adjustment are transformative!
APPLICATION CHALLENGE
- Consider a training program your organization offers.
- What percentage of the design is dedicated to the stakeholders that support pull-through or training transfer (i.e., the manager of the employees participating in the training)?
- How does this training tie to the organization’s objectives?
- What can you do to make that connection more explicit?
