LEVEL 6 - WHAT IS IT?

After many years of conducting impact studies with tens of thousands of trainees, I came to one rather conspicuous conclusion – you can’t measure the true effectiveness of training until you know what the climate is like.  And you can’t figure out what the right climate is like until you know how to measure training impact.  Because the effectiveness of training cannot be measured in a vacuum, without also evaluating the climate that influences that effectiveness, I’ve added what I call a Level 6 “transfer climate” to the traditional five levels of evaluation. 

While the 5 traditional levels are critical to any evaluation strategy, they don’t give you the full picture of training success or failure.  Nor do they tell the most important and dynamic part of the story.  That is, they only provide a static report indicating whether training “worked” or didn’t work in the past tense, but don’t make any suggestions or recommendations on why or how to maximize that impact and make the same training more successful in the future. 

Additionally, a Level 6 can explain the variability of training effectiveness across the organization.  It arms you as the training professional with a critical narrative for your business clients – why does training work for Mary and not for Joe?   By uncovering and explaining this type of variability in training effectiveness, you are able to inform your business partners what THEY can do different as business leaders to drive and maximize their own training dollars.  They are the ones that can create and drive the right “transfer climates” for Mary and Joe back on the job.  After all, leaders spend a lot of money putting their employees through training - isn’t it only fair that we give them some control over the outcome? 

In all my years of presenting Level 1 to 6 evaluation results, I have yet to encounter a business group that does not find the results about their transfer climate the most interesting and informative piece of the presentation.  When I tell my audience I’m giving them Level 1 to 5 results for their training, of course they’re excited - less than 4% of all companies are doing it and even less do it right.  But when I tell them I’m also giving them Level 6 results to explain why training succeeds or fails back on the job, that’s what really gets them to the edge of their seats.  Imagine being able to say to your business partners, “We understand this training was more effective for some employees than others, we know why it was more effective, and this is what you can do to make it more effective in the future.”  Now what business leader wouldn’t be happy with that little tidbit? Welcome to Level 6. 

PLANT YOUR TRAINING and WATER OFTEN

After years of conducting impact and ROI studies for some of the top Fortune 100 companies in the world, it became glaringly obvious to me that the biggest factor predicting the effectiveness  of training programs had little to do with the quality of the training, the method by which it was delivered, or the nature of the audience.  In fact, it had everything to do with the immediate work environments employees were returning to right after their training was done.  Every employee takes his/her new knowledge and skills from training back to their own unique work environment. And depending on all the environmental influences in that work environment, they are either inspired to or discouraged from applying that training on the job. That is, based on these critical environmental influences, the same training program can work wonders for some employees and for others it becomes a complete flop.  And because these post-training "climates" are so diverse and unique for each employee, it is absolutely crucial to measure them, understand which factors are most common across your organization, and which climates will be the most powerful when it comes to maximizing  the impact of your training. 

While these climate factors certainly hold the key to defining a training success or failure, they have also been responsible for making training so hard to measure over the past few decades.  Because there can exist such a huge variety of climate factors within and across any given organization, and because they can influence employees in very unique ways, the results, business impact, and ROI of any one training program can be inconsistent and hard to quantify in any reliable way.  However, by including some form of climate assessment within your measurement strategy, you can evolve the conversation from "How did training work?" to "How do we make it work better?"  By adding this layer of climate measurement, you turn a static post-mortem report about the  impact of training into a living, dynamic report about why training worked better for some rather than others, and how you can improve that impact in the future. 

For this reason, I added a Level 6 - Transfer Climate analysis to the traditional five levels of evaluation.  I call this new level the “transfer climate” because it represents all the factors that can either help or hinder the “transfer” of learning from the training experience back to the job.  As with anything you try to transfer, plant, or replant somewhere else, no matter how strong the leaves or how pretty the flowers, you still need just the right soil with just the right sun, and just the right amount of rain. So too with training, no matter how thoughtful and well-designed the training, you still need to make sure all the right factors are in place to care for and nurture it through that critical and precarious incubation period.  If the climate is suitable for learning transfer, the training sticks, thrives on its own, and ultimately blossoms into fruitful returns.  If the climate is not suitable for learning transfer, the training impact dies there and all your time and resources wither on the vine.  So if you want your ROI to grow, make sure you plant it in the right climate and water it often. 

SCARED TO MEASURE IMPACT?

What's the problem?  Why do organizations and their training groups  end up with nothing to show for their multi-million dollar investments in training?  Why aren’t business leaders making an aggressive and determined attempt to measure and report the true benefits of their training programs?  The short answer is simple - they lack the expertise and confidence to present their results to their C-levels and stakeholders.  In a recent study, it was estimated that over 95% of organizations feel the real need and urgency to demonstrate the impact and bottom-line value of training, but less than 5% feel confident in their ability to measure and report that very same business impact.  The paradox here is that ROI numbers are so important to the business that most training organizations are too afraid to present them.  They want to get it right so bad, and they are so deathly afraid of getting it wrong, that they end up presenting nothing!  This paralysis and lack of data is quickly interpreted as ineffectiveness and only fuels the already prevalent notion, especially amidst the more skeptical business leaders, that most employees’ training doesn’t “work” and is simply a frivolous waste of time and money. 

Ironically, another reason why HR and training groups are not doling out the impact studies is that they have historically been given somewhat of a “free pass” when it comes to proving their worth and justifying their expenses.  Because they represent, advocate, and spend much of their resources on developing the human side of the business, organizational leaders tend to be cautious about taking them to task and demanding irrefutable evidence that every single one of their initiatives are turning a profit.  No leader wants to advance the notion that investing in people is a “bad” idea.  Giving even the slightest hint or inkling that you don’t want to develop and enrich your employees can surely put you on the fast track to early unpaid retirement.  As most leaders realize, human capital and capital gains are better kept in separate conference rooms.   While this historical treatment has been a blessing on the one hand, it may also have over the years, rendered us just a little bit more relaxed when it comes to proving our value to the business.  And although the burning platform has been ignited, and will be burning a lot faster in the future, the sheer lack of fire for all these years has simply left most training organizations inexperienced at the art and science of defining their worth.  

When you put these two factors together (lack of confidence and no experience), you end up with an inevitable scenario where HR and its internal training groups are just not able to compete for resources as well as most of the other functions throughout the organization.  Organizational leaders and stakeholders must allocate their limited capital and make hard decisions about where they want to invest their money.   It should come as no surprise that they will be focused on what will provide the largest and surest return on their investments.  Obviously, the business groups and functions that can show a great ROI track record and present their budgetary needs with the promise of demonstrable returns will be the ones getting the lion share of resources.  

BUILD A BRIDGE AND MEASURE YOUR ROI

In order to show the true benefits of training and ultimately prove that it “works”, we need to build a clear and coherent path from content to results. We need to build a strong solid bridge that tells a story of impact and carries trainees from their original investment to an even higher return on that same investment.  Ideally, we must begin building this bridge before the training (our blueprint or business case which aligns content to business objectives) and then finish building the bridge after training to measure its effectiveness.  Without this bridge to define all the glorious benefits of training, you’ll forever be presenting the only tangible numbers you’re sure to have - the costs. 

In my new article in the 2014 Fall issue of Training Industry Quarterly, I'll show you how to start building this bridge and how to get the biggest "ROI" from your training...


Training Industry Quarterly

http://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014fall/index.php?startid=40              

TRAINING or VACATION?

Would you rather pay for a two-day off-site leadership training program or purchase a weekend vacation package to Venice?


It sounds like a silly question, right? We’d all choose the vacation. But why do we laugh, smirk, and really think it’s such a silly question? If we truly believed that the training “worked” and would make us better, more productive leaders, wouldn’t it be the right investment? After all, being a better leader could surely lead to a promotion in the near future that could mean an increase of $25,000 in annual salary. This could mean ten trips to Europe! The real problem is that deep down, we may all harbor a little skepticism (or a lot) over the ultimate payoff of training and that’s because we’ve never been bombarded (or even slightly peppered) with the evidence of impact and ROI. 

So what’s the problem? Why do employees and their businesses end up with nothing to show for this huge multimillion dollar investment they make in training? Why aren’t organizations making an aggressive and determined attempt to measure and report the true benefits of their training programs?

The short answer is simple—they lack the expertise and confidence to present their results to senior business leaders. In a recent study, it was estimated that over 95 percent of organizations feel the real need and urgency to demonstrate the impact and bottom-line value of training, but less than 5 percent feel confident in their ability to measure and report that very same business impact.

The paradox here is that ROI numbers are so important to the business that most training organizations are too afraid to present them. They want to get it right so bad, and they are so deathly afraid of getting it wrong, that they end up presenting nothing! This paralysis and lack of data is quickly interpreted as ineffectiveness and only fuels the already prevalent notion, especially amidst the more skeptical business leaders, that most employees’ training doesn’t “work” and may just be a significant waste of time and money. 

Perhaps one day, after we've truly started measuring and evaluating the impact of all our training initiatives, and convincing both business leaders (and ALL employees for that matter) that training is in some cases a tremendous investment with incredible returns, maybe then we can ask the "training or vacation" question again.  And we may not ever get employees to stop laughing at such a ridiculous set of options altogether, but if we get them to pause one second longer or laugh just a little bit softer, perhaps we've done our jobs and contributed to the "measurement movement".   

WHY MEASURE?

 

Organizations in the U.S. alone spend over 150 billion dollars on training and employee development every year.  With that type of investment, coupled with the intense focus and unflinching scrutiny most businesses place on the bottom line, one would naturally assume that the “payoff” or ultimate benefits (e.g. increased productivity and revenue) of all these training programs are being rigorously measured and monetized down to the last penny.  Surprisingly, this assumption could not be further from the truth.  One of the biggest conundrums in the corporate world today is why organizations are not measuring the business returns on these colossal training expenditures. Even the most bottom-line–minded businesses, the ones that pinch and squeeze every penny to show profitable growth year over year, seem to be oddly accepting and complacent with their inability to demonstrate an aggressive return on their training investments.  In fact, the vast majority of organizations don’t even take the most fundamental steps to determine if the training is even “working”.  That is, does it actually improve the knowledge, skills and performance it was designed to improve?  Are employees more productive than they were before the training?  Are they doing anything different back on the job? 

Granted, turning all the benefits of training into solid, top-line dollar values is no easy task, but most organizations don’t even try.  They collect little or no data to connect training events to employee behavior, they collect no data connecting employee behavior to business results, and they certainly never come close to building a sound ROI case for their training investments.  When we look at the magnitude of the these training budgets compared to the amount of comprehensive impact studies that are being done, we are instantly faced with the irrefutable fact that other-wise smart, savvy, profit-focused organizations are sending millions of employees through training experiences, spending billions of dollars on training every year, and quite literally have nothing to show for it. 

You don’t need to be a Finance guru to know that without some visible and tangible benefits on the top line you’re always going to have a disappointing and dismal bottom line.  Instead of focusing on the value added by the training, business leaders and stakeholders will always be forced to focus on what they can see and touch - the expense of training.  After all, the only thing they have in front of their faces are the costs.  And to make matters even more challenging, companies have always and will continue to go through great lengths to define all their costs in exhaustive and excruciating detail.  So, using the simplest Benefits Costs Ratio analysis (BCR), what almost all organizations end up with are some very vague or absent numbers above the line and some very real and punishing numbers below the line. What kind of ROI case can you make with this data?  If we ever want senior executives to think of training as a profitable business imperative, instead of a distractive and costly diversion, we need to define these benefits in detail, monetize them, and create powerful stories of impact and ROI.  Put simply, we need to give them “something to show for it”. 

This blog will be devoted to helping all training and business professionals tell their story of impact.