Training Evaluation: Answering the Question, "So What?"
Training sometimes has a reputation: “This is a waste of time.” Or “I’m never going to use this.” These are the kind of sentiments that learning and development (L&D) leaders are often working against, and not always just from the learner who is being pulled from their day-to-day work. Sometimes, it’s executives at the top, the very people that control the purse strings, who hold these same reservations and resentments.
Whose Fault Is That?
It would be easy to scoff at the resistant learner, or lean into frustration against the executive who doesn’t see the value of training, but is it really their fault they have these preconceived notions? Measuring the impact of training is consistently rated as one of the most difficult and poorly executed responsibilities of training professionals, with Training Industry, Inc. Research showing less than 20% of training professionals say their organization has mastered processes for measuring on-the-job application or business impact post-training.
Given training professionals struggle to tell the story of their own impact, is it any wonder that those looking in from the outside aren’t convinced of training’s business value?
What Does “Good” Training Evaluation Look Like? Considering Training Industry, Inc. Research and ANSI/IACET Standards
Training Industry has conducted over a decade of research into what makes a great training organization. That research has identified reporting and analysis as a critically important process capability for great training, but it’s one that many learning professionals and training functions struggle to master. More than 1,500 training professionals rated the following best practices as the most important elements of reporting and analytics:
- Assess or measure learning outcomes. Every training program should have clearly defined learning objectives. These objectives should be assessed or measured to ensure that they have been achieved.
- Assess or measure application or use on the job. It’s great for a learner to have attended a program, and even better if they liked it and learned something, but that doesn’t guarantee they will apply it back on the job. The best evaluation processes give time for new skills to be applied in the workplace and then measure their impact on job performance.
- Assess or measure business outcomes. Just like application post-training, business outcomes aren’t guaranteed just because behavior has changed. It’s critical to assess what impact changed behaviors had on real business metrics.
- Collect course feedback from learners, managers, instructors and other relevant parties. Each party involved in training has a unique perspective and priorities, which means each brings a valuable piece of the assessment puzzle. Learners can speak to their individual experience, but facilitators may have a better bird’s-eye view. Facilitators know what happened in the classroom, but managers have a better view of what happens back at work. No single perspective should be the end-all-be-all of training evaluation.
- Develop meaningful dashboards or reports. Many of the people who need to understand the value of training, especially executives, have about a million and one other things on their plate to concern themselves with besides training metrics. For these busy people, quick, personalized dashboards that cut to the core of their priorities will take your evaluation to the next level. For example, facilitator dashboards can quickly show rosters and pretraining scores to inform their facilitation prep, training team dashboards can show post-course evaluation ratings so that any problems are quickly flagged and executive dashboards can focus on business impact metrics.
- Collect cost data (direct and indirect). The business will want to understand what training is costing them. No impact story can live in isolation without the story of investment, so having a clear understanding of what your programs are costing the business, and being able to justify those costs, will position the training function as a business asset.
- Manage data from multiple sources. Reporting and analysis require synthesizing data from as many sources as possible to build a complete picture of impact. No single metric is perfect, but taken together, factors like course evaluation data, learning management system (LMS) data around use and participation, manager feedback, business performance metrics and other available data can reinforce and validate one another while providing important context to inform decision-making.
Together, these best practices highlight the importance of taking a holistic approach to training evaluation and not relying on assumptions about how learning may translate into impact. These same principles are reflected heavily in training portfolios geared at training professionals. For example, Training Industry, Inc.’s Certified Professional in Training Management (CPTM™) program includes an entire module dedicated to the role of measurement in training management. For a deeper dive, Training Industry offers a two-day certificate course that directly targets measuring, isolating and reporting on the impact of training. If you’re curious about your own measurement capabilities, you can assess yourself with Training Industry’s Training Manager Competency Model Self-Assessment.
Additionally, ANSI/IACET, which sets the Standard for Continuing Education and Training, also emphasizes evaluation and impact measurement. Its standards require organizations to establish performance criteria, verify learner achievement of outcomes, communicate results to learners and maintain documented processes for evaluating both learning events and the training function as a whole.
Overcoming a Lack of Data Access
At this point, you may be thinking, “That all sounds great, but it’s not actually possible for me.” Many learning professionals struggle with not having access to the data they need to help inform their analysis processes. Data can be gatekept by other organizational actors, often for very legitimate privacy concerns. To ensure you have access to the data you need, build relationships with data managers and have a clear purpose for how and why you intend to use the data you are asking for. Additionally, consider where you can anonymize and even aggregate data to minimize personal or sensitive information.
Getting Started: Understanding Measurement Frameworks
If you’re new to training evaluation and looking for a place to get started, you may be happy to hear that you have the benefit of decades of research that have resulted in several models that can provide you with a starting overview and structure. The classic, most well-known model is the Kirkpatrick Model, but there are also more modern frameworks that address some of the blind spots of the Kirkpatrick Model. Training Industry has worked to translate the most up-to-date standards for training evaluation into the easy-to-understand 5-4-3 Model, which outlines five types of consumers/stakeholders of training evaluation data, four reasons to evaluate training that shape the form of both the questions and answers and three classes of metrics that can be used to provide insights to stakeholders.
A Final Thought
Perceived lack of training value is often not a learning problem, but rather a measurement and communication problem. As a training professional, you already know you need to provide best-in-class training, but make sure to never lose sight of the many inquiring minds who want to know, “So what?”
About the Author
Alyssa Kaszycki is a learning product manager at Training Industry, Inc., where she develops courses that help training and development professionals build more effective learning organizations. Her expertise includes learning strategy, course development and evaluation. She also contributes to industry research and thought leadership on emerging topics including artificial intelligence and careers in learning and development.