Training Industry Magazine - Summer 2023 - 41

Why is it " out of this world " ? Because
I recently used it to evaluate training
at the National Aeronautic and Space
Administration (NASA). But before we
go to a galaxy far, far away, let me give
you a little down-to-earth context.
I recently contracted to do training
measurement work for the federal
government. This meant assessing
the impact of training at organizations
like the United States Department of
the Interior (DOI), The Food and Drug
Administration (FDA) and NASA. As this
work began, and we began to scope
the potential impact of the training
(what we call the " story of impact " ),
the critical questions that continued
to arise were " Can we apply a 6-level
model of training evaluation? " and
if so, " What level of impact can we
measure up to? "
The reason why this became a sticking
point was that most organizations that
measure the higher levels of impact (like
Levels 4 and 5) are for-profit businesses
using Level 4 to look for " business
impact, " which typically means an
increase in revenue or cost savings for
the company. This revenue or cost saving
is then compared to the overall cost of
training and a return on investment at
Level 5 can be calculated. This focus on
business impact and ROI clearly wasn't
going to work for agencies that don't
define success in terms of profit and loss.
So how would we apply our 6-level
approach and customize it for this
government training? The general
consensus was that we can certainly
tell our story of impact up to Level 3
(employee behavior changes and onthe-job
improvements that are " mission
critical " ), but when we get to Level 4,
we had to push our thinking away from
how these behaviors impacted the
" business " to how they impacted the
" health of the organization. "
In other words, what organizational
outcomes can we expect to improve by
having all their employees go through
training? It may sound like a semantic
difference, but until we teased out this
distinction between organizational and
business outcomes, the stakeholders
TRAINING INDUSTRY MAGAZINE - SUMMER 2023 I WWW.TRAININGINDUSTRY.COM/MAGAZINE
were not convinced we were shooting
for the right story of impact. Here's
how we defined each level of impact
and how they differed from a purely
business ROI approach.
MEASURING THE IMPACT
OF TRAINING IS NO LONGER
A NICE-TO-HAVE. IT'S AN
ABSOLUTE LEARNING AND
DEVELOPMENT IMPERATIVE
THE LEVELS OF EVALUATION
Level 1 measures participants'
experience
in
the
training. Were
they engaged with the design, the
content and the way it was delivered?
This level of data works for both
business and government, but one
distinction here was our government
stakeholders were especially tuned
into how relevant their training was
to the specific role of participants and
how recommendable it was to peers in
41
https://www.trainingindustry.com/magazine
Training Industry Magazine - Summer 2023

Table of Contents for the Digital Edition of Training Industry Magazine - Summer 2023
