The Data and Measurement Issue - 11

Making sense
of the data
Collaboration
Delta, like many organizations, has a lot
of data that are collected and managed
by siloed groups in various departments.
Those groups analyze and report on the
data to their functional leaders, and
that is often where the story ends.
The hard truth is, no single data source can
answer a complex question. In the parable
of the wise men meeting an elephant in
the middle of a dark night, when each one
examined just one portion of the elephant-
the trunk, the tail, the tusk-independently
of each other, they arrived at the wrong
conclusion, thinking it was a garden hose, a
broom, or a very large spear. The key is to
combine all findings to understand the full
picture: the elephant. In Delta's case, the
" elephant " is the entire pilot training footprint.
Many organizations use simple post-training
surveys to determine the success of their
training programs. Unfortunately, the data
are not reliable to determine training
effectiveness. Learners may be happy with
the training program and provide positive
feedback, but this doesn't reflect their actual
performance in post-learning evaluation
(grades). And if learning doesn't transfer to
tangible business performance (e.g., pilot
flying skills or safe aircraft operation), the
training program is ineffective, even with
stellar post-learning evaluation and learner
feedback. (For an example of learning
evaluation done well, go to page 8.)
Historically, data-pieces of the " elephant " -have been siloed at Delta
across various data management teams. To build a shared collective
understanding of the available data generated, we supported the
team's identification of data owners and channels of communication
across these siloed working groups. By creating collaborative networks
with these executive decision-makers, the team is empowered to design
solutions that address the needs of the business in a meaningful way.
Data integrity
When building a database for analysis, actionable insights are only as good as the supporting data. To see
the entire " elephant, " you need reliable information to guide you toward the correct actions. The consequences
of faulty decisions based on bad data can be extremely costly and dangerous in the airline industry; data
integrity is critical for Delta.
We considered the following steps to build a reliable database to inform actionable insights:
* Identify all available data sources. What data are currently collected? Who owns which data?
Where are the data stored? In what application and format?
* Select and use the relevant data and discard the rest. Which data are relevant to the question we are
trying to answer? In Delta's case, flight performance, flight safety, and pilot errors are directly correlated
to learning and are necessary to understand training effectiveness.
* Identify any missing data showing up as gaps in the big picture. To fully understand Delta's training
effectiveness, we collected feedback from training instructors. Qualitative data from instructors helped
answer questions regarding specific issues during training with training equipment (such as simulators
and flight training devices) that often led to training delays and additional costs.
CONTINUED on next page
PERFORMANCE MATTERS / PG 11

The Data and Measurement Issue

Table of Contents for the Digital Edition of The Data and Measurement Issue