The Data and Measurement Issue - 17

IMPACT
HOW WILL THE OUTCOMES IMPACT THE MEASURED,
THE MEASURER, AND THE CONSUMERS OF THAT INFORMATION?
The measurement activity doesn't start with a survey, a button press,
a movement, or a question. Nor does it end after the data are collected or
analyzed. Framing the measurement activity by asking key questions is just
as important and will help you understand the scope of the challenge and
its impact: Who are you measuring? What does it mean to them? What about
the measurer? What are the data going to be used for and what decisions will
they affect?
A recent TiER1 project required us to measure the development of complex
skills like critical thinking and teamwork in a classroom type setting.
The question was, " Are learners demonstrating proficiency in these types of
skills? " The " why " behind it was to provide developmental feedback to learners,
given that the value of these skills is not immediately obvious-they are more
like enablers than key performance indicators. Here, at least three groups are
impacted: the learners who are measured, the instructors who are measuring,
and the administrators who decide what resources to allocate and how.
Does the impact to each help to achieve or diminish your desired outcomes?
Chances are that the activity itself, whether or not you do anything with the
results, will have some impact. In this case, if the instructors don't see the
value of the activity or see it as undermining their instruction, they may
influence the data negatively by implicitly communicating their skepticism
and distrust of the process. Learners are likely to be influenced by this
skepticism and be less willing to listen to or act on the feedback from the
assessment. Administrators could end up with an inaccurate picture of the
developmental state and trajectory of learners in the classroom, which can
impact the implementation of instructional approaches and the allocation of
classroom resources.
COMMUNICATING PURPOSE AND
IMPACT IN A WAY THAT MATTERS
Start by framing the measurement problem for yourself and all stakeholders.
Knowing the purpose ahead of time will not only help you define scope and
goals but also identify who is impacted and how to communicate with them.
Understanding both the operational requirements or practical constraints and
the state of the science are crucial to answer your questions and purposeful
measurement. It's also critical to understand the art of storytelling and how to
convey the message so that your audience hears, understands, and learns.
Ask yourself, will it make an impact or not? How will you communicate your
purpose and findings in a meaningful way? To make change stick or bring your
strategy to life, everyone affected by the measurement needs to know what
this means for them and clearly understand why it matters. (To learn more
about effective communication of data practices and insights, go to page 28.)
BUILDING TRUST
Meaningful measurement requires understanding the people being measured,
those who are doing the measuring, and those who will use the results of
the measurement activity. When you can clearly communicate the purpose
and impact in ways that matter, you create, for all stakeholders, an opportunity
to build confidence and trust in the process and its outcomes. This
important work will set you up to succeed in activating the strategy that your
measurement activity is designed to support-and it will help you make a
positive impact on your staff and organization.
about the author
Camilla Knott, PhD, is a Principal at TiER1. She leads a portfolio
of work that focuses on developing complex cognitive skills in highrisk
environments. Camilla's work uses well-established and novel
approaches to create innovative solutions that are rooted in both
scientific theory and real-world constraints.
PERFORMANCE MATTERS / PG 17
The Data and Measurement Issue

Table of Contents for the Digital Edition of The Data and Measurement Issue
