Talking Stick - September/October 2018 - 50

of satisfaction with various elements
of our housing program based on
place-based learning communities
and themed housing as compared
with non-programmed living environments." By doing so he was able "to
better understand the impact of various programmatic initiatives and how
they impact sense of community and
belonging in order to positively impact
student persistence."
These internal comparisons are
valuable sources of motivation. Comparing the number of residents who
attend programs or survey response
rates by residence hall floor can motivate residents to attend programs or
increase their response rates. Comparing various subgroups of students can
help a department understand who is
being well served and who may need
additional support. A housing department might also want to look at participation in events by demographic
groups. Because internal benchmarking relies on good comparative groups,
comparing halls with very different
student bodies or contrasting themes
may be misleading. In essence, differences in outcomes may be due to
characteristics of the students rather
than the housing practices, but seeing
comparisons can be useful for staff as
they evaluate their own performance.
Finally, longitudinal benchmarking makes comparisons through time.
This benchmarking type is particularly
useful as part of continuous improvement efforts. The most typical comparisons are year-to-year, especially for
assessments that measure concepts
such as student satisfaction. Longitudinal benchmarking is also useful for
monitoring key strategic outcomes
such as learning or satisfaction because the practices driving learning
and satisfaction are often semester- or
year-long initiatives. However, smaller
calendar units, such as week-to-week
or month-to-month, may be appropri-

TALKING STICK

When
benchmarking
shows that the
department is
out-performing
similar programs,
staff become
excited and
develop additional
pride in their
efforts.

ate for operations areas such as room
changes, facilities, or student conduct.
For instance, a campus may compare
the number of work order requests by
week or month to identify peak periods or times when damages are most
common.
Additionally, longitudinal data
can prove valuable for providing a
perspective for incoming staff. Kevin
Cook, associate director of residential
initiatives at Kansas State University in
Manhattan, explains how longitudinal
data on learning outcomes assisted in
his department's efforts to onboard
new student staff. "I used the longitudinal benchmarking of learning outcome
factors related to the role of our student
staff." By sharing these data, he and his
team were able to show incoming student staff "the potential for their impact
and help provide points of emphasis for
the upcoming academic year."

Longitudinal comparisons are also
good at evaluating the impact of policy
or procedural changes. For example,
assessing students' understanding of a
procedure before and after an educational campaign can demonstrate
that the campaign increased what
they know. One housing department
changed their programming model
and used longitudinal benchmarking
to demonstrate the impact of their
change by reporting improved and
higher levels of student learning over
the course of five years.
Not a Silver Bullet
Despite all the benefits of benchmarking, it is not without its limitations.
Benchmarking is still subject to the
same limitations of most quantitative
assessment efforts. For one, benchmarking cannot make up for a lack of
purpose or goals. To be useful, benchmarking must be linked to important
questions and initiatives. Benchmarking also cannot make up for poorly
designed or poorly implemented
assessment. In other words, poor data
is poor data, so if the measurement
is not clear, not valid, or incomplete,
then comparisons cannot make up
for the problems in the underlying
data. Finally, any data will fail to have
impact if it is not shared in ways users
understand. Like any results, the impact of benchmarking data is reduced
by complex graphs and charts, complicated language, limited access, and
other fundamental reporting mistakes.
Regardless of the type of benchmarking (standards, external, internal, or longitudinal), benchmarking
provides lots of opportunities for
better understanding data and using
data to drive motivation and action
- plus, many departments end up
using multiple types of benchmarking. The choice shouldn't be whether
or not to use benchmarking or even
which type is best. The choice is which

Talking Stick - September/October 2018

Table of Contents for the Digital Edition of Talking Stick - September/October 2018