Talking Stick - January 2012 - 49

conTInued

A SCALE FOR SUCCESS

rubric data, as it “allows for a chance to implement improvement on the spot.” Thinking to the future, Brown reviews data to note where student staff under-perform, information which allows them to “reinforce position training” with in-services throughout the year. Similar to Brown, Albart uses rubric data to inform future student staff trainings, professional staff trainings, and resident assistant course assignments. In order to have a more holistic picture of performance, Piatkowski uses rubric data to supplement RA evaluations filled out by residents. The rubric implementation process does not need to follow these steps exactly, as variations in the process can occur. In fact, Albart notes that the three rubrics they utilize in their department were all created through different processes, each unique to the circumstances and needs they serve. The important common element is to be intentional and thorough throughout the process.

staff, utilizing rubrics to assess departmental processes or programs could create buy-in to this new method of assessment. Once staff members understand and support the use of rubrics, departments can include rubrics in student evaluation. Some departments may decide to holistically adopt rubrics for every assessment tool as a strategic goal to encourage going beyond mere satisfaction or frequency ratings, while others may decide to start small to test the rubric waters without rocking the boat. The next step would be to gather a small working group to brainstorm and develop rubrics. Explore whether analytic or holistic rubrics are most appropriate for the program being assessed. Reach out to colleagues at other institutions to inquire about their assessment strategies and use of rubrics. Collaborate with colleagues at your institution in other departments who might be utilizing rubrics in their assessment tools. These are all important steps in borrowing and adapting from successful models in order to avoid having to re-create the wheel. Training is a critical component for successful assessment of any kind, and utilizing rubrics necessitates an additional training component for staff and student raters. Vital training topics include explaining each description or criterion, how to utilize the assessment instrument, and the rationale behind rubrics as a helpful assessment tool. As mentioned previously, practicing evaluation with the rubric has two beneficial outcomes: it improves the instrument through feedback and increases the reliability through skill development and consistency. Finally, as with any assessment, collecting, analyzing, and sharing and reporting findings are the bottom line. If data is collected, it must be analyzed, disseminated, and put into action for programmatic and/or individual improvement. If individuals are asked to participate in surveys or

evaluations and never hear about the results, they become less motivated to participate in the future. Cynics are created when assessments are conducted and data is collected and then everything is compiled only to sit in a dusty binder on an even dustier shelf. Returning to the participants – whether they are staff or students – to discuss the results and share ideas for improvements underscores the department’s commitment to responding to feedback to make quality improvements that enhance the residential experience. When these steps are followed, rubrics can quickly become a valuable tool in an assessment arsenal. Piatkowski boasts about the benefits of the creation process, saying it “opened up an area of dialogue we never talked about before; it was nice to have those conversations to look to improve elements and make processes that much better.” After going through the time and effort to develop rubrics, Brown encourages staff to “actually use them and look to see how you can adapt them to multiple facets of your program.” Similarly, Albart advises embarking upon rubrics “only if you think it’s going to tell you something or if you are planning to use that information to do something,” which is true for just about any assessment effort. Severy offers encouragement in the process, saying “it’s rare that you have the perfect assessment tool to start, but you can learn as you go, get perspective from people, and go from there.” ts

Making Rubrics an Integral Part of Assessment
Having decided to utilize rubrics, it’s important to identify them as part of the assessment process and then to identify areas where they would be most applicable. As mentioned earlier, there are a number of different opportunities for which rubrics add value to an assessment program. Determining how to utilize rubrics within a unit might mean a focus on staff recruitment, selection, training, and evaluation as a first step; or rubrics could be developed to assess the effectiveness of community programming or to identify floor and hall climate issues related to safety, inclusiveness, responsiveness, and sense of belonging. Think also about how to implement rubric-based assessments. Starting with professional

Joseph D. Levy

is the assistant director of assessment programs at Campus Labs in Buffalo, New York. is the executive director of assessment and research at Colorado State University in Fort Collins. is the dean of students at Colorado State University.

David A. McKelfresh

Jody A. Donovan

january + february 2012

Talking Stick - January 2012

Table of Contents for the Digital Edition of Talking Stick - January 2012