Mindy Davis and Dr. Stefinee Pinnegar, Department of Teacher Education
Research has shown that student-to-student interaction in times of transition is valuable (Kuh, 2005; Light, 2001; Astin, 1993). Kuh (2005) has highlighted the interaction between freshmen and upper-classmen as a high impact strategy in supporting freshmen transition. Yet, research indicates that the benefits received by either the freshmen or the upper-classmen is dependent on the quality of interaction. Since the upper-classmen, usually labeled as peer mentors, have the most responsibility for that interaction, their ability to negotiate these relationships becomes increasingly important to programs that use them. Thus, hiring peer mentors and training them is a crucial concern. Currently criteria to direct selection and training of peer mentors are largely intuitive and haphazard. Other than traditional hiring guidelines and training strategies used for almost any program, there is little guidance for personnel directing college transition programs. Most researchers in this area use externally constructed evaluation tools to explore selection. We wondered what we might learn about evaluation and selection of peer mentors by inviting them into the evaluation process. In our study, we explored what peer mentor constructed rubrics for self-evaluation could teach us about the selection and development of peer mentors. Our overarching question was: What can we learn about peer mentors’ development from their own evaluations of expected peer mentor performance?
The data for this study consisted of the self-evaluation rubrics of 41 undergraduate students (9 males, 37 females) between the ages of 18 and 25, who were enrolled as sophomores, juniors and seniors and were working as peer mentors in a first-year program at Brigham Young University. Of the participants, 23 were new hires; 18 were returning peer mentors.
Peer mentors were asked to read two articles: one focusing on the importance of making quality judgments (Oaks, 2007) and another on five characteristics of great learners (Eyring, 1997). Based on these readings and personal experience, they created individual rubrics to evaluate peer mentor performance across the five characteristics of great learners and in terms of good, better, and best. The mentors were given a sample rubric that showed how to complete the assignment.
There were three stages of analysis in examining the rubrics. First, researchers used a subset of the data and attempted to arrange the peer mentors’ statements about what comprised “good,” “better,” and “best” performance on a single scale. The researchers tried to see if single statements of performance could be arranged from weakest and shallowest to more sophisticated and complex. The researchers found they could not arrange the data in this fashion because more than one dimension seemed to underlie each statement.
Researchers then engaged in a second round of analysis. In this cycle, researchers carefully examined the responses in the category “Welcomes Correction” to uncover and name the
dimensions that underlie each statement. In this analysis, we identified two dimensions: action and motivation. Yet when we attempted to further validate the dimensions by recoding the data on the presence or absence of these dimensions, there was data that could not be coded.
This led to the third round of analysis. In this cycle we looked more holistically at each mentor’s response. We noticed that statements we could not categorize appeared to refer to not only action and motivation but also specified in some way mentors’ reasons for actions and motivations. We labeled this intention. We then developed a coding instrument for the rubrics which provided anchors for identifying extremes from shallow to sophisticated conceptions. This coding tool included a scale with a range from 0 to 5. Using the coding instrument, the evaluators first coded five rubrics across all five characteristics. We conducted and met inter-rater reliability trials working until we reached complete agreement on our coding. We then coded the remaining rubrics. After coding, multi-variate statistics computed the relationship between the codes for Action, Motivation, and Intention and the quality judgment indicators (“Good,” “Better,” and “Best”) across the five characteristics for all the rubrics.
Through the analysis of the 41 peer mentor self-created rubrics, we identified a developmental progression across the dimensions and the characteristics. In regards to the dimensions, we found a developmental progression from Action to Intention to Motivation. In regards to the progression across the characteristics of great learners, we discovered that the most difficult characteristics for peer mentors to develop are Welcomes Correction, Expects Resistance and Overcomes It, and Honors Commitments. These findings suggest that peer mentors develop in Action, Intention, and Motivation and in the five characteristics of great learners.
This research project and the subsequent findings were presented at the annual Conference on the First-Year Experience in Atlanta, Georgia. As a result of the poster, we were invited to submit an article to a whole issue on peer mentor learning for the Journal of The First-Year Experience and Students in Transition. We submitted “Identifying and Measuring Peer Mentor Development: A Qualitative Rubric Analysis” and were informed that the manuscript passed an initial review. The findings of this study were also presented at the McKay School of Education Mentored Research Conference held at Brigham Young University. In addition, the research findings have guided Freshman Mentoring administrators in adjusting selection and training practices.
References
- Astin, W. (1993). What matters in college?: Four critical years revisited. San Francisco, CA: Jossey-Bass.
- Eyring, H. (1997). Child of God. BYU Speeches. Retrieved from http://speeches.byu.edu/?act=viewitem&id=74
- Kuh, G. (2005). Student success in college. San Francisco, CA: Jossey-Bass.
- Light, R. J. (2001). Making the most of college: Students speak their minds. Cambridge, MA: Harvard University Press.
- Oaks, D. (2007). Good, better, best. Ensign, 36(11), 104-8.