Rachel Casper and Faculty Mentor: Grant Eckstein, Department of Linguistics & English Language

Introduction
Applied linguists are interested in the cognitive processes of writing teachers within the subfields
of both second language writing and writing assessment. Such cognitive insights as how a
teacher reads and scores a piece of non-native English speaker (L2) writing can have an impact
on the way in which second languages are taught and assessed in writing classrooms. This is
particularly relevant as increasing numbers of L2 students matriculate into composition courses
traditionally designed for native-English speakers.
Prior research has shown that, across all disciplines, teachers are affected by language in the text
that diminish comprehensibility, fluency, and the efficacy of the argument. These include errors
of grammar and vocabulary usage (Lindsey & Crusan, 2011). Teachers also have demonstrated
ethnolinguistic bias by giving lower scores because of characteristics of the student, such as
student’s names and ethnic background (Rubin & Williams-James, 1997). We also know, from
think-aloud protocol methodologies, where teachers narrate their thoughts while watching a
recording of their eye movements, that raters evaluate native (L1) and L2 texts on different
criteria, even when using the same rubric (Song & Caruso, 1996). That being said, we still do not
know the cognitive processes associated with how composition teachers specifically read, score,
and evaluate L1 and L2 texts. Thus, our research question is: What areas of do composition
teachers attend to when they read and evaluate L1 and L2 texts?
Methodology
Thus, we have gathered first-year composition teachers who evaluate both native and non-native
students, but are not necessarily trained to teach non-native students, in order to discover how
they assess student texts. With an eye-tracker, we can understand teachers’ cognitive processes
during reading and further, we can single out errors in a text that are affecting comprehension
and processing (Anson & Schwegler, 2012). This is done by having participants read
low-proficiency, authentic L1 and L2 texts. Teachers were asked to evaluate each text
immediately after reading it based on a quasi-holistic rubric addressing rhetoric, organization,
word choice, and grammar. The texts were coded by areas of interest (AOI) based on the four
areas on the rubric, for example transitions or grammar errors. The eye tracking data in these
interest areas, which included total dwell time and other measures of early and late reading
processes were then analyzed using t-tests.
Results & Discussion
In the rhetoric section of the text, composition teachers skipped the L1 rhetoric AOIs. This could
possibly be due to the rather straightforward nature of the L1 rhetoric compared to the L2, which
is rather contradictory and confusing, perhaps requiring more attention. In organization,
composition teachers focused more on the L1 organization, spending longer and rereading the
AOIs more. This could be due to the complex nature of the L1 organization and the relative
predictability of the L2 organization by contrast. For word choice, when first reading the text, the
teachers skipped and then returned and reread the word choice. Their pedagogical training
towards word choice errors could influence this as could the complexity of the L1 word choice
over the L2. In grammar, the teachers spent significantly more time in the L1 grammar AOIS,
perhaps because of the more complicated nature of L2 grammar errors.
Conclusions
Composition teachers read the two texts, L1 and L2, differently. Overall, composition teachers
focus on the L1 text and on the texts’ organization. The composition teachers tend to fixate more
on the organization of the L1 text because of its complexity and the relative simplicity of the L2
organizational structures (i.e. clear transition words). While they focus on the organization of the
L1 text, they overlook the rhetoric of the L1 text completely. We believe that this is do to the
pedagogical training of the composition teachers. If they have an understanding of the
organization of the text, the rhetoric might be superfluous ot their evaluation. Besides
organization and rhetoric, the teachers read and evaluate the texts quite similarly.
The fact that organization plays such an important part in differentiating and evaluating the texts,
we suggest that there might be an organizational bias occurring. Once the composition teachers
see the formulaic five paragraph essay of the L2 text the teachers overlook the rest of the text.
This presents and interesting look into how one feature of the text can entirely influence a
teacher’s evaluation. This idea of organizational bias and the fact that eye tracking technology
can be used to discover and describe the cognitive process of teachers as they read and evaluate
texts are the most important findings of our study. Overall, these results, while not particularly
generalizable because of the small sample size, nevertheless may have implications for writing
teachers and administrators in both composition and L2 writing programs, especially if they are
born out in larger follow-up studies. For instances, composition teacher training could focus
more on rhetoric evaluation to eliminate the organizational bias that could disadvantage some
writers.
References
Anson, C., & Schwegler, R. (2012). Tracking the Mind’s Eye: A New Technology for
Researching Twenty-First-Century Writing and Reading Processes.College Composition
and Communication, 64(1), 151-171.
Lindsey, Peggy, & Crusan, Deborah. (2011, December 21). How faculty attitudes and
expectations toward student nationality affect writing assessment. Across the Disciplines,
8(4).
Rubin, D. L., & Williams-James, M. (1997). The impact of writer nationality on mainstream
teachers’ judgments of composition quality. Journal of Second Language Writing, 6(2),
139–154. http://doi.org/10.1016/S1060-3743(97)90031-X