Malena Weitze and Dr. Deryle Lonsdale, Department of Linguistics
An oral testing method, Elicited imitation (EI), has provided helpful insight into the process of language acquisition and language assessment over the past 40 years. EI is an oral test in which the subject hears a sentence, forms a cognitive representation, and then produces a sentence according to that representation (Bley-Vroman & Chaudron 1994, 245). The theory behind EI is that, controlling for utterance length, people can only reproduce an utterance if they understand its syntax (sentence structure). Although there is controversy concerning the use of EI—asserting that it is not the most precise language acquisition assessment tool—currently prevalent methods of second language (L2) acquisition assessment (which are expensive) have produced results that are so broadly defined that learner progress over time is difficult to determine. EI has proved an inexpensive and effective way to assess L2 acquisition and chart it over time.
To further current BYU research regarding EI we examined the work of DeKeyser and Goldschneider (2005:27-66) who focused research on specific morphemes that proved problematic for L2 learners. This research inspired the development our EI test form, a form exclusively focused on determining the significance of tense, aspect, and 1st and 3rd person singular features. We designed our test to examine which syntactic features are most difficult for English language learners overall and which features are most difficult for different L2 learners (difficulty referring to features that influenced test performance on our EI test most significantly).
I. Test Design
We carefully designed a test comprised of 60 items (sentences), separated into five sentence lengths: 6, 9, 12, 15, and 18 syllables. Because we specifically focused our test on tense, aspect, and 3rd and 1st person singular (features) (DeKeyser & Goldschneider 2005: 27-66), we evenly distributed these features among the respective sentence lengths.
We stringently controlled for extraneous syntactic features that were we did not want to measure in this study. We controlled for word complexity, difficulty of prepositions, modals, question words, and complex clauses. Our test was administered to 189 subjects through use of a computerized tool at the English Language Center (ELC) on BYU campus. We referred to the placement system instituted by the ELC—which labeled beginning L2 learners as level 1 and advanced language learners as level 5. This allowed us to analyze the language abilities for students at each language learning level.
II. Item Evaluation
We used a 4-score evaluation system—based on the work of Chaudron, Prior, and Kozok (2005)—to evaluate the EI test responses. The accuracy of these items was evaluated by expert human graders. We first used a binary system to mark each syllable correct (1) or incorrect (0) based on whether or not the subject correctly reproduced individual syllables. For each error we deducted a point. An item with 4 or more errors was given a score of 0.
III. Results
In order to analyze our results we used a stepwise linear regression model. Our overall results indicated that for every language learning level, sentence length was the most statistically significant feature, just as we expected. Sentence length accounted for 80% of the variance (using the adjusted r-squared model) for our test.
There was a gradual progression in subject performance from level 1 to level 5. However, every language learning level dropped significantly in performance for longer sentence lengths; subjects at all levels performed best on the 6 syllable sentences and worst on the 18 syllable sentences. Aside from sentence length, there were other features that proved significant for our overall population. The p-values calculated through our stepwise linear regression, suggested that 3rd and 1st person singular features were significant for all levels. Present tense also approached significance.
We found that different features were more salient for different sentence lengths when we controlled for the sentence length variable in our stepwise linear regression model. The salience of these features varied for level 1 through level 5 within each sentence length. We used the adjusted r-squared measure (a measure which indicated the amount of variance for each feature for the results in the sample) to determine which features were most salient for these specific language levels.
Longer sentence lengths—15 and 18 syllables—were generally more difficult for all language levels regardless of the syntactic features included in those items. Subjects of all language levels performed below 1 on the 4-score scale for these longer items. Regardless of the structures included, shorter sentence lengths—6 and 9 syllables—were generally easier for all language levels; all subjects performed above 3 on the 4-score scale.
We showed that tense, aspect, and 3rd and 1st person singular account for 60% or less of the variance within the sample. Based on this information we determined that there must be other features to consider. We understood that sentence length could not account for any of this variance since we had already controlled for sentence length. We therefore determined that something else must be affecting subjects’ test performance aside from sentence length, tense, aspect, and 3rd and 1st person singular. Because we stringently controlled for extraneous features we knew that other features could not account for the remaining variance within our sample. We determined that further testing would be necessary in order to determine the additional features that accounted for this remaining variance. We suspect that test familiarity, first language (L1), and item order (which now is random) may be some features that could account for the remaining variance within our sample, and we plan to design further tests in order to analyze the effect of these additional features
IV. Publication and Presentation
This project was presented at the Linguistic Association of Canada and the United States held in Las Angeles area in August 2009. A paper submission is currently being peer reviewed for possible publication.
References
- Bley-Vroman, Robert and Craig Chaudron. 1994. Elicited Imitation as a measure of second-language competence. Research Methodology in second language acquisition 245-261.
- Chaudron, Craig, Prior, Matt, and Uli Kozok. 2005. Elicited Imitation as an oral proficiency measure. Paper presented at the 14th World Congress of Applied Linguistics, Madison Wisconsin.
- DeKeyse, Robert and Jennifer Goldschneider. 2005. Natural order of L2 morpheme acquisition in English: A meta-analysis of multiple determinants. Grammatical development in language learning 27-66.