Related Study – Krista Jule U.

Banton

Xiaofei Lu examines one aspect of construct definition that is often included in constructs

underlying speaking and writing assessments. He examines the way in which syntactic complexity is

operationally defined by analyzing three corpus analysis tools that are widely used in research on writing

assessment. Lu begins by carefully unpacking the notion of syntactic complexity and then describes in

detail three prominent analysis tools: the Biber tagger, so-called after its developer, Douglas Biber; Coh-

Metrix, originally developed by Arthur Graesser, Danielle McNamara and colleagues to examine

cohesion in texts (Graesser, McNamara, Louwerse, & Cai, 2004; McNamara & Graesser, 2012); and Lu’s

own Syntactic Complexity Analyzer (Lu, 2010), designed specifically to evaluate syntactic complexity in

second language writing. For language testing researchers who have encountered these tools in their

reading and considered using them in research, this paper provides a welcome analysis of these tools

with implications for their role in operational definitions of syntactic complexity. Furthermore, Lu

provides a useful summary of the linguistic features that have been associated with higher quality scores

on writing in a variety of contexts. Like Römer, Lu argues that findings from corpus analysis might

profitably be used to inform rating scale development. Specifically, Lu points out that many current rating

scales, particularly holistic scales, do not sufficiently distinguish between syntactic variety, on the one

hand, and syntactic sophistication, on the other, both of which contribute to an overall assessment of

syntactic complexity.

References

Biber, D. (1991). Variation across speech and writing. Cambridge: Cambridge University Press.
Biber, D. (2006). University language: A corpus-based study of spoken and written registers (Vol. 23).
John Benjamins.
Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., Cortes, V., Csomay, E., & Urzua, A. (2004).
Representing language use in the university: Analysis of the TOEFL 2000 Spoken and Written Academic
Language Corpus. Test of English as a Foreign Language. ETS Research Report RM-04–03.
Biber, D., & Gray, B. (2013). Discourse characteristics of writing and speaking task types on the TOEFL
iBT® test: A lexico-grammatical analysis. ETS Research Report Series, 2013(1).
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on
cohesion and language. Behavior research methods, instruments, & computers, 36(2), 193–202.
McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied
natural language processing. In P. M. McCarthy & C. Boonthum (Eds.), Applied natural language
processing: Identification, investigation and resolution (pp. 188–205). Hershey, PA: IGI Global.