Computers to mark papers in school trial, reports today’s Sun-Herald.
STUDENTS at a Sydney high school are set to take part in a trial in which computers will mark their English assignments, examining grammar, punctuation, spelling and style.
The e-assessment trial will be conducted among year 11 students at St Andrew’s Cathedral School next term.
Various people, such as the English Teachers’ Association, are watching with interest.
This report in The Dayton Daily News is worth examining, as it deals with the same program (Criterion) to be used in this trial.
The computer grader gave a top score to my banana-eating purple imaginary friend, even if the description made no sense in a middle school-level essay.
But according to the computer, the gibberish-filled essay displayed better writing than an on-topic, sentimental ode to summers past – an essay for which I gave my best effort. As the testing industry hurtles toward more computer scoring of student essays, a Dayton Daily News examination of one of the leading computer scoring programs revealed serious flaws. To control costs, testing companies are spending millions to develop essay exams that do not need human scorers…
To test one such program, the Daily News submitted an essay intentionally filled with nonsense. The Educational Testing Service’s Criterion computer program scored it a 6 out of 6. Another essay, written with earnest effort, was scored a 5 out of 6 from the computer. Critics said the Dayton Daily News ‘ comparisons demonstrate the testing industry may be pushing too hard to cut costs.
James Popham, a retired UCLA professor, former head of a testing company and author of books about standardized testing, finds it “troubling” that major companies use what he said are flawed computer scoring programs even on a limited basis. “What you have demonstrated . . . is that this particular scoring system stinks,” he said. “It suggests that the profit motive has overridden good sense and professional integrity.”
The possibilities are intriguing.
Take any essay generated by the Postmodernism Generator, submit it, and see what Criterion does with it.
A special award must go the this weekend’s Australian for its clear-eyed (as distinct of course from one-eyed) objectivity in the field of education. A range of viewpoints indeed, as we had Kevin Donnelly AND Christopher Pearson on the subject. How fair can you get? Apparently, given his new duties, Keith Windschuttle was too busy this weekend. Perhaps the Oz is a portent of the Radio National of the future.
Thanks to Watchdog of the Wankers for this Quote of the Week.
“Why should those of us who don’t care for the offerings of Radio National, Triple J or the 7.30 Report subsidise others to get their political entertainment commercial-free?” – – Newly government-appointed ABC Board Director and revisionist historian, Keith Windschuttle.
On the subject of educational assessment, do read James Popham, mentioned in that report from Dayton, on “The Trouble with Testing.” Good stuff.