Kevin Kumashiro of the University of Illinois at Chicago recently spoke at my institution. Besides being the author of Bad Teacher!: How Blaming Teachers Distorts the Bigger Picture and a founding member of CReATE, he is also a genius.
Kevin Kumashiro said something profound, powerful, and yet very simple. Let me paraphrase.
“A correct answer on a standardized test is only evidence that a child might have known the answer to the question.”
That’s it! Any extrapolation pushes the bounds of validity. Setting up cause and effect scenarios on the basis of correct answers distorts the results of standardized tests. This also means that isolating causal variables is almost impossible.
Just think about it. How many contributing variables might influence a student’s ability to pick the correct answer on a standardized test? Hell, even if the student gets the right answer we can’t even be sure he or she knew the right answer. There is just as much of a “chance” that they guessed the correct answer.
I wanted to check Kevin’s brilliance and although the results are not scientifically generalizable, I posed the fill in the blank statement below on my Facebook page today.
If a student answers a question correctly on a standardized test we know what…?
Here are the responses I received.
Facebook Friend 1: Ummmm…..that the teacher did a good job teaching?
Facebook Friend 2: that s/he answered that question correctly on that administration of the test. Nothing more.
Facebook Friend 1: My comment was snark…. hope that was clear with the face
Facebook Friend 3: That money, time, and talent were wasted and that children were put through anguish for nothing.
Facebook Friend 4:
A. They know the material
B. They memorized the material.
C. They have really good peripheral vision.
D. All of the above.
E. Some of the above.
F. None of the above.Facebook Friend 5: How about G. They are good guessers.
Facebook Friend 6: a priori, lucky guess, good decoder, bubble coincided w intercept of student’s line art. may or may not coadjute knowing.
Facebook Friend 7: They practiced those “test-taking skills,” so important in life.
Facebook Friend 8: That Michelle Rhee erased the original.
Again, although not scientific, my Facebook respondents seemed to support Kevin’s assertion about answers on standardized tests.
What causes a student to choose the correct answer on a standardized test? All kinds of things. A correct answer (at its best) is evidence that a student knew the answer to the question—end of story!
Therefore any statistical acrobatics (VAMs) that claim to isolate cause and effect relationships should be viewed as suspect at best.
Follow Timothy D. Slekar on Twitter: www.twitter.com/slekar
Recent Comments