I’m not one to think that a child crying is the end of the world. Kids cry for a lot of reasons, many of which are really insignificant. My son cries when his friends have to leave, or when it’s time to put away his toys. So I’m not moved by sporadic claims of tests making elementary school students cry.
But that’s the premise of this article on the reaction to Indiana’s new state tests. Apparently no one told the parents or the teachers that this year’s tests wouldn’t be like last year’s.
The state teachers union called the whole scenario a meltdown, and Indiana officials are now hiring outside experts to review the test. They’ll check whether it was properly designed, and state education officials will investigate how better to communicate to teachers and children in Frankfort and around the state to let them know the difficulty level in advance.
[...]
The Indiana Department of Education wants the outside review to confirm that the test is valid and to ensure no mistakes were made, Superintendent of Public Instruction Tony Bennett said.
A couple of comments: First, it’s a bit too late to call in the external reviewers. That should have been part of the test’s development cycle. Second, and I’m getting tired of saying this, per the 1999 Standards for Psychological and Education Testing, a test cannot be valid or invalid. If someone from the state office actually used those words, they’re only demonstrating that they are unqualified to be crafting the assessment.
But those are relatively minor issues compared to this one:
The state has said it will be fair when it determines the scores required to pass this year and that students shouldn’t worry if they didn’t finish the test.
Huh? Aren’t state tests supposed to compare student performance to the expectations laid out in the state education standards? That’s what we call a criterion-referenced decision. How is it that they have yet to determine the passing scores? Either a student’s performance met the standards or they didn’t. The only reason not to set a cutoff score before giving the test is that they have to see how the students perform first. It wouldn’t be prudent to have 40% or 50% of the students fail.
But then they’re not really comparing student performance to the state standards at all. They’re comparing each student’s performance to the rest of the students’ performances. That’s a norm-referenced approach. Applying norm-referenced scoring techniques to make a criterion-referenced decision is beyond incompetent.
If state test scores are interpreted as criterion-references, they tell us whether students met the state’s standards. This is extremely useful for accountability and improvement. However, if we interpret the scores as norm-references, they only indicate how well students performed relative to their peers. Obviously, for accountability and improvement purposes, these norm-references are useless because there will always be students who score lower than their classmates.
It’s too bad the Indiana Office of Education didn’t bring in outside expertise during their test development. It’s unfortunate that they’re apparently not up to speed on the latest educational testing guidelines. But it is truly frightening that they openly admit to waiting until the test responses are in before declaring the cutoff scores for a crtierion-refernced test. I know of at least one other state that does this, and it’s improper there as well.