High-Stakes Research
by MARGARET E. RAYMOND & ERIC A. HANUSHEK - EDUCATION NEXT
Looks to me as if you've changed the thread topic form a Carville-Rove forum to an education forum. Nonetheless, a quick reaction to this piece.
1. Did the website educationext or whatever publish a response from Amrein and Berliner? If not, why not? It's hard to assess the claims of only one side without arguments from the other side or, alternatively, and none of us will do that, checking the data itself. Serious social science research, and this piece is certainly a part of the dialogue which produces such, is a conversation, a back and forth. If educationnext is simply a propoganda site for one point of view, then we will not see a rejoinder from Amrein and Berliner posted there. And since all the articles you've posted from it to date are only one side, it's fair to assume that's what it is.
2. If part of their argument is that the media gets social science research results wrong, well, welcome to the crowd. I was an up close follower of the wonderful 70s research that was published in places like Newsweek that claimed to support the notion that women should not have careers because if they didn't marry by 35 or so, they had no chance of doing so. Wrong. But it was newsworthy and was grabbed before that conclusion was carefully vetted.
3. As for pejorative comments of the order that these authors are doing rigorous scientific study as opposed to the sloppy stuff of Amrein and Berliner, one's view of that should await some rejoinder from A & B. However, on one occasion, a point they start a section with, it's just not true. They are troubled that A&B chose to drop interval level data of magnitude of differences in favor of binary contrasts of "increase," "decrease," and "unclear." There is nothing more or less scientific involved in such a decision. I happen to prefer the strategy advocated by your authors for the reasons they offer; but the A&B strategy is perfectly acceptable methodology. So long as they make it public. And, apparently, they did.
4. The following point is very interesting. I'll quote it and then comment.
Reporters need not be experts in statistical analysis any more than they must be fully versed in biochemistry or investment-banking regulations. But when a report is commissioned by an organization like the Great Lakes Center for Education Research and Practice, a Midwestern group sponsored by six state affiliates of the National Education Association, it would seem to call for a reasonable dose of skepticism. Why not bring in some outside expertise to review such a report before heralding its arrival? There will definitely be further opportunities for review. After all, the Arizona State shop promises that this is just the first of many annual reports on the impact of high-stakes testing.
Not a bad point. Right. But, given the political place of the Hoover Institution, precisely the same can be said for research commentary (this piece is not original research, rather reporting on the literature) about its work. If one were genuinely serious about the question of "accountability research", one would need to read the literature. Not just quote from one side.
Beginning to wake up. Coffee is working. |
