The first two of a series of posts at Volokh.
Rick Sander (guest-blogging), June 10, 2005
Responding to Critics (1): A New Test of the Mismatch Theory:
The basic argument of Systemic Analysis is simple: if there is a very large disparity at a school between the entering credentials of the “median” student and the credentials of students receiving large preferences, then the credentials gap will hurt those the preferences are intended to help. A large number of those receiving large preferences will struggle academically, receive low grades, and actually learn less in some important sense than they would have at another school where their credentials were closer to the school median. The low grades will hurt their graduation rates, bar passage rates, and prospects in the job market. This is what I call the “mismatch effect.”
My paper tested this idea by comparing the outcomes of whites (who generally receive small or no admissions preferences from law schools) with blacks (who generally receive large, race-based preferences) to compare the outcomes of students who start with similar credentials. My results are robust and, as I’ll discuss in coming days, have withstood criticism pretty well. But I and everyone else agree that it would be preferable to compare blacks with other blacks. In other words, the ideal control group for examining blacks who receive large racial preferences would be a group of blacks who received smaller preferences, or no preferences at all.
As I discuss in my Stanford “Reply to Critics”, such a comparison group not only exists – we now even have data on their outcomes. After Systemic Analysis had gone to press, Ian Ayres and Richard Brooks at Yale pointed out that the Law School Admissions Council, in one of the surveys administered to students in its Bar Passage Study (a major source for my paper), had asked the students in detail about how they applied to, and selected, the law school they attended. About ten percent of the 1800-odd blacks in their study reported that they had chosen to pass up their “first-choice” school even though they had been admitted to that school. Most of these students apparently went to a lower-choice school because of financial aid offers or for geographic reasons. The data suggests that these black “second-choice” students had credentials substantially closer to those of their classmates. Compared to other blacks, these blacks closed nearly half the credentials gap.
These “second-choice” students are not a perfect control group, of course – no one was randomly assigned to attend schools offering different levels of racial preference – but it is about as good a chance to test the mismatch theory as we are likely to have for some time. If the theory is right, then the second-choice students should have better outcomes: higher graduation rates and more success on the bar. In the table below, I make predictions about how the blacks going to their second-choice schools should perform, based on simple linear assumptions (if blacks going to second-choice schools close one-third of the credentials gap with their classmates, they should close a proportionate amount of the outcomes gap, once one controls for index differences).
If the theory is wrong, in contrast, then of course the blacks going to second-choice schools should have about the same outcomes as blacks who took full advantage of the preferences they were offered. In the data presented below, we’d expect the blacks going to second-choice schools to do slightly better, since they somewhat better index scores than the average black law student (but this difference alone would only close about one-eighth of the gap in outcomes).
The actual outcomes look like this: SEE CHART volokh.com
These are pretty remarkable results. The “mismatch” predictions are either right on target or, in some cases, too low. The differences in success rates between black law students generally and those going to their second-choice schools are huge. As with everyone else, the black second-choice students’ outcomes depend heavily on their grades. But these blacks are substantially less mismatched than other blacks, and they get substantially higher grades (they average about ten percentile points higher in their classes – another outcome exactly in line with predictions).
Many critics of Systemic Analysis, when they come to the question of why black law students have such low graduation and bar passage rates, either offer no explanation or rather wearily suggest a “something about race” problem. These data offer a very clear example of how well blacks can perform.
There are two sorts of objections one might raise about this data. First, are the samples involved large enough to produce statistically significant, reliable results, or could these results somehow be a fluke? And second, is there some way that the blacks going to second-choice schools are systematically different (other than their slightly higher credentials) from other black law students? I think the answers are (a) the results are very reliable and (b) there are no alternative explanations for these results. But these require slightly longer explanations, and I’ll elaborate in my next post.
"Responding to Critics (2): “Second-choice” students
This is the second in a series of postings further explaining my work on the use and effects of racial preferences in law schools, and responding to critics of my work. One of the central claims in my research is that black law students are often “mismatched” by large racial preferences, placing them at schools where they do poorly and actually learn less than they would at a school with a smaller preference or no preference at all.
On Friday, I posted a new analysis that strongly corroborates the “mismatch” story: for a large sample of blacks admitted to law schools, those who passed up their “first choice” law school and went to a lower-ranked school – in other words, going to a school where they would have been admitted with a smaller preference – had dramatically better outcomes (grades, graduation, and bar passage) than blacks who made no such choice. Today I want to address some questions raised by this analysis.
First, are the results significant and reliable? The database for this analysis includes 1,757 black students entering law school in 1991. Just under one-tenth of these students (171) were admitted to their first-choice law school but chose to go to another school. This is a pretty large sample, and it means that any outcome where the success rate of the two groups of blacks is more than six or seven points apart (e.g., 80% vs. 87%) will be statistically significant. Pretty much all of the outcomes for black second-choice students are, in fact, better than the outcomes for other black students, by at least that margin (and sometimes by as much as 20 percentage points). So the answer to the first question is a resounding Yes.
Second, are there differences between the black second-choice students, and other black law students, that might account for their different rates of success? There is one important difference – the blacks who chose their second-choice school have, as a group, slightly higher average credentials than other black students. That difference accounts for about one-seventh of their higher performance. Otherwise, the black second-choice students are largely indistinguishable from other blacks at the outset of their law school careers. They are about equally likely to have a parent who attended law school (6% for the second-choicers vs. 7% for other blacks), to have a “burning desire” to become a lawyer (30% vs. 30%), to be “very concerned” about getting good grades (89% vs. 88%), and to believe they experienced discrimination during college (68% vs. 64%).
The factor that makes second-choice blacks truly different is simply that they are less mismatched with their classmates than other blacks are. Because they have turned down their “first-choice” school, they are at a school where, on average, their “academic index” is only 93 points below the class mean, compared with a 140-point deficit for other blacks. This in turn means that they get significantly higher grades, on average – and that, in all likelihood, makes all the difference for their future outcomes.
Going back to the technical discussion, controlling for differences in entering credentials makes one of the six interesting outcomes for these two groups statistically insignificant (ultimate bar passage). But the other five (first-year grades, third-year grades, graduation rate, first-time bar passage, and rate at which matriculants become lawyers) are significant, and all six outcomes are much higher for the second-choice blacks. One can debate what the proper controls should be – which factors and comparison groups provide the fairest comparison – but I have seen no analysis in which the second-choice blacks do not substantially outperform the comparison black group, and in which at least some of the differences are highly statistically significant.
Moreover, since the findings of the mismatch theory came from an entirely different analysis (comparing blacks and whites), but predict with great precision the actual improvements in outcomes for the black second-choice students, it would be hard to imagine a more compelling confirmation of its basic theses.
Responding to comments:
“Mahan Atma” says the results are “nonsense” because the blacks going to second-choice schools are not randomly selected; without randomization, there can be no true statistical significance. Not so. It is of course possible to determine the signficance of a difference between two groups that have not been randomly selected—all that significance in this context means is that the difference almost certainly is not due to randomness, but to some real distinction between the two samples. The crucial issue then is what variable accounts for this difference. The point of all regression analysis in the social sciences is to control for plausible differences that might explain why two groups have different outcomes. I find that when one uses these controls, the performance gap between the black second-choice students and others is largely intact – and statistically significant.
“Michael” contends that the BPS dataset is too noisy to be useful; some respondents do not understand the questions properly and miscategorize themselves. But I counted as “second-choice” students only those who said that they had been admitted to more than one law school, and who did not attend their first-choice school for an identified reason (usually geographic or financial constraints). Moreover, we can accurately estimate the size of the mismatch these students faced at their schools. Certainly it’s possible that some of the students I’ve identified as black “second-choice” students had their hearts set on going to UCLA, but went to their second-choice, Harvard, because Harvard offered them more money. But there can’t be many such students (or the average size of the mismatch these students face wouldn’t show up as being as small as it does), and to the extent such noise exists in the data, it simply implies that the results were strong enough to show through that noise.
“Donald” and several others wondered how the “second-choice” effects would play out for whites. I discuss this issue in some detail in my “Reply to Critics”. Here’s a short answer. The substantial number of whites who indicated they turned down their first-choice school (largely for the same reasons as blacks) tended to end up with a “positive mismatch” – that is, they had higher credentials than most of their classmates. This led, predictably, to higher grades in law school – well above the class median. In the top half of the class at most law schools, however, there isn’t much difference in graduation and bar outcomes – the vast majority of students graduate and pass the bar. Consequently, the benefits from a “positive mismatch” are a lot smaller than the harms of a large “negative mismatch”. So “theory” predicts that whites going to second-choice schools will see little if any improvement in graduation and bar passage rates, and that’s borne out by the data. (The white “second-choice” students may see significant job market benefits, but I haven’t tested that idea yet.)
More coming up
volokh.com |
