These pieces originally appeared as a weekly column entitled “Lessons” in The New York Times between 1999 and 2003.
[THIS ARTICLE FIRST APPEARED IN THE NEW YORK TIMES ON MARCH 29, 2002]
News Analysis: Fuzzy Data on Race
New York State’s new data that show school test scores by race and income give a useful picture of the yawning achievement gap between black and white pupils. But the data are too crude to accomplish a larger goal: identifying problem schools as a step toward improving them.
Compiling separate test results for different races and income groups, part of a new federal education law, is intended to serve two purposes. First, because learning occurs in families as well as schools, test data for race and income groups permit fairer comparisons. For example, children whose families can afford housing with quiet space, for study, or who have many books at home, will often have higher scores than low-income students, even if their schools are equally good.
The second purpose of breaking down scores demographically is to throw a spotlight on schools that systematically leave some children behind. A school can have good average scores if its white pupils score higher than similar pupils elsewhere, even if its minority pupils score lower than similar pupils elsewhere. The new data should help identify schools like these, schools that ignore disadvantaged children by hiding behind scores of those who are better off.
Such reporting is part of President Bush’s education plan, and New York is carrying it out. But its reach is farther than its grasp. The new data are too imprecise to permit pure comparisons of school quality. To avoid comparing schools that are only superficially similar, policy makers will have to supplement the data with more expensive and nuanced observations.
Consider the family income breakdown. New York’s data separate students who participate in the federal lunch program from those who do not. But the program’s cutoff – family income equal to 185 percent of the poverty line – is so high that it cannot distinguish severely disadvantaged students from those without great hardship.
In New York, roughly 20 percent of children are from poor families, whose incomes are below about $18,000 for a family of four. Such children, many of whom are ill-housed, malnourished and from troubled homes, must overcome much greater obstacles to learning than those experienced by children at the top of the low-income range, with incomes of about $33,000.
Policy makers would err if they concluded that a school filled with working-class children outperformed one with poor children simply because it had higher scores, even though pupils in both places got subsidized lunches.
Another problem with using lunch data to assess disadvantage is its inaccuracy, particularly for whites. Poor white families are more likely to be temporarily poor, from a spate of bad luck. Poor black families are more likely to have been poor for a long time.
Because schools are not able to monitor families’ movements in and out of poverty, white children who were previously poor often continue to receive lunch subsidies. This happens less often with black children whose poverty is longer-term. So whites are more likely to be economically secure than blacks, even when lunch eligibility is the same.
Even with better income data, racial breakdowns can be misleading. Most people think black and white children should perform similarly if their family incomes are similar. Educators are puzzled by the fact that on average, whites score higher than blacks even if their family incomes are the same.
But income alone is a poor proxy for social class. Black middle-income parents have lower socioeconomic status on average than white middle-income parents, and this affects children’s achievement.
Because blacks are, over all, poorer than whites, middle-class blacks are more likely to have poor siblings and parents than whites earning the same income. More often, middle-class blacks help to support struggling relatives, and this leaves less income to devote to children for summer camp, better housing or other spending that aids achievement.
Black professionals are also more likely to be first in their families to achieve middle-class status. For young white adults, home ownership is often spurred by parents who help with down payments. First-generation middle-class families cannot get such help. So it is not surprising that black home ownership is lower than whites’, even at the same income levels. This leaves many black middle-class children living in more distressed neighborhoods than whites whose parents earn the same.
Family structure also affects learning. In patterns that originated in the forced breakup of families during slavery, black children are less likely to be raised in traditional nuclear families than whites. Consider a black child raised, in part, by a grandparent. Even if the child’s mother graduated from high school or college, the grandmother is less likely to have done so than a white child’s grandmother.
While grandparents may provide nurturing support, children partly raised by less-educated grandparents will be exposed to less complex grammar and vocabulary than children raised only by college-educated parents. This also leads to more difficulty in school for black than for white children, even when both sets of parents are educated and have middle-class incomes.
In many schools, black children’s poor performance may have nothing to do with these demographic confoundings, but result from unfocused school leadership, less adequate teachers, less access to challenging curriculum, poor disciplinary climate or placement in separate classes that warehouse low achievers. Data alone can’t tell where failure results from such school practices that are amenable to reform.
So New York’s new data can only be a warning, and may be misleading. To judge schools accurately, officials would have to send teams of knowledgeable outsiders to observe teachers, examine student work and determine if a school’s instruction and disciplinary climate are likely to stimulate the highest possible learning in each group, regardless of home advantage.
Analyzing test data is a first step, but only a first step, toward making such determinations.
