The Chronicle of Higher Education, January 21, 2005, Friday

Copyright 2005 The Chronicle of Higher Education
The Chronicle of Higher Education

January 21, 2005, Friday

SECTION: STUDENTS; Pg. 27

HEADLINE: Scholars Say College Admissions Offices Misuse Advanced Placement Data

BYLINE: DAVID GLENN

DATELINE: Philadelphia

BODY:
College-admissions officers should be cautious when weighing Advanced Placement courses on applicants' high-school transcripts, according to a working paper presented here at the annual meeting of the American Economic Association this month.
The paper asserts that the mere act of taking AP courses in high school -- as distinct from scoring well on the official AP tests -- does not predict that a student will perform well in college.
The paper's authors -- Kristin Klopfenstein, an assistant professor of economics at Texas Christian University, and M. Kathleen Thomas, an assistant professor of economics at Mississippi State University -- analyzed the records of more than 28,000 students who graduated from Texas high schools in 1999 and who enrolled that fall in the state's four-year public universities.
The researchers posed two central questions: Did the Texas students who took Advanced Placement courses in high school have higher grades in their first year of college than their non-AP peers? And were they less likely to drop out before their sophomore year?
The answers turned out to be no and no. In Ms. Klopfenstein and Ms. Thomas's statistical analysis -- described in a paper titled "The Advanced Placement Performance Advantage: Fact or Fiction?" -- all of the variance in the students' grade-point averages and dropout rates was explained by the familiar predictors of college performance: high-school grade-point averages, SAT scores, parents' education and income, and the proportion of experienced teachers in the students' high schools. After such variables were accounted for, the AP courses on the students' transcripts did not have any extra predictive power of their own.
"AP experience may serve as a signal of high ability and motivation," the authors write, "but it does not by itself indicate superior academic readiness."
Quality vs. Quantity
Most of the best-known research about the Advanced Placement program -- including studies conducted under the auspices of the College Board, which creates and administers the official AP tests -- has tried to answer a different set of questions. Those studies have generally found that AP tests are a reliable equivalent to first-year college examinations, and that institutions should feel comfortable in awarding college credit to students who score well on the tests.
The new Texas study does not challenge such classic findings about the AP program. Instead, the authors express concern that some school districts and college-admissions offices behave as if simply taking an AP course, regardless of one's performance on the AP test, will help a student do better in college.
In an interview, Ms. Thomas said that she generally supports the recent expansion of the AP program and similar efforts to make high-school curricula more rigorous. She worries, however, that some school districts may be paying too little attention to the quality of their AP courses, and that high schools and colleges may have policies that are not warranted by solid evidence.
In particular, Ms. Thomas cited the University of California's policy of giving extra weight to AP courses when calculating applicants' high-school grade-point averages. The university considers an A in an AP course to be worth 5.0 points, as opposed to 4.0 for an A in a standard course. The university grants such extra weight whether or not the student chose to take the official AP test at the end of the course.
Ms. Thomas fears that admissions policies like California's -- together with various incentives offered by federal and state governments -- might be prompting some high schools to slap together ill-designed AP programs with poorly trained teachers.
The AP program has expanded very rapidly. In 1980-81, according to the College Board, 133,702 high-school students took Advanced Placement exams. In 2003-4, the number was 1,017,396.
"We suspect that there's a lot of variance in the quality of AP programs," Ms. Thomas said. "As schools move to increase access, unless their principals and administrators really understand what goes into developing that program, we can just envision cases where suddenly a teacher hears, 'Guess what? You're the new AP teacher.'"
Many districts have moved to add AP courses, Ms. Thomas said, in the wake of a 1999 study by Clifford Adelman, a senior research analyst at the U.S. Department of Education, who found that "intensity of high-school curriculum" is a strong predictor of students' college performance.
Ms. Thomas does not dispute Mr. Adelman's general insight, but she argues that for certain districts with large numbers of struggling students, AP courses might not be the best way to make the curriculum more rigorous. AP courses often require teachers to move rapidly through a huge amount of material, she said, and "sometimes covering a topic in depth is sacrificed for covering breadth."
Predicting Success
The Texas study comes on the heels of a similar paper that was released last month by scholars at the University of California. Saul Geiser, a visiting scholar at the University of California at Berkeley's Center for Studies in Higher Education, and Veronica Santelices, a Ph.D. candidate at the University of California at Berkeley's School of Education, analyzed the records of 81,445 students who enrolled as freshmen in the university system from 1998 to 2001.
Mr. Geiser and Ms. Santelices found that the admissions program's "bonus point" for high-school AP courses "bears little or no relationship to college performance." Like Ms. Klopfenstein and Ms. Thomas, the California scholars found that simply taking AP courses is not a valid predictor of college grades or persistence.
The California authors also found, however, that the subgroup of students who took the official AP tests, and did well on them, performed very well in college. Strong scores on AP tests, in fact, were a better predictor of students' sophomore-year grade-point averages than any other variable except their high-school grade-point averages. (Ms. Klopfenstein and Ms. Thomas were unable to perform a similar analysis because they did not have access to the Texas students' AP-test records. The College Board might soon provide them with those data, however.)
At the conference panel here, Ronald G. Ehrenberg, a professor of industrial and labor relations and of economics at Cornell University, and director of the Cornell Higher Education Research Institute, cited the importance of Ms. Klopfenstein and Ms. Thomas's line of research.
He also said, however, that the study's design might have missed some important potential effects of taking Advanced Placement courses. For example, he asked, "What's the impact of having taken an AP class, or classes, on where students go to college?"
In a telephone interview, Trevor Packer, executive director of the College Board's Advanced Placement program, said that while he has serious reservations about the Texas paper's methodology, he welcomes both the Texas and California studies.
"More and more people have felt that AP classes are something on a student's transcript that would be meaningful in the admissions process," he said.
That belief, he continued, should be scrutinized carefully because the College Board has established only the much narrower claim that students who score well on AP tests should be awarded college credit in those subjects. The Texas and California studies, he said, might be useful steps toward assessing whether taking AP courses, per se, is an accurate predictor of college performance.
Mr. Packer said he wished, however, that the Texas paper had distinguished between students who did well on AP tests and those who did not. "Until Klopfenstein and Thomas separate their sample into at least two different groups," he said, "it's impossible for them to make this claim that they're making that AP doesn't impact overall college success."
Further Analysis
Ms. Thomas said that she and Ms. Klopfenstein are eager to conduct such a study, and hope to receive data from the College Board that would allow them to do so. But she also said that it was valid for them to look at AP students as a single group, without regard to their test scores, because that is exactly what some important institutional actors, including the University of California admissions offices, have done.
Mr. Packer said he also worries that the Texas study's assessment of dropout rates might be flawed because Ms. Klopfenstein and Ms. Thomas considered all students who disappeared from the Texas records in 2000 as having dropped out. It is possible, Mr. Packer said, that some of those students might have transferred to private colleges or transferred out of state.
Ms. Thomas replied that national statistics suggest that very few Texas students were mislabeled in that way. In any case, she added, such a flaw would not affect her analysis of students' first-year grade-point averages.
As for Ms. Klopfenstein and Ms. Thomas's concern that the quality of AP courses is sometimes weak, Mr. Packer conceded that it is a legitimate worry, but said that more-detailed studies were needed. "What would really help administrators," he said, "would be to understand what sort of AP classrooms do impact students' college success."
"Administrators need research-based help in understanding what configuration of the AP classroom is helpful for expanding access and providing increased opportunity and inclusivity, without watering down the course," he said.
Such studies are indeed planned by Ms. Klopfenstein and Ms. Thomas. They would like to use their Texas data to identify particular AP programs that perform improbably well, given the school district's resources or demographics. Ms. Thomas said that studies of such programs could explain how they succeed, providing a lesson for other districts.
The College Board, meanwhile, plans to publish new guidelines this year about the training that AP teachers should receive, Mr. Packer said.
Ms. Klopfenstein and Ms. Thomas's paper has been submitted to a journal but has not yet completed the peer-review process.