Largest Federal After-School Program Shows Mixed Results in Most Scientifically Rigorous Study of After-School Programs to Date

2/6/2003

From: Mark Dynarski, 609-275-2397, mdynarski@mathematica-mpr.com or Joanne Pfleiderer, 609-275-2372, jpfleiderer@mathematica-mpr.com both of Mathematica Policy Research, Inc.

WASHINGTON, Feb. 6 -- To meet growing public demand for safe and supervised after-school activities for middle school and elementary school students, Congress authorized in 1994 a national program that today provides 7,500 rural and inner-city public schools in 1,400 communities nationwide with school-based academic and recreational activities. The first-year findings from a study conducted by Mathematica Policy Research, Inc., reveal that while the 21st Century Community Learning Centers changed where and with whom students spent some of their after-school time and increased parental involvement, they had limited influence on academic performance, no influence on feelings of safety or on the number of "latchkey" children, and some negative influences on behavior.

The program-funded at $1 billion in 2002-has shown limited academic impact overall, but some gains in social studies grades for elementary students and math grades for middle school students. For middle school students, the study found no increase in completion of homework assignments but greater teacher satisfaction with completed homework. The findings apply only to school-based programs that are part of the 21st Century program and do not necessarily extrapolate to all after-school programs. Mathematica is continuing to study programs that are part of the evaluation and will prepare more reports based on another year of follow-up data and visits.

The study, 'When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program', was conducted by Mathematica and its partner, Decision Information Resources, Inc., for the U.S. Department of Education. Additional support for data collection and analysis came from the Charles Stewart Mott Foundation. The study is the largest and most rigorous examination to date of school-based after-school programs. It includes a nationally representative sample of 62 middle school centers in 34 school districts. It also examines 34 elementary school centers in 14 school districts nationwide.

"While research has evaluated other after-school programs, this study is one of the few that is consistent with the principles of scientifically based research set out in the No Child Left Behind Act," says Mark Dynarski, senior fellow at Mathematica and director of the research effort. "The study is unique in the large number of after-school programs included and in the application of rigorous techniques that rely on control and comparison groups to measure impacts. The initial findings indicate that significant work remains to be done to develop after-school programs that improve children's academic, personal, and social skills."

Key findings of the report include:

-- Limited Academic Impact. At the elementary school level, the study found significant gains in social studies grades for participants; gains in other subjects were not significant. On average, programs had no impact on whether elementary school participants completed their homework or completed assignments to their teacher's satisfaction. In middle school, the study found small improvements in participants' math grades and higher teacher satisfaction with their completed homework. A subgroup analysis revealed larger grade point improvements and less absenteeism and tardiness for black and Hispanic students.

-- Reduction in Parent Care and No Change in "Latchkey" Children. Programs reduced the proportion of students being cared for by parents and by older siblings, and increased the proportion being cared for by nonparent adults. Programs did not reduce the percentage of students in self-care ("latchkey" children).

-- No Improvements in Safety and Behavior. Programs did not increase feelings of safety after school. Middle school participants were more likely to report that they had sold drugs and somewhat more likely to report that they smoked marijuana (although the incidence was low). Participants also were more likely to have their property damaged. (These data were not collected for elementary school students.)

-- Increased Parental Involvement. More middle school parents volunteered and attended open houses or parent-teacher organization meetings. Parents of elementary school participants were more likely to help with homework or ask about classroom activities.

-- Negligible Impact on Developmental Outcomes. No impacts were seen on students feeling they were better able to plan, set goals, or work with a team. Middle school participants were less likely to rate themselves as "good" or "excellent" at working out conflicts with others.

Program Characteristics: The 21st Century Community Learning Centers combined academic help with recreational activities and cultural and interpersonal enrichment. The study found that few students attended the school-based programs daily, with middle school students averaging 32 days during the school year and more than half attending fewer than 25 days. Elementary students attended an average of 58 days during the year. While a third of program coordinators and three out of five staff members were teachers, many worked only a few days a week and for short periods rather than continuously throughout the year. Programs spent about $1,000 per student, equivalent to about a 16 percent increase in educational spending per student. The average grantee ran three or four centers, serving nearly 700 students. Over half were minority students, with two-thirds of the hosting schools designated as high-poverty. Since 1998, Congress has mandated that these programs include academic activities.

To receive a copy of the full report, please contact Jackie Allen at (609) 275-2350 or go to http://www.mathematica-mpr.com.

Mathematica, a nonpartisan research firm, conducts policy research and surveys for federal and state governments, foundations, and private-sector clients. The employee-owned company, with offices in Princeton, N.J., Washington, D.C., and Cambridge, Mass., has conducted some of the most important studies of education, health care, welfare, employment, nutrition, and early childhood policies and programs in the U.S. Mathematica strives to improve public well-being by bringing the highest standards of quality, objectivity, and excellence to bear on the provision of information collection and analysis to its clients.