by Matt Rosenberg October 2nd, 2013
There’s not yet any evidence the City of Seattle’s Family Support Program for at-risk students in Seattle Public Schools is improving academic outcomes, and changes in the program’s focus, worker training and performance metrics appear necessary, say University of Washington researchers who recently published their findings in the journal Advances in School Mental Health Promotion.
Part of the larger Families and Education Levy renewed repeatedly by Seattle voters since 1990, the Family Support Program at the time of the study’s asessment in the 2010-11 school year employed 38 workers. It served 1,081 primary client or “focus” students – mostly in grades K-5 plus about a tenth in Grades 6-8 – and another 1,545 students who got some assistance. It’s operated by the city’s Office of Education, which funded the UW assessment that has been unfolding in a series of reports over the last several years.
Trying To Get At Readiness To Learn
FSP or “focus” students are chosen for special help from among indicators on household income, standardized test scores, attendance, behavior, and parental involvement but the exact reason or reasons they’re selected or not weren’t available to the researchers, making impossible any highly precise comparisons with similarly at-risk kids not enrolled in FSP, the study said. The purpose of the program is to indirectly promote academic success by encouraging family involvement, and access to a wide rage of support services. It also seeks to “improve teacher and school administration perceptions and understanding of students and families,” stated the study authors, who are from the Division of Public Behavioral Health and Justice Policy in the Psychiatry and Behavioral Sciences Department of the UW School of Medicine.
Support services promoted through the FSP include connections to academic counseling or assistance (although not tied to any timely student-specific academic performance data), plus mentoring, home visits, transportation, and organizing of events. The program secures for some students donated clothing, corrective lenses and food, and provides referrals to mental health and substance abuse counseling, as well as “connecting families to recreational opportunities, identifying safe transportation options, and securing stable housing,” according to the study.
Improved Access to Services, Parental Engagement
UW researchers found anecdotal evidence the program appears successful in some of its soft-focus emphases. More services were available to targeted students and nearly half of parents reported being more involved in school-related activities such as going on field trips or enrolling their children in after-school programs. In focus groups some teachers, administrators, FSP workers and families said they believed the program improved “academic functioning.”
Scant Evidence of Academic Gains
But the study also found no quantitative evidence that the City of Seattle’s Family Support Program is leading to improved academic outcomes; based on a lack of substantial improvement between the 2009-10 and 2010-11 school years in key indicators such as attendance rates, frequency of disciplinary actions, and standardized test scores.
Attendance rates stayed the same and nearly identical between FSP and non-FSP students over the two years, at 93.4 percent for the former and 95.1 percent for the latter. FSP students were less likely in each of the years to meet state testing standards in core subject areas. In math 23 percent of the FSP students’ state test scores were improved or at least or at least “positively stable” compared to the year before but 77 percent either were “negatively stable” or worse. In reading 30.3 percent were improved or positively stable while 69.7 percent were negatively stable, or worse. On improvements in disciplinary action, non-FSP students outperformed FSP students.
The study reported, “In general and on average (FSP-enrolled) youth did not improve from their prior school year and they did not demonstrate (statistically significant) rates of change when compared to youth who were not in the FSP. For disciplinary actions, youth in the FSP actually demonstrated a sharper deterioration (more disciplinary actions) than youth not receiving services from the FSP.”
None of this is failsafe evidence the FSP program in Seattle and others like it elsewhere in the U.S. don’t work, say researchers, but they add it’s likely changes are needed in Seattle, including in assessments.
Better Analytics Needed
A more telling analysis involving comparison of FSP students with carefully matched “control” groups was not possible for several reasons, researchers said, including that “the specific reasons for for student referral and participation (or non-participation) were not available.” It was strongly evident though, through “an almost perfect negative correlation,” that school-level rates of poverty predicted lower school-level rates of performance on the key metrics used in the study. Researchers called this a very worrisome sign.
Tighter Focus On Academic Challenges
The UW research team suggested several implications from the study. One was that there may need to be a stronger focus on direct academic assistance. “It might be that the best way to improve academics is to have an explicit focus on academic instruction and tutoring, and that attempting to address possible non-academic barriers to school success is too far removed from academic outcomes.” A related observation was, “It could be that family support can affect academic outcomes, but that (FSP worker) activities were not adequately connected to academic outcomes…the activities of the program may have been too diffuse or diverse to impact academic outcomes.”
Less Turkey, More Eyeglasses?
Executing plans to help kids get eyeglasses or to school on time may be much more helpful than other activities such as “sourcing Turkeys for Thanksgiving” or getting donated clothes, researchers noted. The study also reported that while training of Seattle FSP workers during the 2010-11 observation year addressed topics such as homelessness, child welfare services, mental health, gangs, engagement of fathers and safety during home visits, none focused on “addressing academic achievement.”
Better Worker Training, Student Data Needed
The study found that for Seattle’s and other public schools across the nation, family support programs need to be better designed and evaluated through a clearer focus on academic progress activities and related staff trainings, plus use of “evidence-based school interventions such as ‘Check and Connect‘ which monitor…academic functioning…and link the student to appropriate interventions” based on identified academic challenges. In focus groups that were part of the study, Seattle FSP workers told researchers “they had not been trained on how to identify and engage in activities that would be tied to academic outcomes.”
Additionally, the program’s data systems “were not capable of providing timely feedback to (workers) on the academic progress of students they served.”
Assessing the academic benefits of such school-based programs is a growing trend as required academic progress benchmarks ratchet up in states across the U.S. A report issued last November by the state-funded Washington Institute for Public Policy on the Education Advocacy Program of the state Department of Social and Health Services to help foster children succeed in school found that while foster kids in EAP fared better than non-EAP foster kids on cutting unexcused absences, and while their households better minimized disruptive multiple moves during the school year, the grade point averages and graduation rates of the EAP-enrolled students were no better than other at-risk foster kids not in EAP.