Greater Orange News Service

OUSD’s “Marriage Encounter Program” Approach
vs.
U.S. Department of Education’s Real Scientific Rigor

A Principal working for a firm she recommends that then gets multiple year contracts; pirated works copied without the owners’ permission; top administrators comparing the program to “marriage encounter training”; 40% of the consultant contracts cut back with no reciprocal program cuts: Welcome to the Focus on Results program in the Orange Unified School District. The Focus on Results educational consultant program in OUSD has cost taxpayers over two million dollars in educational tax funds over the last four years that include years with budget cuts of popular programs. Orange Net News is producing this yearlong exclusive news analysis series that examines the OUSD program; the financial implications; and the worth of the program to the taxpayers that paid for the controversial consultant program.

At the same time Orange Unified School District (OUSD) administrators were comparing their Comprehensive Reform Model called Focus on Results to Marriage Encounter Training, the independent non-profit American Institutes for Research (AIR) in association with the U.S. Department of Education (USDE) was using authentic scientific criteria developed by What Works Clearinghouse (a USDE research arm) to publish a study that evaluates the quality and effectiveness of 22 widely adopted Comprehensive Reform Model programs.. The study was released two months ago in November. The study evaluated programs that had extensive research on their effectiveness. Focus on Results was not one of the models in the study.

Since 1999 the American Institutes for Research (AIR) has operated the USDE funded Comprehensive School Reform Quality Center that has pioneered research driven consumer-friendly research reviews of education programs. For the Comprehensive School Reform Models Study, the AIR reports it conducted “extensive reviews of about 800 studies to rate the models”. Reviewing the 800 studies with scientific criteria, the AIR study rated the models in five categories of quality and effectiveness that including the ability to improve student achievement. The study framework was developed in consultation with an Advisory Group made up of leading experts and researchers that aligned the study with the No Child Left Behind requirement for scientifically based evidence.

Of the 22 reform program models examined none received the study’s highest possible rating. However, two received the second highest “moderately strong” Category 1 rating (showing “evidence of positive effects on student achievement”): Direct Instruction’s Full Immersion Model (based in Eugene, OR) and Success For All (Baltimore. Md.) Five models met Category 1 “moderate” standards (noticeable evidence of positive outcomes); eight met a “limited” standard (some evidence of positive outcomes, but more research is needed); and seven received a “zero” rating indicating meaningful studies attesting to their success were unreliable or unavailable. None of the programs received a “negative” rating which would mean that no research was available to review (like with Focus on Results) or strong evidence demonstrated negative effects.

The most significant information in the report for Orange Unified (aside from modeling authentic scientific research rigor for OUSD Administrators and OUSD Trustees) was the relatively low costs for the most successful programs, including those with the highest effectiveness according to authentic scientific criteria developed by USDE researchers. The two highest rated programs cost significantly less per year than OUSD’s yearly investments in Focus on Results (which has no rigorous scientific based criteria).OUSD has paid as much as $250,000 a year for Focus on Results in elementary schools, plus another $250,000 a year for secondary schools. According to the AIR report, the Direct Instruction K-8 program (operating since 1968) costs $74,000 annually compared to the $250,000 a year price tag for the OUSD K-6 schools alone with Focus on Results. In addition, Direct Instruction’s program had high effectiveness in both literacy and math in K-8 programs. In OUSD, Focus on Results only focus is “literacy”. Success for All (operating since 1987 in over 1400 schools) was the other top rated program. Success for All had scientifically proven results with its program that focused on three areas: literacy, math and social studies. The yearly cost for this scientifically proven program (with a three area focus) is only $88,580.

In November, about the same time AIR’s comprehensive scientifically rigorous study was released, OUSD Assistant Superintendent Cheryl Cohen gave her long awaited report to the OUSD Trustees on the effectiveness of the $2 million dollar OUSD Reform Model program Focus on Results. Having no realistic data or scientific rigor to support the program, Cohen instead compared the $2 million taxpayer supported Focus on Results program to “marriage encounter training” prompting surprise and outrage across the greater Orange communities. At the same meeting, OUSD Trustees Kathy Moffat, Kim Nichols, and Wes Poutsma continued to defend the value and cost of the $2 million dollar program despite the obvious lack of comprehensive scientific rigor, or related data in the Cohen Report.

In her many appearances before the OUSD Trustees supporting OUSD’s $2 million investment in Focus on Results, Cohen has recounted that the program in OUSD originated when some OUSD principals were introduced to the program by their involvement in the Los Angeles Times’ Reading by Nine initiative and brought it to her attention (Focus on Results is no longer listed as a participant in the Reading by Nine program CLICK ON http://www.latimes.com/extras/readingby9/who.html). In Cohen’s accounts, she has never mentioned that one of those OUSD principals at the time was also a Senior Consultant for the Focus on Results program. That principal’s school, Prospect Elementary is still listed on the Focus on Results website as a “Case Study” for the Reading by Nine program (CLICK ON: http://www.focusonresults.net/results/ourresults_readingby9_b.html ). Since 2003, Prospect’s state test scores have declined (last year it posted OUSD’s biggest state score drop) and it is now designated by the State of California as one of OUSD’s many Underperforming Schools.

A major factor in wasting huge amounts of educational tax dollars on unproven programs is the fact that district administrators across the county do not research the data available for school reform programs, and school boards routinely act as rubberstamps for district bureaucratic requests. In 1998, Sam Stringfield (the principal research scientist at John Hopkins University Center for Social Organization of School and considered a top researcher in educational reform models) provided guidance in his widely circulated papers to help school administrators select an improvement program. Stringfield’s much renowned writings give guidance on how to analyze a reform program using three basic questions: 1) Does the program meet specific researched objectives? ; 2) How strong is the research backing the program?; 3) Are the financial and human costs beyond the resources of your district?

It appears none of this widely known best practice model for choosing a comprehensive reform program was used by the OUSD Administration in choosing Focus on Results. Years after this important work was published, the OUSD administrators, admittedly on just the advice of a few district principals (one who worked for the firm), committed themselves to an unproven program (at the time of OUSD’s initial investment, Focus on Results had only been around for two years) with no research rigor in a way contrary to leading research on the best practices available for choosing reform programs. When community pressure forced the OUSD Administration to justify the spiraling expense, the OUSD Administration has tried to manipulate data to prove their investment was worth it. Unfortunately with no research to back them, the rationale with the most “rigor” was for them to compare a $ 2 million dollar educational taxpayer investment (while proven programs like class size reduction and music programs were slashed) to “marriage encounter training”.

This month AIR is scheduled to release its report on secondary reform programs.

To view Part 1, 2 and 3 of this exclusive Orange Net News report CLICK below
PART 1
PART 2
PART 3

Research Links (Click On the link provided):
The AIR/ CSRQC Elementary CSR Model Report
Information about the American Institutes for Research (AIR)
Information on USDE What Works Clearing House
Information on Comprehensive School Reform Quality Center
Information about Reading By Nine
Prospect Case Study by Focus on Results
Information on Comprehensive School Reform Quality Reports
AIR Press Release on CSR Report
Sam Stringfield CSR Guidance (1998)
The latest USDE Research on another OUSD Program in use- SAXON MATH