First, it's worth noting that the review discussed here was published February 1, 2007 and the original press release from Positive Action Inc. was released December 15, 2006 (yes, 2006). The matter was brought to the attention of your editor in a press release received April 11, 2007 from Mike Keller at MarketingWorks. Your editor's conclusion?  The marketing world now understands that serious evaluation is important to sales, the What Works Clearinghouse is as close as the industry gets to Consumer Reports, and in this case old news is still good news.

–––––––––––––––––––––––––––––––––

Improving students’ behavior improves their academic performance, test scores, and the overall environment of a school and a community. Using this approach, the Positive Action program has earned the distinction from the U.S. Department of Education What Works Clearinghouse as the only evidence-based character education program in the nation to achieve the top rating of “positive effects.” These “positive effects” were demonstrated in both the behavior (average of +19 percentile points) and academic outcome (average of +15 percentile points) domains.
 
Positive Action earned its top rating based on two studies: a randomized experimental trial in Hawaii and a high quality matched-control study in a large school district in the southeastern United States. Both research studies evaluated academic progress and behavior in schools using the Positive Action program. The research was conducted by Dr. Brian Flay, Professor, Department of Public Health, and Oregon State University. The research compared elementary schools participating in the Positive Action program to comparable elementary schools not using the program. These are the results according to the WWC Improvement Index.
 
• Standardized achievement test scores in reading and math: average of +16.5 percentile points
• Grade retention: +36 percentile points
• Suspension rates: average of +19.5 percentile points
• Substance use outcomes: average of +22.25 percentile points
• Violence rates: +18 percentile points
• Absentee rates: +10 percentile points....

Educators from participating schools have heralded the evidence-based character education program as one that gives students skills for a lifetime of achievement. Suzee Fujihara, a teacher from Lihikai Elementary in Maui, Hawaii, applauded the effects of Positive Action, noting that the program “gives children strategies to change negative attitudes to positive ones. The children are actually taught how to help themselves, and others, to be positive when negative thoughts and actions are present. Positive Action… makes them more independent in solving their problems and gives them the feeling of power to change their attitudes and actions themselves.”
 
Press Release, April 11.

––––––––––––––––––––––––––––––––––
(One nit: What about the program Too Good for Drugs and Violence? See WWC summary of character education reviews.)

If NCLB's requirement that federal funds be used only to purchase educational programs grounded in "scientifically based research" (SBR) meant programs with a "positive effect" rating from the What Works Clearinghouse (WWC), Positive Action would have a something of a lock on its category. 

Today, SBR does not have that meaning. But the WWC is an arm of the Department of Education (although its work was "outsourced" to the American Institutes for Research and the Campbell Collaboration, and the contract is now the subject of competition). At the moment, WWC's stamp of approval is helpful. Going forward, we have every reason to believe it will become more like a requirement. 

Getting beyond this program....

For now the "positive effect" bar is quite low - one positive evaluation done at any point in time of some version of the program that would stand up to a PhD dissertation defense. The bar is bound to rise - for example to include a requirement for research related to the program offered today, a minimum sample sizes, and student demograhics respresenting the intended market. But programs that get approval before the changes are made are bound to get some kind of reprieve from the new standards.

(To get a better sense of review stasndards, read our summary of the WWC's review of Positive Action in the "K-12 Program Review" section of the April 17 issue of New Education Economy® - our weekly review of professional reading available for free download at edbizbuzz until June 1.  Download an example of K-12 Program Review here. )

Given this reality, school improvement firms that can't jump today's hurdle should find that PhD candidate quickly. But every firm needs to think about evaluation as an ongoing cost of doing business, like paying taxes or hiring new staff. Management teams with foresight will go one step further, understanding that quality needs to be "built into" school improvement programs rather than "bolted on."