Whatever its shortcomings, reports from the Department of Education's What Works Clearinghouse (WWC) are the closest the federal government comes to telling us the instructional programs that meet NCLB's definition of scientifically-based research (SBR). It's certainly the best reference point for state and local education agencies considering whether a program they are planning to purchase with federal funds meets NCLB requirements. And that fact is not missed by school improvement industry providers, who - also lacking any definitive reference - understandably employ any kind of positive WWC review in their marketing and sales efforts.

Edbizbuzz readers might consider how the ill-fated Reading First program would have turned out had the WWC issued its August 13 Topic Report Beginning Reading some years ago. The report is worth at least quick look.

We looked at 887 studies of 153 programs that qualified for our review. Of these, 51 studies of 24 programs met our evidence standards, 27 without reservations and 24 with reservations. The remaining 129 programs had no studies that met the WWC evidence screens. Of these, 92 programs had one or more studies that were reviewed and did not meet WWC evidence screens. Thirty-seven programs did not have any outcomes studies. In looking at the four outcome domains for the 24 interventions, 10 interventions had positive effects or potentially positive effects in all the outcome domains addressed in their studies (see table 1). Eleven interventions had a combination of positive or potentially positive effects in one or two domains while having mixed, negative, or no discernible effects in other domains. Three had only mixed effects or no discernible effects across domains.

Another point for reflection is why so many providers have done so little evaluation of their programs since Reading First began. Clearly evaluation didn't/doesn't matter to sales.

Finally, readers will see that some programs we all believed were well-evaluated - Direct Instruction, for one - didn't pass WWC review. What's going on? Direct Instruction was not well-researched? Or WWC standards are not appropriate?

More on all this later.