EducationNews.org

The Fordham Foundation has recently released a report written by Sol Stern entitled: Too Good to Last: The True Story of Reading First. In this on going series of interviews, Reid Lyon responds to questions about Reading First and what transpired regarding this program. Since some of his responses are quite lengthy, this interview will be on going in nature over the next few days.

1) Recently, the Fordham Foundation has raised some concerns about what the heck has happened to the Reading First Program. What has been your initial reaction?

1. Both Sol Stern's report and the stance of the Fordham Foundation reflect a good deal of courage and common sense in unpacking the political issues surrounding the Reading First program. For the first time, someone began to examine the motives and behavior of vendors, legislators, and agencies that seem bent on killing a program that helps kids learn to read.I am now in the midst of writing a book on the role of science in guiding educational practices and policies and Reading First is used as one example of how that process can work.Within this context, I am reviewing the complaints and allegations lodged against the Reading First staff tasked with implementing the program.I am also reviewing all the OIG reports and congressional testimonies and remarks made by congressional members. Frankly I am surprised that no one to date has done a systematic analysis of all the complaints and allegations and the specific evidence of wrongdoing that substantiates the allegations.To be sure, the OIG report addressing the role of the Department and the Reading First office in administering the program alleges a number of errors of commission and omission on the part of the staff but the veracity of the evidence seems weak – at least to me.Indeed, when you examine the complaints and the OIG reports you find that many of the allegations are based on misinformed opinions of why the Reading First program was developed, what its specific goals were, and how the legislation guided the efforts of the Reading First staff at the Department of Education in implementing the program.

My review thus far shows that there were instances where the implementation of the program suffered from a lack of resources and support for the Reading First office and the lack of specific guidelines developed for contractors as well as those developed by contractors.But these are neither ethical nor legal issues, and the claims made by some that the Reading First Program had a particular fondness for some programs over others or that there were actual conflicts of interest simply are not supported.The fact that two reading programs with actual evidence of effectiveness lost market share indicates just how careful the feds were to avoid going beyond what the law authorized and impinging on states' authority to make their own decisions. At the same time, a significant weakness in the Reading First law was the lack of explicit procedural and ethics guidelines vis-à-vis the functions of the contractors. I am not sure where the heck the Department's lawyers were during the implementation planning and procedural phases, but apparently minimal guidance was provided by that very important office. I think that there were errors of omission in not ensuring that everyone on the implementation side understood conflict of interest issues – particularly perception of conflict issues. However, I do not believe there were errors of commission where particular individuals manipulated the system to gain financially.

The Stern Report and the Fordham call for accountability will likely shed light on the political attacks on Reading First by congressional members as well as understanding the motives of those whose programs were not adopted by districts and schools and who then blamed the Reading First office and Chris Doherty.When you review the complaints of someone who had been thought of as a responsible scholar—Bob Slavin—you find the complaints characterized by a lack of appreciation for evidence, a surprising use of innuendo and gossip to smear people in a way that resembles a National Inquirer story, and simple untruths.The Stern Report and the Fordham Foundation correctly pointed out that many of these untruths led to the slashing of the budget for a program that was doing very good things for the poor kids around the country while at the same time increasing funding for the Title I program – A program that has never undergone serious evaluation of effectiveness and was and is simply an entitlement program that has squandered billions of tax payer dollars over the years.

2) Is Reading First an Effective Program?

When Reading First became law in 2002, its goal was to improve reading achievement for all students -- but particularly and especially for those youngsters whose futures have typically been limited because poor kids in low performing schools are harder to teach and not expected to excel. Reading First levels the playing field. While it still has work to do, recent reports from both the OMB and the GAO appear to indicate that the program is well on its way to achieving this goal.

There are several fundamental issues relevant to the "effectiveness" question.When Bob Sweet did the heavy lifting on drafting the congressional language for Reading First and I provided the scientific input, we both knew effectiveness would be tied to several factors.We knew that the power of Reading First in improving reading abilities among youngsters typically disenfranchised from effective education would require the integration of at least three essential ingredients.First, no matter how good programs, materials, and/or instructional strategies are, they must be provided to students by a well prepared teacher under conditions that ensure implementation fidelity.Limited gains will be observed and sustained if programs are not accompanied by these conditions.For this reason, substantial funds were available through the Reading First legislation to provide systematic professional development in SBRR and its implementation.Likewise substantial funds were available for assessments and instructional materials based on SBRR.Note that the funds for professional development were available to all eligible schools in a district – not just those identified as Reading First schools.

Second, we knew that the ability of individual districts and schools to monitor implementation fidelity and to rapidly strengthen implementation when indicated was enormously critical – I can't emphasize how important implementation fidelity is to program success and the ability to scale up programs like Reading First.

Third, we knew we would have to build in a very robust independent program evaluation to determine for which kids Reading First programs were having a positive effect.This third ingredient – the independent evaluation – was so critical that Bob Sweet ensured that $25 million dollars PER YEAR were provided so that the resources and talent would be available to carry out a high quality evaluation EVERY year.But, for reasons still not understood – at least by me – the Department of Education did not move forward rapidly to identify the independent evaluator or set in motion the infrastructure to carry out this essential part of the law.

Indeed, the delay was so substantial that Bob Sweet, then a senior congressional staff member with the House Committee on Education and the Work Force, wrote letters of concern to the Secretary of Education and the Director of IES to determine why the delay was occurring.To us the delay was irresponsible and unconscionable.Bob had worked extremely hard to ensure that enough money was available to do a world class evaluation on a yearly basis and the delays ensured that whatever was finally planned would most likely be rushed and insufficient.And I believe that the evaluation that is now underway will not do the comprehensive job we had anticipated.For example, if I am correct, the current evaluation will only examine student performance for the first year of Reading First implementation.If Reading First is similar to other complex funding programs, it takes at least a year or more for implementation to solidify and become consistent.Second, the data sources and types of data available for the evaluation vary by state thus limiting the conclusions that can be drawn. Third, to my knowledge, there was not time to ensure that robust measures of implementation fidelity were developed, standardized, and applied to all districts incorporating Reading First programs into their instructional day.Fourth, it is probable that comparisons will be made to non-Reading First schools who were also receiving Reading First support for professional development – without appropriate controls, these types of confounds will pollute the data.It is the case that hindsight is a luxury but if I had the opportunity to do things differently, I would have argued strongly that the technical assistance mechanisms and the evaluation design had to be developed and in place before funds were distributed to states.In many cases, Reading First funds were distributed to states that did not have a plan for professional development and systematic implementation.To be honest, Texas was one of those states.

Bottom line; I believe that the Department squandered the opportunity to conduct a rigorous evaluation that would have provided specific guidance with respect to improving the program. A program as complex as Reading First has to have an impeccably designed and comprehensive evaluation that can address the complexity of the program and its implementation as it seeks to determine effectiveness.I do not believe the current evaluation will accomplish that. This is unfortunate particularly since, for the first time in my experience, enough money was available to carry out such an evaluation.I am not sure how much the current evaluation costs but I doubt it is worth $25 million.Where did all that money go? Federal bureaucracy can screw anything up and frequently does. This is one sad example.

Given this, you have to turn to individual states to learn what they are finding vis-à-vis improvements in reading among those youngsters eligible for Reading First programs.Shep Barbash has done the most comprehensive and detailed examination of these data and summarized them elsewhere. His review points to significant gains in reading abilities among children in many Reading First schools across the country.However, as I mentioned, different states use different measurements so it is impossible to examine trends across states and determine the conditions under which Reading First is having an effect.I would predict that any states that are showing gains have provided robust professional development and ensured that continuous assessment of student progress is provided systematically and continuously.I would also predict that implementation fidelity is present.The GAO and OMB reports provided a glimpse of how states and districts are viewing the value of the program and their results were very positive.

I believe Reading First has had a significant impact in addition to that observed in individual schools and classrooms. In my visits to almost every state since Reading First came into law, it is clear that the language of reading instruction has changed.It is hard to go into a school these days or to a state department of education and not hear educators discussing research-based reading instructional approaches, progress monitoring, systematic professional development, and implementation issues.There is an astounding common knowledge among teachers and educational leaders about the results of the National Reading Panel and what constitutes SBRR.In essence, the profession is now developing a systematic common professional language that is being used among teachers, educational leaders, and policy makers in addressing the needs of children who are struggling with reading.Since language has been frequently reported to drive thought, I see this as a major step forward.For the first time, there is actually systematic movement in the publishing sector to develop products based on the results of converging scientific evidence and states review these products for alignment with SBRR.The science of reading has come a very long way since the early 1990's and that is reflected in current policies and practices.

Published April 2, 2008

Dr. G. Reid Lyon, an internationally recognized authority in educational issues announced the development of SYNERGISTIC EDUCATIONSOLUTIONS (SES) a consulting company to advise in the implementation of evidence-based assessment and instruction practices, professional development programs, development of education policy at local and state levels, and the development of assessment and evaluation programs for colleges and departments of education preparing for regulatory and accreditation activities. Prior to his most recent position as Executive Vice President for Research and Evaluation at Higher Ed Holdings, Dr. Lyon was the Chief of the Child Development and Behavior Branch within National Institute of Child Health and Human development (NICHD) at the National Institutes of Health (NIH) from 1992 until 2005. In 2006 Dr. Lyon was named one of the ten most influential people in American education during the last decade by the Editorial Projects in Education Research Center (Education Week) for his work in ensuring that scientific research occupies a central role in educational practices and policy.    He also currently serves as a distinguished research scholar in the school for Behavioral and Brain Sciences and the Center for Brain Health at the University of Texas-Dallas.  The website address for SES is www.reidlyon.com