Interview with Reid Lyon: Reading First is the largest concerted reading intervention program in the history of the civilized world.

Michael F. Shaughnessy
Senior Columnist EducationNews.org
Eastern New Mexico University

Question 1: The Reading First Impact study has recently been released. In a previous interview you had predicted that the Department of Education's Evaluation would not show significant effects for the program because of a number of possible problems with the study. Can you elaborate on your initial concerns?

Yes, my major concerns are:

· The Impact Evaluation was delayed significantly in its design and implementation;

· The evaluation treats Reading First as both a policy and programmatic independent variable that did not lead to significant differences between Reading First and non-Reading First schools on a measure of reading comprehension. But it is what teachers do in imparting instruction that improves reading comprehension if the instruction is implemented with fidelity, not the overarching policy. As it stands now, we know very little about the impact of different instructional programs on the relationship between specific program characteristics and development of all reading components – not just reading comprehension.

· The probability that the sample studied in the Impact evaluation is representative of all Reading First schools is limited; Because of the delay, the evaluators could not randomly select a nationally representative sample. Far too few Reading First schools (119 from a total of 5,880) were studied given the overlap (contamination) between Reading First and non-Reading First schools with respect to what was occurring instructionally in the classroom.

· I share with Mike Petrilli from the Fordham Foundation the concern that the delay in designing and implementing the evaluation made it impossible for the states that won the first grants to participate in the study given that there programs had started before the evaluation was initiated. One could hypothesize that these states began their Reading First implementations with solid implementation plans in place.

· I also share with Petrilli the concern that the schools selected for study were the ones that just barely won grants under the program, which were compared to schools that just barely missed funding. (Schools are ranked according to various criteria, such as poverty, need, etc.). Petrilli points out that the schools where you would expect the greatest impacts from Reading First are the poorest ones, enrolling students who are further behind in reading–schools that would have been ranked at the top of the priority list. These schools weren't included in the study;

· Because of the delay the current impact study scope and depth departed significantly from the scope and comprehensiveness of the evaluation mandated in Section 1205 of the Reading First legislation. I have outlined the requirements of the evaluation mandated in congressional language in Section 1205 below;

· The findings of no significant differences in reading comprehension outcomes presented in the Interim Report are difficult to interpret. This is because, as noted earlier, many non-Reading First schools were implementing the same programs and professional development opportunities as the Reading First schools. This impact evaluation is not a true experiment which could have certainly been done given the tremendous financial resources allocated for the evaluation. As Tim Shanahan has pointed out, the comparisons made were not Reading First with non-Reading First schools, but Reading First with less-Reading First schools.

· The interim report does not provide evidence that the impact study examined specific relationships between each of the essential components of reading and overall reading proficiency as mandated in task 1 in Section 1205 of the law. All that is known from the interim report is the amount of time spent in instruction for each of the components. It is not known the extent to which limitations or strengths in each of the components influenced reading comprehension;

· The interim report does not provide evidence that the impact study measured the extent to which different materials (including assessments and programs) improved reading proficiency as mandated in task 5 in Section 1205 of the legislation. Thus, it is impossible to determine which students benefitted from which materials (assessments and programs) and under which conditions;

· The interim report does not provide evidence that the impact study measured whether specific assessment instruments and strategies helped teachers identify specific reading difficulties as mandated in task 6;

· The interim report does not provide data on the effects of professional development on the improvement of reading proficiency as mandated in task 7 of Section 1205 of the congressional language;

Question 2: You mentioned that the delay in initiating the design and implementation of the evaluation limited its scope and capability to produce interpretable findings and inform improvements. Can you explain this issue in more detail?

Yes.Bob Sweet and I felt that a comprehensive and rigorous evaluation of the Reading First program was essential. As such, we crafted 10 primary analyses and measurements that an independent evaluator was required to carry out (again see Section 1205 of the law). Specifically the law required:

1) An analysis of the relationship between each of the essential components of reading instruction and overall reading proficiency.

(2) An analysis of whether assessment tools used by State educational agencies and local educational agencies measure the essential components of reading. 

(3) An analysis of how State reading standards correlate with the essential components of reading instruction.

(4) An analysis of whether the receipt of a targeted assistance grant under section 1204 results in an increase in the number of children who read proficiently.

(5) A measurement of the extent to which specific instructional materials improve reading proficiency. 

(6) A measurement of the extent to which specific screening, diagnostic, and classroom-based instructional reading assessments assist teachers in identifying specific reading deficiencies.

(7) A measurement of the extent to which professional development programs implemented by State educational agencies using funds received under this subpart improve reading instruction. 

(8) A measurement of how well students preparing to enter the teaching profession are prepared to teach the essential components of reading instruction.

(9) An analysis of changes in students' interest in reading and time spent reading outside of school

(10) Any other analysis or measurement pertinent to this subpart that is determined to be appropriate by the Secretary.

No doubt, these are complex analyses requiring research designs and methods (including sampling strategies) appropriate to the evaluation targets. Because of the complexity of the required evaluation, the law provided $25 million dollars PER YEAR over a six year period (total = $150 million) to ensure that the evaluation research tasks could be accomplished. The amount of funds set aside for the external evaluation was arrived at following a survey of evaluation researchers who were asked to identify the cost of the most rigorous evaluation possible. Twenty-five million dollars per year was the arrived at figure. It is unclear how much the current evaluation cost nor is it clear where the bulk of the funds set aside for the evaluation were used. The law stated that such funds "may" be used for technical assistance and that could be the case.

I anticipate that many will argue that the scope the evaluation mandated in the legislation is far too ambitious for an external evaluation study. We wanted it to be very ambitious. But this was not to be as noted in this footnote of the report:.

"The Reading First Impact Study was originally planned as a randomized control study, in which eligible schools from a sample of districts were to receive Reading First funds or become members of a non- Reading First control group. The approach was not feasible, however, in the 38 states that had already begun to allocate their Reading First grants before the study began. Furthermore, in the remaining states, randomization was counter to the spirit of the Reading First Program, which strongly emphasizes serving the schools most in need (p. 7)."

The delay led to an inability to design the study and address the evaluation targets specified in the law.Had the evaluation been ready to be implemented in the early stages of Reading First the amount of critical information that we would have learned would have increased dramatically.

Some of the evaluation targets delineated above would have required a different design and focus than that which was employed in this impact study. For example, the evaluators would have to carry out systematic measurement of proficiency in all domains of reading development noted in the legislation and to ensure the use of the same measures across all Reading First sites – not just time spent in instruction for each of the domains. Moreover, tests of the effects of specific materials (assessments, programs) on reading outcomes would require designs capable of showing causal effects.

Unless the final Impact Study report addresses the required analyses and measurements articulated in the legislation and listed above, it is not the study that the Congress intended.A significant amount of information will have been lost. More importantly, it will not be possible to answer a fundamental question:For which students are which instructional materials (programs) most beneficial under well defined conditions (professional development, implementation fidelity, and so on).

This is not a trivial issue because many of the measurements and analyses that we felt were essential to the accurate interpretation of any findings, positive or negative, do not appear to have been carried out. Again, to be fair, this was due to the delay in getting started.

To elaborate on a couple of points I made earlier, the current design used to test the evaluation questions identified in the Interim Reportis not optimal for addressing task number 5 (see above) mandated in the legislationwhich would be best examined using a randomized controlled study.

In the Interim Report, specific programs and materials were not identified although it is well known that core programs will differ in their alignment with SBRR, the content presented, their scope and sequence, and relative coverage and emphasis on each of the five domains of reading subsumed within the Reading First legislation.

Tim Shanahan has pointed out that while differences in the effects of core programs would have been identified, the frequent revision of commercial programs based on state adoption requirements would render any findings obsolete following the next adoption.

This is a legitimate argument but one I disagree with.Examining program-specific effects on outcomes would have clearly yielded objective information for a publisher to use in any future revisions. The evaluation we included in the congressional language could have could have also provided critical information on whether particular programs were more likely to be implemented with fidelity, and whether particular program assessment and instructional characteristics were influential in producing specific student outcomes. This would have been possible given that every state and participating LEA identified the specific programs they were using.

Also recall that the majority of core programs adopted by states and schools had not been tested for effectiveness because of a congressional decision to change that criterion to the softer "based on SBRR" criterion.Addressing task 5 had the potential for providing effectiveness information for each program whether they are commercial basals, supplemental programs, or comprehensive school models as they were implemented during the initial Reading First six year term. Many of these programs had made claims of effectiveness.It is important to test those claims.

What I continue to argue to the consternation of commercial basal program publishers and private vendors is that at some point, education must become more serious about holding commercial publishers of educational materials accountable for the effectiveness of their programs. Establishing effectiveness and defining explicitly the conditions under which programs, approaches, and strategies are effective takes the basic effectiveness variable off the table.A focus can then be placed on determining why programs are effective in one context and not in others. This moves the analysis to issues related to teacher familiarity with, and competence in providing the program, and the essential implementation factors that can make or break a program's effectiveness – proven or not.

Let's cut to the chase –Are the core programs adopted by states and districts actually effective and under what conditions and do the effectiveness data support market claims?

We don't know. Remember, and I am repeating myself, the majority of core reading programs adopted by states and districts had never actually demonstrated effectiveness. That was not required in the law following Congressional revisions of the initial language. The programs only had to be "based on SBRR". For years, commercial programs have given lip service to determining the effectiveness of their products and had the evaluation addressed all of the targets mandated in the law, more information would have been available to determine whether these claims were valid.

Question 3: You have made the point that this impact evaluation cannot draw strong conclusions about the lack of significant differences between Reading First and non-Reading First schools on reading comprehension outcomes. Can you explain this in more detail?

Tim Shanahan said it best when he stated that "the comparisons were not Reading First with Non-Reading First schools, but Reading First with less-Reading First". There are several reasons for this.

When Bob Sweet and I crafted the legislation, we knew that SBRR would be a foreign concept to many as would using continuous data collection to differentiate instruction.For this reason, substantial funds (20% of the Reading First allocation) were to be used for professional development. However, we stipulated that these funds could be directed toward both Reading First and non-Reading First schools.

In some cases, states added their own funds to augment Reading First professional development programs. For example, Alabama and Arizona were among several states that dedicated substantial state funds to ensure that "Reading First Like" programs were implemented in all elementary schools.

Programs were created in several states by the federal government to provide professional development and technical support to non-reading first schools to develop their capacity to implement Reading First assessment and instructional criteria. An analysis reported by Tim Shanahan indicates that 60 percent of Reading First and non-Reading First schools were following the same curriculum by the third year of implementation.Shanahan also reported that school districts like Aurora, IL and Syracuse, NY required all of their non-Reading First schools to adopt the same reforms as Reading First schools using local money.

It does not appear that the current impact study specifically addressed this overlap in the evaluation, although the final report may present these data.In many cases, one would expect Reading First and non-Reading First schools to be more similar than alike in their impact on reading outcomes.Again, and I am beating a dead horse, the comprehensive external evaluation mandated in the congressional language would have allowed for fine-grain control and analysis of these confounds.

Question 4: Some commentators have suggested that the absence of significant differences between Reading First and non-Reading First schools was a result of too much emphasis on basic skills in Reading First schools. Is this observation accurate?

If I understand the data presented in the interim report correctly, it is not accurate.Ironically, the interim report presents data that the majority of instructional time in Reading First schools was focused more on comprehension than phonics. First grade classroom instruction included 21.4 minutes on phonics and 23.6 minutes on comprehension.Second grade classroom instruction included approximately 29.2 minutes of instruction on comprehension and approximately 14 minutes on phonics. Thus the conclusion that comprehension outcomes were not significant because of a focus on basic skills does not seem to square with these data.

In addition, the results indicated that less emphasis was placed on phonemic awareness (a basic word level skill), fluency and vocabulary than comprehension and phonics with vocabulary being absolutely critical for comprehension as is background knowledge. – But you can't engage vocabulary to determine meaning unless you get the words off the page accurately and fluently. As the IES director pointed out that the emphasis on comprehension, which again was greater than an emphasis on phonics, may not have been structured enough.I would agree that this is the more defensible possibility – Time does not guarantee effective instruction. It may also be the case, that students did not engage in sufficient wide reading.It could be that more instructional time was required to demonstrate effects on comprehension But remember, it is likely that many Reading First and non-Reading First school were implementing the same approaches leading to null findings.

Question 5: Some commentators have indicated that one explanation for the lack of significant differences is that scientifically based reading instruction does not work.Do the results from the interim study support that conclusion?

No, that conclusion is scientifically incorrect, and how one reaches such aconclusion from a study yielding null results escapes me.In their reviews of the research, both the NRC In their reviews of the research, both the NRC Report on beginning reading and the NRP report indicated that the elements of reading development that were addressed in Reading First were essential for proficient reading and that the effectiveness was mediated by systematic and direct instruction in the components.

The effectiveness of SBRR programs evaluated using appropriate experimental designs and methods have been replicated several times. In fact, a study published in the highly regarded Reading Research Quarterly by Mathes et al. and which won the Albert J. Harris Research Award from the International Reading Association found that explicit and comprehensive reading instruction was significantly effective in dramatically reducing reading failure. There are many studies that support this conclusion.

Moreover, replications of the NRP Report results have also stood up to scrutiny when the replications are designed appropriately. For example, two recent invited papers published in the Elementary School Journal concluded that there was no significant difference in the effects of systematic versus unsystematic phonics instruction as reported in the NRP. However, a re-evaluation of these two studies published in the archival and peer reviewed Journal of Educational Psychology, confirmed the findings of the NRP while pointing out why the ESJ design was not appropriate for the question under study. The JEP study was not an invited paper, was submitted through the formal peer review process, and underwent rigorous evaluation prior to publication.

Given that instruction guided by SBRR has been consistently found to be effective, the questions that must be addressed is whether the core programs used in Reading First schools were completely aligned with SBRR, whether teachers were adequately prepared to assess and teach the content, and whether the programs were implemented with fidelity. These issues are especially important for a study yielding no differences, or null results. There is an axiom in research that the null hypothesis cannot be proven. What this means for the RFIS is that if the results show no differences by virtue of the intervention, it does not mean that the intervention is not effective. The task is to determine why the null results might have emerged.

The IES Director pointed out additional reasons that could explain the results that have merit.He suggested that the instruction may work but it was not sufficient enough to significantly impact comprehension even if it improved decoding skills, reading fluency, and vocabulary. This is indeed possible.Had all of the evaluation targets mandated in the law been specifically addressed, data that could inform this possible interpretation would have been available.

Question 6: Some have argued that the Reading First schools should have shown a greater impact on comprehension because more time was allocated for reading instruction in Reading First schools. Does this conclusion have merit?

I don't believe so but I certainly stand ready to be corrected.The interim report presented data that the actual time spent in daily instruction in the five reading domains in Reading First classrooms was in the 59 minute range and about 50 minutes in the non-Reading First schools. While this is statistically significant, I have my doubts that 9 minutes of actual daily instruction will produce a great deal of difference.

Moreover, it is surprising that the instructional time in these schools is far below what state level Reading First implementations require, which hovers around the 90 minutes of core reading instruction and an additional period (usually 20- 40 minutes of supplemental instruction. So here we have a situation involving two groups of schools selected because they are high poverty and low achievement. One group is funded through Reading First; the other group is not. The Reading First implementation results in 9 minutes of additional core instruction and there is no mention in the report of the supplemental instruction that should have increased time on task to approximately 120 minutes. Understanding these null results requires a close examination of why the implementation was not more effective in these schools.

Some in the press have emphasized that the weekly total of instructional time favors Reading First schools with close to an hour in additional instruction, but time spent in daily instruction is a more compelling indicator of instructional intensity. For high risk students in high risk schools, an additional 9 minutes of daily core instruction is not going to make a dent in their development of reading skills. So all we know is that 59' of instruction (presumably in an unspecified basal) is no better than 50' of presumably unspecified instruction. In Reading First schools,a few more minutes were spent in phonics instruction, but even this amount of time in explicit phonics is not adequate for students at risk for reading problems, especially when the total amount of time in other components is insufficient.

The Sample is of great concern.It is important to understand the nature of this variation in instructional time between the schools in the study sample and those that were not representedin the study as it applies to sampling issues. This issue must be addressed aggressively because a great deal can be learned to improve the program from a more comprehensive analysis of allocated instructional time.

Question 7: In your view, has the press provided accurate interpretations of the results presented in the interim report?

Some reports have been fair and others not so fair. To be sure, helping the public understand the details we have discussed in the interviews is difficult given the complexity of the issues surrounding research design, sampling issues, departure from the evaluation scope mandated in the law and so on.

That said, I would have liked to see more in the press informing the public that any conclusions drawn must be interpreted with a significant degree of caution and also provided some facts that would have explained the need for caution. Many of these caveats are easy to explain, but somehow they never seem to be emphasized when reporting on educational research findings.

In some of the press accounts there were the usual "hooks" that I suppose are meant to titillate the reader. For example, one press account reported that thousands of students were involved in the study which is not the case. The unit of analysis in the study, I believe, is the schools, so power is determined by the number of schools in the sample, not the number of students. As sections of the report make clear, the number of schools is not adequate to assess many questions involving differences with the two groups of schools evaluated for the study. This problem is puzzling since the budget allocated for the evaluation should have been more than adequate to expand the sample size.

Some outlets referred to Reading First as a "phonics based" program which it is clearly not (read the legislation for goodness sake). Some outlets continue to make this mistake which might be because they want to kick up the flames in the "reading wars". It is hard to understand the press's continued love affair with the phonics-whole language dichotomy other than its eye-catching appeal or that they believe the readership cannot understand that reading development requires the integration of several essential reading skills.

This may be a wacky analogy, but maybe it can be explained like the performance of an eight cylinder car engine. When all cylinders are firing, performance is good. If one or more cylinders are misfiring, then performance suffers.Each cylinder can be thought of as a necessary but not sufficient reading component that must be integrated with all the others to produce reading proficiency.

There was the usual mention that Reading First was based on the NRP findings, which, in part, it was.However, the NRP is frequently characterized as a "phonics based" Report which of course it is not (read it) and a "Bush Initiative" (it was convened during the Clinton Administration).

You also see in all press accounts a reference to the Reading First "Scandal" with blasts from Miller and Obey charging that the program was rife with conflicts of interest and conflating this inaccurate statement with the null findings in the interim report. The press has a responsibility to report these statements by congressional members.But the press should also report there has been no finding of actual conflicts of interest by the OIG or the Justice Department and that the specific allegations of individuals with particular programs have not been supported by the OIF investigation.

Question 8: Is it your perception that there is an unusual degree of venom directed toward Reading First from members of the reading community and will the interim report fuel this antagonism?

There is no doubt that those opposed to Reading First and SBRR will be energized in their protests against the program given the findings presented in the Interim Report. I doubt that many will have read the report but will pay more attention to the press accounts which many, as pointed out earlier, do not provide the necessary interpretation caveats.

It is hard not to be taken aback by the degree to which many in the reading community want to see Reading First fail, particularly when the programs, methods, and approaches advocated by this constituency or that constituency have never come close to any systematic impact evaluation of the scope implemented with Reading First.

Interestingly, similar attacks have not been made against the Title I progam.Somehow the press and the congress has been silent on the fact that the Title I program, which has served as a massive entitlement funding source to districts and schools, is allocated approximately $3 million dollars a year to ostensibly "evaluate" a $15 billion dollar program – and that evaluation only measures whether money gets from point A to point B, not whether it is effectively helping students who are reading significantly below grade level. Again, no education program to date has ever undergone any evaluation approaching the scope of the current Reading First Impact study and certainly has come even remotely close to the comprehensive evaluation we actually required in the Reading First legislation (see Section 1205).

Question 9: Are there other factors that we should take into account when interpreting the interim report?

Indeed, in all the angst surrounding the Reading First program it seems we have not attended to the fact that it is a relatively new initiative characterized by massive implementation challenges and with substantial growing pains that would characterize any very complex initiative implemented in the public schools. If you visit as many Reading First schools as I have and talk with as many Reading First directors and teachers as I have, the complexity associated with implementation of both the policy and the instructional programs becomes readily apparent. It is more the rule than the exception that during the first two years of Reading First implementation in districts and schools, teachers were first learning to understand, administer and use the results of assessments to inform instruction.

As they were learning thesenew concepts,they were also taking part in state reading academies to learn more about  the foundation of SBRR (in 5 areas of reading in k-1, in 4 areas  of reading in 2-3).

In addition, as they were learning and using new assessments and taking part in professional development academies and workshops, they were simultaneously learning how to use a new approach to instruction and how to integrate core program instruction with additional interventions when required to meet individual student needs.This was done at the same time they were learning about center activities, grouping students for instruction and aligning and using supported classroom libraries.

It is important to ask whether any program that has added this amount of new learning to a teacher's other responsibilities including going to IEP meetings, attending parent conferences, preparing for their instruction in math, social studies and science, serving on school wide committees and a host of other tasks could demonstrate substantial gains after only two years. What is amazing is that despite this unbelievable load, Reading First teachers and their leaders rose to the occasion and have done and are doing a superb job. Also note that the GAO and OMB reports show that they feel that this job is essential and that it is having a major impact.

Question 10: What types of additional information would be helpful in drawing conclusions about the effectiveness of instructional interactions that take place within Reading First and non-Reading First schools?

The quality of the implementation of both policy and the instructional approaches emphasized by the policy must be measured. There are many questions that would have to be posed and answered to obtain a clear picture of implementation fidelity, but a few come immediately to mind.

First, it would be informative to understand whether the quality of the infrastructure (state and district leadership, professional development, building level management, assessment and evaluation capacity, etc.) actually reflect what was described in state and district grants.The fact is, it takes time to build the infrastructure at the state, district, and school levels that is essential to support the implementation of the program. We need to better understand how different Reading First schools managed this process.

An understanding of the quality of the infrastructure and its relationship to implementation fidelity is critical in interpreting outcome data.Schools could be grouped according to high and low quality infrastructure (this sounds easier than it is) holding programs constant to get some idea of how infrastructure impacts outcomes.

Third, outcome data should be collected from schools using different programs, holding constant the quality of the infrastructure to help determine the conditions under which programs reading proficiency.

There are many more issues that have to be addressed to obtain a comprehensive understanding of what has to be done to implement a policy and its constituent programs successfully in the complex world of schools. There is a tremendous amount we can learn from this Reading First Impact Study. It certainly points out the complexity of attending to the multiple confounds that can reduce interpretation of the data.It provides information about the program and the evaluation itself that can lead to improvements in both.It reflects the very hard work and excellent abilities of the evaluation team and their commitment to carry out the best evaluation possible given the amount of time to design, implement, analyze the data, and report the results. The Reading First program has been in existence only a short period of time but we have already learned where many of its features can be improved. The evaluation carried out reflects a commitment to evidence based education and accountability.Hopefully it will inspire policymakers and scientists to carry out equally robust evaluations of policies, and educational and instructional programs.

In closing, let me say this.Reading First is the largest concerted reading intervention program in the history of the civilized world. Most importantly, it is one of the few Federal State-Grant Programs to undertake a rigorous impact evaluation. We set aside $25 million dollars per year for six years to carry out the most comprehensive evaluation of an education program to date. Unfortunately significant delay in designing and implementing the evaluation did not allow it to address many of the evaluation targets specified in the legislation (Section 1205), making interpretation of the data difficult and reducing the potential to inform specific improvements. That said, most evaluations of formula funding programs can only tell us how the money was spent, nothing about impact, theory of action, or why we might not have gotten the results that we expected. The Title I program for example costs the taxpayer $15 billion dollars per year with $3 million dollars set aside for an evaluation of whether funds get from point A to point B.Title I has never been evaluated using appropriate designs and methods to determine the extent to which the program effectively helps students who are reading significantly below grade level.Now that is a scandal!

Putting aside the current impact study's shortcomings and its departure from the scope of the mandated comprehensive evaluation in the law, the key is to use any trustworthy information from this evaluation to understand what worked and what didn't so that the intervention and its implementation can be improved and the desired impacts obtained.

This is not a cause for mourning and political opportunism, but a cause for deliberation and careful consideration of all the possible explanations - ineffective treatment, poor implementation, diffusion of funds, active treatment in the control condition, and many other factors.

It is also a time to be very careful in drawing conclusions from this study and to be very clear about its limitations in making inferences about the success of the policy and the success of the instructional model emphasize in the model. It has been the bane of education to implement policy with very little research foundation and very little effort at rigorous evaluation. Change is hard!

This said, my specific answer to your first question is yes, my initial predictions were realized and there were other concerns that emerged in my review of the Interim Impact Study Report. I have no doubt that IES and the contractors who carried out the study did the best they could to conduct a fair and objective evaluation and I am impressed by the hard work that went into it.It is very possible that the final impact evaluation report will address all of the concerns I identify here.If it does, I believe we will have learned a substantial amount of information that can explicitly guide improvements.If it doesn't, we still have learned a great deal.It is also the case that some of the shortcomings of the evaluation may be because we did not write the legislation precisely enough such that one could readily understand the absolute necessity to carry out a comprehensive impact evaluation. I think that we did write Section 1205 clearly within the constraints of conference committee negotiations on particular language used, but if not that is another improvement that must be carried out.

Dr. G. Reid Lyon, an internationally recognized authority in educational issues announced the development of SYNERGISTIC EDUCATIONSOLUTIONS (SES) a consulting company to advise in the implementation of evidence-based assessment and instruction practices, professional development programs, development of education policy at local and state levels, and the development of assessment and evaluation programs for colleges and departments of education preparing for regulatory and accreditation activities. Prior to his most recent position as Executive Vice President for Research and Evaluation at Higher Ed Holdings, Dr. Lyon was the Chief of the Child Development and Behavior Branch within National Institute of Child Health and Human development (NICHD) at the National Institutes of Health (NIH) from 1992 until 2005. In 2006 Dr. Lyon was named one of the ten most influential people in American education during the last decade by the Editorial Projects in Education Research Center (Education Week) for his work in ensuring that scientific research occupies a central role in educational practices and policy.He also currently serves as a distinguished research scholar in the school for Behavioral and Brain Sciences and the Center for Brain Health at the University of Texas-Dallas.The website address for SES is www.reidlyon.com

Published May 5, 2008

June 4, 2008 An On–Going Discussion with Reid Lyon


Comments (69)

Nancy Salvato
Said this on 7-5-08 At 07:35 am
The key to what Reid Lyon was trying for is in the following quote: Establishing effectiveness and defining explicitly the conditions under which programs, approaches, and strategies are effective takes the basic effectiveness variable off the table.A focus can then be placed on determining why programs are effective in one context and not in others. This moves the analysis to issues related to teacher familiarity with, and competence in providing the program, and the essential implementation factors that can make or break a program's effectiveness
Sandra Nance
Said this on 7-5-08 At 09:27 am
I have been in education for over 25 years, 4 of those with Reading First. This is the first reading initiative I have been involved in that actually works. With this program, struggling readers in high poverty schools are finally receiving the explicit, systematic reading instruction needed to become proficient.
Patricia Mathes
Said this on 7-5-08 At 12:41 pm
The interview with Reid Lyon on the Reading First impact study pointed out the complexity of the implementation issues associated with the program. If you review the Interim Report, it becomes clear that the design of the study and its methodology, while strong in some ways, did not allow for an understanding of the many factors that influenced the results. It is also not clear why the evaluation did not design the study to address the number of essential evaluation questions Lyon listed in the interview. As it stands now, the findings are too ambiguous to base any policy decisions on.



Jessica Lindl
Said this on 7-5-08 At 05:02 pm
Dr. Lyon highlights the most important challenge the education community faces today: "it is what teachers do in imparting instruction that improves reading comprehension if the instruction is implemented with fidelity, not the overarching policy." How do we as a nation work together to improve implementation fidelity by better supporting and preparing our teachers? Eliminating incorporating SBRR into instruction is not the answer. Eliminating policy is not the answer. Declaring right or wrong instructional approaches without SBRR is not the answer. Working together and focusing on the needs of our teachers and our students is, let's start now.
Patricia Mathes
Said this on 7-5-08 At 05:36 pm
The interview with Reid Lyon on the Reading First impact study pointed out the complexity of the implementation issues associated with the program. If you review the Interim Report, it becomes clear that the design of the study and its methodology, while strong in some ways, did not allow for an understanding of the many factors that influenced the results. It is also not clear why the evaluation did not design the study to address the number of essential evaluation questions Lyon listed in the interview. As it stands now, the findings are too ambiguous to base any policy decisions on.

Eric J. Ban, Ed.D.
Said this on 8-5-08 At 02:02 am
Surprisingly Clear Purpose
Systematic Approach for Understanding
Beneficial Recommendations for Kids
Solid Policy
Smart Funding
Strong Implementation & Results
Thank you for doing the right things and doing things right!!
Cindy
Said this on 8-5-08 At 10:50 am
There is no doubt that Reading First has impacted our district. We have two reading Reading First schools and three non Reading Frist schools. Our state has done an excellent job of providing our teachers with quality professional development that we as a district have shared with the non Reading First schools. If you were to compare our Reading First schools with the non Reading First schools, you would see that they are the same. All of our schools use Success for All and provide similar reading interventions. (We sought other grant funding to support our non Reading First schools.) Our scores across the district on state testing continue to improve each year. Fidelity to a comprehensive reading program, plus intensive professional development for teachers, has really made a difference. Reading First was the shot in the arm we needed to make continued growth in our district.
Mary Dean
Said this on 8-5-08 At 11:17 am
Before reading this article, I wrote a letter to Margaret Spellings regarding the study in question. Below is my letter.
I was disappointed to open my local newspaper last Friday, May 2, to find an article entitled
Margie Gillis
Said this on 8-5-08 At 12:05 pm
Lyon's interview points out several fundamental confounds in this interim
impact evaluation report that make any valid interim interpretation of the
data literally impossible. It is very disappointing that the evaluators did
not carry out the comprehensive evaluation they were actually supposed to
do. By not explaining more clearly the limitations of their evaluation, they
have departed from standard scientific practice and common sense.
It has done more of a disservice than provided clarity.


Yvonne Siu-Runyan, Ph.D.
Said this on 8-5-08 At 01:34 pm
Question: How is success evaluated? By merely sounding out nonsense words? This is not reading. What about comprehension and developing lifelong readers, writers, and thinkers? Hey, follow the $$$, and ask, "Who is getting rich and at whose cost?"

*******Reid Lyon's response: There are three points to be made in response to this question. First, the ability to derive meaning from print should be the primary purpose of learning to read. If the writer of the question had time to review my congressional testimony in 1997, this was the major point I made at that time, the major point I made before the House Committee on Education and the Work Force in 2004 and the major point I made in testimony in all the years in between. . The fact is that getting to meaning is pretty complex particularly if your experiences with oral language and literacy concepts are limited during the early childhood years.

Youngsters who are read to frequently and who engage consistently in oral language interactions with adults and other kids during the early years typically come to school with robust vocabulary, background knowledge and the linguistic building blocks essential for developing word level skills. These kids move relatively effortlessly from print to meaning because they learn to read the words rapidly and relate what is read to what is known(background knowledge) and understood (vocabulary).

This set of experiences is not the same for kids from disadvantage where access to books and language and literacy interactions with adult are sometimes in short supply. For these kids, the ability to acquire word level skills is more challenging and even when reading accuracy and fluency improves, comprehension remains limited because of limitations in vocabulary, background knowledge, and content knowledge. It would be wonderful if all youngsters could bypass learning about the linguistic building blocks of our language and the ability to read words quickly and fluently and go directly to meaning, but our language does not work that way.

You frequently hear from many who are more ideologically driven that teaching word level skills produces word callers and that the love of reading is only inspired when the youngster is immersed in rich literature where meaning is derived from exposure to the literature. That would be wonderful, but it is hard to love something you cannot do, and there simply is no way to get to meaning unless you read words accurately and fluently.

The Reading First legislation emphasized these point several times ensuring that applicants for funding under the law had to select and implement comprehensive reading programs that included instructional opportunities and interactions to develop phonemic awareness, phonics, fluency, vocabulary, and comprehension. It is correct to say that reading nonsense words is not reading
Doug
Said this on 8-5-08 At 02:50 pm
The initial evaluation of Follow Through, the largest educational experiment ever conducted at the time, suffered from the same unrefined analysis as the RF study. Subsequent analyses looked at different approaches within Follow Through and found significant effects. Hopefully the upcoming reports (or later re-analyses) will allow for interactions to be identified.
Debora Scheffel
Said this on 8-5-08 At 04:22 pm
The Lyon interview identifies a number of factors that need to be taken into account when interpreting the Reading First Impact study. I am not at all confident in the findings of "no significant differences" between Reading First and non-Reading First schools until it is understood the degree of overlap in curricula, professional development, and additional state funding for implementing Reading First-like programs in non-Reading First schools. A high degree of caution when interpreting the findings is in order.
Jo-Anne Gross ,President,Remediation Plus Systems
Said this on 8-5-08 At 11:08 pm
Thank You again Reid Lyon!
We need to help support teachers transition to teaching explicit systematic synthetic phonics (based on the NICHD Research)beginning with teaching teachers the phonemes of the English Language and their articulation so they can inform their students who struggle about this breakthrough research and show them how breaking words into phonemes then to phonics and learning to blend them for reading and segment them for speliing can in as little as 3 months make a non reader into a person able to read at least 3 grade levels higher fluently and as the journey continues they can read more and more text ,learn more vocabulary and comprehend more and more because they CAN read.Why would anyone in their right mind want to hurt children by denigrating and threatening this empirical research,this exciting forward thinking legislation and stop the movement towards improved success for students and teachers .There is a scent in the report I am afraid that appears to want to vindicate the disproven view that phonics stops comprehension-this war being played out by the educrats on both sides of the Reading War must be stopped for the well being of children-have you ever watched and met the little angels that get stress disorders in grade 2 because they can`t read due to faulty instruction that includes no systematic phonics?
That would stop this ridiculousness in it`s tracks.
Rowe Young
Said this on 9-5-08 At 12:06 pm
The article it self is good, that research to identify individual difference was explored is poor.In testing for visual and movement laterality difference, our experiments identified a very important difference between a large majority of unsuccessful students and a large majority of teachers in the research.
First of all, to explain what was happening, the project among other things was to seeking to identify lateral behavioral characteristics that identified reading disabled students from non reading disabled students.
One of the key factors that was found, was identifying how a subject used their hands in a bi-manual way.
During the course of the project, it was observed that many individuals use a bottom hand to exclusively control the directional rotation movement. When this one variable was identified, we blindly found previously identified LD students were often found to preform in this manner. These were interestingly, a less reading disabled group than LD students who were found to preform the task with an exclusive right hand turn on the top of the activity. We hypothesised that the more disabled group had difficulties having to do with auditory factors, not motor behavior. (see previously submitted abstract) Teachers, especially female who participated, often also used the right hand exclusive turning method on the variable (7 possible methods available). Could it be, that exclusive right top operational handed thinkers have a difficult time communicating with inverted rotational operational thinkers in a class room setting?
Marion
Said this on 9-5-08 At 05:37 pm
As a classroom teacher, I would love to know what is the best way to teach reading (i.e. to have a SBRR program). However, even with a wonderful program, I need the support of the school administration so that I have the materials that I need for the program. I must have books! Lots of books! Where is the money for books!
Jennifer Baumann
Said this on 12-5-08 At 11:35 am
What another political ploy. I open up the opportunity for any of these researchers to gather information from our Reading First School. We border two Indian reservations and have been struggling for years to increase student achievement, Reading First has changed this for our district. Not only have our reading scores improved, but the climate in our school have changed tremendously. It would be disheartening to see the funding cut for this program, it has opened up so many great opportunities for us.
James Herman
Said this on 12-5-08 At 06:05 pm
As Director of Reading First in Tennessee, I would like to comment on the Reading First National Impact Study.

The Reading First Impact Study Report seems to agree that non-Reading First schools are using Reading First principles in their reading programs. If this is true
Carole
Said this on 13-5-08 At 02:27 pm
I appreciate Mr. Lyon's careful and balanced review of the impact study. I have worked with 12 RF schools for 4 1/2 years providing technical assistance. I have seen progress that is documented by assessment data and by observation. The progress in schools does vary according to the fidelity of implementation and the degree of support from administration. However, all twelve schools have made measurable progress! I also know that teachers in non RF schools are taking advantage of PD opportunities funded by RF, divisions are implementing strategies promoted by RF in non RF schools, and the dialogue created by this endeavor has had a positive impact on education. I think most elementary educators are looking at reading instruction in a more critical manner and promoting a more balanced approach. I think policy makers and Congress need to look critically at data and consider using the mandated funds for assessing the impact on instruction of the program in a scientific manner before throwing away the progress that has been made. The children of our nation deserve the best education we can provide. When we have information to guide us in making better decisions we would be well served to use it.
Said this on 13-5-08 At 05:40 pm
As a Reading First Literacy Leader in Tennessee, I have seen the changes that have been made with the help of Reading First. Non Reading First schools are beginning to adopt principles of teaching and Scientifically-based reading that were brought to their attention by Reading First. I was teaching at a non Reading First school before this, and there was no mention of any of the priciples of good teaching that Reading First brings. The politicians need to bring this back into education and out of the political arena. I also do not see how they can conclude that these differences are NOT because of Reading First. Reading First students are making more gains than there non Reading First counterparts. For once, the Congress needs to think of someone other than themselves. Also, why have they not evaluated how Title I money is spent? They might be suprised to see the results.
Carla Jeffers
Said this on 13-5-08 At 09:45 pm
As the elementary supevisor in a very rural county in East Tennessee I can't explain the positive impact that Reading First has had with my teachers and students. When I first became a supervisor 10 years ago the first thing my teachers begged for was professional development that was on-going and sustainable. I wrote REA grants and we became one of the first districts to adopt the Guided Reading concepts. I continued that work with Reading First. I'm not saying I agreed with everything that Reading First required and it took some adapting on my teachers' part to adjust BUT it helped me to continue the great PD opportunities that I had started with REA. The program is still too new to truly understand the long term impact it will have on how teachers teach reading. I do wish the politicians would allow us, the education experts, to decide what works. There is no other profession that outsiders decide what is best for the professionals that have been trained in their field. How many of you would want a computer technician telling a medical doctor how to perform an operation? Leave education to the educators!
Dick Allington
Said this on 14-5-08 At 08:51 am
Gosh, Reid Lyon and crowd are beginning to sound a lot like the whole language people these days. The federal study demonstrated the RF did not improve reading achievement. Period. So now we get lots of reasons why this study is not a valid study and the why the results are wrong. That is exactly what the WL folks have been saying for decades. And they too provide lots of assertions and anecdotal episodes that WL did work, in their schools, with their kids. Personally, I'm not surprised that RF did not improve reading achievement. What this study demonstrates is the same thing that a previous 4 large-scale studies have demonstrated: Nothing the feds improse improves anything. What is missing from federal policy is a focus on teacher quality. Teacher effects are always, aqnd always have been, larger than any program effect. But once again the feds seem to think that expert teaching can be packages and delivered via commercial programs. Hasn't worked before, won't work now. Training teachers to use packaged materials does not develop expertise in teaching kids to read. What the focus on packaged programs does do though is mightily enrich those folks who produce the packaged products -- while neither teachers expertise or instruction nor kids' reading achievement improves. Time to begin to accept a half-century's worth of federal research data and look for a different solution to America's reading problems.
Debbie Frey
Said this on 14-5-08 At 09:11 am
In TN, only the neediest schools received the Reading First Grants. Therefore, if there is no gap in reading achievement between these formally low performing schools and the rest of the schools in the state, than Reading First is a definite success because it has raised those at-risk students' scores.

In TN, every elementary school and district is being required to implement the strategies recommended by the National Reading Panel; therefore, the achievement gap should definitely be shrinking because the strategies are the same, everywhere.

Schools and teachers do not embrace and implement programs with the same intensity. There will always be schools with staff members who adhere to instructional and intervention guidelines and those who don't.

I will be happy to compare the comprehension scores of our students with any other in the state or nation. We don't settle for the 25% retell score (comprehension) that is recommended by the designer of DIBELS, Roland Good. Rather, we encourage our students to reach for 75% or better!

Everyone must remember that Reading First is only a grant program and its requirements were based on thousands of research studies that were reviewed by the National Reading Panel. We shouldn't be calling these strategies "Reading First" requirements. These are the best Research-Based-Practices for teaching reading to students.

The two Reading First schools in my county qualified for the Reading First Grant because of high poverty and TCAP scores at third grade at 38% Proficiency. Within a year of implementing research based instruction our schools jumped into the high 70's for student Proficiency in Reading after 1 year and now both schools consistently score in the high 90's. We have significantly reduced the number of students being referred for special education. Nearly all of our students with IQ scores above 65 are performing at grade level in reading.

The harm of this report is already spreading in our county as the new elementary supervisor is questioning the use of scientifically based reading research in our schools to drive instruction.
Wandaleen Adams
Said this on 14-5-08 At 10:12 am
Reading First has improved the reading proficiency greatly in our school system. The professional development and Reading First Academies provide teachers with opportunities to be trainined in the reading components that build reading skills. The politicians need to take a good look at the first important step for educating students--teach them to read. The Reading Coaches have performed their duties and supervised the reading program. It is essential that we have reading specialists involved in the teaching of reading. Thanks to the Reading First program, our teachers have achieved increased learning for teaching all students to read.
judy
Said this on 14-5-08 At 10:37 am
I have been a REA and a Reading First Literacy Leader. I feel that RF has had a tremendous impact on our school. The structure, explict teaching and interventions provided through Reading First have helped our faculty and students to truly put reading first in their classrooms. I am very concerned as to what will happen if Reading First is not here.
Gilbert Lay(Principal)
Said this on 14-5-08 At 10:59 am
My school has participated in Reading First for the past four years. My teachers have benefited tremendously from the Professional Development. Our students have steadily progressed the entire time. I feel the program has boosted our test scores tremendously. We are a very rural school with a poverty level of 89% free/reduced lunch. Our students in k-3 have about all benchmarked since the 2nd year of Reading First. I would like to state that I feel the program has been very helpful to my school, students and teachers. I would like to see RF continued at full funding.
Eli Dokson
Said this on 14-5-08 At 12:25 pm
Ours is a small rural district, just completing our 3rd year of the Reading First program. Its impact has been dramatically positive in our PreK -12 school, not just in terms of students'improvement in reading, but in school climate throughout the building. Students are excited about reading, teachers are excited by the students' progress and have embraced the profesisonal development work, as hard and all-consuming as it often is. The work being done in K-3 through Reading First has influenced instruction throughout the school. Our principal has become a disciple for using assessment data to help each and every student in every grade and every subject. We are just beginning this process in the upper grades, but the Reading First model continues to provide inspiration. It's working.
Patrick Riccards
Said this on 5-5-08 At 12:09 pm
This is an important discussion of Reading First that is not being heard. If we've learned anything from the past five or six years, it is the importance of fidelity when it comes to scientifically based reading research and its implementation. How can IES and ED believe that the program has been implemented faithfully and with fidelity if it has spent less than 10% of what was committed by Congress to evaluate the program?

Clearly, more work needs to be done to evaluate the effectiveness of RF. Even IES has found that teachers are changing their practice to align with SBRR. That is a huge step forward.

From this interview and other articles from the past week, it is clear we should be looking at those schools and those districts that needed RF the most, not from those that barely qualified for the program.

SBRR works. The program was carefully developed, and the science carefully scrutinized. Clearly, we need to do the same for assessment of the program. Dr. Lyon has some good ideas to get us there.
Reid Lyon
Said this on 14-5-08 At 10:50 pm
Come on Dick Allington, how in the world can you interpret that something did not work when the sampling and implementation problems were so huge and not accounted for. Give me a break. If Reading First did or did not work, let
Amanda Wagner
Said this on 15-5-08 At 11:42 am
I would respectfully request that Mr. Allington participate in the 90 hours of professional development for Reading First teachers in the state of Tennessee. As a Literacy Leader in the Memphis City Schools system, I have accepted the responsibility of improving the teaching of reading at my school. Our 90 hours of professional development have nothing to do with 'reading in a box' products. Mr. Allington might be surprised to know that the teachers in my school frequently pull their research and presentations from the excellent sources found in I.R.A.'s "The Reading Teacher". Primary teachers who accept the challenge of working in high-poverty urban districts have long been victims of (literally)every Tom, Dick, and Harry's 'Reading in a Box' program to improve test scores. This paradigm was put in place LONG before REA or Reading First entered the fray. The true beauty of the Reading First grant in Tennessee is that the LEA's are given the choice of SBRR core programs. As for progress made under Reading First - our school has gone from a "High Priority" AYP status to "Good Standing" under the Reading First grant. Our 3rd graders have met or exceeded district and state proficiency rates in Reading/Language Arts during the course of Reading First. Was this grant not designed to assist 'failing' schools meet district and state Reading goals? If so, then Reading First has done the job. As one who spent their entire career in the school with the children - there is no doubt in my mind that amongst the plethora of programs, initiatives, superintendents, politicians, researchers, commentators, and other assorted folks who haven't seen the inside of an urban classroom for quite a while - this was government money well spent. Rest assured, our Reading First teachers will remain excellent teachers long after Congress decides what the new 'magic pill' for urban schools will be. They will remain excellent teachers because of the work and training done under the Reading First Grant in Tennessee.
Nancy Lay
Said this on 15-5-08 At 01:42 pm
I have been a Literacy Leader at a small rural school for the past four years and I have experienced firsthand the impact Reading First has had on our students as well as our teachers. It has puzzled me to no end as to why anyone could be against this program. Since the onset of Reading First at our school we have seen tremendous gains in reading scores. Our teachers now understand how to deliver SBRR reading instruction that is intentional and explicit and systematic. Our teachers are analyzing data and working as a team to improve scores for all students. Guess what?! It's working! Our students are making necessary gains to improve their reading skills. I am very pleased with the fact that last year our 3rd grade students scored 100% proficient or advanced on the Tennessee Comprehensive Assessment Program (TCAP). Is that not our overall goal, to have all students reading on grade level by the end of third grade? Also, Special Education referrals have been cut by 65% at our school. Could that possibly be because students are learning to read? I don't know how in-depth the researchers went with individual student performances, but that is what we analyze at our school. Every child is different and learns differently, so this must be taken into consideration when you are looking at lesson delivery and effectiveness. The same stands true for schools. Every school is different-- program fidelity, teacher attitude, etc. must be taken into consideration when conducting a study as to the effectiveness of a program.

James Herman
Said this on 15-5-08 At 02:25 pm
Reid,
Thank you for keeping the focus on the study and not someone's ideology.
Jim Herman
Julie Shirer
Said this on 15-5-08 At 09:38 pm
I am a literacy leader at a Reading First School in TN. Reading First has been such a positive for us. Our teachers are better trained and more prepared for teaching children to read than ever before. The instruction is explicit and all strategies are research based. Through the funding of Reading First we are able to meet the educational needs of all our children...the academically gifted and challenged and the socioeconomically advantaged or disadvantaged. We have a unique situation at our school...we serve children from families of doctors, lawyers, college professors, teachers, fast food workers, and the unemployed. THEY ARE ALL POSITIVELY AFFECTED BY READING FIRST. Based on this years' scores, most students have an upward trend in their reading progress. This has been evident the last several years in our standardized testing as well as our progress monitoring.
Reading First has allowed us to provide high quality professional development for our teachers which has spilled over to the other grade levels at our school. Most schools in our district have seen the growth in our children academically and are following the Reading First philosophy as much as they possibly can without the benefit of the funding. We have been able to provide reading tutors for our students and excellent materials to increase student achievement. Isn't this the goal...to increase student achievement? Then, certainly, Reading First has been a success for us.
Zig Engelmann
Said this on 16-5-08 At 07:41 pm
Comment #29 was written by Dick Allington, and he is guilty of several Allingtonisms, which are serious distortions of the history of education. One Allingtonism is the assertion that the last four large-scale studies the Feds have funded have shown that nothing works and that teacher effects are always
christinna landon
Said this on 17-5-08 At 10:44 am
Someone posted that we need more reading experts. Stop parents from being able to sue themselves. So much money is being wasted by greedy parents!!! We could be paying teachers to teach kids to read. Now they have become law assistants and actuaries. I don't remember the last time a workshop in curriculum was given. We are hell bent on how to stay out of court!
The districts are not at fault....the laws and lawmakers need to come down from their ivory towers and smell the roses from the rank and file.....Who is watching the store.
Christinna
joanne Yatvin
Said this on 17-5-08 At 11:09 am
In his analysis Reid Lyon reveals the many reasons why any large scale, mandated educational reform does not--and cannot--work. To wit, there is widespread misunderstanding of both the principles and the process of implementation; instructional fidelity is impossible; commercial vendors sell snake oil; school districts buy snake oil; professional development connected to RF is superficial; many professional developers are unqualified; RF has been sold as a phonics program; you can lead a horse(kid, teacher) to water but you can't make him drink. In additon, RF has its own special weaknesses that Lyon refuses to recognize. Two major ones are: there are several other essential components of reading, just as important as the five identified by the NRP and deified by RF; one size never fits all.

Fianlly, Allington has it right: it's the teacher, not the program, that creates readers.
Jill Kerper Mora
Said this on 17-5-08 At 12:38 pm
The policy-wonks involved in Reading First are missing the point. RF supports an erroneous paradigm of the reading process. Perhaps this stems from the National Reading Panel
Brett Dillingham
Said this on 17-5-08 At 01:47 pm
It's hard to take Reid Lyon seriously when it comes to the science of reading. I attended the Improving American Schools Act conference in 2001 where Reid spoke at a keynote. He said "It
Bonniwell
Said this on 17-5-08 At 02:14 pm
I have worked in three school districts in three different states since NCLB was started, and have watched as various versions of Reading First were implemented. While it may have benefited some districts that had poor reading programs to begin with, one obvious characteristic is obvious in all cases; high reading scores and/or great improvements in reading as measured by Reading First-approved instruments such as PALS and DIBELS are not indicators of good comprehension. I frequently work with students who supposedly read above grade level, but who cannot read and discuss books that are on or even below grade level. Too many children have learned that learning to read means responding to lists and multiple choice evaluations. For them, reading is not about discovery and pleasure and learning. It is instead endless opportunities to be measured and to risk falling short of some benchmark.

The real question is: what is being measured, and how useful is the measurement?
Pamela Perkins
Said this on 17-5-08 At 09:41 pm
Mr. Lyon: How do you explain Margaret Spelling's statements about the huge successes and magnificent improvements based upon Reading First when there are none?
Linda Ellis
Said this on 17-5-08 At 11:46 pm
The reason RF has failed is because there is no such thing as a one-size-fits-all curriculum. Dr. Lyon is wrong in his accusation that colleges of education do not properly prepare teachers. I've taught at four different colleges of education where students received between l8 and 21 hours of reading coursework. Yes, they got phonemic awareness and phonics. They also received training in assessment that guides instruction and how to match the instruction to the individual needs of students. In these programs, they were field-based where they learned with struggling readers. They learned that a one-size-fits-all curriculum does not work because every child is different and every struggling reader struggles in different ways. They taught strategies based on the needs of their students. However, in well over a thousand case studies that I have done through my students with struggling readers, overwhelmingly the students were overusing the graphophonic cueing system and underusing semantics and syntax. Yes, the strategy that works best for struggling readers is teaching them to use the context--"What would make sense there?" Teacher candidates were amazed to see their case study students take off when they were taught just a few strategies--contextual and structural analysis--when they provided strong models for them through the read aloud, echo reading, choral reading, etc. and when they gave them time to practice. They left my class convinced that when they go More recently, because of the emphasis on speed and accuracy, students face new challenges. They left my classes convinced that in their own classrooms they had to set up classrooms where they could differentiate instruction to meet the needs of individual students. The problem is that these future teachers need a great deal of support once they leave the colleges of education to continue to implement best practices. Like any profession, they must continue to learn and they learn most when they get into the classrooms with their students.They don't need to be handed programs. Support networks like the Literacy Collaborative provide that support. The International Reading Association forever has recommended that every school have a certified reading/literacy specialist on each campus to provide professional development and support. Richard Allington is right. It's the knowledgeable teacher who makes the difference, not commercial, one-size-fits-all programs. That's why RF hasn't worked. The emphasis must be on creating and supporting knowledgeable reading teachers.
Reid Lyon
Said this on 18-5-08 At 08:55 am
I very much appreciate that Dick Allington took the time to contribute to the discussion on the Impact Study interim report. Nothing will move forward without input and discussion from all perspectives. He correctly pointed out that whole language advocates have relied on anecdotes to support the effectiveness of their approach. He and I agree that anecdotes reflect opinion and not evidence. However the issue of evidence within a whole language context goes beyond anecdotes. A more salient point is that if one proposed to whole language advocates that they participate in a study comparing that approach with other approaches, the response would be that such a comparison would be irrelevant given their opposition to the measurement of outcomes, and in a more extreme position, their conclusion that evidence was in the eye of the beholder. The typical argument vis-à-vis the measurement issue was that outcome measures employed in objective studies were not an authentic proxy for what transpired in the classroom. Unfortunately no specific alternatives to reliable and valid measurement was forthcoming from whole language advocates, making it difficult to understand exactly what was transpiring in whole language classrooms. I don
Priscilla Gutierrez
Said this on 18-5-08 At 11:20 am
Gee, let's see what Bush's phonics czar had to say in testimony before Congress back in 2001:


"On the basis of a thorough evidence-based review of the reading research literature that met rigorous scientific standards, the National Reading Panel (NRP), convened by the NICHD and the Department of Education, found that intervention programs that provided systematic and explicit instruction in phonemic awareness, phonics, guided repeated reading to improve reading fluency, and direct instruction in vocabulary and reading comprehension strategies were significantly more effective than approaches that were less explicit and less focused on the reading skills to be taught (e.g., approaches that emphasize incidental learning of basic reading skills). The NRP found that children as young as four years of age benefited from instruction in phonemic awareness and the alphabetic principle when the instruction was presented in an interesting and entertaining, albeit systematic manner."

Interjection: The studies used by the NRP that Lyon is referring to only showed a very small effect size on a very small percent of the population -specifically at-risk children in kindergaren and first grade. Elaine Garan did a thorough analysis of this. However, the direct phonics instruction did not contribute to later reading comprehension. If comprehension is at the core of reading, then why the need for massive doses of direct phonics instruction?

"The President's reading initiatives have been developed on the basis of the best scientific evidence and knowledge relevant to reading development, reading difficulties, and reading instruction currently available."

Interjection: The so-called Houston studies that formed the basis for using phonics as "surgical strikes" i.e. the cure for reading problems had a number of problems including lack of random sampling (students from the best schools in Houston were taught using Open Court and were the only treatment group that got additioning tutoring) and the fact that the data did not support the findings. Researchers such as Gerry Coles who went back and looked at students' performance on comprehension in second grade found the 'loaded' treatment groups actually fared worse on comprehension ability. What an incovenient truth!

"Of particular importance within the President's reading initiatives is the requirement that funding for State and local educational agency Early Reading First and Reading First programs is contingent upon objective and rigorous peer review of the grant applications that are submitted."

Interjection: Reading First scandal, anyone? Conflicts of interest and blatant bias, anyone? They can't crash our party, anyone? Let's beat the shit out of them, anyone?

"The President's reading initiatives ensure that locally determined and implemented programs for the assessment and evaluation of programmatic effectiveness are at the core of this critical program."

Interjection: Locally determined? Tell that to all the states and districts who were forced to use specific programs tied to the TAC Directors who wouldn't approve grants unless their own programs and/or assessments were used.
At the initial Reading First Literacy Academies, attendees were given lists of reading programs and told that if they selected a particular program that sic had ties to the TAC Directors, their grants were virtually assured of getting approved.

Reading First was nothing more than a huge profit scam for a select group. Many of the NRP's actual findings got highjacked. The Department of Ed's summary that was widely distributed to school districts contradict the actual findings, including the value and use of phonics which the NRP cautioned against placing too much emphasis or importance upon.

And so after 5 years of phonics and commercial programs being touted as scientifically research-based, and being crammed down people's throats , NOW we are talking comprehension and caution?

What a sinverguenza...
Reid Lyon
Said this on 18-5-08 At 12:19 pm
Joanne Yatvin
Stephen Krashen
Said this on 18-5-08 At 02:38 pm
Concerning whole language

I have re-examined the literature in which whole language is compared to other approaches, including studies that were part of the National Reading Panel
S Johnson
Said this on 19-5-08 At 01:29 am
We need to follow England's lead. Read The Rose Report...

In the U.S., it looks like Core Knowledge's Reading Program is getting it right...
http://coreknowledge.org/CK/schools/KTR/index.htm

http://www.thewritingcode.com/pages/transcripts/pi...

"The teaching of reading is controversial. We've had the language wars
in this country over how to best teach kids to read. There have been
the proponents of the look-say method in which children are supposed
to pick up on the shape of an entire word. That was the idea behind
those dreadfully boring Dick and Jane readers that I grew up with. There's the whole language method in which children are immersed in a text rich environment and are spontaneously supposed to pick up on the ability to read the way they spontaneously learn to speak. There's the
method of phonics, in which children are drilled on correspondences
between sounds and letters of the alphabet. I think that any successful reading technique has to begin with an understanding of the logic of our alphabet, and also has to recognize that learning to read is not like learning to speak." -Steven Pinker,
Johnstone Family Professor in the Department of Psychology at Harvard
University

http://www.edge.org/q2007/q07_9.html

"We need to push harder for an education system that teaches evidence-based decision making while we hold our public leaders to a higher standard and less partisan behavior as we attempt to tackle some of the historically most difficult challenges facing the future
of humanity." - J. Craig Venter, Human Genome Decoder; Director, The
J. Craig Venter Institute

The Effects of Synthetic Phonics Teaching on Reading and Spelling
Attainment: A seven year longitudinal study

http://www.scotland.gov.uk/Resource/Doc/36496/0023...

Independent Review of the Teaching of Early Reading: Final Report, Jim
Rose, March 2006

http://www.standards.dfes.gov.uk/phonics/report.pd...

How phonics became easy as a-b-c: A report on how young children in
England should be taught to read is expected to endorse a
phonics-based approach.

http://news.bbc.co.uk/2/hi/uk_news/education/47946...

Reading system goes into schools

http://news.bbc.co.uk/2/hi/uk_news/education/67652...

Study spells success for phonics

http://news.bbc.co.uk/2/hi/uk_news/scotland/714781...

The 'Simple View of Reading'

http://www.standards.dfes.gov.uk/primaryframeworks...

The Usefulness of Brief Instruction in Reading Comprehension Strategies

http://www.aft.org/pubs-reports/american_educator/...

Jo-Anne Gross
Said this on 19-5-08 At 08:25 am
Congratulations To S Johnson for mentioning the Rose Report in England-Unlike the U.S. legislation which sadly leaves room for continuous Educrat debate at the expense of vulnerable children,the Rose Report created a response for the well being of vulnerable children and Tony Blair`s government legislated the use of explicit systematic synthetic phonics in the elementary years.The study responsible for the early years was so compelling that they abruptly ceased it in grade 1 to allow the control group to also learn how to read and spell.
No one is discussing that phonological awareness as a precursor to phonics teaching is so brilliant and even though it`s phonology like phonics,the teaching of reading beginning with the segmenting of oral sounds makes teaching reading a very precise science,like learning the scales of the piano in order to eventually be able to play Mozart..there is truly no difference..This dogged determination to argue with this methodology is so harmful to your nation-in Canada we don`t have your legislation but many are embracing Dr Lyon`s NICHD Research synthesis as well as the Rose report to work towards improving our literacy for all kids especially kids at risk,our First Nations students and the20% of the population referred as Dyslexic-what we have witnessed undoubtedly is that most students are successful when taught this way to begin with.From my perspective in Canada your arguments are narcissistic and shockingly harmful to kids-if you can read discombobulated in Grade one you can learn orally what it means..and use it contextually properly in Grade 1 as well as read it-the arguments are so ridiculous.....
vern
Said this on 19-5-08 At 10:21 am
Hello All,
The truth is the sequence of instruction is important.

Please respond if you have been teaching in a Reading First School for 4-5 years and you have undertaken professional development as you learned new program. At the heart of Reading First is improved teacher knowledge, stronger instructional tools.

It is very interesting for me as a teacher in an RF school to read the comments of those on the sidelines talk about what I do.

We have undertaken a new approach, combined improved teacher professional development and we are expected to teach reading, math, science, social studies and be a social worked to each child.

If any of you know a better way to do this, get in an RF school and help.

I hope to see all of you, and the members of congress that cut this funding spend 180 days doing thsi very hard work.

Is it pefect, no! Is it better than what was, YES!

To all of you, great teachers learn new information every day. All teachers do not have rich abilities, so sequences support this development.

Join the fun of teaching in an RF school. Time to get off the sidelines and get into the classroom, everyday for 180 days, year in and year out.
Reid Lyon
Said this on 19-5-08 At 11:17 am
Dr. Mora makes several key points that I agree with. First, I also feel that comprehension is the reason we engage in reading instruction. Instruction in necessary linguistic and cognitive domains provides the means to an end
Stephen Krashen
Said this on 19-5-08 At 01:29 pm
Re: Comments 52 and 53:
Much of the enthusiasm for intensive phonics in England is based on the results of a single study done in Clackmannanshire, Scotland. A careful reading of the study shows that children taught with synthetic phonics were not
Stephanie Swims
Said this on 19-5-08 At 02:56 pm
As a teacher in a Reading First School for 5 years, I can assuredly say that my approach to teaching reading has been fundamentally changed.
I was trained as a whole language teacher in college, but never truly understood what a child needs to grasp in order to be a successful reader.

Reading First professional development has completely changed my understanding of the reading process along with giving me a greater appreciation of the reading product.

Now, when I am asked how or why I am teaching a certain concept I can confidently state that "research shows...". SBRR allows teachers to feel empowered to teach as they already knew how, with the research to back up what they are doing.

As long as I teach, regardless of grade level or program, I will always be cognizant of the 5 Basic Elements of Reading (Vocabulary, Phonics, Phonemic Awareness, Comprehension and Fluency) and their importance in reading instruction.
Post a Comment
* Your Name:
* Your Email:
(not publicly displayed)
Reply Notification:
Approval Notification:
Website:
* Security Image:
Security Image Generate new
Copy the numbers and letters from the security image:
* Message: