Opposition to Teach For America coming to Seattle

From: Joan Sias
Date: Mon, Nov 1, 2010 at 2:10 PM
Subject: about TFA proposal
To: schoolboard AT seattleschools.org
Cc: Seattle Public Schools Superintendent, Joan Sias

Dear Directors,

I have reviewed the Mathematica report that the Superintendent cited in her School Board Aciton Report in which she proposes that SPS enter into a contract with Teach for America.

My conclusion is that any person who holds up this report as evidence that TFA will have a positive impact on the achievement gap in SPS' high poverty schools does not understand this study nor its limitations.

The Superintendent is remiss for failing to inform the Board of the existence of the more recent and more relevant peer-reviewed study published by the Great Lakes Center for Education Research and Practice (http://www.greatlakescenter.org/docs/Policy_Briefs/Heilig_TeachForAmerica.pdf; attached as "Heilig_TeachForAmerica.pdf" ).

By citing this non-peer reviewed Mathematica report, the Superintendent seems to be suggesting that if SPS hires TFA test prep instructors to work in our high poverty schools, we can expect to see similar results as demonstrated in the Mathematica study.

I have several reasons for cautioning the directors from accepting this proposition.

1.  This study is not peer reviewed.

This study is paid for by three organizations that have a track record of promoting education reform

• The Smith Richardson Foundation  http://www.srf.org/grants/domestic.php  "Education and school reform have been central to the Foundation’s grant making in recent years, including efforts to assess the effectiveness of various school reform initiatives, such as charter schools.   
• The William and Flora Hewlett Foundation  http://www.hewlett.org/newsroom/12-major-foundations-commit-500-million-to-education-innovation
• The Carnegie Corporation

Members of these foundations were allowed to review the report before it was finalized, and may have influenced the final report.

2. This study has a major design flaw.  

The study formed two distinct control groups, to which TFA test prep instructors were compared. The control groups both contained sizable proportions of school teaching staff that were not certificated, and did not complete a bachelors or masters degree in education.  (statistics from page 28 of the pdf)

• Only 55% of the teachers in the full control group included teachers that possessed at least a bachelor's degree.
• Only 33% of the teachers in the "novice control group" included teachers that possessed at least a bachelor's degree.
• Only 64% of the full control group posseses regular certificates.
• Only 31% of the novice control group possessed regular certificates.

 

2. The results are weak.

• With the 100 classrooms (from 17 schools) included in the study, TFA test prep instructors have a small statistically significant effect on mean math acheivement, over the single academic year of the student, as measured by ITBS.
• from page 14 of the pdf: "[C]ontrol class students experienced “normal” achievement growth, shown in Figure 2 by a horizontal gray line. In contrast, the average TFA class student increased in rank from the 14th to the 17th percentile over the same period."
• For a graphical representation of the main effects, see Figure 2 on page 14 of the pdf.
• TFA test prep instructors have no statistically significant effect on mean reading achievement, as measured by ITBS.
• Compared to the control group (and which included a high proportion of teachers lacking a bachelors' degrees and regular certificates) TFA test prep instructors appear to cause a large increase in the standard deviation of the reading achievement (c.f., Figure VI.5 on page 52 of the pdf, page 41 of the printed report.  
• This means that even though there is no change in mean achievement, a proportion of students are doing better in reading than had they had a control teacher;
• while this would seem a positive benefit, the converse is also true:  A proportion of students are showing poorer achievement than had they had a control teacher instead of a TFA test prep instructor.  This I find to be an undesirable effect of letting TFA test prep instructors take the place of certified teachers.
• If we were to compare TFA recruits to teachers with at least a bachelors degree and with at least fours years of experience, it is likely that the performance of the TFA recruits would be even more unfavorable.

If the study had compared TFA recruits to certified teachers with at least four years of experience, the very modest effect of TFA test prep instructors on math achievement might well have disappeared:

On page 50 of the pdf (page 33 of the report) the report authors present results of additional comparisions. These additional comparisons are more relevant than the major results given in this report.

• The small TFA effect in math was reduced by more than 1/3  (i.e., from 3.1 NCE to 1.9 NCE)  when the researchers compared TFA test prep intstructors to certified teachers only; 
• The small TFA effect in math was reduced by about 25% percent (from 2.4 NCE to 1.8 NCE) when the 1st year TFA test prep intstructors  (rather than all TFA recruits) were compared to the original control group (that control group having a substantial proporiton of underqualified teachers). 

3. This study is not very relevant to SPS.

• This study concerned a small number of schools (17) that suffered from strong shortages of qualified applicants for teaching openings. Overall, about 40% of teacher in these schools were veteran teachers (having four or more years of experience), 40% were TFA test prep instructors (with 2 years teaching experience, on average), and about 20% were "novice" teachers.  Overall, in these schools, a high proportion of both veteran teachers (45%) and an even higher proportion of novice teachers (67%) lacked a bachelor's degree.
• The small effect of TFA recruits on math acheivement detected in this study would likely be smaller in a schools that have an adequate pool of qualified non-TFA applicatants for teaching staff openings.

  

This study brings out a major factor that the Board should consider in its decision:  The poor retention rate of TFA hires.

• We find this footnote on page 49 of the pdf: " In our sample, there were five TFA teachers in their third year, one in their fifth year, and one in their sixth year. Of the rest, 15 were in their first year of teaching, and 22 were in their second year.
• Thus, we see that in this study, of the 43 TFA test prep instructors included in the study, only 6 were recruits that had chosen to remain working ast test prep instructors beyond the initial two-year commitment period.

Why should a Board Director take into account the retenion rate?

• TFA recruits do not stay employed at the school for as many years as individuals who seek teaching jobs through the normal route.
• When a school has a policy to seek and hire TFA test prep instructors, there is an increaesd probabiity that these problems will also develop: 
• increased overall attrition rate of school staff
• long term increase in recruitment and training costs
• decrease in the mean years of experience of teaching staff
• opportunity for students to develop stable, trust-based relationships with school staff that has the potential to continue for many years is diminished

Another look at the data presented in Table VI.2 (page 49 of the pdf):

Comparison	Effect (expressed as NCE's)
ALL TFA's versus uncertified controls	3.12
All TFA's versus veteral controls:	2.71
All TFA's versus all controls (only 45% having at least a bachelor's degree)	2.43
ALL TFA's versus certified controls	1.92
ALL TFA's versus certified controls with at least three year's experience	???? (not reported)

About this this table:

• This table (based on results on page 49 of the pdf) shows that certification matters. The effect of TFA recruits diminishes as the qualifications of the non-TFA staff improve.
• The report does not provide the data that lets us draw a conclusion as to whether certified teachers with experience are more effective than certified teachers with limited experience. (hence the question marks in the last line of this table).
• It stands to reason that the effectiveness of certified teachers will increase as their experience increases. 
• The more recent peer-reviewed report by Great Lakes Center Kay http://www.greatlakescenter.org/docs/Policy_Briefs/Heilig_TeachForAmerica.pdf indicates that experience is an important factor, and that the last number in the table, if available, would be close to zero.

When TFA recruits are introduced into a system that does not suffer a shortage of qualified applicants, it is likely that i high poverty schools, the proportion of teachers that are both certified and have at least several years of experience will decrease.  This will likely lead to poorer outcomes for students at high poverty schools. If it is true that, on average, certified teachers with more experience have the best impact on students, then it behooves the District to try to create policies, programs, and supports that help to make teachers want to continue teaching in these buildings.

In summary,

When SPS hires TFA test prep instructors, they are making a choice against choosing individuals who already have a demonstrated commitment to the teaching profession, or who desire to embark on a professional teaching career, and have a much higher likelihood of remaining in the profession, remaining employees of SPS, and of becoming in the not-too-distant future a member of the corps of certified SPS teachers having at least four years of teaching experience.

SPS does not lack for a pool of qualified non-TFA applicants, many of whom intend to continue to live in this community for the long term, may have already demonstrated through working as a substitute teacher an interest in becoming an SPS teacher, and who may already possess considerable skill at teaching and managing a classroom..

It is hard to understand why SPS would prefer TFA applicants to such persons., especially given that a majority of TFA hires will not remain employed beyond two years, and do not for the most part intend to choose teaching as their profession