Teach Secondary Issue 13.4

Methodological challenges The Teacher Choices feasibility trials have thus far underscored the importance of close collaboration with teachers at every stage of the research process. Despite some exciting initial findings, however, there remain some methodological challenges associated with the Teacher Choices trials. For example, running causal evaluations of teacher practices means having to identify specific outcomes that we would expect the practice to influence, and evaluating these with sensitive assessment measures. When standardised test measures lack the necessary precision, evaluators like NFER have opted to create and validate bespoke assessments, or collect teacher- developed topic tests in order to evaluate the impact of Teacher Choices. T eachers regularly make decisions in their daily practice to try and improve pupil learning, with limited evidence available to support their choices. To try and help with this, we introduced a new approach to evaluation in 2019 called Teacher Choices, which sought to generate relevant and actionable evidence that could inform the dilemmas that teachers routinely face. After some disruption during the pandemic, we’re now launching a new series of Teacher Choices evaluations and taking stock of what we’ve learned so far. In Autumn 2023 we published the first set of Teacher Choice feasibility studies, including ‘AWinning Start’, which compared two approaches to lesson starters in Y8 classes. This study, along with others at both primary and secondary level, aimed to assess whether real-world, practice-based questions could be successfully evaluated using a randomised controlled trial (RCT), and whether teachers could integrate a discrete set of approaches into routine teaching without disruption. The findings have left us highly encouraged. First, we found that teachers were highly motivated to take part. Participation in these trials aligned with schools’ professional development priorities to apply relevant, evidenced-based practices, and also supported a desire shared among teachers to directly contribute to the evidence base. We also learned that short and simple teacher guides were sufficient for teachers Evaluations FOR ALL Christine Kelly and Faizaan Sami look at how the EEF’s new stream of evaluations can give teachers better evidence to support their everyday practice to understand the trial, and apply the choice approaches with fidelity. Striking a good balance between strict prescription and flexibility within provided guidance is no easy feat. We therefore gathered input fromEEF content specialists, as well as practitioners from our Research Schools Network, to distil the guidance down to the core elements of practice, and had teachers use their best judgement when ‘localising’ the approach to their contexts. Finally, the pilots provided a useful test case for disseminating findings to the wider sector. We want to help teachers quickly digest which approaches are more impactful and find out about participants’ experiences of implementation, so that they canmake more informed choices when replicating the chosen approaches in their own classroom. WHAT NEEDS DOING - AND WHAT’S TO COME... Multidisciplinary expertise Integrating RCTs into real-world practice entails navigating some level of variation. Not all teachers instruct the same topics in the same sequence, and pupils in classrooms may not be grouped in the same way across settings. Such differences in contexts challenge our ability to test pupils across settings and produce a standard impact – but they also present unique opportunities for working alongside teachers to try and identify innovative testing solutions. Every newTeacher Choice project will therefore be guided by a Study Advisory Board comprising methodological, subject and practice experts, who will be able to provide fresh perspectives in order to help inform the evaluation. Next steps Three more Teacher Choices evaluations have just completed their scoping phase – including ‘Using Examples to Teach Grammar to Year 7 (Cognitive Science Teacher Choices)’. This evaluation will seek to support teachers in establishing which modelling technique that uses examples is most effective for teaching English grammar. We’ve also partnered with the Hg Foundation to fund a Teacher Choices trial examining whether ChatGPT-informed lesson and resource preparation is more effective in reducing teacher workload than traditional preparation methods. ABOUT THE AUTHORS Christine Kelly is the EFF’s methodological innovation lead and Faizaan Sami is its evaluation manager; for more information, visit educationendowmentfoundation.org.uk 15 teachwire.net/secondary R E S E A R C H