Over the last seven days several research related articles in the TES have caught my eye. First, there was Joe Nutt saying - 'Good research is good - but experience is better' with research often so indigestible as to be of little use to teachers. Then, there was an article by Martin George asking whether ‘edtech’ is immune from rigorous research, given that pace of technological change makes the usual evidence-gathering on effectiveness redundant. Finally, we have Professor Barbara Oakley saying that too many education researchers ‘do not do research that is founded on the scientific method,’ resulting in a crisis of replicability. In other words, when teachers and school leaders are wishing to use to research evidence, the evidence doesn’t exist, or if it does, it’s neither comprehensible or replicable
Now it’s fair to say that there are no simple or easy answers to the questions these articles raise. However, at the level of the school when teachers report on a disciplined inquiry or some form of collaborative practitioner inquiry, there is something which can do i.e. use a reporting checklist - to improve the quality of reporting and in doing so make the research more accessible and useful to both themselves and colleagues.
One such checklist is the TIDieR (Template for Intervention Description and Replication) Checklist - Hoffmann, Glasziou, et al. (2014) – I’ve adapted to report on an school-based intervention which provides one-to one support for pupils studying GCSE English.
TIDiER Checklist – One to one support for pupils studying GCSE English
1. NAME – Provide the name or the phrase which describes the intervention.
Additional one-to-one support for pupils studying GCSE English.
2. WHY – Describe any rationale, theory or goals of the essential elements of the interventions
The provision of additional support may lead to an improvement in individuals performance in GCSE English examinations, with more pupils gaining grade 4 or better.
3. WHO – Describe the participants and how they were selected for the intervention
The participants were Y11 pupils in a mixed sex comprehensive school, where examination results are consistent with national averages. and will below average numbers of pupils receiving the pupil premium.
Twenty pupils - out of a total of 150 pupils studying GCSE English - were identified by English teachers as being on grade 5/4 borderline for GCSE English were asked to attend the activities associated with the intervention. The twenty pupil included 12 boys and 8 girls.
4. WHAT - Materials: Describe any physical or informational materials used in the intervention, including those provided to participants or used in intervention delivery or in training of intervention providers. Provide information on where the materials can be accessed (e.g. online appendix, URL).
Existing teaching resources were used – with teachers pooling resources . Additional resources were also created to respond to specific teaching problems as they emerged. These were also shared.
5. WHAT : Procedures: Describe each of the procedures or activities, and are processes used in the interventions including any enabling or support activities.
The night before their scheduled session all pupils involved in the intervention received a text message reminding them of the time and place of their support session.
6. WHO PROVIDED: For each category of intervention provider – teachers, pastoral support, teaching assistant etc describe their expertise, backgrounds and any specific training given.
All teachers (five) within the English Department were used to providing the one-to-one support to pupils. No additional raining was provided.
7. HOW: Describe the mode of delivery of the intervention – large group teaching, small group teaching, one to one, online etc.
Additional support was provided on an one to one to basis to individual pupils
8. WHERE: Describe the location where it occurred, any necessary infrastructure or other features
The sessions were provided each day (Monday – Thursday) after school between 4.00 pm and 4.45 pm and held individual teacher’s base rooms.
9. WHEN and HOW MUCH – Describe the number of times the intervention was delivered and over what period of time including the number of sessions, their schedule, and their duration
Each pupil was scheduled to receive 10 sessions – spread over 12 weeks, commencing in February and ending the middle of May. Each session was expected to last 45 minutes.
10. TAILORING Was the intervention planned to personalised or adapted for the needs of a particular group – if so, then describe what, why, when and how.
Those pupils allocated to the programme were provided with a personalised programme of work – which was devised after discussion between the pupil, the class teacher and the teacher providing additional support.
11. MODIFICATIONS If the intervention was modified during the course of the study, describe the changes – what, why, how and when.
Due to staff absence – 1 member of staff was absent for the period of the intervention – those sessions were delivered by a teaching assistant
12. HOW WELL : Planned : If intervention adherence or fidelity was assessed , describe how and how and were any strategies used to maintain/develop fidelity.
It was hoped that pupils would attend on average 8 sessions. Where sessions were missed, emails were sent to the both the GCSE English teacher and group tutor to ask them to remind pupils to attend future sessions. Where teachers were not available to take the planned sessions, support was provided by a teaching assistant
13. HOW WELL : Actual : If intervention adherence or fidelity was assessed , describe the extent to which the intervention was delivered as planned
The mean number of sessions attended was 7. Seven pupils (35%) attended ten sessions, six pupils attended nine sessions (30%) with three pupils attending one or less sessions. The remaining four pupils attended between six and eight sessions.
14. OUTCOMES : Actual : What outcomes were obtained
19 out of 20 pupils gained at least a grade 4 in GCSE English
15. DISCUSSION : What has been learnt and is relevant internally and externally
The provision appeared to have made an impact – as in the previous academic year only 50% of a similar group pupils gained at least a grade 4
Each member of staff involved had to commit around 30 hours of additional time to support the innovation and was only possible due to their commitment
Other activities – which could have taken place in after school meetings had to be delayed till later in the year.
All the staff involved were experienced and effective practitioners – the model may need to be adjusted for a different profile of staff
Consideration needs to be given whether small group support should be provided for pupils
Limitations
Of course, when you use a checklist there are drawbacks. Although the check-list might will help you to report on the intervention, it still might not capture all the complexity of what has happened. By adopting a check-list may lead a reduction in creativity in the ways in which teachers report on interventions. Adopting a check-list may also be perceived as increasing the workload of teachers
And finally
There are no easy answers when it comes to addressing some of the problems with using research evidence. That said, regardless of whether you are someone who is producing research evidence or using it to help bring out about improvements for pupils, if you are conscientious, judicious and explicit in your use of evidence, you will not go far wrong.
References
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., Lamb, S. E., Dixon-Woods, M., McCulloch, P., Wyatt, J. C., Chan, A.-W. and Michie, S. (2014). Better Reporting of Interventions: Template for Intervention Description and Replication (Tidier) Checklist and Guide. BMJ : British Medical Journal. 348.
Humphrey, N., Lendrum, A., Ashworth, E., Frearson, K., Buck, R. and Kerr, K. (2016). Implementation and Process Evaluation (Ipe) for Interventions in Education Settings: A Synthesis of the Literature. Education Endowment Foundation, London.