As a school research lead one of the things that you will have to get grips with is the debate over whether randomised controlled trials (RCTs) can make any meaningful contribution to understanding ‘what works’ in educational settings. Helpfully Connolly, Keenan, et al. (2018) have recently had published systematic review on the use of RCTs in education, which seeks to address four key criticisms of RCTs: it is not possible to undertake RCTs in education; RCTs are blunt research designs that ignore context and experience; RCTs tend to generate simplistic universal laws of ‘cause and effect’; and that they are inherently descriptive and contribute little to theory. So in this rest of this post I will provide extracts from the systematic review, examine the review’s answer to the questions posed, identify some missed opportunities, and finally, make some comments about RCTS and the work of school research leads.
The school research lead, 'factfulness' and 10 reasons why we may be getting things wrong
Now it just so happens that I’m reading the late Hans Rosling’s recently published book: Factfulness: Ten reasons we’re wrong about the world – and why things are better than you think. Rosling argues that when asked simple questions about global trends in the world, people systematically get the answers wrong. However, the world despite all the challenges we are facing, would appear to be in a much better state than we think. Unfortunately, we have a tendency to worry about everything all of the time, rather than adopting a world view based on facts. As a result, we lose our ability to focus on those things that matter most. Instead, we should adopt a stance of factfulness which involves the stress reducing habit of only carrying opinions for which you have strong supporting facts.
To help us to adopt a stance of factfulness – Rosling suggests that we adopt 10 rules of thumb
- Gap – recognizing that when a story talks about a gap – this paints a picture of two separate groups, with a gap in between. The reality is often not polarised – usually the majority are in the middle – just where the gap is supposed
- Negativity – recognizing when we get negative news, remembering that information about bad events is much more likely to reach us – when things are getting better we don’t often hear about them
- Straight line – recognizing the assumptions that a line will just continue straight – and that such lines are rare in reality
- Fear – remember that frightening things get our attention – and recognising that these things are not necessarily the most risky
- Size – recognizing when a lonely number seems impressive (small or large) and remembering you could get the opposite impressions if it were compared with or divided by some other relevant number
- Generalization – recognizing when a category is being used an explanation, and remembering that categories can be divided into sub-categories
- Destiny – recognizing that many things (including people, religions, cultures, countries) appear to be constant just because change is happening slowly
- Single – recognizing that a single perspective can limit your imagination
- Blame – recognizing when a scapegoat is being used and remembering that blaming an individual often steals the focus from other plausible explanations
- Urgency – recognizing when a decision feels urgent and remembering that it rarely is
And finally
Please don't think I'm saying all is rosy in English schools and the important trends are all heading in an upward direction. Rather I'm just asking that you understand how the grounds on claims are being made, the nature of the claim, the warrant for the claims, whether than warrant has strong backing, whether qualifications to the claim, and whether there any rebuttals.
Evidence-based practice and instructional coaching - why the research evidence is not enough.
Just this week, @DrSamSims wrote a very well-argued blogpost giving four reasons why instructional coaching is the most well-evidenced form of CPD, and which concludes that All schools that aspire to be evidence-based should be giving it (instructional coaching ) a go. However, from the perspective of what evidence-based practice is all about – the last sentence in the blogpost is badly flawed. First, it’s not for researchers to tell school leaders and teachers what they should or should not be doing in their schools and classrooms . What school leaders and teachers prioritise in their schools and classroom is down to their professional judgment. Indeed, the last sentence of Sam’s blogpost is grist to the mill for the opponents of evidence-based education. Second, the role of research evidence in evidence-based practice is to provide the backing for warrants. As such, research evidence plays an indirect role in developing an arguments supporting the use of the intervention - Kvernbekk (2016). Third, even if you think that instructional coaching meets an obvious priority for your school i.e. supporting the improvement of teaching learning – that does not mean your school should automatically do it. Ideally, school leaders would use a disciplined process to work out whether what worked ‘there’ is going to work ‘here’.
So in the rest of this post I am, once again, going to lean on the work of Kvernbekk (2016) to examine the process you might wish to undertaken before adopting even the ‘most evidenced’ intervention.
Read MoreModels of organisational development - What type of culture does your school or college have?
So in this post I’m going to write about Patterson, Nolan, et al. (2011) and two models of organisational development - perform or perish and responsive and relational which were developed from research undertaken in the health sector and which are described in Table 1.
Perform or perish
|
Responsive and
relational
|
Pace: Quick fix, short term, process driven, pushing and
fixing
|
Relational and
responsive Complexity: Longer term, focus on
people and perceptions, brokering
|
External: Top down agenda, local context largely
overlooked, off-the- shelf, one-size fits all approaches applied
|
Locally contextual factors fully acknowledged and
addressed, solutions tailored to situation, existing models modified
accordingly
|
Select few determine goals and direction of change
|
All groups including users/carers involved in deciding
goals and direction of change
|
Punitive and transactional leadership style from top,
little unit level leadership
|
Empowering, inspiring and transformational leadership
style at all levels, especially unit
|
Metrics matter: Superficial, often quantitative targets
for success, e.g. patient flow
|
Meaning matters, relational, dynamic qualitative
‘indicators’ of success, peoples’ experiences
|
Scored
|
Profiled
|
Impoverished change environment results and the ‘senses’
are reduced
|
Enriched change environment results and the ‘senses’ are
enhanced
|
The school research lead and critical thinking
Table 1 Specific abilities underlying critical thinking in
medical practice
|
|