During this Thursday's #UKEdReschat there was a lively discussion hosted by @StuartKime which focussed on the implementation of research and the ingredients associated by with successful implementation
However, for me this cycle of implementation misses out a fundamental step - how to bring together all the evidence and then make a decision about how to proceed. For as (Alonso-Coello
et al., 2016)
Often
the process that decision-makers used, the criteria that they consider and the
evidence that they used to reach their judgments are unclear. They may omit important criteria, give undue
weight to some criteria, or not use the best available evidence. Systematic and transparent systems for
decision-making can help to ensure that all important criteria are considered
and that the best available research evidence informs decisions. P1
Adopting some form of 'evidence' framework can have a number of benefits for school leaders. (Alonso-Coello
et al., 2016) subsequently identify a number of such benefits, which I have adapted for the use in a
school-setting.
- You and your fellow decision-makers, will have an improved understanding of the advantages and disadvantages of the various actions being proposed
- Helping ensure that you include all important criteria in the decision-making process
- Providing you with a concise summary of the all best available evidence – be it research evidence, school data, stakeholder views and practitioner expertise
- Helping colleagues will be in a better position to understand the decisions by senior leadership teams and the evidence supporting those decisions
As mentioned in a previous post evidence-based school leaders make explicit the criteria they use to make a
decision. In the context of your school,
these criteria may well change depending on what domain and sub-domain of
school leadership and management you are concerned with (Neeleman,
2017). The criteria for making decisions about
teaching and learning – may well be different to the criteria you apply to
making financial decisions. In addition,
you may want to take into account whether criteria are adjusted for different parts
of the organisation. In the context of
your school, the criteria being applied at say level of the Board of MAT Trust,
may well be different to how the criteria are applied at Key Stage 1 in a
primary school. Using (Alonso-Coello
et al., 2016) as a
starting point, let’s look at some of the criteria that could be applied to
decision-making (see Figure 1)
Figure 1 Evidence to Decision Template
Furthermore, application of the above framework needs to be seen in the context of the strength or otherwise of the evidence - be it research, practitioner expertise, school data or stakeholder views. However, that is another discussion which I will explore in a future post.
And finally
Having reflected on the EEF school improvement cycle - it seems to me insufficient attention is being paid to how you turn evidence into a decision and the processes necessary to support evidence-based decision-making - and this represents a fundamental flaw in the five stage process put forward. Evidence-based decision making is so much more sophisticated than a simple model of priorities, external research, implementation, evaluation and mobilisation - and involves critical appraisal of multiple sources of evidence, aggregation of that evidence, and subsequent integration of the evidence into the decision making process.
References
Element
|
Criteria
|
Priority of the problem
|
Is the issue an important problem for which a remedy is sought and
that can be locally implemented?
|
Benefits
|
How substantial are the desirable anticipated effects?
|
Costs
|
How substantial are the undesirable anticipated effects?
|
Certainty of the evidence
|
How robust and secure are the different sources - research, practitioner expertise,
stakeholder views and school data - of evidence?
|
Balance
|
Does the balance of the desirable and undesirable effects favour
the intervention or the comparator?
|
Resource use
|
How large are the resource requirements – attention, time, money,
professional learning?
Does the balance of costs and benefits favour the intervention or
the comparator?
|
Equity
|
What impact does the decision have on educational equity? Will it help close gaps in attainment?
Are there important ethical issues which need to be taken into
account?
|
Acceptability
|
Are there key stakeholders – teachers, parents, trustees, who
would not accept the distribution of the benefits, harms and costs?
Would the intervention adversely affect the autonomy of teacher,
department, school or MAT?
|
Feasible
|
Are there important barriers that are likely to limit the
feasibility of implementing the intervention (option) or require
consideration when implementing it?
Is the intervention or strategy sustainable?
|
Additional comments and recommendation
|
Furthermore, application of the above framework needs to be seen in the context of the strength or otherwise of the evidence - be it research, practitioner expertise, school data or stakeholder views. However, that is another discussion which I will explore in a future post.
And finally
Having reflected on the EEF school improvement cycle - it seems to me insufficient attention is being paid to how you turn evidence into a decision and the processes necessary to support evidence-based decision-making - and this represents a fundamental flaw in the five stage process put forward. Evidence-based decision making is so much more sophisticated than a simple model of priorities, external research, implementation, evaluation and mobilisation - and involves critical appraisal of multiple sources of evidence, aggregation of that evidence, and subsequent integration of the evidence into the decision making process.
References
ALONSO-COELLO, P., OXMAN, A. D., MOBERG,
J., BRIGNARDELLO-PETERSEN, R., AKL, E. A., DAVOLI, M., TREWEEK, S., MUSTAFA, R.
A., VANDVIK, P. O., MEERPOHL, J., GUYATT, G. H. & SCHÜNEMANN, H. J. 2016.
GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent
approach to making well informed healthcare choices. 2: Clinical practice
guidelines. BMJ, 353.
NEELEMAN, A.-M. 2017. Grasping the scope of
school autonomy: a classification scheme for school policy practice Belmas.
Stratford on Avon, England