In search of feasible fidelity

In the University of York publication “Better Evidence-based Education: Evidence-based policy and practice”, David Andrews looks at the process required for evidence programmes to be adopted by schools. Commitment to fidelity is vital, so educators must assess whether this will be feasible for the intervention they are considering.

He urges educators thinking about adopting evidence-based approaches to weigh the costs and benefits for themselves. In particular he stresses that benefits need to be “practically significant” as well as “statistically significant”.

His list of the costs and benefits to be assessed includes the following:

Direct costs

  • How much do the materials cost, and do they duplicate other materials that need to be purchased?
  • How much does the training and ongoing support cost?
  • How much time will it require?
  • Will there be a need for additional personnel?

Indirect costs

  • How much teaching time will be required from personnel and pupils?
  • How much time will be expended in gaining teacher and staff support?
  • How much prep time should teachers expect outside the classroom? Are there hidden costs?
  • How much political capital will be expended in changing to a new approach?

Direct benefits

  • What are the academic gains expected?
  • What non-academic gains are expected, for example attendance, discipline, parent engagement or reduced academic disruptions?
  • Are there direct savings of teaching time?

Indirect benefits

  • What are the expected improvements in staff climate and morale?
  • What are the expected benefits in retaining teachers and staff?
  • What are the expected benefits from the general skill development of teachers?

He also states “Sustaining the adoption can be more difficult than choosing and getting started.” While it is important not to judge an intervention prematurely, and to allow sufficient time to gain results, if ‘early wins’ can be identified, these are crucial to maintaining teachers’ enthusiasm, and helping sustain the programme.

Another consideration for Mentor is that if schools are failing to adopt evidence-based interventions which have a direct impact on pupil learning and achievement , for example reading ability, there will be an additional challenge in getting them to adopt programmes where the main benefit (e.g. reduced alcohol consumption) is perceived to lie outside schools’ ‘core business’.

Advertisements

One thought on “In search of feasible fidelity

  1. Interesting stuff Claire. What I’ve heard from researchers who are trying to evaluate prevention programmes is that recruitment can be made much easier if the intervention has the backing of those who should know – having the NHS say this is a ‘good thing’ seems to grease the wheels very nicely.

    But, finding ways to sustain programmes, particularly where there is a cost to the school is going to be a real issue. The things that I hope would help are:

    • Positive feedback about the tangible difference the programme is making (both qualitative and quantitative); and it would help if some of those outcomes are to first order issues for schools (attendance, behaviour, standards)
    • Ongoing support that is valued by teachers and managers
    • The positive interest of those external eyes that make a difference to schools (Ofsted, DfE, local politicians and policy people)
    • Prizes – who can resist being honoured, especially if it includes a celebrity!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s