Skip navigation

Delivering better health care

 

What can go wrong when you are implementing evidence-based practice? Some lessons from the development process.

Michael Dunning, Editor, ImpAct

Considerable effort has been devoted in the last ten years to questions about clinical behaviour and changing clinical practice. Extensive research and development programmes have been funded. All of the latter have involved projects tackling specific clinical conditions and working from the basis of robust evidence of clinical effectiveness. These efforts have had a number of labels over the years - moving through from medical (and clinical) audit, to clinical effectiveness and now to clinical governance. While the labels change the concern remains to improve the quality of care for patients. Despite all this effort, definitive answers remain elusive although some practical lessons are emerging.

The first paper in this series starts from the premise that development work might tell us what we should do. But it will also, if people are scrupulously honest and open, help us learn what we should not do, perhaps the more important lessons from the work. After a note about context, eleven traps to avoid are described.

Where did we start?

In the early 1990s, as interest in evidence-based practice grew a few of us were starting to ask questions about speeding up the process of change. Could we find ways to implement proven clinical practices and thus bring the benefits to patients more quickly? The focus would be on organisational development - designed to secure the implementation of evidence for a specific clinical condition across populations. This seemed to have attractions and could complement efforts to promote evidence-based practice within the clinical community. It would represent a management approach to service improvement.

Discussions across the (then) Oxford region argued that one of the measures of success of health authorities as commissioners of health care should be - Are they commissioning the right things? This prompted the creation of one of the first major development programmes in this field: Getting Research into Practice (GRiP). Henry McQuay and I created the GRiP project in Oxford in 1993. Four health authorities in the region worked with us to tackle a range of clinical issues. GRiP demonstrated that change was possible and identified a series of steps that people might follow. Other development programmes followed, including:

In the North West - with the NHS Executive Research and Development Directorate and working through Research Liaison Groups.

In London again with NHS Executive, Research and Development in a regional programme to mirror a national Research and Development programme focused on methods to promote the implementation of research findings in the NHS.

At the King's Fund with the Promoting Action on Clinical Effectiveness (PACE) Programme.

More recently in the West Midlands, with Linda Dunn from the Partnership for Developing Quality, we created a short training session to get over key messages about managing change. The aim was to help participants understand the overall process - rather than the individual activities, ie to create a manageable picture of the task overall. Later papers in this series will build on that work.

These initiatives recognised that answers to all the questions about changing clinical practice are not available. Andy Oxman's work in the early 1990s reminded us that there are no magic bullets (to implement change) but that a multi -faceted approach (using a set of linked interventions) was more likely to be successful. The trick was to make the best use of what is already known and believe that change was possible - although it might be difficult. The challenge was to bring together the knowledge from research with the practical experience to develop a practical model for general use. The unknown was not an excuse for not trying.

A clearer picture about how to do this type of work is emerging - even though some important questions remain. It was most recently summarised in an effectiveness bulletin produced by the Centre for Reviews and Dissemination at York University in 1999.

The work of dedicated research groups, such as those involved in the Cochrane Collaboration ( Effective Practice and the Organisation of Care - EPOC ) remains important.

The following notes are based on observations of many local projects but out of respect to those involved, do not point the finger to where things went wrong. The eleven traps are:


Eleven traps to avoid

Spending too much time looking for evidence.

The immediate reaction of many people when they first get involved in an implementation project is to set out to find the evidence - that is commission and/or undertake their own searches. They often say We need to find the most up to date evidence . But this is fraught with problems. It ignores the fact that the evidence is common across the NHS: it's not a local issue. Beware also of clinicians who have their own approach to evidence - they know what they like (this could be papers by colleagues from medical school or from colleagues they have met at professional conferences!) but this may not be evidence within current understanding.

Many people devote too much time and effort looking for the right evidence. It may be increasingly easy to find papers through Internet searches of Medline etc but interpreting them is another story. Defining evidence is a serious business. It requires skill to find and review research papers. It is not something to be undertaken lightly. A better starting point is a review produced by a reputable source.

Experience has shown that it is wise to focus implementation initiatives on clinical topics where the evidence is robust and generally non-controversial.


Avoiding creating glossy guidelines

Once the team has agreed a focus for their work and assembled the relevant evidence discussion will turn to views about how the service should look in the future. What are the key elements and decision points in the new service? Views will emerge about whether the aim is to create a guideline, a protocol, a pathway etc. It can be fun designing flow charts and guidelines - but glossy presentation may not impress clinicians. In fact it might irritate them.

The challenge is to persuade clinicians about the value of the evidence so decide whether a guideline may or indeed may not help that process. It is easy to write down what others should do, but it is much harder to persuade them to do it. Conserve your energy for activities, such as training and education sessions, that will make a real difference.

Remember that the evidence base is evolving and will change over time. Don't waste energy and effort on creating something elaborate that may be soon out of date.


Assembling the wrong team

Most people prefer to work with people they know and like - but have they the right skills and contacts? You will need people who can contribute to the clinical discussions, people who can handle the training aspects and people who can influence the deployment of resources. There is a danger of believing that only people with a clinical background can contribute.

Whatever topic you tackle it will have an impact on the level of service - such as diagnostic services or the supply of particular drugs or dressings. All organisations have strict timetables for resource allocation processes so make sure that people are involved in your team that can help you through that maze. Fitting in to the budget setting timetable is important.

Think carefully about the skills and experiences you will need. Make sure that you include managers and those who can influence resource allocations. Don't leave them out. Remember they need time to change budgets.


Expecting people to give up their time for you

People who set up and lead implementation projects are inevitably enthusiastic. They are prepared to make sacrifices to create space in their diary for the work. But all people in the organisation will not feel the same. Why should I give up my time for you? - is a question you will have to be ready to answer. It cannot be avoided if you are to assemble the range of skills and experience you will need to make the work a success.

Experience has shown that senior commitment to initiatives is a prerequisite of success. It can free up time in people's busy schedules and legitimise their contribution.

Bear in mind that everyone will not share your enthusiasm for the extra work. Time spent early in the process to get the commitment of senior manage is well spent.


Assuming that everyone will re-act the same

Most people are aware of the work of Rogers when they talk about innovation and change. The language of innovators and laggards will be familiar. But people often fail to take this into account when they are planning their project. They expect everyone to react to their messages the same. They do not consider the context within which they will be working and the implications of whether some clinicians they hope to influence will be receptive or hostile.

Time taken to assess the likely reaction to your initiative is time well spent: create a contextual analysis. Determine where to start - with clinicians at the co-operative end of the scale. Starting with difficult people will wear you out before you start. Throughout this process avoid language that labels people as difficult. Those you think may be difficult may turn out to be some of your strongest allies.

It is wise to start with clinicians that are likely to be sympathetic to your cause. Early success will give you confidence and results to help you persuade more resistant colleagues.


Ignoring the impact on services

It's easy to get locked into a mindset that sees the only challenge is to change clinical behaviour: a belief that it is about information and education. Many people channel most of their energy into this aspect of their work. They lose sight of the need to change service levels - such as access to diagnostic services. It can often take as long to achieve these changes as those in clinical behaviour.

A link with planning and budgeting timetables and the engagement of the appropriate managers in the discussions will be important (see above). Careful assessment of the scale and pace of change will ensure that progress on the two aspects of the work keeps in step. Don't let one aspect of the work outpace the other.

There is no point in persuading clinicians to change their practice if the service cannot cope with additional demand. Make sure any difficult resource issues are tackled early in your project.


Keeping people in the dark

Everyone in the NHS is busy. The amount of information flowing about the service continues to grow. Diaries are perpetually full and offices are overflowing with paper. Try to ensure that your initiative is not buried under this mountain. You have to keep the attention of those you seek to influence. Too many people think that once they have engaged people they will retain their interest: this is a fallacy. They will not be holding their breath to hear from you.

Communications is an issue you overlook at your peril. Draw up a schedule that helps the team be clear about how their message gets to different groups. Remember the choice of messenger is essential: will people believe and trust them? Steps need to be taken to ensure the consistency of the message. Make the best use of existing communications systems and avoid the need to create new meetings and paperwork.

Don't prompt the question - is that project still going? Make sure that those affected by your work get enough - and not too much - information to keep them in touch with your progress and what it means for them.


Leaving patients out of discussions

People have a sense that they should involve patients, somehow, in their efforts but many are still unsure how to do this. So they put it off. They overlook the positive impact patients can have on their efforts and that patients can contribute. They have a false sense that they need a representative patient/s.

Patients can be strong advocates for change. Moreover they are no longer shy about demanding effective care from clinicians. A variety of mechanisms (such as focus groups and patient panels) have been shown to be effective in gaining an understanding about what patients think about the changes proposed.

Don't put off involving patients because you are not sure how to involve them. They will want the same as you - effective practice locally.


Assuming that staff will turn up to training sessions

Time and space will have to be found to enable busy clinicians to learn about the initiative and what it will mean for them. Allow time to discuss the evidence and its impact on current practice. Prepare carefully for these sessions: plan the presentation of evidence and the structure of the sessions and find ways to engage participants in discussion. Research shows that a talking heads approach with lectures but little or no time for discussions is not likely to work.

Don't overlook the need to ensure that clinical staff can get away from their commitments Can staff attend your training sessions? Don't assume the answer is yes. Have you checked rotas with their clinical managers? If workshop sessions prove difficult a series of individual tutorial for individual clinicians may be the best way forward.

Progress will depend critically on the extent to which you are able to get the message over to busy clinicians. It will grind to a halt if effective education and training is not organised for clinical staff.


Forgetting to provide stickers for patient records

In the heat of a short-term project it is easy to lose sight of the longer-term gaol: to ensure that improvement in the quality of care endures. The problems beyond the project may slide of the agenda. A smooth transition from a project to routine practice should be the objective.

Research has demonstrated that simple steps like creating reminder systems as part of patient record can sustain change. Similarly make sure that someone takes responsibility for maintaining the supply of new material (such as referral forms and or leaflets for patients). Finally, staff turnover is perpetual in the NHS: there is little stability. Make sure that on-going induction training for new staff continues to promote the changes.

Don't let your efforts become just another dead project where the improvements you achieve are eroded over time. Put arrangements in place to sustain the changes in the long-term.


Making the same mistake twice

Creating a project can be exciting and rewarding. It should be approached as a learning opportunity for the team and the organisation. All too often it is possible to see in retrospect what was done but not how it was done. That experience is often soon forgotten.

Build time for reflections into the agenda for project meetings. An opportunity to look back over the work so far and note those things that went well and those that didn't. From the outset encourage honesty and emphasise that mistakes hidden are learning opportunities missed. Look for ways to help other local colleagues learn from your endeavours - can they learn from your mistakes?

Encourage honesty and a willingness to discuss failure: it is unrealistic to expect to be right all the time.


Conclusion

Managing a local implementation project can be a rewarding and educational. Things will go wrong, opportunities will be missed and mistakes will made. Don't most of us learn more from things that we get wrong rather than those things we get right? Everything will not go smoothly, but learning from others can help speed up the process. The lessons from others that informed this paper are intended for this purpose.

Good luck.

MD/November 2000