Skip to main navigationSkip to main content
The University of Southampton
Public Engagement with Research

Reviewing your public engagement activity

Once you have planned your project/activity evaluation, you need to ensure that you are regularly reviewing the plan with regard to project development, unexpected events and new opportunities. There are a few questions you should keep in mind at all times to ensure that you are evaluating effectively.

Image of reports stacked in a pile with a magnifying glass

Some funders are very relaxed in their approach to evaluation and outcomes reporting, but increasingly you will be asked to evidence how the money you have been given has added value to your field of research, has increased awareness of the issue among the public, or has impacted upon their behaviour. With this in mind, the below guidance directs you to some of the larger funding bodies in the UK, and what they recommend for evaluation.

Arts and Humanities Research Council (AHRC)

The AHRC recommend using a logic model for engagement and evaluation planning, and they use the Kirkpatrick Model for levels of potential impact:

Reaction - the initial response to participation

Learning - changes in people's understanding, or raising their awareness of an issue

Behaviour - whether people subsequently modify what they do

Results - to track the long-term impacts of the project on measurable outcomes

Reaction

You may want to set objectives regarding things like perceived levels of enjoyment and usefulness. You can assess reactions in three main ways: by getting people to write down their response (usually by questionnaire); by talking to them one-to-one or in focus groups; by observing them.

If you want to know whether people enjoyed the project/found it useful/learnt something, you can also find out what they particularly did or didn’t enjoy, what was most and least useful and what they would change and why. You can also get information on the environment; e.g. comfort of the venue, quality of refreshments.

The easiest time to get initial reactions is when people are taking part in the project. It may also be worthwhile to get a more considered response a short time after the actual interaction when people have had time to reflect.

Funders appreciate evaluation strategies that provide feedback on lessons learned, good practice, successful and unsuccessful approaches. If you understand why something went wrong, it can help improve things for the future – a ‘lessons learned’ section will enable better practices to evolve.

Learning

You can find out quite easily what, if anything, people think that they have learnt from your programme/project within the reaction data: you can ask them to tell you what they think they have gained, and whether they have a more complete view or understanding of the issue.

Behaviour

Tracking and measuring changes in behaviour is resource-intensive: you’ll need to know what the baselines were and will need some sort of ongoing contact to monitor change. You might rely on self-evaluation, but you may want independent verification. Either way, you will need resources and expertise capable of delivering this sort of evidence.

Results – long-term impact

Tracking people with whom you have engaged over an extended period is the most straightforward way of assessing long-term impact. However if you only track the people you engaged with, there is no ‘control group’ to allow you to ascribe changes to your project rather than to other influences. The resource implications for this are considerable – it is only practical for large scale projects with budgets to match.

Reporting results

You should carefully consider the evidence you have collected, thinking about what it tells you. Negative outcomes should not be ignored – they may be helpful in providing ‘lessons learned’ for future programmes/projects. The positive and negative findings from an evaluation should be fed back into the decision-making process for future programmes/projects. An example template for reporting back to funders is given at Annex 3.

In addition to providing a report for your funders, you may also consider reporting your findings in other ways to a wider public. Perhaps you could put highlights from the evaluation on your website, or publish some case studies of exemplary work conducted during the programme/project.

Once the evaluation is completed, you may also like to consider the process itself. There may be things you have learnt from the process and things you would like to change for future evaluation cycles – perhaps your aims were too vague so you would like to think about making them more measurable in the future; or a monitoring tool worked particularly well, and you now have a questionnaire template to adapt for future use.

Source: http://www.ahrc.ac.uk/documents/guides/understanding-your-project-a-guide-to-self-evaluation/

Research Councils UK (RCUK)

RCUK contributed to this 'Evaluation - Practical Guidelines' handbook, which outlines:

  • Building an evaluation strategy
  • Gathering data
  • Data handling
  • Reporting

In addition, there is further advice on evaluation questionnaires and further reading among the document's annexes.

Economic and Social Research Council (ESRC)

For any public engagement activity to be successful, it is important to plan all elements. This guide takes you through the different steps you should consider to make your activity successful.

Public engagement activities can take place at any stage of a research programme, for example:

  • Project start-up: involving stakeholders at this stage of the process can help shape the research agenda. This ensures that the research tackles pertinent issues.
  • Preliminary findings: sharing preliminary findings with key groups not only increases awareness but can tease out issues, helping shape later stages of research or analysis.
  • Project end: sharing and testing research findings both with stakeholders and with other groups who might be interested in the research, including the general public. This raises awareness of the research and of social science; it can potentially enable the outputs to be used more widely and have greater impact.
  • Other times: public engagement activities don’t have to be linked to specific projects. For many groups, meeting and working with a researcher is a valued experience and provides a unique opportunity to understand research findings and processes. It may also provide new, unexpected research opportunities.

Public engagement projects can also provide insight and direction for future research.

Source: http://www.esrc.ac.uk/public-engagement/public-engagement-guidance/guide-to-public-engagement/

Natural Environment Research Council (NERC)

NERC have a comprehensive guidance document based on their 'Pathways to Impact', which you can access here: http://www.nerc.ac.uk/funding/application/howtoapply/pathwaystoimpact/pathwaystoimpact-policy/

Privacy Settings