Evaluating the impact of programs and initiatives is increasing critical for NGOs with a community focus. But for some smaller organisations, this can often be a time- and resource-intensive exercise. Many NGOs struggle to complete larger evaluations often due to limited budgets and resources, and so many are finding ways to embed quality data collection, reporting and reflection into their everyday operations.
There are a range of useful tools and techniques that organisations can use for cost-effective evaluation, shared at a recent Sydney seminar on NGOs and Evaluation on a Shoestring hosted by the NSW Regional Network Committee of the Australasian Evaluation Society.
The head of an NGO in the health sector spoke about the organisation’s experience doing just this. They shared some of the tools and approaches that they use to conduct evaluation on a budget, including:
- subscribing to survey software like SurveyMonkey which enables them to store hard copy survey responses and create simple reports
- using volunteers to directly collect hard copy survey responses at events, and completing associated data entry
- using existing events, community activities and forums as opportunities to collect data and embedding focus groups, short surveys or other feedback mechanisms
- building existing staff capacity to collect, analyse and report on data – many staff already have skills in interviewing, analysis and communications
- providing small incentives to encourage participation in surveys/feedback via their newsletter.
These efforts complement the investments that the organisation has made in recent years to become accredited under the QIC Health and Community Services Standards and subscribing to the Results Based Accountability online services to help monitor performance.
Embedding an evaluation mindset
They also suggested that it is a common mistake for NGOs to treat evaluation like ‘going on a diet’, where they complete a small number of large and expensive evaluations every now and again (particularly driven by supporter or stakeholder needs) and then lapse into bad practice. Rather, good evaluation is about a healthy lifestyle of evaluative thinking!
Participants in the seminar also talked about the challenges of meeting expectations, from funders and other stakeholders, to systematically report on program outcomes. They see the benefits of what they do every day, but it’s much harder to convincingly tell these complex stories and quantity their impacts. Doing this well requires a thoughtful, strategic approach to evaluation.
Tips and techniques for success
When approaching evaluation projects:
- It’s important for NGOs to be aware of financial and resourcing constraints and communicate any challenges to stakeholders and evaluation project members when designing a project or methodology
- It’s paramount that project members understand what the purpose of the evaluation is and develop approaches designed to fulfil this purpose in an efficient way
- It may be useful to consider whether there are opportunities to conduct some parts of the evaluation in-house, which could involve using experts to design a methodology, and then letting NGO staff carrying out the evaluation themselves
- It's critical to keep in mind the skill levels and evaluation experience of NGO team members who are supporting evaluation activities, and to build their capacity in this area
Byline:
- Sophie Duxson, Social Research at the UTS Institute for Public Policy and Governance
Interested in learning more about best practice evaluation?
Contact:
Social Research, Policy and Consulting team
UTS Institute for Public Policy and Governance
ippg@uts.edu.au