What is service monitoring?

Monitoring involves the regular collection and recording of data relating to the delivery of a service. The data includes process type measures such as recruitment Eg., the number of families attending a service; reach or the extent to which people who access a service are representative of the target population; and implementation and fidelity which relate to the extent to which the key ingredients of a service have been delivered and received by participants. Monitoring uses standard service data (as set out in data requirements and a process flow), which should be reported regularly throughout the course of the service delivery.

Monitoring will answer questions such as:

•       How many families is the service seeing?
•       What are the demographics of the families participating in the service?
•       Is the service reaching their target group?

Monitoring data can be reviewed within a framework of performance indicators, with the aim of supporting the services’ implementation and data capture.

Why is monitoring an important part of evaluation?

Monitoring is important for the practical running of a service because it allows you to identify challenges to delivery and fix problems fast. Monitoring supports evaluation by:

•       Providing clarity on what we are evaluating
•       Identifying issues with data collection so that support can be put in place early

What is implementation evaluation?

Implementation evaluation helps us to understand what exactly is being delivered, and most importantly whether the service is being delivered as intended. It moves beyond monitoring and answers more specific questions. Implementation evaluation is directly linked to the service’s logic model and examines the degree to which the inputs, activities, and outputs are working in practice. In addition to  establishing whether the service is deliverable, an implementation evaluation will also determine:

•       Whether the service is acceptable to the target population
•       Whether the service is ready for further evaluation

Why is implementation evaluation vital?

While it may be tempting to leap straight to evaluation of impact (or effectiveness evaluation), it is important to consider what it means to be an effective service.

Yes, an effective service needs to have a positive effect on outcomes, but it also needs to engage the right people and keep them engaged long enough for them to benefit. And it needs to be implemented as planned – with high quality delivery and fidelity to the model. Without the foundation of engagement and fidelity, there’s unlikely to be a change in outcomes.

A robust implementation evaluation is a vital step towards an effectiveness evaluation as it:

•       Establishes readiness for further evaluation
•       Helps to explain why there are/aren’t the expected changes in outcomes