A logic model is a visualisation of how a service or programme works, detailing the resources and activities that are needed to make it happen and what is expected as a result. It aids evaluation by identifying the steps of implementation that need to be evidenced and what outputs and outcomes are expected to be seen. This can help to identify data that needs collecting and evaluation questions that are helpful to ask.
Monitoring is a normal part of running any service, it simply means checking that critical elements of the service are going to plan/running as expected and making changes where problems are identified. In the context of evaluation this is part of the process of for an implementation evaluation and relates to the inputs, activities, and outputs set out in the logic model; it uses basic delivery data, such as group attendance, staffing numbers, and service user demographics. Often monitoring is focussed on key performance indicators, and data are collected to establish whether those are being met and is checked on a regular basis.
Performance indicators, or key performance indicators (KPIs), are measures used to monitor the degree to which a service is successfully delivered. They support decision making and help determine whether a service is being delivered as planned, needs more support, or requires a commissioning/contract review.
A document that sets out the specific steps involved in delivering a service or part of a service and what information or data are collected at each of those steps. Its purpose is to describe the collection and movement of data through a service, who is responsible for each step, what systems are used, how data are transferred from one system or team to another where necessary.
A qualitative evaluation provides the opportunity to explore the experiences of those using a service. Qualitative evaluations use words rather than numbers to discover what may contribute to a service being successful or not
Describes the way or ways in which participants are brought into a service. This should cover how people first hear about the service, whether they need to be referred or can self-refer/sign up independently, who could and should be referring to the service (Eg., Health visitors). It should go on to set out the steps involved in transferring a referral/self-referral into a participant – who receives the referral, what information needs gathering, what efforts should be made follow-up on referrals, when a referral becomes a participant or is abandoned.
This is a rating system; Red (targets not being met, serious concerns), Amber (falling short of targets); Green (targets met, everything going to plan). RAG ratings are based on proportions rather than absolute figures for greater consistency and standardisation when monitoring multiple services.
A theory of change (ToC) shows how services are intended to work, how you move from a perceived problem to a planned outcome and why it is believed this will work. It can help think about the working of a service in a specific context, communicating with people the plans for the service, and it will feed into the production of a logic model, evaluation questions, and evaluation plans.
Some models for services are manualised. This means that details have been set out for how it is delivered, not just that group sessions will take place with parents, but what the content is, how it is to be delivered and at what stage of a child’s life it can be delivered.
A service design group is a collection of people who are responsible for taking a service from idea to launch. They will be invested in the development of the service in a range of ways and will bring different skills and expertise to the process. The group may be convened by a commissioner, a service delivery organisation, a public service, a research body or another organisation, but the group will need to consist of people from all of these.