Lesson 1: Monitoring vs. Evaluation

LESSON TEXT

Monitoring and evaluation are spoken as one phrase — 'M&E' — but they're two different activities that answer different questions, and confusing them leads to measuring the wrong things.

Monitoring is the ongoing, routine tracking of what you're doing and whether it's on plan. It's continuous and operational: Are activities happening as scheduled? Are we reaching the number of people we intended? Are we within budget? Monitoring answers 'are we doing what we said we'd do?' It's the equivalent of watching the dashboard while you drive — frequent, real-time, focused on staying on track.

Evaluation is the periodic, deeper assessment of whether what you're doing is actually working and worth doing. It's less frequent and more analytical: Are we achieving the results we intended? Is the effort producing real change? Should we continue, adjust, or stop? Evaluation answers 'is this actually making the difference we hoped for?' It's the equivalent of stepping back to ask whether you're driving to the right destination at all.

The distinction matters because organizations often do lots of monitoring and almost no evaluation. They diligently track activities and outputs — workshops held, people trained, materials distributed — and conclude they're succeeding because the activities happened. But activity is not achievement. You can run every workshop on schedule and change nothing. Monitoring tells you the activities occurred; only evaluation tells you whether they mattered. A program that monitors well but never evaluates can look busy and successful while accomplishing nothing real.

VIDEO SCRIPT

[On camera] Monitoring and evaluation — we say it as one phrase, "M&E," but they're two different things answering two different questions. Mix them up and you measure the wrong stuff. Monitoring is ongoing, routine tracking of what you're doing and whether you're on plan. Continuous, operational. Are the activities happening on schedule? Are we reaching the numbers we planned? Are we on budget? It answers: "are we doing what we said we'd do?" It's watching the dashboard while you drive. Evaluation is the periodic, deeper look at whether what you're doing actually WORKS and is worth doing. Less frequent, more analytical. Are we achieving the results we intended? Is this producing real change? Continue, adjust, or stop? It answers: "is this actually making the difference we hoped for?" It's stepping back to ask if you're even driving to the right place. And here's why it matters: organizations do tons of monitoring and almost no evaluation. They track workshops held, people trained, materials handed out — and conclude they're winning because the activities happened. But activity is not achievement. You can run every workshop perfectly and change nothing. Monitoring says the activity happened. Only evaluation tells you if it mattered.

Lesson 2: The Results Chain — Activities to Impact

LESSON TEXT

To measure whether something works, you need a clear model of how it's supposed to work — how your effort is meant to lead to the change you want. The results chain is that model, and it's the backbone of serious impact measurement. It has four links, and confusing them is the most common measurement error there is.

Inputs and Activities

Inputs are the resources you put in — money, staff, time, materials. Activities are what you do with them — running a training, delivering a service, building something. These are the easiest to measure and the least meaningful on their own, because doing an activity proves nothing about whether it helped.

Outputs

Outputs are the direct, countable products of your activities — number of people trained, services delivered, items distributed. Outputs tell you the activity produced something tangible. But note: an output still isn't a result. '200 people trained' is an output. Whether those 200 people can now do anything differently is a separate question.

Outcomes

Outcomes are the changes that result from your outputs — changes in knowledge, behavior, conditions, or capacity among the people you're serving. '200 people trained' is an output; 'of those trained, most are now applying the skill in their work' is an outcome. Outcomes are where real results begin, and they're much harder to measure than outputs — which is exactly why many organizations stop at outputs and call it success.

Impact

Impact is the deeper, longer-term change your outcomes contribute to — the ultimate difference in people's lives or in the problem you set out to address. It's the hardest to measure, the slowest to appear, and the most affected by factors outside your control. Impact is the point of everything, and also the murkiest to prove.

The chain runs: inputs → activities → outputs → outcomes → impact. Each link is progressively more meaningful and progressively harder to measure. The great temptation, everywhere, is to measure the easy early links (activities, outputs) and claim credit as if you'd achieved the hard later ones (outcomes, impact). '5,000 people reached' sounds like impact; it's an output. Learning to see where a claimed result sits on this chain — and being honest that reaching people is not the same as changing their lives — is one of the most valuable habits of mind in all of impact measurement.

VIDEO SCRIPT

[On camera] To measure whether something works, you need a model of how it's SUPPOSED to work. That's the results chain — the backbone of real impact measurement. Four links, and mixing them up is the most common measurement mistake there is. Inputs and activities: the resources you put in, and what you do with them. Money, staff, running the training. Easiest to measure, least meaningful — doing an activity proves nothing. Outputs: the direct countable products. People trained, services delivered, items handed out. Tangible — but still not a result. "200 people trained" is an output. Whether those 200 can now do anything differently? Different question. Outcomes: the actual changes from those outputs. Knowledge, behavior, conditions. "200 trained" is output; "most of them now use the skill at work" is outcome. That's where real results begin — and it's much harder to measure. Which is exactly why organizations stop at outputs and call it a win. Impact: the deep, long-term change your outcomes contribute to. The real difference in people's lives. Hardest to measure, slowest to show, most affected by things outside your control. Chain runs: inputs, activities, outputs, outcomes, impact. Each one more meaningful and harder to measure. And everyone's tempted to measure the easy early links and claim credit like they hit the hard later ones. "5,000 people reached!" sounds like impact. It's an output. Reaching people is not the same as changing their lives.

Lesson 3: Baseline, Midterm, Endline

LESSON TEXT

To know whether something changed, you have to know what it was like before — and check again during and after. This is the logic of baseline, midterm, and endline evaluation, the standard rhythm of measuring a program or initiative over its life. Each answers a distinct question, and skipping any of them cripples your ability to know what happened.

Baseline — the starting point

A baseline measures the situation before you begin, establishing where things stood at the start. Without it, you can never credibly claim change, because you have nothing to compare against. If you don't know that reading scores were at a certain level before your program, you can't show they improved because of it. The baseline is the most commonly skipped and most sorely missed measurement — because by the time people want to prove impact, the 'before' is gone forever. Measure the starting point before you start; you can't go back and collect it later.

Midterm — the course correction

A midterm evaluation checks progress partway through, while there's still time to adjust. Its purpose is less to judge and more to learn and correct: is this working so far? What should we change? A midterm turns a program from a fixed bet into something that can course-correct on evidence — the core cycle from Course 1, applied mid-flight. Programs that skip the midterm can only discover failure at the end, when it's too late to fix.

Endline — the verdict

An endline evaluation measures the situation after the program, and compares it to the baseline to assess what changed. This is where you learn whether the intended outcomes and impact materialized. The endline answers the big question — did it work? — but only if you have a baseline to compare it to. An endline without a baseline can describe the final state but can't demonstrate change.

Together these form a before-during-after arc that lets you see change, correct mid-course, and judge results honestly. The single most important practical lesson: establish your baseline before you begin. Everything else depends on it, and it's the one measurement you can never recover if you miss it.

VIDEO SCRIPT

[On camera] To know whether something changed, you have to know what it was like before — and check during and after. That's baseline, midterm, endline. The standard rhythm of measuring a program. Each answers a different question, and skipping any one hurts. Baseline — the starting point. Measure the situation BEFORE you begin. Without it, you can never credibly claim change, because you've got nothing to compare to. Don't know where reading scores were before your program? You can't prove they improved because of it. And here's the painful part — the baseline is the most-skipped and most-missed measurement, because by the time people want to prove impact, the "before" is gone forever. You cannot go back and collect it. Measure it before you start. Midterm — the course correction. Check progress partway through, while there's still time to adjust. Less about judging, more about learning: is this working? What do we change? It turns a program from a fixed bet into something that corrects on evidence. Skip it, and you only discover failure at the end — too late. Endline — the verdict. Measure after, compare to baseline, see what changed. Did it work? But only if you have that baseline. An endline alone describes the ending — it can't prove change. Before, during, after. And if you remember one thing: establish the baseline before you begin.

Lesson 4: The Attribution Problem

LESSON TEXT

Here is the hardest and most important idea in impact measurement, and the one most often ignored: even if things got better, how do you know YOU caused the improvement? This is the attribution problem, and taking it seriously separates honest impact measurement from wishful storytelling.

Suppose you run an employment program, and a year later, more participants have jobs. Success? Maybe. But the economy also improved that year. Other organizations were working in the same area. Some participants might have found jobs anyway. The improvement is real, but how much of it was you? This is not a technicality — it's the whole question. Claiming credit for change you didn't cause is the most common form of dishonesty in impact measurement, usually sincere rather than deliberate.

The core difficulty is the counterfactual: what would have happened without your program? You can measure what did happen with the program, but to isolate your effect you'd need to compare it to what would have happened otherwise — a world you can't directly observe. All rigorous impact measurement is, at heart, an attempt to estimate that counterfactual.

There are ways to get closer. A comparison group — people similar to your participants who didn't receive the program — gives you a glimpse of the counterfactual: if their outcomes improved just as much, your program probably wasn't the cause. This is the logic behind rigorous evaluation designs. Full experimental rigor isn't always possible or appropriate, but even asking the question — 'what would have happened anyway, and how would we know?' — dramatically improves the honesty of any impact claim.

Recall correlation versus causation from Course 2: the attribution problem is that exact idea, applied to your own work. 'We ran a program and things improved' is a correlation. Concluding 'therefore our program caused the improvement' is the same leap we warned against — and it's especially seductive when the program is yours and you want it to have worked.

You won't always be able to prove attribution rigorously, and that's acceptable — what's not acceptable is ignoring the question and claiming credit as if the counterfactual didn't exist. The honest practitioner always asks: what else could explain this change, and how much of it can we genuinely attribute to what we did? That humility is the mark of real impact measurement, and it's rarer and more valuable than any technique.

VIDEO SCRIPT

[On camera] Now the hardest idea in impact measurement — and the one most people dodge. Even if things got better… how do you know YOU caused it? That's the attribution problem. And taking it seriously is what separates honest measurement from wishful storytelling. Say you run an employment program, and a year later more participants have jobs. Win? Maybe. But the economy also improved that year. Other groups worked the same area. Some folks would've found jobs anyway. The improvement's real — but how much was you? That's not a technicality. That's the whole question. Claiming credit for change you didn't cause is the most common dishonesty in this field — and usually it's sincere, not deliberate. The heart of it is the counterfactual: what would've happened WITHOUT your program? You can see what happened with it. To isolate your effect, you'd need to compare against a world you can't directly see. All rigorous impact measurement is really an attempt to estimate that invisible world. You can get closer — a comparison group, people like your participants who didn't get the program. If they improved just as much, it probably wasn't you. Full rigor isn't always possible. But just ASKING — "what would've happened anyway, and how would we know?" — massively improves your honesty. Remember correlation versus causation? This is that exact idea — applied to your own work. And it's most seductive when the program is yours and you really want it to have worked. You won't always prove attribution. That's okay. Ignoring the question and claiming credit anyway — that's not.

Lesson 5: Designing Honest Indicators

LESSON TEXT

An indicator is a specific, measurable sign that tells you whether something is happening. Choosing good indicators is where measurement succeeds or fails in practice — because what you measure shapes what you do, and the wrong indicators quietly steer an organization toward the wrong things.

Measure what matters, not just what's easy

The strongest pull in all of measurement is toward the easy indicator over the meaningful one. Counting workshops held is easy; measuring whether participants' lives improved is hard. So organizations measure the easy thing and gradually mistake it for the goal. Fight this: choose indicators that reflect the outcome you actually care about, even when they're harder to measure, and be suspicious of any indicator that's suspiciously convenient.

Beware indicators that can be gamed

Any indicator that becomes a target tends to get gamed — people optimize the number rather than the underlying reality. If a clinic is judged on 'patients seen,' it may rush visits to boost the count while care suffers. When choosing an indicator, ask: if people optimized purely for this number, would that actually produce the result we want, or could they hit the number while missing the point?

Use a few good indicators, not many weak ones

It's tempting to measure everything, producing dozens of indicators nobody uses. A handful of well-chosen indicators that genuinely track your key outcomes beats a sprawling scorecard. The same discipline of subtraction from the dashboards course applies here: fewer, better, actually-used.

Combine numbers with stories

Quantitative indicators tell you how much and how many; they often can't tell you why or what it meant to people. A small amount of qualitative evidence — real accounts from the people affected — gives numbers meaning and catches things the numbers miss. Honest measurement usually blends both: the number for scale, the story for understanding.

Good indicators are honest indicators: they measure what genuinely matters, resist gaming, stay few enough to use, and combine the quantitative with the human. Designing them well is a real skill, and it's central to knowing — rather than assuming — whether your work makes a difference.

This completes the measurement course, and with it, one of the disciplines at the very heart of BIIM's expertise. You now understand not just how to make good decisions, but how to know whether the things you decided to do actually worked — which is what turns a well-meaning organization into an effective one.

VIDEO SCRIPT

[On camera] An indicator is a specific, measurable sign that tells you whether something's happening. And choosing good ones is where measurement lives or dies — because what you measure shapes what you do. Wrong indicators quietly steer a whole organization toward the wrong things. Measure what matters, not just what's easy. The strongest pull in all of measurement is toward the easy indicator over the meaningful one. Counting workshops is easy. Measuring whether lives improved is hard. So people measure the easy thing and slowly mistake it for the goal. Be suspicious of any indicator that's suspiciously convenient. Beware indicators that can be gamed. Any number that becomes a target gets gamed — people optimize the number, not the reality. Judge a clinic on "patients seen" and it rushes visits while care suffers. Ask: if someone optimized purely for this number, would we get what we actually want? Use a few good indicators, not many weak ones. Same subtraction discipline as the dashboards — fewer, better, actually used. And combine numbers with stories. Numbers tell you how many; they can't always tell you why or what it meant. A few real accounts from the people affected give the numbers meaning and catch what they miss. Number for scale, story for understanding. Honest indicators measure what matters, resist gaming, stay few, and blend the human with the numerical. That's how you KNOW — not assume — whether your work makes a difference. And that's the heart of what BIIM does.