Missing Data: MCAR, MAR, MNAR

You sent a donor satisfaction survey to 2,000 people. Eight hundred responded. The average satisfaction rating is 4.1 out of 5. The board is pleased. But should they be? The answer depends on a question nobody asked: why didn't the other 1,200 respond?

If those 1,200 just happened to be busy that week and their satisfaction would have looked about the same as the people who did respond, then 4.1 is a reasonable estimate. But if the unhappy donors were the ones who didn't bother filling it out, then the real average is lower. Maybe much lower. The number you reported isn't wrong, but it only describes the people who showed up. And the people who show up aren't always representative of everyone.

Statistics distinguishes three reasons data goes missing, and the reason matters far more than the amount.

Missing Completely at Random (MCAR) means the missingness has no connection to anything in your data. A batch of paper surveys fell behind a filing cabinet. A database glitch deleted random rows. The 1,200 who didn't respond are a random slice of all 2,000 donors. In this case, the 800 responses you have are an unbiased sample of the whole group. Your 4.1 average is trustworthy. You have less data than you wanted, which means wider confidence intervals, but no systematic distortion. MCAR is the best-case scenario, and it's also the rarest.

Missing at Random (MAR) means the missingness is related to something you can observe, but not to the missing value itself. Younger donors might be less likely to fill out a survey regardless of how satisfied they are. If younger donors also tend to give smaller gifts, your survey over-represents high-value donors. The satisfaction scores you collected aren't biased by satisfaction itself, but they are biased by age and gift size. You can fix this if you know the pattern. If you have age or gift data for everyone (responders and non-responders), you can weight the responses or adjust your analysis to account for the imbalance. MAR is common and manageable, but only if you have the right auxiliary data to correct for it.

Missing Not at Random (MNAR) is the scenario that should keep you up at night. The missingness is directly related to the value you're trying to measure. Dissatisfied donors skip the satisfaction survey because they've already disengaged. Donors who gave small amounts skip the "how much did you give?" question because they're embarrassed. Program participants who dropped out don't complete the exit survey. In each case, the missing data would have looked systematically different from the data you have. No amount of weighting or adjustment fully fixes this, because you can't observe the pattern that's driving the missingness. The best you can do is acknowledge it, bound the possible bias, and try to reduce non-response in the first place.

These three categories shape how much you should trust any analysis built on incomplete data. A frequency distribution of survey responses looks perfectly fine with MCAR data missing. Under MAR, the shape is skewed but correctable. Under MNAR, the shape is misleading in a way you can't fully see.

Donor surveys are the obvious case, but missing data appears everywhere. Volunteer hours that go unlogged because volunteers forget. Event attendance counts that miss people who left early. CRM records where the donation channel field is blank because data entry was rushed. Email engagement data that's missing for recipients whose email clients block tracking pixels. In each case, ask yourself: is the data missing for a reason that's connected to what I'm trying to measure?

When data is missing, the first question isn't "how much?" It's "why?" The mechanism behind the gaps determines whether your remaining data tells a true story or a flattering one.

See It

Switch between the three missing data mechanisms. Watch how the observed average (solid line) shifts away from the true average (dashed line) depending on why the data is missing.

Reflect

Think about the last survey your organization sent out. What was the response rate? Do you have any reason to believe the non-responders were systematically different from the responders? What data do you have about both groups that might help you check?

When you see a metric in a report that's based on incomplete data (and most metrics are), do you know which type of missingness is most likely at play? How would your interpretation change if the missing values were concentrated among the people with the worst outcomes?