Nonprofit Impact Measurement, Evaluation & Evidence Strategy Consultant
I help nonprofits turn mission, programs, data, lived experience and reporting obligations into an evidence system that improves decisions, earns confidence and tells the truth about what changed.
A finished evaluation plan is not required. Bring the grant requirement nobody can interpret, the dashboard nobody uses, the outcomes nobody defined, the story leadership wants to publish or the spreadsheet that has become a retirement home for numbers. I will help identify the decision underneath it.
Good work and provable work are not automatically the same thing
A nonprofit can change lives and still struggle to explain how. It can also generate beautiful charts that never establish whether anyone is better off. Impact measurement belongs between those conditions: rigorous enough to support judgment, practical enough to survive contact with the organization doing the work.
Boards ask whether a program deserves more resources. Funders ask whether commitments were met. Program leaders need to know what to improve by Tuesday, not only what looked respectable in last year's report. Development teams need credible proof without turning every participant into a testimonial. Communities need evidence that reflects their priorities rather than a distant institution's convenience. These are related questions, but they are not the same question.
I often encounter a familiar scene: leadership has a count of services delivered, a donor wants outcomes, the CRM uses different categories than the program database, and the annual report needs to go to design next week. Everyone is working hard. Nobody owns the definition of success. Adding another survey at that point usually creates more motion, not more knowledge.
That distinction matters for conversion as much as compliance. Credible evidence can strengthen grant proposals, major-gift conversations, board confidence, strategic planning, community partnerships, recruiting, advocacy, reputation and public understanding. Weak claims create the opposite effect. A suspiciously perfect number can make serious work look less believable than an honest finding with limitations.
Measurement, monitoring, evaluation, research, audit and assurance answer different questions
Confusion begins when a familiar word carries a different promise for every person in the room. A shared vocabulary protects the organization from buying the wrong work, demanding evidence the design cannot produce or publishing a claim stronger than the method supports.
| Term | What it does | The decision it can support |
|---|---|---|
| Performance measurement | Tracks selected indicators against goals, standards, baselines or targets on a continuing basis. | Is performance moving in the intended direction, and where does leadership need to investigate? |
| Monitoring | Observes implementation, participation, resources, outputs, risks and conditions while work is underway. | Is the program operating as intended, reaching people and responding to emerging problems? |
| Program evaluation | Uses a designed inquiry to judge or understand a program's relevance, implementation, results, value or improvement needs. | What is happening, why, for whom, under which conditions and what should change? |
| Impact evaluation | Examines causal effects by comparing observed results with a credible estimate of what would have happened without the intervention. | Did the intervention cause the observed difference rather than merely coincide with it? |
| Needs assessment | Studies a population, place, problem, assets, gaps and priorities before or during program design. | Which needs are important, who experiences them and what response fits the context? |
| Research | Builds generalizable or transferable knowledge through methods appropriate to the research question. | What can be learned beyond the immediate management of one program or organization? |
| Audit | Examines financial statements, controls, compliance, performance or other defined subject matter against formal criteria. | Does the subject satisfy the applicable audit objective and professional standard? |
| Assurance | Provides a formal conclusion intended to increase confidence in information against identified criteria. | Can intended users rely on a qualified practitioner's assurance conclusion? |
| Theory of change | Explains how and why activities are expected to contribute to change, including mechanisms, assumptions and context. | Is the strategy plausible, and which links in the reasoning require evidence? |
| Logic model | Organizes resources, activities, outputs and outcomes into a concise program map. | What does the program do, and how are near-term actions connected to intended results? |
| Indicator or KPI | Defines a specific signal used to monitor a condition, process, output, outcome, quality or risk. | What should be observed consistently, by whom, how often and within which boundary? |
| Dashboard | Brings selected measures, thresholds, context and decisions into an understandable operating view. | What needs attention now, and which action should the viewer consider? |
| Learning agenda | Prioritizes the questions an organization intends to answer over time through data, reflection, evaluation and research. | Which unknowns matter most, and what sequence of evidence-building is realistic? |
The U.S. Government Accountability Office separates performance management from stronger evaluative evidence. Its 2026 summary explains that ongoing performance management can act as an early-warning system, while process, outcome and impact evaluations answer deeper questions about implementation, results and causal effects. That distinction is just as useful in a nonprofit boardroom as it is in public administration.
My role has a deliberate boundary
I help nonprofits design evidence strategy, define questions, map program logic, specify indicators, improve reporting, prepare for evaluation and select qualified specialists. I do not present this strategic work as an independent evaluation, formal audit, assurance engagement, institutional review determination, statistical certification or causal proof. When the question requires that authority, I help leadership frame the need and engage the right professional.
The problem is rarely that the nonprofit has absolutely no data
Most organizations already possess case notes, attendance, financial records, grant reports, surveys, stories, web analytics, donor information and years of institutional memory. The trouble is that those materials were created for different purposes, collected under different definitions and stored in systems that have never been introduced to one another.
Outputs impersonate outcomes
Meals distributed, workshops delivered and people enrolled describe valuable activity. They do not automatically establish changed health, stability, knowledge, behavior, income, belonging or opportunity.
Every grant invents a vocabulary
The same participant becomes a client, beneficiary, household, learner or encounter depending on the report. Totals cannot be compared until the definitions and counting rules agree.
Metrics arrive before decisions
A leadership team selects attractive measures without identifying who will act on them. The dashboard becomes decorative evidence that measurement happened.
The starting condition is missing
Without a meaningful reference point, an end-of-year number may be impossible to interpret. Retrospective baselines can help, but they should not be presented as perfect substitutes.
The organization claims the whole change
Participants live inside families, schools, markets, health systems, public policies and changing communities. A nonprofit can contribute materially without being the sole cause.
Only reachable people are measured
Follow-up data may represent the participants easiest to contact. Those who left, moved, disengaged or faced the greatest barriers disappear from the apparent success rate.
The numerator looks precise
Duplicate records, changing categories, inconsistent entry, missing fields and denominator drift can produce a polished percentage resting on a surprisingly soft foundation.
The outcome matures after the grant
Behavior, housing stability, educational progress, ecological recovery and systems change may take longer than a twelve-month reporting calendar allows.
Participants pay for the evidence
Repeated surveys, intrusive questions, inaccessible forms and uncompensated storytelling can shift the cost of accountability onto people with the least institutional power.
Staff learn to protect the number
When every missed target feels punitive, teams become cautious about documenting complexity. A system meant to encourage learning begins rewarding silence.
Finance and program data never meet
Leadership cannot examine cost, intensity, reach and outcomes together because each lives in a separate platform, spreadsheet or definition.
The story outruns the evidence
Communications wants clarity, fundraising wants urgency and program staff knows reality is conditional. Without a claims process, optimism can become overstatement.
More collection is not the automatic cure. Sometimes the strongest decision is to retire a weak indicator, stop asking a burdensome question, repair a definition or acknowledge that the available evidence cannot answer the desired claim. Measurement maturity includes knowing what not to measure.
Twelve connected choices turn scattered information into organizational learning
The 2024 CDC Program Evaluation Framework is practical and nonprescriptive. It emphasizes context, program description, focused questions, credible evidence, supported conclusions and action. That final word matters. Findings that never influence a decision are expensive institutional trivia.
Decision
Name the resource, program, strategy, renewal, design or communication decision the evidence must inform.
Intended users
Identify the people who will interpret and act, along with communities and participants affected by the conclusion.
Context
Document population, place, history, program stage, policy, economy, environment, partnerships and conditions outside organizational control.
Program theory
Explain the pathway from need and resources through activities, mechanisms, outputs, outcomes and longer-term change.
Learning questions
Prioritize a small set of answerable questions instead of accepting every curiosity as an evaluation requirement.
Indicators
Specify signals that fit the question, including definitions, calculations, sources, frequency, ownership and limitations.
Reference points
Select baselines, targets, benchmarks, thresholds or comparison groups appropriate to the judgment being made.
Collection design
Choose records, surveys, interviews, observation, public data or other methods that balance credibility, ethics, access, burden and cost.
Data governance
Assign purpose, consent, access, quality, correction, retention, security, stewardship and vendor responsibilities.
Analysis
Define how quantitative and qualitative evidence will be checked, disaggregated, interpreted and tested against alternate explanations.
Communication
Match detail, uncertainty, story, visualization and recommendation to the board, funder, staff, community or public audience.
Action and learning
Record the decision, owner, timing, adaptation and question that remains open after the report is delivered.
A measurement system is an operating system
It changes role definitions, intake, program delivery, staff training, technology, board materials, grant commitments, communications review and budgeting. Treating it as a reporting project produces a report. Treating it as an organizational capability produces better judgment.
The arrow between an activity and an outcome is where the real argument lives
A logic model can show what enters a program, what the program does and what it hopes will happen next. A theory of change goes deeper by explaining why the sequence should work, which mechanisms matter, what must be true and which outside conditions can strengthen or break the pathway.
A useful logic model can include
- the need or condition being addressed;
- resources, partners and capabilities;
- activities and delivery model;
- direct products or outputs;
- short-, intermediate- and longer-term outcomes;
- priority populations and geographies;
- contextual factors and major assumptions.
A useful theory of change also explains
- the mechanism expected to produce change;
- how participants encounter and respond to the intervention;
- preconditions required at each stage;
- alternate pathways and unintended effects;
- the contribution of partners and surrounding systems;
- where uncertainty is greatest;
- which evidence could challenge the theory.
Suppose a workforce nonprofit offers credential training. Completion is an output. Employment may be an intermediate outcome. Sustained earnings, advancement and stability sit farther downstream. The theory must account for employer demand, transportation, childcare, disability access, documentation requirements, local wages, discrimination, economic cycles and whether the credential actually changes hiring behavior. A straight arrow labeled “jobs” conceals most of the strategy.
The model should match program maturity. A pilot may need to test reach, acceptance, implementation and early signals. A stable program with sufficient data may support outcome evaluation. A changing coalition working across institutions may need contribution analysis, developmental learning or systems mapping. Demanding causal proof from a young, adaptive program can be as unhelpful as asking a mature program only how many brochures it distributed.
Common theory-of-change failures
The donor's diagram
The model is built for an application, stored after submission and never used to design the program or interpret evidence.
The missing mechanism
Activities point directly to distant outcomes with no explanation of what changes in knowledge, access, behavior, capacity or conditions.
The obedient world
Every assumption favors success. Policy changes, partner capacity, market shifts and participant constraints are excluded because the boxes looked cleaner without them.
The universal participant
The model assumes every person encounters the same pathway despite differences in language, geography, disability, age, trust, risk and resources.
The program-only lens
Partners, public agencies, families, employers and community networks disappear, allowing the organization to claim change produced through a larger ecosystem.
The permanent theory
Contradictory findings are treated as reporting problems instead of reasons to revise assumptions and program design.
A metric is not defined because everyone recognizes its label
Terms such as served, engaged, retained, placed, housed, healthy, trained and successful sound obvious until two staff members calculate them. An indicator dictionary turns the label into an operational agreement.
| Specification | Question it answers | Example of hidden ambiguity |
|---|---|---|
| Name and purpose | Why does this measure exist, and which decision uses it? | A number copied into reports because it has always been there. |
| Operational definition | Exactly what qualifies and what does not? | Does “served” mean enrolled, attended once, completed or received a material? |
| Numerator and denominator | Which quantities create the rate? | A completion rate using starters one quarter and eligible participants the next. |
| Unit and population | Is the count people, visits, households, cases, acres, organizations or events? | One person appears as six service encounters. |
| Time and geography | Which period and place does the value represent? | Calendar-year intake compared with fiscal-year outcomes. |
| Source and method | Where does the information originate and how is it collected? | Self-report, staff judgment and administrative confirmation blended together. |
| Disaggregation | Which groups or locations need separate analysis? | An overall gain conceals widening differences in access or completion. |
| Frequency and owner | Who updates, checks and interprets the measure, and when? | A dashboard refreshes monthly from a field nobody maintains. |
| Target or threshold | What value prompts attention, adaptation or escalation? | Leadership celebrates an increase without defining whether it is meaningful. |
| Limitations and version | What can the measure not prove, and when did its definition change? | A revised survey is charted as though it were identical to the old instrument. |
A balanced indicator portfolio
Inputs show resources. Activities show what the organization does. Outputs show direct products and reach. Outcomes examine changes in knowledge, behavior, capacity, access or conditions. Quality measures test whether delivery meets a meaningful standard. Fidelity measures ask whether the program occurred as designed. Efficiency connects results and resources. Equity measures examine differences in access, experience and outcomes. Sustainability considers whether benefits or capabilities can last.
Leading indicators can provide earlier warning. Referral completion, attendance consistency, service wait time, participant experience or partner responsiveness may signal a future outcome. Lagging indicators show results after time has passed. Both are useful if leadership understands the relationship. Turning an early signal into a promise of long-term impact is not useful.
Baselines, targets and benchmarks
A baseline describes a starting condition. A target states an intended level or direction within a defined period. A benchmark provides an external or internal point of comparison. A threshold triggers attention or action. These are different tools. A national average may be irrelevant to a small rural program, and last year's value may be a poor baseline after a policy or eligibility change.
Targets should be ambitious enough to guide effort and credible enough to retain meaning. If a missed target creates punishment without inquiry, the system encourages gaming. If every target is revised downward after the fact, it stops guiding anything. Leadership should document why a target was set, what changed and which response follows.
“After the program” does not automatically mean “because of the program”
A participant outcome can improve after receiving services while many other influences operate at the same time. Responsible evidence strategy separates what happened, what the program plausibly contributed and what a sufficiently strong design can attribute to the intervention.
Description
Records who participated, what was delivered and what conditions or outcomes were observed. It does not isolate the program's effect.
Contribution
Tests whether the program's role is plausible and meaningful within a larger causal system, using theory, evidence, alternate explanations and stakeholder knowledge.
Attribution
Estimates the difference caused by the intervention against a credible counterfactual, often requiring specialized design, data, statistical expertise and independence.
A counterfactual is an estimate of what would have happened without the intervention. Randomized designs can create one under suitable conditions, but they are not always ethical, practical or necessary. Quasi-experimental approaches, comparison groups, interrupted time series, regression methods and other designs may be appropriate when led by qualified evaluators. A pre-post change among participants alone does not rule out maturation, selection, outside events, regression to the mean or other explanations.
This does not make ordinary monitoring worthless. Descriptive performance information can identify reach, delays, implementation problems, changing demand and promising patterns. The key is matching the strength of the statement to the strength of the design. “Eighty percent of participants completing the follow-up reported greater confidence” is different from “the program increased confidence by eighty percent,” and both differ from “the program caused sustained employment.”
Honest contribution language often improves credibility. Nonprofits rarely act alone, and pretending otherwise can erase partner organizations, families, public systems, volunteers, employers and participants themselves. Shared credit is not smaller impact. It is a more accurate account of how change happens.
The method should follow the question, maturity, risk and intended use
Evaluation is not a single product called “the evaluation.” Different designs answer different questions. Leadership should resist commissioning the most impressive-sounding method before deciding what it needs to learn.
| Approach | Primary question | When it may fit |
|---|---|---|
| Needs assessment | What conditions, assets, gaps and priorities exist? | Before program design, geographic expansion or major resource allocation. |
| Evaluability assessment | Is the program clear, stable and documented enough to evaluate? | Before investing in a significant evaluation or promising evidence to a funder. |
| Developmental evaluation | How should an emerging or adaptive initiative learn while it evolves? | Innovation, complex systems, coalitions and models without a fixed implementation path. |
| Formative evaluation | How can design or early delivery improve? | Pilots, launches, new audiences and programs still resolving major assumptions. |
| Process or implementation evaluation | Was the program delivered, accessed and experienced as intended? | When outcomes are weak or uneven and leadership needs to understand delivery. |
| Outcome evaluation | Are desired changes occurring, for whom and under which conditions? | Established programs with defined outcomes, sufficient follow-up and usable data. |
| Impact evaluation | What effect did the program cause compared with its absence? | High-stakes effectiveness questions with suitable design, scale, ethics and expertise. |
| Economic evaluation | How do costs relate to results or alternatives? | Resource choices requiring qualified cost-effectiveness, cost-benefit or related analysis. |
| Participatory evaluation | How can affected people meaningfully shape questions, methods and interpretation? | When relevance, legitimacy, local knowledge and power require shared authority. |
| Equity-focused evaluation | How do access, experience, benefit and harm differ across groups and systems? | Whenever aggregate results may conceal exclusion, burden or unequal outcomes. |
The OECD's six evaluation criteria offer useful lenses: relevance, coherence, effectiveness, efficiency, impact and sustainability. OECD also warns against using them mechanically. The purpose, stakeholders and context should determine which criteria matter and how they are interpreted. A small community initiative does not become more rigorous because every possible criterion was forced into a forty-page scope.
When independence matters
An external evaluator can bring methodological depth, neutrality, specialized subject knowledge or confidence for a consequential decision. Independence may be required by a funder, regulator, research protocol or governance process. It can also protect staff from being asked to grade their own design. I can help define evaluation questions, organize existing evidence, assess readiness, build a request for proposals, compare scopes and support implementation without representing strategic preparation as the independent conclusion.
The American Evaluation Association's current principles emphasize systematic inquiry, competence, integrity, respect for people, common good and equity. Those principles also call for transparent methods, limitations, conflicts and use of emerging technologies. An organization selecting an evaluator should examine fit with the program, methods, community, ethical demands and intended use, not merely choose the proposal containing the most advanced vocabulary.
A number can show a pattern; a conversation can reveal the mechanism
Quantitative and qualitative evidence are not competing teams. Each can be rigorous or careless. The right mix depends on the question, population, program, decision and resources.
Quantitative evidence can help examine
Reach, frequency, duration, completion, changes over time, distributions, subgroup differences, relationships, costs, thresholds and patterns across larger numbers of observations. Quality depends on definitions, sampling, instruments, missing data, measurement error and analysis.
Qualitative evidence can help explain
Experience, meaning, barriers, mechanisms, context, unintended effects, implementation, trust and why a pattern differs across people or places. Quality depends on question design, selection, facilitation, documentation, coding, interpretation, reflexivity and transparency.
Mixed methods can connect breadth and depth. Survey results may show that attendance declines after the third session; interviews may reveal that the schedule conflicts with a new work shift or that transportation stops running. Administrative records may show higher completion at one location; observation may identify a delivery practice worth testing elsewhere. The value comes from integration, not simply placing a chart beside a quotation.
Sampling and missingness are strategic issues
A sample must be appropriate to the conclusion. Convenient respondents may not represent the full population. Small numbers can still generate valuable qualitative learning, but they may not support stable percentages or subgroup comparisons. Missing data is rarely random by default. Participants with unstable housing, language barriers, safety concerns, limited connectivity or negative experiences may be harder to reach, which means the most convenient dataset can systematically exclude critical reality.
Disaggregation should be purposeful and safe. It can reveal inequity, geographic variation or different pathways, but small cells may expose identities or create unstable conclusions. Categories should reflect legitimate analytic need, respectful language, participant choice and applicable guidance. Collecting sensitive information because a software field exists is not evidence strategy.
Technology should support the evidence model, not quietly invent it
A CRM, case-management platform, survey tool or business-intelligence product can improve consistency and access. It cannot decide which outcomes matter, repair a broken theory of change or turn an ambiguous field into truth.
Program and case systems
Eligibility, intake, service plans, touchpoints, referrals, participation, milestones, follow-up, consent, case notes and outcomes.
Fundraising and CRM
Donors, grants, restrictions, communications, campaigns, relationships and stewardship, connected only where a legitimate purpose exists.
Finance and grants
Budgets, expenses, awards, allocations, subawards and reporting periods needed to interpret resources alongside delivery and results.
Surveys and listening
Participant feedback, pre-post instruments, partner experience, staff insight, interviews, focus groups, complaints and community input.
Digital and communications
Website behavior, email, search, social, content use and conversion signals that show attention or action, not mission impact by themselves.
Public and partner data
Census, health, education, environmental, economic, administrative and partner sources that add context without replacing local knowledge.
The governance layer
Each material field needs a purpose, definition, owner, source, permissible use, access rule, retention period, correction method and quality expectation. Unique identifiers and matching rules require special care. An accidental duplicate can inflate reach. An aggressive matching process can create privacy or misidentification risk. Integration should move only the information necessary for an authorized purpose.
Human-subjects research, quality improvement and ordinary program operations can carry different requirements. The HHS Office for Human Research Protections notes that some projects involving research require institutional review board consideration. A nonprofit should obtain qualified legal, privacy, research-ethics and institutional guidance when its activity may cross that boundary, especially with vulnerable populations, health information or plans for generalizable research.
Accessible collection is part of data quality. If a survey, dashboard or report excludes people using assistive technology, limited-bandwidth connections, translation or alternative formats, the evidence may reflect the interface more than the population. The W3C Web Content Accessibility Guidelines 2.2 provide a current technical reference for digital accessibility.
Where AI may help
AI can assist with document organization, approved qualitative coding, translation support, missing-field detection, routine explanations, draft summaries, anomaly review and knowledge retrieval. It may help a team find patterns deserving human investigation. It should not silently decide which community is worthy, label participant risk, fabricate missing observations, infer protected characteristics, generate unsupported impact claims or replace expert evaluation judgment.
The NIST AI Risk Management Framework offers a voluntary structure for governing, mapping, measuring and managing AI risk. For nonprofit evidence work, an AI use case should identify the purpose, source data, affected people, validation method, human reviewer, override, documentation, monitoring, incident response and accountable owner. Sensitive information should not be placed into a tool merely because the interface makes pasting easy.
A dashboard should tell the viewer what deserves attention, not prove that software has colors
The best dashboard begins with role and decision. A program manager, CEO, board member, funder and public audience need different levels of detail, timing, context and authority. Combining every need into one screen usually produces a crowded compromise nobody trusts.
| Dashboard | Useful content | Decision it supports |
|---|---|---|
| Program operations | Intake, wait time, capacity, attendance, completion, referrals, follow-up, exceptions and service quality. | Where should a manager intervene this week? |
| Outcome learning | Outcome trends, cohorts, dose, subgroup differences, context, uncertainty and follow-up coverage. | Which part of the model appears to work, stall or require investigation? |
| Board strategy | Mission outcomes, reach, cost, major risks, equity, evidence strength, targets and decisions requiring governance. | Does the portfolio still deserve its direction and resources? |
| Executive portfolio | Programs, geographies, funding concentration, capacity, outcome signals, partner health and organizational priorities. | Where should leadership allocate attention and investment? |
| Grant performance | Award goals, required indicators, baselines, targets, reporting status, spending and supported narrative. | Is the organization meeting commitments and preparing an accurate report? |
| Community access | Demand, eligibility, referral source, geography, language, disability access, wait time, participation and exit. | Who is reaching the program, who is not and where does the pathway break? |
| Partner network | Referrals, handoffs, service gaps, shared outcomes, communication, capacity and unresolved dependencies. | Which relationships strengthen or constrain the participant journey? |
| Fundraising evidence | Validated outcomes, program capacity, funding need, restricted use, evidence dates and approved claims. | Which accurate proof can support a donor conversation or proposal? |
| Public impact | A concise set of current, understandable findings with definitions, sources, limitations and stories. | What can the organization responsibly tell the public? |
| Data quality | Missingness, duplicates, timeliness, unusual values, definition changes, instrument completion and ownership. | Can leadership trust the evidence feeding every other view? |
A useful dashboard includes interpretation. Targets, thresholds, notes, denominator changes and program events prevent a viewer from inventing a story around a line. A sudden decline may reflect staff turnover, a changed intake rule, a seasonal pattern or a genuine failure. The screen should make investigation easier rather than present every movement as verdict.
My KPI dashboard design consulting covers dashboard strategy across organizations. Nonprofit impact consulting addresses the evidence and decisions behind the display. When forecasting, scenario modeling or advanced prediction is the main need, my predictive analytics consulting can provide additional support.
One evidence base can support many reports without forcing every audience into the same document
A board needs governance signals. A program team needs operating detail. A funder needs the commitments and learning attached to an award. A donor needs an accurate reason to care. A community may need access, accountability and an understandable response to what it shared.
Board reporting
Connect mission, strategy, resources, outcomes, evidence quality, risk and decisions. Directors should not need to excavate a hundred program metrics to find the three issues requiring governance.
Grant reporting
Map each commitment to its definition, source, period, owner and narrative. Explain material variance instead of cosmetically repairing it after the deadline.
Public impact reporting
Use current findings, plain language, responsible visuals, accessible formats, partner credit, participant dignity and clear limits on what the evidence establishes.
Development communication
Give fundraisers an approved evidence library with dates, populations, calculations, source notes, claim strength and language that can be used without distortion.
Program learning
Provide findings early enough to change delivery. A perfect annual presentation is not a substitute for a timely conversation about a failing referral process.
Community accountability
Return findings in forms people can access and use. Explain what changed, what did not, who decided and how participants can challenge an interpretation.
For organizations receiving federal awards, 2 CFR 200.301 addresses goals, outcomes, indicators, targets, baselines, data collection and performance reporting. It also states that agencies should not require information unnecessary for performance measurement and evaluation. Specific award terms still govern, and qualified grants, legal and compliance professionals should interpret obligations.
Form 990 program-service reporting creates another public evidence surface. The IRS instructions ask organizations to describe significant program accomplishments and encourage objective measures when available. That does not turn the tax return into an impact evaluation, but it does mean inconsistent public claims can travel farther than the annual report.
A practical claim ladder
What the organization did
Delivered, funded, convened, trained, contacted, distributed or referred, using clear units and periods.
Who participated
Reached, enrolled, attended, completed or responded, with counting rules and limitations.
What was observed
Reported, recorded or measured change among a defined population at a defined time.
What the work contributed
A plausible role supported by theory, triangulated evidence, context and analysis of alternatives.
What the program caused
A causal effect supported by a suitable impact-evaluation design and qualified interpretation.
What remains uncertain
Missing data, selection, timing, context, instrument limits and alternate explanations stated plainly.
The organization does not need to sound timid. It needs to sound accurate. “Partner organizations and 640 participants helped test a new approach that improved completion during the pilot” can be powerful. It also leaves room for the evidence, people and conditions that made the result possible.
Evidence without humanity is forgettable; humanity used without care is exploitation
Stories help a board, donor or public audience understand what a measure means in a life, place or institution. They should add context and mechanism, not decorate a number or pressure participants to perform gratitude.
A responsible story process considers informed consent, comprehension, privacy, safety, future discoverability, power, compensation, review, withdrawal and how refusal affects services. A person may agree to an internal learning interview without agreeing to a public video indexed forever. A parent may consent for a child under conditions that require additional specialist review. A survivor's identity may remain inferable even after a name is removed.
Selection also matters. The most camera-ready success story may not represent a typical pathway. Publishing only exceptional outcomes can distort expectations and hide barriers affecting everyone else. A story can be explicitly presented as an illustration rather than proof. Multiple shorter perspectives may reveal variation better than a single heroic arc.
My storytelling and authority work connects complex evidence to clear narrative, while my digital marketing consulting connects that narrative to audiences, channels and conversion. Impact measurement provides the evidence and supports the claims underneath the story.
There is no universal nonprofit impact score
The examples below are prompts, not ready-made metric menus. The appropriate outcomes depend on mission, program theory, population, geography, stage, resources, ethics, data access and the decision at hand.
Human services
Access, wait time, continuity, participant-defined goals, referral completion, stability, safety, service quality and the conditions affecting follow-up.
Food security
Reach and distribution alongside reliable access, dietary fit, dignity, utilization, household conditions, referral networks and changing local need.
Housing and homelessness
Placement is only one point. Retention, housing quality, returns to homelessness, income, service connection, landlord relationships and market constraints may matter.
Youth and education
Participation, belonging, knowledge, attendance, persistence, academic or career milestones, family context, safety and longer-term pathways.
Health nonprofits
Reach, screening, navigation, care access, adherence, experience, health outcomes and disparities, with qualified clinical, privacy and research guidance.
Disability organizations
Access, self-determination, participation, accommodation, service quality, employment, community living and outcomes defined with disabled people.
Conservation and environment
Acres and events may sit beside ecological condition, species, water, stewardship behavior, policy, resilience, community benefit and long time horizons.
Arts, museums and culture
Attendance can be joined by access, learning, artistic development, representation, cultural stewardship, visitor experience and community connection.
Advocacy and coalitions
Reach, participation, relationships, narrative change, capacity, policy movement and institutional response require contribution thinking rather than simplistic causation.
Veterans organizations
Connection, navigation, employment, health access, housing, family outcomes and trust should respect service context and participant privacy.
Foundations and grantmakers
Portfolio alignment, partner experience, grant burden, organizational capacity, community voice, learning and the funder's contribution to field outcomes.
Associations and membership bodies
Member value, participation, professional outcomes, standards, chapters, advocacy, workforce conditions and institutional influence.
Environmental and systems work is especially familiar to me. My background in Earth and Environmental Sciences makes me attentive to scale, lag, interacting causes, feedback, uncertainty and outcomes that refuse to fit a neat annual calendar. My conservation and environmental consulting and science and deep-technology work provide useful adjacent context.
Evidence becomes trustworthy when ownership is specific
| Role | Useful responsibility | Failure to avoid |
|---|---|---|
| Board | Govern mission outcomes, material risk, strategy, resources, evidence quality and the decisions requiring director authority. | Micromanaging data entry or accepting ceremonial success stories without limitations. |
| CEO and executive team | Set organizational learning priorities, align functions, resource capacity and act when evidence challenges a favored strategy. | Demanding proof while leaving measurement nobody's actual job. |
| Program leadership | Define delivery reality, interpret context, maintain measures and use findings to improve service. | Being treated as a data supplier instead of an evidence user. |
| Development and grants | Translate funder commitments into feasible requirements and use approved findings accurately. | Promising a measure before program and data leaders confirm it can be produced. |
| Finance | Connect resources, restrictions, periods and costs with program activity and outcome questions. | Assuming financial categories already map cleanly to program evidence. |
| Communications | Shape accessible narrative, visual explanation, consent and claims review for each audience. | Turning an association into causation because the headline sounds better. |
| Data and technology | Maintain architecture, integrations, access, quality controls, documentation and vendor accountability. | Letting platform defaults decide the organization's theory of change. |
| Evaluation partner | Bring methods, independence, ethics, analysis and interpretation suited to the question and context. | Receiving an impossible scope after every public promise has already been made. |
| Community and participants | Shape relevance, questions, acceptable collection, meaning, interpretation and use where authority is genuine. | Being consulted after the measures are fixed or used only as story material. |
For deeper work on board structure, executive relationships and the difference between governing and advisory roles, see my board of directors consulting and advisory page. The board does not need every metric. It needs a defensible view of whether mission strategy is producing useful results, where evidence is weak and which decision belongs at the governance level.
Evidence strategy from the first question through practical use
Measurement diagnostic
Review strategy, programs, commitments, indicators, collection, platforms, reports, claims, roles, capacity and current decision use.
Theory of change
Clarify need, mechanisms, assumptions, pathways, partners, context, expected outcomes, unintended effects and open questions.
Logic-model development
Build an understandable map connecting resources, activities, outputs, short-term change and longer-term outcomes.
Learning agenda
Prioritize questions by strategic value, urgency, feasibility, evidence gap, risk, stakeholder need and expected use.
Indicator architecture
Create outcome sets, definitions, formulas, reference points, disaggregation, sources, owners, limitations and version control.
Data and system mapping
Trace collection through CRM, program, survey, finance, grant, digital and reporting systems to identify gaps and duplication.
Dashboard strategy
Define users, decisions, views, measures, thresholds, context, refresh, drill-down, accessibility and governance requirements.
Board evidence design
Connect mission outcomes, strategy, resources, risk, evidence strength and decisions in a governance-level view.
Funder reporting system
Map award commitments, definitions, sources, dates, owners, variance explanations, approvals and reusable core evidence.
Evaluation readiness
Clarify the program, questions, records, ethics, stakeholders, resources and scope before a formal evaluator is engaged.
Evaluator selection support
Develop requirements, requests for proposals, evaluation criteria, interview questions and implementation coordination without prejudging findings.
Qualitative listening design
Structure interviews, focus groups, partner input, community feedback, documentation, consent, coding and interpretation around defined decisions.
Impact narrative system
Create claims levels, source notes, approval, story consent, partner credit, visual explanation and reusable evidence for communications.
AI evidence governance
Define permitted uses, sensitive data boundaries, validation, human review, transparency, monitoring and accountable ownership.
Team and board facilitation
Build shared definitions, resolve conflicting expectations and turn difficult findings into explicit organizational choices.
Fractional evidence leadership
Provide continuing senior guidance across programs, grants, data, dashboards, communications, evaluation partners and implementation.
A project may focus on one program, a portfolio, a grant requirement, a dashboard, an evaluation-readiness process or the complete organizational evidence model. Retained and fractional work can support adoption across reporting cycles. I can build frameworks and requirements, facilitate decisions, write content, guide vendors and work beside internal teams rather than disappearing after a presentation.
I understand evidence as science, management, communication and institutional behavior
I am Dr. Rob Urban. I founded Paper Boat Media, and clients work with me directly as their senior advisor. My value here is not pretending that one discipline owns the whole problem. Impact evidence crosses program design, research thinking, executive judgment, technology, fundraising, governance and public narrative.
Earth and Environmental Sciences
Doctoral training taught me to define variables carefully, work across interacting systems, respect scale and time, test assumptions, interpret imperfect evidence and state uncertainty without surrendering judgment.
U.S. Marine veteran
Military service reinforced preparation, accountability, role clarity and learning after action. Evidence should change the next operation, not merely document that the last one occurred.
Seven-time national bestselling author
Writing taught me how to preserve complexity while making it understandable. A credible impact narrative needs both disciplined evidence and a human reason to care.
My work spans executive strategy, nonprofit growth, associations, boards, fundraising, analytics, AI, marketing, reputation, public affairs, environmental issues, community participation and storytelling. That range helps when the apparent dashboard problem is really a strategy problem, the reporting problem began in a grant promise or the data-quality problem reflects how staff experience the program.
I know the edge of the work. Qualified evaluators, statisticians, auditors, assurance practitioners, accountants, attorneys, privacy professionals, subject-matter experts, institutional review boards and research-ethics leaders should own determinations that require their authority. I can help the organization ask a better question, prepare better evidence, select the right specialist and use the result intelligently.
Start with the consequential decision, then earn the measure
Understand mission, programs and pressure
I examine strategy, funder commitments, stakeholder expectations, current reports, data, platforms, capacity, timeline and the decision creating urgency.
Trace the evidence already present
Work follows definitions, collection, handoffs, systems, analysis, approval and communication to reveal gaps, duplication and quiet workarounds.
Define theory, users and questions
Leadership clarifies the program logic, intended evidence users, priority questions, ethical boundaries and claims the design must support or decline.
Create the measurement architecture
I build indicators, specifications, sources, dashboards, reports, roles, governance, requirements and an achievable implementation sequence.
Find ambiguity before scale makes it expensive
A pilot can expose confusing definitions, inaccessible collection, staff burden, integration gaps, unstable calculations and reports that fail the intended decision.
Connect evidence to operating rhythm
Training, meeting cadence, grant workflow, board materials, communications review and ownership turn the design into normal practice.
Act, document and revise
The organization records what the evidence changed, which uncertainty remains and how the next cycle will improve the program and measurement system.
Evidence must respect the place, population and system producing it
Paper Boat Media is based only in DeLand, Florida. The locations below are markets I serve through remote collaboration, blended engagements and on-site work when the program, community, board or evidence process benefits from being in the room.
- DeLand
- Ocala
- Daytona Beach
- Lake Mary
- Palm Beach
- Orlando
- Space Coast
- Miami
- Sarasota
- Boca Raton
- Ponte Vedra Beach
- Naples
- Washington, D.C.
- Denver
- Nashville
- New York City
- Charleston
- Silicon Valley
- Los Angeles
- Dubai
- United Kingdom
- Abu Dhabi
A Volusia County human-services organization, an Orlando health foundation, a Washington policy nonprofit, a New York cultural institution, a Silicon Valley technology philanthropy and an international NGO should not inherit identical assumptions. Definitions, consent, available data, law, infrastructure, culture, geography and stakeholder authority change the evidence design. Sound reasoning can travel. Context-free metrics should not.
Authoritative frameworks informing this page
These sources support terminology, evaluation thinking, federal performance requirements, ethics, accessibility and technology-risk boundaries. They do not replace guidance specific to a grant, jurisdiction, profession, population or research protocol.
- CDC: Program Evaluation Framework
- U.S. GAO: Assessing and Improving Program Effectiveness
- W3C: Web Content Accessibility Guidelines 2.2
- IRS: Instructions for Form 990
- American Evaluation Association: Guiding Principles
- NIST: AI Risk Management Framework
- eCFR: 2 CFR 200.301 Performance Measurement
- UK FCDO: Evaluation Policy and OECD-DAC Criteria
- HHS OHRP: Quality Improvement and Research Questions
- U.S. GAO: Program Evaluation Key Terms and Concepts
- CDC MMWR: Program Evaluation Framework, 2024
Nonprofit impact measurement and evidence strategy questions
These 30 questions cover the distinctions, design decisions and working relationships most likely to matter before an engagement.
Measurement and evidence fundamentals
What does a nonprofit impact measurement consultant do?
I help leadership connect mission, program theory, outcomes, indicators, data, dashboards, evaluation plans, reports and public claims. The work can include diagnosis, facilitation, architecture, writing, requirements, implementation guidance and coordination with qualified evaluators or specialists.
What is nonprofit impact measurement?
Nonprofit impact measurement is the organized use of evidence to understand progress, outcomes, contribution and learning connected to mission. It includes more than counting activity and should be designed around decisions the organization can actually make.
What is the difference between an output and an outcome?
An output records a direct product of activity, such as workshops delivered or participants enrolled. An outcome describes a change in knowledge, behavior, capacity, access or condition. Both can matter, but one should not be presented as the other.
Is impact measurement the same as program evaluation?
No. Continuing measurement monitors selected indicators, while evaluation uses a focused design to answer defined questions about context, implementation, outcomes, value or causal effects. A measurement system can support an evaluation without becoming one.
Does every nonprofit need an impact measurement system?
Every nonprofit benefits from enough evidence to govern mission, improve programs and communicate responsibly. The scale should match the organization's programs, risk, resources, commitments and decisions. A small organization does not need an imitation research institute.
Theory of change, indicators and claims
What is the difference between a theory of change and a logic model?
A logic model concisely maps resources, activities, outputs and outcomes. A theory of change explains why the pathway should work, including mechanisms, assumptions, context, alternate pathways and outside conditions.
How should a nonprofit choose impact indicators?
Begin with the decision, theory of change and priority learning question. Then assess relevance, validity, feasibility, burden, ethics, timing, data quality, disaggregation and who will own interpretation. Familiarity alone does not make an indicator useful.
How many KPIs should a nonprofit track?
There is no universal number. Use a focused set for each role and decision, supported by deeper diagnostic measures when needed. A board, program manager and funder should not receive identical dashboards merely because the data exists.
What is a baseline?
A baseline describes the starting condition against which later observations may be compared. It must use a defined population, measure, period and method. A retrospectively assembled baseline can be useful if its limitations are explicit.
Can a nonprofit say that its program caused an outcome?
Only when a suitable evaluation design and qualified analysis support causal attribution. Many organizations can responsibly describe observed outcomes and explain a plausible contribution without claiming that the program alone caused the change.
Evaluation planning and independence
When should a nonprofit hire an independent evaluator?
Consider an independent evaluator when a funder requires one, the decision is consequential, causal or specialized methods are needed, internal conflicts could affect credibility or leadership needs expertise the organization does not possess.
What is evaluation readiness?
Evaluation readiness means the program, purpose, questions, theory, delivery, records, stakeholders, ethics, resources and intended use are clear enough to support a credible evaluation. A readiness review can prevent an expensive study from answering the wrong question.
Can you conduct a formal impact evaluation?
I do not represent this strategic consulting as a formal independent or causal impact evaluation. I can prepare the organization, clarify questions, organize evidence, develop requirements, help select qualified evaluators and support leadership in using their findings.
What is a process evaluation?
A process or implementation evaluation examines how a program was delivered, reached participants and operated in context. It can reveal whether weak outcomes reflect the theory, access, fidelity, capacity, experience or implementation.
What is an impact evaluation?
An impact evaluation estimates a program's causal effect by comparing observed results with a credible estimate of what would have happened without the intervention. It requires a suitable design, evidence, ethics and qualified methodological judgment.
Data, dashboards and reporting
What should a nonprofit impact dashboard include?
Include only measures, thresholds, context and limitations needed by the intended user. Depending on the decision, the view may cover reach, access, delivery, outcomes, cost, equity, partner health, data quality, targets and actions.
Can you help design a nonprofit KPI dashboard?
Yes. I can define users, decisions, measures, specifications, source systems, thresholds, refresh, annotations, accessibility, governance and adoption. Technical implementation can be coordinated with the organization's platform and qualified providers.
How can a nonprofit reduce reporting burden?
Create a reusable core evidence model, map funder-specific additions, standardize definitions, collect information once when appropriate, retire unused fields, integrate systems carefully and ask only questions tied to a legitimate decision or requirement.
What should a board impact report show?
Directors generally need mission outcomes, strategic progress, resource context, material risk, evidence quality, disparities, major assumptions and decisions requiring governance. Detailed operating measures should appear only when they explain a board-level issue.
Can the same evidence support grants, fundraising and public reports?
Yes, when the core definitions, sources and limitations remain stable. Each audience needs different detail and framing, but the organization should not quietly change the meaning or strength of a claim between a grant report and a donor campaign.
Ethics, stories and AI
How should nonprofits use participant stories?
Use meaningful consent, clear intended use, privacy protection, respectful framing, appropriate review and the ability to decline without affecting services. Present a story as an illustration unless the evidence supports a broader conclusion.
Should participants be compensated for evaluation input?
Compensation may recognize time and expertise, reduce participation barriers and improve fairness. The design should consider budget, context, undue influence, benefits rules, organizational policy and qualified ethical or legal guidance.
Can AI analyze nonprofit impact data?
AI can assist with bounded tasks such as approved coding, summarization, translation support, anomaly review or document retrieval. Sensitive data, bias, validation, transparency, human review, security and accountability must be addressed before use.
Can AI write an impact report?
AI can support drafting, but accountable people must verify every fact, calculation, quotation, claim, limitation, consent condition and recommendation. A fluent paragraph does not prove that the source data or interpretation is correct.
Does program evaluation require an IRB?
Sometimes a project may constitute human-subjects research or fall under institutional, funder or legal requirements. The appropriate research-protection office, institutional review board, counsel or qualified specialist should make that determination for the specific activity.
Working with Dr. Rob Urban
Do nonprofit clients work directly with Rob?
Yes. I personally advise executives, boards and cross-functional teams. Qualified evaluators, statisticians, attorneys, privacy professionals, accountants or technical specialists can join when their authority is required.
Can this begin as a focused diagnostic?
Yes. A diagnostic can examine one program or the organizational evidence system, identify the highest-risk gaps and produce a prioritized path before leadership commits to a larger build.
Do you work on projects, retainers and fractional engagements?
Yes. A defined project can solve a specific architecture or reporting problem. Retained and fractional work can support implementation, reporting cycles, evaluator coordination, board use and continuing organizational learning.
Which geographic markets do you serve?
Paper Boat Media is based in DeLand, Florida. I serve clients across Florida, established national markets and selected international markets through virtual, blended and on-location work.
Can the engagement start before leadership knows the exact scope?
Yes. Begin with the decision, pressure, confusing report, disputed metric or outcome the organization cannot explain. I can determine whether the constraint is strategy, theory, definition, collection, technology, evaluation, communication, governance or capacity.
Build evidence strong enough to guide the work and carry the story
The organization may have meaningful outcomes and weak reporting, extensive data and little agreement, a funder asking for causal language, a board buried in activity counts or a story that deserves more support than the current evidence can provide.
You do not need to know exactly what you need. Explain the program, the pressure, the decision and what feels unreliable. I will trace the problem across strategy, evidence, systems, people and communication, then define the most responsible next move.