Capacity building-Mexico:Theories of Action and establishing an evaluation framework

As part of our project conducting a participatory assessment of a  foundation’s new grantee portfolio in southern Mexico, I’m now in the Yucatán, conducting site visits to groups on the peninsula.

To recap, our goals for the project are:

 1) Establish a monitoring baseline, which will retrospectively provide a snapshot of grantees’ progress to date based on stated program goals, and provide a baseline for future monitoring and evaluation (M&E).

 2) Support grantees’ development of their own monitoring capacity, including strengthening their data collection and organizational learning strategies. An overarching goal is for grantees to think about evaluation not only as a reporting requirement but instead as an important tool for optimizing their work.

Through a condensed theory of action mapping exercise, we identified outcomes for each grantee’s project with a particular focus on those that grantee’s felt could be achieved during the current grant period. We also collaboratively identified key indicators for these short-term outcomes. We found that the process was important for grantees, as they often described their outcomes in a way that lacked precision, was too conceptual, and/or used terminology that was unclear to those external to the project.  In addition, we found that grantees often misconstrued activities/products (outputs) as outcomes. To address this, we worked with grantees’ project teams to understand this important difference and focus on outcomes, or how their target communities would actually be different upon completion of the project. We had grantees articulate outcomes in terms of changes in conditions, institutions, behaviors, understanding, relationships, abilities, policies, etc.

We believe – and grantees told us – that our interventions added value in the following ways:

• Uncovering gaps in logic where groups had not discussed or explored their assumptions and realized they should include earlier priority outcomes and discuss what activities would be important to achieve them. A common example of an early priority outcome that some groups had not considered is “community buy-in,” or a community’s appropriation of the grantee’s particular project. Another common outcome to consider that emerged from discussions was moving from access to use. For example, it is not enough to simply provide access to new types of fresh vegetables; communities actually need to prepare and eat these vegetables if the desired outcome of improved nutrition and increased food security is to be achieved. While the access-use distinction may seem self-evident, often grantees had not considered how their projects would support this linkage programmatically.

• As a process for “putting on paper” what was in the heads of project staff, but not well articulated. In addition, we were told that the visual map was valuable as a communication tool, for their own staff, funders, etc.

Some important points to keep in mind when implementing this process yourself:

1) Given the short time-frame (3-4 hours max),  it’s important to come to the meeting with a set of shorter and longer term outcomes,  on which to build during the meeting with project staff.  While we could have identified short and long terms outcomes from proposals, pulling together a first map was significantly expedited by use of the pre-visit survey that asked staff directly for short and longer term outcomes. Send this draft ahead of time; ask staff to consider, briefly, what is missing, mischaracterized, etc.

2) The Theory of Action process is most valuable when all of key staff are present. At the end of the day, the process, and the discussion that arises, is more valuable then any final “product”

3) We learned that explaining the difference between outcomes, activities, and outputs, was important even for experienced groups. It was helpful to try and push groups by asking questions such as “How are people actually different or transformed because they attended the workshop”? Through these questions, and by providing examples, groups began to grasp this difference.

4) Given our goals, we asked staff what outcomes they thought could be achieved over the term of the grant. Then, we asked each participant to put two colored dots on the outcomes we felt were most important for reaching longer term goals. This process of prioritizing outcomes helped the grantee understand if their strategy had the appropriate focus, and for us,  what what were the key outcomes to track moving forward.

5) We asked groups to also add any external factors or outcomes to the maps that could help or hinder their progress, but that were out of their control. We sensed that this was helpful in illuminating which factors might need to be part of the “map” in order to reach outcomes at a higher level.

6) Our final step was identifying which outcomes the groups felt could be achieved (in part or whole) during the grant period. To these outcomes we added indicators.   Even the more savvy groups found it challenging to come up with qualitative outcomes. They wanted to look at their computers and reports; they seemed a bit unsure at times when they couldn’t find answers. Essentially, we found that groups just needed to be given confidence that their instincts were on the right track. Often, we told them to just think about their own personal perspective on what they would see, observe, notice, hear, etc. that would tell them that  there was progress toward this outcome, or if instead the outcome wasn’t happening at all.