Capacity building-Mexico:Theories of Action and establishing an evaluation framework

As part of our project conducting a participatory assessment of a  foundation’s new grantee portfolio in southern Mexico, I’m now in the Yucatán, conducting site visits to groups on the peninsula.

To recap, our goals for the project are:

 1) Establish a monitoring baseline, which will retrospectively provide a snapshot of grantees’ progress to date based on stated program goals, and provide a baseline for future monitoring and evaluation (M&E).

 2) Support grantees’ development of their own monitoring capacity, including strengthening their data collection and organizational learning strategies. An overarching goal is for grantees to think about evaluation not only as a reporting requirement but instead as an important tool for optimizing their work.

Through a condensed theory of action mapping exercise, we identified outcomes for each grantee’s project with a particular focus on those that grantee’s felt could be achieved during the current grant period. We also collaboratively identified key indicators for these short-term outcomes. We found that the process was important for grantees, as they often described their outcomes in a way that lacked precision, was too conceptual, and/or used terminology that was unclear to those external to the project.  In addition, we found that grantees often misconstrued activities/products (outputs) as outcomes. To address this, we worked with grantees’ project teams to understand this important difference and focus on outcomes, or how their target communities would actually be different upon completion of the project. We had grantees articulate outcomes in terms of changes in conditions, institutions, behaviors, understanding, relationships, abilities, policies, etc.

We believe – and grantees told us – that our interventions added value in the following ways:

  • Uncovering gaps in logic where groups had not discussed or explored their assumptions and realized they should include earlier priority outcomes and discuss what activities would be important to achieve them. A common example of an early priority outcome that some groups had not considered is “community buy-in,” or a community’s appropriation of the grantee’s particular project. Another common outcome to consider that emerged from discussions was moving from access to use. For example, it is not enough to simply provide access to new types of fresh vegetables; communities actually need to prepare and eat these vegetables if the desired outcome of improved nutrition and increased food security is to be achieved. While the access-use distinction may seem self-evident, often grantees had not considered how their projects would support this linkage programmatically.
  • As a process for “putting on paper” what was in the heads of project staff, but not well articulated. In addition, we were told that the visual map was valuable as a communication tool, for their own staff, funders, etc.

Some important points to keep in mind when implementing this process yourself:

1) Given the short time-frame (3-4 hours max),  it’s important to come to the meeting with a set of shorter and longer term outcomes,  on which to build during the meeting with project staff.  While we could have identified short and long terms outcomes from proposals, pulling together a first map was significantly expedited by use of the pre-visit survey that asked staff directly for short and longer term outcomes. Send this draft ahead of time; ask staff to consider, briefly, what is missing, mischaracterized, etc.

2) The Theory of Action process is most valuable when all of key staff are present. At the end of the day, the process, and the discussion that arises, is more valuable then any final “product”

3) We learned that explaining the difference between outcomes, activities, and outputs, was important even for experienced groups. It was helpful to try and push groups by asking questions such as “How are people actually different or transformed because they attended the workshop”? Through these questions, and by providing examples, groups began to grasp this difference.

4) Given our goals, we asked staff what outcomes they thought could be achieved over the term of the grant. Then, we asked each participant to put two colored dots on the outcomes we felt were most important for reaching longer term goals. This process of prioritizing outcomes helped the grantee understand if their strategy had the appropriate focus, and for us,  what what were the key outcomes to track moving forward.

5) We asked groups to also add any external factors or outcomes to the maps that could help or hinder their progress, but that were out of their control. We sensed that this was helpful in illuminating which factors might need to be part of the “map” in order to reach outcomes at a higher level.

6) Our final step was identifying which outcomes the groups felt could be achieved (in part or whole) during the grant period. To these outcomes we added indicators.   Even the more savvy groups found it challenging to come up with qualitative outcomes. They wanted to look at their computers and reports; they seemed a bit unsure at times when they couldn’t find answers. Essentially, we found that groups just needed to be given confidence that their instincts were on the right track. Often, we told them to just think about their own personal perspective on what they would see, observe, notice, hear, etc. that would tell them that  there was progress toward this outcome, or if instead the outcome wasn’t happening at all.

 

 

 

 

Trackbacks

  1. […] than working with the grantee project staff on their Theory of Action, we also visited the project beneficiaries. Our goals were to understand what has happened to date, […]

  2. […] In our work, we’ve modified and evolved the ToA/ToC process to suit the needs of the groups we work with and have integrated other tools, particularly to build on ToA for action planning. (See Basic Steps and Tips and Lessons Learned). While we found  discussions on the logical linkages between activities and shorter and longer term outcomes as well as the broader vision to be of value, it is often challenging for organizations to embed the ToA in their organizational planning and reflection practices without support and guidance. Getting participants to take active roles and even leadership in the process can be helpful. We designed this facilitation guide to helps groups to lead their own ToA Processes. While at some point in the process ToA is best facilitated by an external consultant who understands the process well, the guide can help groups get started using easy to follow steps. In addition, the guide can support initial ToA meetings in distant locations that can then feed into larger ongoing organizational processes for multinational or multi-site organizations or collaborative initiatives aiming to develop an meta level or umbrella theory. We also describe a ToA approach for a group interested in linking directly to action planning, and created an early post here how we used ToA as part of a participatory assessment for a donor.  […]

Speak Your Mind

*