Evaluating Communities of Practice & Social Learning

During one of the days of the KM4Dev Seattle convening  we discussed the CoP Value Creation Framework.

The framework identifies five stages of value creation for social learning – Immediate value (what you experience), potential value (what you get out of it), applied value (what you do with it) , realized value (results), and transformative value (radical changes in you or your culture). There are also a useful guide which sets out the indicators for each stage

The framework establishes how value is being created very early on in the process, not just at the end, with realized or transformative value. I can see the framework as being very helpful both in providing guideposts for establishing a Theory of Change for social learning and communities of practice, as well as for exploring and making a case for value to funders and others. Frameworks provide something to “hang” findings on and help provide legitimacy, relevance and cohesion to evaluative claims.

The framework helps articulate the value that you hope to create both for participants, their organizations, and the world beyond.

Some of the key evaluation questions in the contexts where we work (social justice initiatives, women’s rights, movement building, etc. might include:

• How do you hope people will benefit from participation in Communities of Practice/social learning activities? What shifts in knowledge, capacity, relationships, understanding do you hope to see? How will people’s framing and analysis of the issues shift? How is their perspective of systems-change deepened and refined?

• How will organizations benefit from their staff’s participation in Communities of Practice? (Knowledge, capacity, stronger/more effective strategies, engagement of new actors, new perspectives?)

• How will communities/the world benefit from the existence of Communities of Practice? (For example, will programs and project better reflect community needs and visions?) The framework discusses  value in terms of “strategic conversations with stakeholders”, and we would push this concept a bit further. In our experience the engagement of new actors and bringing to the mix new voices and  perspectives (particularly those most marginalized) is where we can really disrupt the system and make space for something new to emerge!   As such, we might specifically look at questions such as, “Will there be new connections made that will better be able to transform the existing power structures?”)

Tools/Approaches for evaluative social learning activities:

• Outcome Mapping
• Outcome Harvesting 
• Most Significant Change and  Most Significant Change Lite
• Surveys with closed and open questions (both to capture progress towards expected results as well as unexpected changes)
• Participatory method to look at how people’s lives, organization, community might have changed because of CoP (Before/after). Some examples are here 
• Use of Dedoose. Preload key expected indicator codes from the CoP Value Creation Framework adding additional codes for unexpected changes.Analysis allows for assessment of progress according to thematic codes (indicators) as well as looking for relationships between codes (such as between types of CoP activities and value created) as well as patterns according to according to codes, respondent types. The program allows for useful visualizations.