How+to+approach+assessing+the+impact+of+networks?


 * // How to approach assessing the impact of networks? What’s challenging? What needs to be assessed? What principles should inform network impact assessment? //**

While the number of funders investing in and experimenting with networks and network approaches is growning, there is limited //evidence// to make the case that networks work. Many of the funders pioneering network centric grantmaking have an intuitive sense that network benefits – connectivity, trust, reciprocity – are critical to most social change endeavors. However, the science behind making this case is in its early days. This is mostly because it’s a really hard nut to crack.

Assessing the impact of networks is hard for a number of reasons. First of all, different participants will often have different reasons for participating in a network (and similarly, funders fund networks for multiple reasons). This makes it hard to align around and clarify desired outcomes – and/or figure out how to assess progress towards multiple sets of outcomes. Second network impact tends to be hard or impossible to see in its entirety. Networks are decentralized, complex, and constantly changing systems often with lots of players. This makes it difficult to measure causality or even simply track activities that may be resulting in meaningful impact. And, some of the most powerful network impact may be unexpected. How do you measure the impact of emergent, self-organized action? Finally, it can take a really long time to achieve measurable impact.

Added to these challenges, there is a lack of capacity to do the work. Networks themselves, which are mostly driven by volunteer participation, have a little capacity for a reflective practice of impact assessment. The tools for assessing emergent systems are lacking. (Social network analysis does seem to be promising in some cases, though it can be resource-intensive.) And, despite the unique nature of networks, many funders and other stakeholders expect measurements according to conventional criteria.

Challenges aside, what needs to assessed? Consider evaluating impact at three different levels: connectivity, overall network health, and field-level outcomes. [i]


 * Connectivity: what is the nature of relationships within the network? Is everyone connected who needs to be? What is the quality of these connections? Does the network effectively bridge differences? Is the network becoming more interconnected? What is the network’s reach?
 * Overall network health: how healthy is the network along multiple dimensions (participation, network form, leadership, capacity, etc.). How have participants been impacted by the network?
 * Field level outcomes: what progress is the network making on achieving its intended social impact (e.g. policy outcomes, innovative products)? How do you know?

As for methodology, there is no silver bullet and, as mentioned, the tools are lacking. However, there are a few guiding principles to keep in mind when assessing the impact of networks: [Note: add more on the importance of comparative data: pre/post network and benchmarking across networks More on network effects]
 * Be clear at the outset about: the network’s value proposition(s); why people are taking part; who the network is accountable to; and the donor’s role in the network.
 * Ask good questions from the outset. Be clear about what will focus your analysis and interpretation of the data. For example, don't dive into an exploration of network metrics like density or connectedness without first thinking about whether density is good, bad or irrelevant for your network.
 * Gather data from diverse perspectives inside and outside the network.
 * Emphasize learning over near-term judgment, given long time horizon for many networks.
 * Focus on meaningful contribution toward impact, rather than attribution.
 * Use network evaluation as part of an ongoing dynamic process of network learning and adaptation; build capacity to conduct self-evaluation. This is important for both effective assessment and overall network health.
 * Share learning from the evaluation broadly—within the network and beyond.

[i] Madeline Taylor and Pete Plastrik in [|//Net Gains//], and Bruce Hoppe and Claire Reinelt in “[|Social Network Analysis and the Evaluation of Leadership Networks]”