Stage 1 of the W3 Project was a ground-breaking study that applied a systems-thinking approach to help better understand the role of peer work in a public health response.
During this stage, we partnered with ten peer organisations and programs working in the HIV and hepatitis C sector. Together with peer workers, we developed a program theory to describe and evaluate peer work: the W3 Framework.
This post outlines how we did this and what we learned.
Background to the W3 Project
The aim of the W3 Project — also known as the ‘What Works and Why (W3) Project’ — was to improve our understanding of the peer response to HIV and hepatitis C.
In our work before the W3 Project, we found that most research in the area:
… continues to look at one intervention at a time in isolation from its interactions with other interventions, the community, and the socio-political context of their implementation. To understand and evaluate the role of a combination of interventions, we need to understand not only what works, but in what circumstances, what role the parts play in their relationship with each other.Brown et al (2015)
Affected communities constantly adapt to new opportunities (and barriers) to prevent transmission of HIV and hepatitis C. As a result, peer-led programs must constantly adapt to these changes in tandem with their communities. What’s more, they operate simultaneously at multiple levels: individual, network, community, and structural. However, the contracting and evaluation of programs often struggle with this complexity.
The W3 project wanted to really shift the capacity of community-based and peer-led programs to capture this complexity in their evaluations. In order to do this, we needed a better way to describe what the program is (or should be) achieving. To that end, we needed a better program theory to guide our evaluation and investment decisions.
Stage 1 had three objectives.
- Develop a better understanding of how peer-led programs work
- Establish a framework to evaluate the role and contribution of peer work
- Develop methods to best capture and share insights from practice
Methods and results
Systems thinking and a participatory approach
When using a systems-thinking approach, the aim is to simultaneously examine:
- The big picture
- The individual pieces that make up the picture
- The complexity of non-linear relationships, influences, and emergent effects
In order to do this, we held a series of 18 workshops with our partners. The workshops ranged from one to two days each. Some were with single organisations and some with up to four organisations. All in all, more than 90 people were involved.
We drew on the experience and expertise within peer programs to create systems maps of their work. We created four systems maps together with peer workers from programs led by:
- Gay men and other men who have sex with men
- People who use drugs
- Sex workers
- People living with HIV
These maps showed the complex flows of knowledge and influence that underpin the effectiveness of peer programs within their communities and the health sector and policy environment. (You can see three maps in the resource list below).
Developing a framework for understanding peer work
Looking at the systems maps, we realised that — despite the differences in the peers, their goals, and the specifics of their work — there was notable similarity across them:
- They are all integral parts of their communities
- They are all active within the health system and policy environment
- In order to remain relevant, they constantly adapt their work to reflect changes within their environment
- This ‘environment’ includes their communities as well as the health system and policy environment
- Their work impacts not only their communities but also the health sector and policy environment
- Above all, peer skill is key to their effectiveness
This realisation then led us to identify four key functions that peer-led programs fulfil. In time, these functions became the basis of the W3 Framework.
Tailoring evaluation indicators and developing tools
After developing the W3 Framework, we wanted to know how it could help community-based and peer-led organisations in practice. In particular, we wanted to know what would be feasible within their (often limited) available resources.
We worked with nine of the W3 project partners to develop tailored indicators under each of the four functions. We also worked with peer-led projects across seven organisations to pilot a range of different tools for gathering insights against the indicators.
Although these trials were small in scope, we did gain some valuable insights:
- The strength of the W3 Framework is in its potential to help build and integrate knowledge across programs, organisations, and the sector as a whole
- The W3 Framework is useful as a sophisticated program theory for planning and evaluating peer work
- Peer insights from peer work are valuable assets and sources of real-time knowledge and information
- Applying the W3 Framework requires a cultural shift towards prioritising and valuing peer insights
- At least in the short term, if tools are to be sustainable, they need to align closely to current practices and opportunities
If you’re keen to know more about Stage 1 of the W3 Project, check out the following links.
W3 Project Stage 1 reports
This report includes:
- A one-page executive summary
- An overview of the project and the W3 Framework
- A summary of the feasibility trials of the indicators and tools
The appendices include:
- A detailed methods section
- A detailed report on the feasibility trials
- A listing of the W3 presentations (national and international)
- Copies of selected tools
System diagrams and draft indicators
These documents provide detailed descriptions of the system logic diagram and draft indicators developed with the W3 Project partners. (They should be read together with the above reports.)
- PWUD peer service provision and policy participation system logic and draft indicators
- PLHIV peer leadership and policy participation system logic and draft indicators
- GMSM peer network targeted health promotion system logic and draft indicators
W3 Framework toolkit guide for the feasibility trials
This document was used to support the development of tools during the feasibility trials.
Selected journal articles
- A systems thinking approach to understanding and demonstrating the role of peer-led programs and leadership in the response to HIV and hepatitis C: Findings from the W3 Project
- The power of peers: W3 Framework for evaluating the quality and influence of peer-led programs
- Characterising the policy influence of peer-based drug user organisations in the context of hepatitis C elimination Full article (subscription required) | Pre-press version (free access)
- Using systems methods to elicit complex program theories Full article (subscription required) | Pre-press version (free access)
Poster series: Findings from the What Works and Why Project
Stage 1 of the W3 Project received funding from the Australian Government Department of Health.