Stage 3 of the W3 Project was a national study of the impact of Australian peer-led HIV and hepatitis C programs.
In this stage, we pooled resources and data from selected peer-led responses in multiple states across Australia. The aim was to build an evidence bank that showed how peer-led responses were achieving and sustaining impact in their work across the four W3 Functions. This has helped us generate stronger and clearer evidence of what works, and why, and provide insights and guidance for supporting peer-led organisations to achieve and demonstrate their full impact and value.
- Background to Stage 3 of the W3 Project
- W3 Project Stage 3 purpose
- Overview of methods
Background to Stage 3 of the W3 Project
Australia’s National HIV and Hepatitis C Strategies have consistently highlighted the importance of community involvement, particularly through peer-led approaches. These initiatives, orchestrated and executed by individuals most affected by HIV and Hepatitis C, are deeply rooted within affected communities and play a pivotal role in addressing these health concerns.
Drawing from their strong influence within communities and on overarching health systems, these peer-led endeavours encompass a wide spectrum of activities. These range from peer service delivery, like HIV rapid testing and needle and syringe programs (NSPs), to peer health promotion and leadership roles in policy reform. The effectiveness of these initiatives lies in their ability to resonate with and make a significant impact within communities, and their ability to influence both health systems and policies.
However, a challenge arises when evaluating the full extent of the impact of peer-led approaches. Traditional evaluation mechanisms tend to focus on individual-level factors, often overlooking broader system-level impacts. This makes it challenging for peer-led responses to demonstrate the full extent and value of their contributions.
The third stage of the What Works and Why (W3) Project attempted to address these challenges. We worked in close collaboration with six peer-led organisations from across four Australian states. These organisations were led by people living with HIV (PLHIV) and people who use drugs (PWUD).
W3 Project Stage 3 purpose
Building on the work of the first and second stages, this third stage set out to create a consolidated evidence base that captured the scope, quality, and overall impact of peer-led work within the public health response to HIV and Hepatitis C. By pooling together evidence about our partner organisations, the project sought to pave the way for more informed funding, oversight, and evaluations of future peer-led work.
Overview of methods
The impacts of COVID-19
The onset of the COVID-19 pandemic required us to adapt our research methods substantially. These adaptations were primarily driven by the practical challenges of the pandemic and the priorities of our partner organisations.
Extended lockdowns and travel restrictions throughout 2020 and 2021 significantly curtailed our ability to undertake planned activities, such as in-person workshops.
Furthermore, our partner organisations were at the front lines of the COVID-19 response, concentrating their efforts on protecting and assisting their communities. This inevitably impacted their capacity to fully engage in all research activities.
In light of these circumstances, we made the following key shifts:
- Changed methods: All activities from 2020 to 2021 were transitioned online. To support partner organisations, we redirected our methodology to use existing evidence, reducing the need for them to produce and collate new data.
- Adjusted study period: Initially, our research covered the 2020–2021 timeframe. As the pandemic persisted, we recognised that this window would not help us understand the typical work of these organisations. We therefore expanded our study period to encompass 2018–2022. This allowed us to incorporate insights from both pre-pandemic conditions and the evolving response and recovery phases.
- Refined study scope: Our initial plan encompassed a wider range of peer work to include the impact of peer-led programs overall as well as specific types of peer-led programs. However, given the above challenges, and the extended study period, we narrowed our focus to the role, impact, and value of peer-led organisations.
To guide our work, we used the Locate evidence, Evaluate evidence, Assemble evidence, inform Decisions (LEAD) framework. This choice was driven by the framework’s ability to guide the collection of diverse and meaningful evidence. It’s especially adept at addressing complex public health problems by adopting a systems-thinking approach.
Although the LEAD framework appears linear, in practice, it was a dynamic and ongoing process. This is because we looked for evidence in diverse sources and contexts. As we uncovered, assessed, and gathered new evidence, we continuously revisited and improved previous steps. This was a continuous learning process that allowed us to improve our work over time.
We developed evaluation indicators based on the W3 Functions through a three-round consultation process using a modified Delphi method. Input was sought from representatives of peer-led organisations and government health departments. We described this work in our report ‘W3 Project: Creating a set of evaluation indicators for peer-led work’. Indicators were further refined throughout the subsequent stages of the LEAD framework.
The process began with preliminary identification and brainstorming sessions with partner organisations. Desktop research and targeted meetings with partner organisations assisted in further pinpointing data sources. Additional meetings with partner organisation staff provided a more comprehensive understanding of their work and helped to bridge evidence gaps.
As we located evidence, we assessed it for its reliability. This included verifying source credibility and trustworthiness, ensuring accuracy and validity. We also looked at the strength and usefulness of the evidence in demonstrating each indicator, and in demonstrating work within each W3 Function.
We synthesised the evidence using the framework analysis method. This involved getting to know the data, organising it against the evaluation indicators, and identifying patterns and trends.
This step involved interpreting the assembled data to understand the connections and the overarching story it told about the work of our partner organisations. We identified barriers and enablers to demonstrating the system-level impact of investing in peer work. We also developed recommendations to help leverage these enablers and mitigate against the barriers.
Tailored evaluation indicators for peer-led work
We developed two sets of evaluation indicators:
- Organisation-level indicators for use by peer-led organisations
- Program-level indicators for use by peer-led programs in either peer-led or mainstream organisations
The lists include broad indicators that measure quality/process and impact across all the W3 Functions. They also include examples of ways peer workers might be able to go about demonstrating each indicator.
As we began to locate and sort evidence against the organisation-level indicators, we made further changes. The final list was more concise with fewer gaps and less overlap. Due to the impacts of the COVID-19 pandemic, however, we were unable to trial the program-level indicators.
Consolidated evidence base for the system-level impact of peer-led work
In total, we collated over 1,100 pieces of evidence. This evidence demonstrated that peer-led organisations had a clear commitment to long-term and sustained community engagement. They invested significantly in cultivating relationships and partnerships across the health and other sectors. They also demonstrated the outcomes of drawing on community insights and sector relationships to constantly adapt and refine their work. Furthermore, they brought influential peer expertise and insights to policy and strategy discourse.
Overall, we found that investing in peer organisations can exert system-level influence, not only within communities but also within policy and health systems, thereby strengthening the overall public health response to HIV and hepatitis C.
2021 community report: Creating a set of evaluation indicators for peer-led work
2022 community report: Locating evidence against the W3 indicators for peer-led work
2023 community report: The impact of peer-led work in Australia’s HIV and hepatitis C response
Conference presentations and posters
- Structural and system impact of peer work: using the W3 Framework to improve and demonstrate the full value of peer-led work – presentation at the 2023 International HIV Coinfection + Viral Hepatitis Elimination Conference
- Using the W3 Framework to improve and demonstrate the full value of peer-led work with people who use drugs – presentation at the 2023 Harm Reduction International Conference
- Using the W3 Framework to improve and demonstrate the full value of peer-led work with people who use drugs – workshop at the 2023 Harm Reduction International Conference
- Proving the power of peers: Creating a practical toolkit to enable peer/community-led organisations to evaluate their work and better understand, demonstrate, and improve their impact – Poster at the 2022 Joint Australasian HIV&AIDS + Sexual Health Conferences
- Developing practical evaluation indicators for programs led by people living with HIV and people who use drugs – poster at the 2021 Joint Australasian HIV & AIDS and Sexual Health Conference
Stage 3 of the W3 Project receives funding from the Australian Government Department of Health
Our other research projects
This post was first published on 10 June 2021. We have edited the post to keep it up to date.