The W3 Project, Stage 3 (2020-2023)

Stage 3 of the W3 Project was a national study of the impact of Australian peer-led HIV and hepatitis C programs.


In this stage, we pooled resources and data from selected peer-led responses in multiple states across Australia. The aim was to build an evidence bank that showed how peer-led responses were achieving and sustaining impact in their work across the four W3 Functions. This has helped us generate stronger and clearer evidence of what works, and why, and provide insights and guidance for supporting peer-led organisations to achieve and demonstrate their full impact and value.

The image shows the original banner from the W3 Project website. The background is a blurred image of a crowded streetscape. W3 Project is written in white letters in a black box. Underneath is a red line. Under the red line, it says, “Understanding what works and why in peer-based and peer-led programs in HIV and hepatitis C”.

Background to Stage 3 of the W3 Project

Australia’s National HIV and Hepatitis C Strategies have consistently highlighted the importance of community involvement, particularly through peer-led approaches. These initiatives, orchestrated and executed by individuals most affected by HIV and Hepatitis C, are deeply rooted within affected communities and play a pivotal role in addressing these health concerns.

Drawing from their strong influence within communities and on overarching health systems, these peer-led endeavours encompass a wide spectrum of activities. These range from peer service delivery, like HIV rapid testing and needle and syringe programs (NSPs), to peer health promotion and leadership roles in policy reform. The effectiveness of these initiatives lies in their ability to resonate with and make a significant impact within communities, and their ability to influence both health systems and policies.

However, a challenge arises when evaluating the full extent of the impact of peer-led approaches. Traditional evaluation mechanisms tend to focus on individual-level factors, often overlooking broader system-level impacts. This makes it challenging for peer-led responses to demonstrate the full extent and value of their contributions.

The third stage of the What Works and Why (W3) Project attempted to address these challenges. We worked in close collaboration with six peer-led organisations from across four Australian states. These organisations were led by people living with HIV (PLHIV) and people who use drugs (PWUD).

W3 Project Stage 3 purpose

Building on the work of the first and second stages, this third stage set out to create a consolidated evidence base that captured the scope, quality, and overall impact of peer-led work within the public health response to HIV and Hepatitis C. By pooling together evidence about our partner organisations, the project sought to pave the way for more informed funding, oversight, and evaluations of future peer-led work.

Overview of methods

The impacts of COVID-19

The onset of the COVID-19 pandemic required us to adapt our research methods substantially. These adaptations were primarily driven by the practical challenges of the pandemic and the priorities of our partner organisations.

Extended lockdowns and travel restrictions throughout 2020 and 2021 significantly curtailed our ability to undertake planned activities, such as in-person workshops.

Furthermore, our partner organisations were at the front lines of the COVID-19 response, concentrating their efforts on protecting and assisting their communities. This inevitably impacted their capacity to fully engage in all research activities.

In light of these circumstances, we made the following key shifts:

  • Changed methods: All activities from 2020 to 2021 were transitioned online. To support partner organisations, we redirected our methodology to use existing evidence, reducing the need for them to produce and collate new data.
  • Adjusted study period: Initially, our research covered the 2020–2021 timeframe. As the pandemic persisted, we recognised that this window would not help us understand the typical work of these organisations. We therefore expanded our study period to encompass 2018–2022. This allowed us to incorporate insights from both pre-pandemic conditions and the evolving response and recovery phases.
  • Refined study scope: Our initial plan encompassed a wider range of peer work to include the impact of peer-led programs overall as well as specific types of peer-led programs. However, given the above challenges, and the extended study period, we narrowed our focus to the role, impact, and value of peer-led organisations.

W3 Framework

We used the W3 Framework as the foundational theory for our project. Using the W3 Functions helped us develop evaluation indicators that could capture the full system-level impact of peer-led work.

LEAD Framework

To guide our work, we used the Locate evidence, Evaluate evidence, Assemble evidence, inform Decisions (LEAD) framework. This choice was driven by the framework’s ability to guide the collection of diverse and meaningful evidence. It’s especially adept at addressing complex public health problems by adopting a systems-thinking approach.

Diagram showing the LEAD Framework as a flowchart surrounded by 2 phrases. The first phrase is 'systems perspective'. The second phrase is 'opportunities to generate evidence'. The flowchart shows 5 steps. 1: Specify questions. 2: Locate evidence (identify and gather the types of evidence that are potentially relevant to the W3 Indicators). 3: Evaluate evidence (apply standards of quality as relevant to different types of evidence). 4: Assemble evidence (select and summarise the relevant evidence according to consideration for its use). 5: Inform decisions (use evidence to develop recommendations and guidance to support decision-making and future evaluation).
Diagram adapted from Figure 3-1 ‘The Locate Evidence, Evaluate Evidence, Assemble Evidence, Inform Decisions (L.E.A.D.) Framework for obesity prevention decision making’ in ‘Bridging the evidence gap in obesity prevention: a framework to inform decision making’, by the National Academy of Sciences, 2010, Washington, DC: The National Academies Press.

Although the LEAD framework appears linear, in practice, it was a dynamic and ongoing process. This is because we looked for evidence in diverse sources and contexts. As we uncovered, assessed, and gathered new evidence, we continuously revisited and improved previous steps. This was a continuous learning process that allowed us to improve our work over time.

Specifying questions

We developed evaluation indicators based on the W3 Functions through a three-round consultation process using a modified Delphi method. Input was sought from representatives of peer-led organisations and government health departments. We described this work in our report ‘W3 Project: Creating a set of evaluation indicators for peer-led work’. Indicators were further refined throughout the subsequent stages of the LEAD framework.

Locating evidence

The process began with preliminary identification and brainstorming sessions with partner organisations. Desktop research and targeted meetings with partner organisations assisted in further pinpointing data sources. Additional meetings with partner organisation staff provided a more comprehensive understanding of their work and helped to bridge evidence gaps.

Evaluating evidence

As we located evidence, we assessed it for its reliability. This included verifying source credibility and trustworthiness, ensuring accuracy and validity. We also looked at the strength and usefulness of the evidence in demonstrating each indicator, and in demonstrating work within each W3 Function.

Assembling evidence

We synthesised the evidence using the framework analysis method. This involved getting to know the data, organising it against the evaluation indicators, and identifying patterns and trends.

Informing Decisions

This step involved interpreting the assembled data to understand the connections and the overarching story it told about the work of our partner organisations. We identified barriers and enablers to demonstrating the system-level impact of investing in peer work. We also developed recommendations to help leverage these enablers and mitigate against the barriers.

Achievements

Tailored evaluation indicators for peer-led work

We developed two sets of evaluation indicators:

  • Organisation-level indicators for use by peer-led organisations
  • Program-level indicators for use by peer-led programs in either peer-led or mainstream organisations

The lists include broad indicators that measure quality/process and impact across all the W3 Functions. They also include examples of ways peer workers might be able to go about demonstrating each indicator.

As we began to locate and sort evidence against the organisation-level indicators, we made further changes. The final list was more concise with fewer gaps and less overlap. Due to the impacts of the COVID-19 pandemic, however, we were unable to trial the program-level indicators.

Consolidated evidence base for the system-level impact of peer-led work

In total, we collated over 1,100 pieces of evidence. This evidence demonstrated that peer-led organisations had a clear commitment to long-term and sustained community engagement. They invested significantly in cultivating relationships and partnerships across the health and other sectors. They also demonstrated the outcomes of drawing on community insights and sector relationships to constantly adapt and refine their work. Furthermore, they brought influential peer expertise and insights to policy and strategy discourse.

Overall, we found that investing in peer organisations can exert system-level influence, not only within communities but also within policy and health systems, thereby strengthening the overall public health response to HIV and hepatitis C.

Publications

Project reports

2021 community report: Creating a set of evaluation indicators for peer-led work

2022 community report: Locating evidence against the W3 indicators for peer-led work

2023 community report: The impact of peer-led work in Australia’s HIV and hepatitis C response

(Executive summary and appendix with full reference list also available.)

Conference presentations and posters

Partners

PLHIV-led organisations
PWUD-led organisations

Funding

Stage 3 of the W3 Project receives funding from the Australian Government Department of Health


Our other research projects


This post was first published on 10 June 2021. We have edited the post to keep it up to date.

Leave a Comment

Your email address will not be published. Required fields are marked *