Library

The National Roundtable on Evaluation of Multilevel/Combination HIV Prevention Interventions had the goals of examining the present state of the art of multilevel and combination HIV prevention interventions, both domestically and internationally; to define the significant challenges and scientific gaps in current evaluation methods and identify the most promising methodological approaches to address these gaps; and to guide the future agenda for HIV prevention research. To address these methodological gaps, we must combine the methodological and statistical rigor associated with clinical trials, the conceptual framework of implementation science, the on-the-ground strategies of programmatic monitoring and evaluation, and the strengths of pre- and post-intervention mathematical modeling. In looking at the HIV epidemic in the US, the group discussed current initiatives guided by the National HIV/AIDS Strategy and the increased optimism over treatment as prevention. We noted that considerable progress has been made in developing core metrics to evaluate outcomes along the “treatment-as-prevention cascade” that could be captured through public health surveillance—number of new HIV cases detected and proportion linked to care, retained in care, on active treatment, and virally suppressed. Our understanding of the optimum package of interventions with regard to both effectiveness and efficiency remains incomplete. Progress was reported, however, in the collection of process data at the local level to better assess how to improve programs. Devising epidemic impact measures to quantify reductions in HIV incidence attributable to combination interventions remains challenging, mostly due to barriers to testing impact through methods like community cluster randomization in the US. Looking globally, the group discussed a number of planned clinical trials of combination interventions also spurred by optimism over treatment as prevention. Common elements of combination approaches included expansion of voluntary counseling and testing, adult male circumcision, prevention of mother-to-child transmission, and management of sexually transmitted infections, along with expanding ART treatment. Outcome measures were generally framed in terms of the treatment-as-prevention cascade, though these data are not available from current surveillance systems, pointing to the need to create improved systems of data collection. The most common approach to measuring epidemic impact was clustered community randomization, with incident infections measured through cohorts or newer cross-sectional, multi-assay algorithms. Recommendations from the roundtable include the following:

• A new coalition of interventionists, implementation scientists, public health program and surveillance specialists, mathematical modelers, and behavioral scientists is needed to adequately address the evaluation of multilevel/combination HIV interventions at the community-level.
• The use of the conceptual frameworks of the HIV prevention continuum and engagement-in-HIV-care cascade should be used in structuring evaluation of combination HIV interventions.
• Common public health surveillance systems to evaluate combination HIV prevention interventions at the community level are recommended, and this capacity should be further developed internationally.
• Mathematical modeling before, during, and after multilevel/combination HIV interventions should be incorporated in the design, implementation, and interpretation of intervention results.
• Because an emphasis on efficiency as well as effectiveness from implementation science is helpful, costing and cost-effectiveness evaluations of combination HIV prevention interventions are recommended and are important to policy makers.
• Use of innovative trial and observational study designs outside of the traditional randomized, controlled trial paradigm should be used to account for the complex multilevel and combination nature of new HIV prevention interventions, and emerging design and analysis methods (e.g., stepped-wedge designs, adaptive trial designs, causal inference modeling of “natural experiments”) should be considered to address the challenges of community-level effectiveness evaluation.
• Because social factors and human behaviors are integral factors all along the HIV care and treatment cascade, it is crucial to include social and behavioral science in the design, implementation, and evaluation of combination interventions (e.g., community engagement and mobilization interventions).
• Mixed methods, including qualitative data collection (e.g., key informant interviews with implementers, in-depth interviews with target population members), are recommended to increase our understanding of how and why interventions are successful or not.
• Increased funding opportunities for methods development, whether as standalone projects or as supplements to large trials, is recommended as is funding for career development in methods research (e.g., methods-focused K awards).

There have been many reactions to the release of preliminary data suggesting a significant increase in HIV incidence in San Francisco. Many are alarmed, some are saddened, and more than a few have a great deal of anger. The one reaction nobody has had is surprise. While a few don’t like the numbers and will argue about the analysis of them, there has been little shock that there appears to be a range of 750 – 900 new infections this year. That the majority of these infections are occurring in men who have sex with men (MSM) is also not surprising to most observers of or members of the San Francisco gay community. The increase in numbers is based on several realities: The AIDS epidemic changed forever when new treatments were unveiled in Vancouver in 1996. The perception of AIDS as a death sentence, already suspect among many gay men, is gone. If people see HIV as a death sentence, they make certain choices about risk. We know that when the perception of HIV becomes one of a chronic, manageable illness, people make different choices. The perception of HIV has changed on the streets and in the minds of MSM. Most HIV prevention efforts have not caught up with that change. Prevention and health education efforts, which rely on death or danger-avoidance for motivation, may no longer resonate with gay men. Studies indicate that high risk sexual behavior is increasing among MSM in San Francisco. Another factor contributing to the higher estimate of HIV incidence is that the population of gay men in San Francisco is larger than was previously thought. Because of recently improved data collection, such as survey work done for the Gay Men’s Health Study and some groundbreaking work with homeless gay men in San Francisco, we now know that gay men comprise between 15-20% of the adult male population of San Francisco. This is a significant increase from the data available in 1997. In San Francisco, one can look at gay men’s communities as being divided into three distinct groups, all of whom need HIV prevention. The first group is those who have eliminated high-risk behavior from their lives. These people have a thorough knowledge of HIV transmission and risk reduction techniques. They have chosen to eliminate risk based on personal decisions about their risk and need. Despite many years of risk reduction education and media, a second group of people have engaged in high-risk behavior throughout the epidemic. This has not changed. For these men, decisions about perceived risk are outweighed by their needs for identity, intimacy, pleasure, or other considerations. They know about risk, have made choices, and engage in behavior at the level of risk that they believe to be appropriate. A third group of MSM has increased their risk behavior recently. This group makes situational decisions about risk behavior. These decisions are based on their knowledge 3 and understanding of HIV transmission, the perceived risk of the behavior in question, and the stated or presumed HIV status of their partner. For nearly 20 years, we have conducted HIV prevention and education based on several theories, including cognitive learning theory. However, there are many issues that mediate people’s risk assessment and behavior. The behaviors in questions are basic, often urgent, and engaged in without complex cognitive analysis. A brochure can be informative on Tuesday morning; in a moment of intense passion on Friday night, a different analysis occurs. Substance use before and during sex is also a factor. Additionally, for some individuals chronic loneliness, isolation, and alienation lead to remedies, which may include highrisk behavior. The pursuit of pleasure in a society where discrimination can make life painful can be another motivation for pre-cognitive, high-risk behavior. There are many complicated and compelling reasons why a person chooses his own acceptable level of risk. In the era of HAART (highly active anti-retroviral therapy), many men make an assessment based on their firm belief that living with HIV is an acceptable option. Others make a determination about the potential for transmission from a given partner based on information (or perception) about his HIV status or viral load. The fact is, there is an increase in risk behavior amongst MSM in San Francisco. The AIDS epidemic has entered a new and complex era. It is time for prevention efforts to be based on these new realities. In the next section, we will outline an 11-point Action Plan to revitalize the HIV prevention programs of the City and County of San Francisco. No increase in HIV infections is acceptable, regardless of the reason. This new plan will allow us to break this epidemic by realistically equipping MSM to take control of their health.

Because of threats to government funding, community based organizations that provide HIV prevention programs diversify their sources of income by seeking non-governmental funding, including funding from foundations. This survey of foundations across the United States asked questions about the amount of money they spend on AIDS and HIV prevention, the types of prevention projects and populations their grants served, and the sources of AIDS information used by the foundation staff. The results were then compared to the types of projects known to be effective and to the populations most at risk for HIV infection. These organizations funded a diverse range of HIV prevention programs, with a total of 583 grants worth over $8.1 million. This represents almost a quarter of all AIDS-related funding by foundations in the United States.

• HIV prevention grants represented 52% of all AIDS-related grants
• The average HIV prevention grant was for almost $14,000, ranging from $750 to $1.4 million.
• Public policy, capacity building, outreach and technical assistance were among the most frequently funded HIV prevention programs.
• Some useful HIV prevention programs, such as condom distribution and needle exchange, were rarely funded. • Women and youth received 25% of the funding for HIV prevention.
• More grants targeted the general population than drug users, gay men or ethnic minorities, although these groups contain more people at risk for HIV infection.
• Print media, site visits and colleagues were the main sources of AIDS related information for funders.
• Better sources of information are available to help funders gauge the effectiveness of proposed programs.
• Several factors (such as using culturally relevant language, providing creative rewards for participants, designing flexible programs and creating a forum for open discussion) that make HIV prevention programs more effective are provided for use when evaluating programs.