Context: Key populations (KP) in Togo, including men who have sex with men and female sex workers, experience disproportionately high rates of HIV prevalence. In response to this situation, USAID/West Africa and FHI360 established the PACTE–VIH regional HIV/AIDS project, which utilized mobile health (mHealth) to send reminders to enrolled participants on ways to reduce the risk of HIV infection. While mHealth programs are known to be effective in improving HIV–related health behavior outcomes, the role of length of time enrolled in a mHealth program is not well understood.

Methods: To assess whether there was a relationship between time enrolled in the PACTE–VIH mHealth program and adoption of HIV prevention behavior, we use data from an operations research study conducted in Lomé, Togo, in 2016 with 1005 KP and employ logistic regression analysis.

Results: The odds of adopting prevention behaviors were higher with each additional month enrolled in the program (OR: 1.62; 95% CI 1.19–2.20). Fully adjusted (age and marital status) logistic regression models demonstrated no significant difference in odds of adopting prevention behavior between enrollment times of less than six months and any category greater than six months.

Conclusion: Time does matter for adoption of HIV prevention behaviors. However, more studies are needed to better understand what length of time enrolled in a program truly affects behavioral outcomes.

Keywords: regression analysis, adoption, populations, virally, approaches, cascade, environmental

ART, anti–retroviral therapy; KP, key populations; USAID/WA, U.S. Agency for International Development/West Africa Mission; HIV, human immunodeficiency virus; PTC, prevention, treatment and care; FSW, female sex workers

Despite progress in controlling the spread of the human immunodeficiency virus (HIV) in sub–Saharan Africa, West Africa still experiences high rates of HIV infection among certain vulnerable subpopulations.¹ Togo has a mixed HIV epidemic, with a countrywide HIV prevalence of 2.5% and higher prevalence estimates of 13.0% among men who have sex with men (MSM) and 11.7% among female sex workers (FSW).² Improved HIV prevention, treatment and care (PTC) of these key populations (KP) is essential to achieving the 2020 UNAIDS 90/90/90 goal: 90% of HIV positive individuals aware of their status, 90% of those positive being on sustained anti–retroviral therapy (ART), and 90% of those in care being virally suppressed by the year 2020.³ Accomplishing these goals will depend on engaging KP at each step along the PTC cascade.

Various short–message service (SMS) and social media campaigns have been implemented in low– and middle–income settings in an effort to connect with individuals living with or at–risk for HIV infection.⁴ These interventions include using mobile health (mHealth) approaches, including SMS or text messaging and online channels such as social media, to link at–risk populations with HIV PTC services.⁵ In the context of HIV treatment, mHealth programs have demonstrated success in improving health outcomes by promoting medication adherence.⁶ Evidence suggests that mHealth programs can improve viral suppression with SMS reminders to take antiretroviral therapy (ART) and attend scheduled appointments.⁷ This evidence indicates high feasibility and potential for adoption of prevention and care actions as a result of mHealth components of HIV programs.^8,9 

A systematic review by Gurman et al. highlights the dearth of evaluation research in mHealth, specifically in the context of developing countries.⁴ Many of these previous mHealth interventions have prioritized HIV treatment, and few have evaluated outcomes relating to prevention behavior, including HIV testing and consistent condom use.¹⁰ Furthermore, the mechanisms behind how mHealth programs might work have not been thoroughly explored.¹¹ Prior HIV mHealth studies have suggested investigating longer–term interventions, since short–term ones may not yield definitive results.¹² One mHealth study on alcohol use examined how receiving more SMS reminders over time led to reduced reports of alcohol consumption.¹³ A cluster randomized trial using a six–month SMS intervention to reduce HIV risk behaviors in South Africa, Zimbabwe and Mozambique showed no effect on sexual risk taking behaviors, but a positive effect on HIV testing.¹⁴ A recent meta–analysis on the effect of text message reminders for HIV–related compliance concluded that text message reminders can be an important strategy to increase HIV care compliance; text message reminders significantly reduced rates of non–attendance, increased drug adherence and significantly improved CD4 count and viral load among those in treatment.¹⁵ While the evidence suggests that text messages have an effect on behavior, the length of time exposed to text messages was not reported. To date, there are no studies examining time exposure to HIV–related SMS interventions and related HIV behavioral outcomes.

In this study, we examined the association between the duration of enrollment in an SMS intervention with the adoption of HIV prevention behavior. We question whether there was a dose–response between duration of time enrolled in the program and adoption of prevention behaviors. Understanding the potential effects of enrollment duration on HIV prevention behavior outcomes can help programs make effective use of their resources and tailor their interventions to be more efficient. Results from this study can provide insights into the role of various components of HIV mHealth programs and explore which components may lead to desirable behavioral outcomes.

The Regional HIV/AIDS Prevention and Care project, known in West Africa as PACTE–VIH, was a five year project implemented by FHI360 between 2012 and 2017 and funded by U.S. Agency for International Development/West Africa Mission (USAID/WA). The program introduced information and communications technology (ICT) targeting KP: mHealth and social media approaches were employed to achieve a greater reach through online and network–based forums.^5,10 The SMS program served as an alert system to effect behavior change and improve adherence to ART through the following: reminding KP living with HIV to take ARVs as prescribed; reminding KP to consistently use condoms and lubricants during sexual intercourse; inviting KP to utilize PTC services and seek early treatment for STIs; and reminding KP to check their HIV status every 3months. The overall goal of the PACTE–VIH project was to optimize high quality, comprehensive HIV prevention and support services for KP, as well as to provide data on the effectiveness of these types of behavior change activities.

To guide our study we utilized a framework (Figure 1) to depict how enrollment in SMS interventions could lead to adoption of prevention behavior and what key factors could potentially contribute to this adoption. The response of an individual after receiving a SMS message could be affected by the frequency of messages and the perceived usefulness of the message. Variability between individuals is expected given that while some individuals will need multiple messages each month to adopt a behavior, others might react immediately after one or two messages in the same week. In addition, the usefulness of the message might be associated with its content, how it is phrased, the language used and other factors that may or may not trigger a response to action in individuals. It is also possible that both frequency and usefulness of messages interact in a way that is optimal, leading to an immediate positive (e.g. read, pay attention, adopt behavior, seek services) or negative (e.g. read, disregard/ignore, delete after reading) reactions. Other factors that can potentially influence the likelihood of adoption of prevention behavior, even in an ideal scenario of optimal frequency and usefulness, include socio–demographic, economic and environmental factors, including privacy related issues.

Figure 1 Framework for assessing the effect of SMS on HIV related behavior.

Data for this analysis come from an operations research study conducted in Lomé, Togo. The aim of the operations research was to assess the role of mHealth in improving HIV knowledge, outreach, and linkage to prevention and care services among KP. PACTE–VIH targeted MSM and FSW to receive SMS messages containing information described above. SMS were sent out twice weekly with a different message in each instance. One example message is as follows: “Protect yourself and your partner. Always use a condom and lubricant anytime you have sexual intercourse.”

Peer educators who were part of PACTE–VIH program, also connected data collectors for this study to randomly chosen MSM and FSW in the urban and peri–urban Lomé region. These peer educators conducted the following types of recruitment: contacted KP enrolled in the PACTE–VIH program to participate in the study; invited enrolled participants to ask their friends who were exposed to the PACTE–VIH program to participate; and accompanied data collectors to “hotspots” where FSW work and could be recruited. KP aged 18years or older was eligible to participate. Data collection took place over 30days in July and August of 2016.

The study interviewed a total of 1,005KP who responded to a structured questionnaire that asked about cell phone and internet usage, HIV knowledge, prevention practices, and opinions on the PACTE–VIH mHealth program. Data collectors were trained to not lead the respondents during the interview and responses were recorded as provided by study participants. All respondents gave signed consent, and no personal information or identifying data were obtained as part of the interview. Ethical approval for this study was provided by the Togolese Ethical “Comité de Bioéthique pour la Recherche en Santé” of the Ministry of Health and Social Protection (AVIS N° 19/2015/CBRS) on June 30, 2016. Approval was also provided by the University of California, Berkeley Center for Protection of Human Subjects (CPHS #2016–03–8583). For this paper, the authors used a sub–sample, including 312 respondents who ever received the PACTE VIH HIV–related SMS between November 2014 and June 2016.

The primary independent variable was time in months enrolled in the mHealth program. The questionnaire recorded the month and year of respondents’ first and last HIV–related SMS received. Therefore, enrollment time was constructed by subtracting the date of first SMS from the date of most recent SMS received. Due to a non–normal distribution, we log–transformed the continuous variable for initial analyses (see Figure 3 in Results). The final analysis used a categorical version of enrollment time in order to address the non–normal distribution of the continuous variable, and to examine how adoption of prevention behavior may vary across intervals of several months. The final variable was categorized into four levels: less than 6months, 6–11months, 12–23months, and 24months or more.

The dependent variable, adoption of prevention behavior, refers to the respondents reported action after having received the message (reminders to get tested, reminders to use condoms and lubricant). The variable was coded as a binary “yes/no” variable; “yes” indicated the respondent followed through with the action suggested by the SMS, otherwise they were categorized as “no.” All the responses were spontaneous, not prompted.

Control variables (covariates) included respondent’s age, socioeconomic status, marital status, education, and type of key population (MSM or FSW). Socio–economic status was represented by an index, constructed with principal component analysis (PCA) using variables that described technology ownership (owning a computer or smartphone) and transportation ownership (owning a car, motorcycle, and bicycle), with Cronbach’s Alpha reliability coefficients estimated. The final socio–economic status index was divided into three categories (terciles), ranging from lowest to highest.

Out of 1005KP surveyed, 313 received PACTE VIH HIV–related SMS, of which 123 were unable to recall the length of their exposure period: start date (84.6%), end date (3.3%), or both (12.2%). We used regression–based multiple imputation to account for the missing data in the independent variable, enrollment time. The process of multiple imputations drew values from a distribution of the observed data and assigned them to the 123 missing observations over five iterations.¹⁴ five total datasets were created and analyzed separately in the regression analysis, with the results from each dataset pooled to give one final result.

Data analyses relied on chi–square test and logistic regression. Chi–square test allowed us to explore bivariate relationships between dependent and independent variables of interest. Logistic regression models enabled us to assess the relationship between number of months exposed to the mHealth program and adoption of prevention behavior before and after controlling for other covariates. Final models were adjusted for covariates with chi–square p<0.05 in the bivariate analysis between dependent variable and covariates. To assess differences in observed and imputed data, we conducted the same analyses with only the observations that had no missing data (N=189). All analyses were performed in Stata.

Figure 2 diagrams the observations included in the final analytical sample. Of the 1,005 total KP interviewed in the operations research, 313KP reported receiving HIV–related SMS. One observation was dropped due to erroneous data entry. The remaining observations were used in the final analyses; 123 missing values were imputed for a final sample of N=312. The analysis using only respondents with no missing data excluded these 123 observations, for an analytic sample of n=189.

Figure 2 Flow chart of exclusion criteria, missing data, and final analytical sample.

Prior to imputing the missing data, we assessed the distribution of respondents by enrollment time among those with no missing data (n=189). Figure 3 displays the frequency of respondents as a function of enrollment time, by number of months enrolled. The majority of respondents reported an enrollment time of less than 24months, with the greatest number of respondents reporting an enrollment time of 8months. The distribution of months enrolled as a continuous variable was skewed slightly right as a result.

Figure 3 Frequency of observations by enrollment time expressed in months, represented as number months enrolled in the PACTE-VIH program.

As previously described, enrollment time was ultimately divided into four categories in order to examine adoption of prevention behavior as a function of time enrolled in the PACTE–VIH program by several month intervals. We first examined proportions and bivariate statistics of study covariates by the primary independent variable, enrollment time, across category of months enrolled (Table 1). These data include only the 189 respondents with no missing data. Our analytic sample included disproportionally more MSM. Study participants were predominantly young, with the majority less than 30 years of age; most of the participants had never been married and had secondary education; and the majority of the participants in the sample were enrolled in the program for more than six months, with only 16% were enrolled less than six months (Table 1).

Chi–square analysis demonstrated a significant difference in proportions across enrollment time categories for type of KP and wealth, at a significance value of p<0.05. More FSW (74.19%) were enrolled for less than 6months than MSM, while MSM accounted for over 80% of KP enrolled for any category at 6months or more. KP in the lowest socio–economic status tercile accounted for 61.29% of those enrolled in the PACTE–VIH program for less than 6months, as compared to those in the highest socio–economic tercile, where 72.34% of KP was enrolled for 24months or more. The other study covariates including age, marital status, and education, were not significantly different across the enrollment time categories (p>0.05).

Analysis of preventive behavior by the length of exposure, excluding individuals with missing values (Table 2), shows that slightly over half of respondents (52.91%) reported adopting prevention behavior after receiving HIV–related SMS messages. Forty–seven respondents (24.87%) were enrolled in the mHealth program for 24 months or more, while the largest proportion of participants were enrolled for a period of 12–23months (33.86%). Chi–square tests demonstrated a significant difference between two or more categories of enrollment duration and adoption of prevention behavior at the 0.05 significance cutoff (p=0.029).

When examining the study covariates, we found that about two–thirds (67.72%) of respondents in the complete case dataset were aged less than 30years, and nearly 80% were MSM. Slightly less than half of all respondents (42.86%) were in the highest wealth category. Most respondents (87.83%) had never been married, and only 21.16% reported receiving a higher education. Based on chi–square tests, there were significant differences at p<0.05 between adoption of prevention behavior and one or more categories of KP, wealth, and marital status. All study covariates displayed in Table 2 with p<0.05 were included in the final adjusted regression models.

For the regression analyses, our final study sample included 312 observations. We used the enrollment time variable with the 123 missing observations imputed. We initially ran logistic regressions on unadjusted models of enrollment time on adoption of prevention behavior, with enrollment time as a log–transformed continuous variable and as a 4–level categorical variable. Table 3 displays the unadjusted odds ratios of these regressions. As a continuous variable, there was a 2% increase in the odds of adopting prevention behavior with each month increase in time enrolled in the program, but this result was not statistically significant (OR: 1.02; 95% CI: 0.99–1.04). After assessing the same model with enrollment time as a categorical variable, we found the same results; there was no significant difference in odds of adopting prevention behavior with each category increase, even though all categories show increased odds of prevention behavior compared to those enrolled less than 6months.

Despite a lack of significance in the primary relationship between enrollment time and adoption of prevention behavior, we ran logistic regressions with models adjusted for study covariates in order to observe the potential influences of these factors on prevention behavior. Multivariate regression results for categorical enrollment time on adoption of prevention behavior are shown in Table 4. Consistent with the unadjusted findings, we found that none of the enrollment duration categories had significantly different odds of adopting prevention behavior as compared to those who were enrolled for less than 6months.

When observing the relationships between study covariates and adoption of prevention behavior, we found that respondents in a higher socio–economic tercile had higher odds of adopting prevention behavior (OR: 2.45, 95% CI p–0.026), while having ever been married was associated with lower odds of adopting prevention behavior (OR: 0.24; 95% CI 0.10–0.58 p–0.001).

In addition to running multivariate regressions with the imputed data of the enrollment time variable, we performed the same analyses with only observations with no missing data (n=189) to compare findings. The results from the unadjusted model indicated a significant association between enrollment time and adoption of prevention behavior. The odds of adopting prevention behavior were higher with each additional month enrolled in the program (OR: 1.62; 95% CI: 1.19–2.20). When assessing enrollment time as a categorical variable, the unadjusted results demonstrated that all three enrollment categories had significantly higher odds of adopting prevention behavior, as compared to individuals enrolled for less than 6months (p<0.05).

The results were attenuated and lost significance after adjusting for common confounders, such as age and marital status. The fully adjusted regression model showed that there was no significant increase in odds of adopting prevention behavior, as compared to an enrollment time of less than 6months. These results are displayed in the supplementary tables (Tables S1 & S2) in the appendix.

The aim of this study was to determine whether enrollment time was associated with adoption of HIV related prevention behavior. Understanding if length of time enrolled in a study influences behavioral outcomes could inform program interventions to send mHealth reminders in a manner that results in positive behavior change without overusing limited resources.

The results from our study demonstrated that participants who received HIV–related SMS for a longer period of time displayed no statistical significant difference in adopting prevention behavior as compared to those who received SMS for less than 6months. The regression results from the observations with no missing data corroborated our findings from the primary analysis that used multiple imputations to account for the missing values; fully adjusted models in both sets of analyses demonstrated no significant difference in adoption of prevention behavior with increased number of months enrolled. These findings indicated that KP who were enrolled in the PACTE–VIH program for any period of time greater than 6months were just as likely to adopt prevention behavior as KP enrolled for less than 6 months. In addition, it emerged that being married and level of wealth were significant factors contributing to adoption of prevention behavior even when time in the program was accounted for, assumptions that we included in the framework presented in Figure 1.

The results from our study differed from a prior mHealth study that explored number of SMS received as a primary exposure.¹³ Although not an HIV study, the text4baby mHealth trial on alcohol consumption found a significant decrease in odds of self–reported postpartum alcohol consumption with exposure to a greater number of SMS messages, as compared to fewer messages.¹³ The results from this dose–response study contrasted ours; while Evans et al. found that receiving SMS more frequently resulted in desirable behavior change, our study demonstrated a similar level of adoption of prevention behavior across varying enrollment times. Further research is needed to understand by what specific processes and in which contexts mHealth programs can improve HIV prevention behavior, and if maintaining enrollment for more than six months to a year would be a beneficial use of resources.

Though longer time enrolled in the PACTE–VIH mHealth program may not have resulted in differences in adopting prevention behavior in our study sample, wealthier, or ever married KP were more likely to adopt prevention behavior as compared to their less wealthy and never married counterparts. These trends are common among HIV prevention programs; a meta–analysis by Albarracin et al, demonstrated that certain socio–demographic factors, such as age, can determine how likely program participants are to engage with behavioral interventions.¹⁶ Accounting for these differences within KP subpopulations by targeting especially vulnerable groups, who may not be as likely to engage with mHealth, could also potentially improve the effectiveness of such programs.¹⁷

Receiving SMS reminders in HIV mHealth programs is known to improve health treatment outcomes,⁷ and internal studies demonstrated that the act of receiving SMS messages through the PACTE–VIH mHealth program improved HIV testing behaviors and treatment adherence among KP. Prior studies have found that participants who receive SMS reminders had significantly higher adherence to HIV treatment and subsequent viral suppression, as compared to those who received no SMS reminders.^9,18,19 These programs may have been effective because of the rapport established between participants and health providers,¹¹ rather than as a result of participants receiving repeat SMS reminders. In the PACTE–VIH program, the role of the peer educators was instrumental in engaging KP within the program and encouraging participants to come in for face–to–face appointments. Behavior change is most likely to occur when subjects are presented with interactive and personalized resources.²⁰ Therefore, sending participants HIV–related SMS over an indefinite period of time may not serve its intended purpose of improving HIV prevention behavior in KP sub–populations. Instead, HIV programs in developing countries may have more success with behavior change outcomes by involving peer educators to engage with hard–to–reach, at–risk populations.²¹

This analysis provided important insights into how enrollment time in a mHealth program may, or may not, influence prevention behavior outcomes. Engaging peer educators to provide a more collaborative experience could improve KP adherence to HIV PTC interventions, particularly if mHealth harnesses technology to engage of all sub–groups of KP, including youth or those of lower socio–economic status, to participate in these programs. Programs that integrate these aspects into their interventions may be more effective than repeated SMS messaging over a long period of time. However, there needs to be more high–quality evaluations to assess the effect of mHealth on HIV behavioral outcomes before we can make definitive programmatic recommendations on the ideal enrollment periods of HIV mHealth interventions.^4,16 Based on the results of this study, it seems like participants may only need 6months to gain the HIV knowledge they needed to make decisions about prevention behaviors. Hammering the message home in the first 6months may work, and further exploration is needed to ascertain the ideal conditions under which this short amount of time in a program could lead KP to adopt and maintain HIV associated prevention behaviors. Ideally, participants could be segmented and tracked on how they proceed after receiving an SMS. Not everyone is expected to act on the SMS messages they receive immediately, and among those that do respond, we should expect a distribution similar to the diffusion of innovation curve with innovators, early adopters, early majority, late majority and laggards.²²

The data collection for this study can be considered challenging, especially for MSM in the context of Togo. The study did not have a list of program participants from which a representative sample could be drawn; while FSW were recruited in their “hotspots”, MSM were referred to the study by peer educators and by other MSM study participants. One major limitation of this study was the large number of missing values in the primary independent variable. However, the theory–based method of multiple imputations accounted for any bias that may have otherwise occurred in an analysis that excluded all observations with any missing data. Another limitation from the data collection process was the self–reporting of the outcome variable, adoption of prevention behavior. Due to the cross–sectional design of the study, data collectors were not able to verify whether participants did indeed follow the suggested prevention behaviors, such as testing for HIV every 3months, after receiving the HIV–related SMS message. This could lead to over–reporting of the outcome as a form of social desirability bias, especially because the peer–educators, who also promote the prevention behaviors, were recruiting respondents. Social desirability bias among high risk HIV subpopulations has shown to be associated with an increased reporting of prevention behaviors.²³ However, the lack of significant results in the final models in our study indicates the outcome was likely not over–reported.

Time does matter for adoption of prevention behaviors after a mHealth intervention. The length of program time required is still to be determined, especially for KP. While this study found no significant difference in adopting preventative behavior between those enrolled for less than 6 months and those enrolled longer than 6months, the findings of this study demonstrated the opportunity for mHealth interventions in HIV prevention. Given that mobile technology is becoming more common in behavior health interventions, future studies must aim to provide a greater evidence base for the implementation of mHealth programs to ensure benefits are maximized. Longitudinal studies would enable better records of exact dates of enrollment and censorship of study participants, and specific outcomes such as testing frequency and self–reported condom use could be measured more accurately. Ultimately, having more data on effectiveness of enrollment time will allow programs to tailor their efforts to allocate resources efficiently and scale–up efforts to reach vulnerable populations most at–risk of HIV infection.

United States Agency for International Development (USAID)/West Africa as part of the Evidence for Development (E4D) Project.

None.

Author declares that there are no conflicts of interest.

1. Papworth E, Ceesay N, An L, et al. Epidemiology of HIV among female sex workers, their clients, men who have sex with men and people who inject drugs in West and Central Africa. J Int AIDS Soc. 2013;16(S3):18751.
2. National Council for the Fight against AIDS and Sexually Transmitted Infections (CNLS). Second-generation sero-behavioral survey of men who have sex with men in Togo in 2017. Lome: CNLS; 2015.
3. Joint United Nations Programme on HIV/AIDS. 90-90-90: an ambitious treatment target to help end the AIDS epidemic. Geneva: UNAIDS; 2014.
4. Gurman TA, Rubin SE, Roess AA. Effectiveness of mHealth behavior change communication interventions in developing countries: a systematic review of the literature. J Health commun. 2012;17(S1):82–104.
5. Riley P. mHealth in West Africa: A Landscape Report. USAID. 2014.
6. DeKoekkoek T, Given B, Given CW, et al. mHealth SMS text messaging interventions and to promote medication adherence: an integrative review. J clin Nurs. 2015;24(19-20):2722–2735.
7. Lester RT, Ritvo P, Mills EJ, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet. 2010;376(9755):1838–1845.
8. Mechael PN. The case for mHealth in developing countries. Innovations: Technology, Governance, Globalization. 2009;4(1):103–118.
9. Nachega JB, Skinner D, Jennings L, et al. Acceptability and feasibility of mHealth and community-based directly observed antiretroviral therapy to prevent mother-to-child HIV transmission in South African pregnant women under Option B+: an exploratory study. Patient Prefer Adherence. 2016;10:683–690.
10. Catalani C, Philbrick W, Fraser H, et al. mHealth for HIV treatment & prevention: a systematic review of the literature. Open AIDS J. 2013;7:17–41.
11. Chi BH, Stringer J. Mobile phones to improve HIV treatment adherence. Lancet. 2010;376(9755):1807-1808.
12. Mbuagbaw L, Thabane L, Ongolo-Zogo P, et al. The Cameroon Mobile Phone SMS (CAMPS) trial: a randomized trial of text messaging versus usual care for adherence to antiretroviral therapy. PloS one. 2012;7(12):e46909.
13. Evans W, Nielsen PE, Szekely DR, et al. Dose-response effects of the text4baby mobile health program: randomized controlled trial. JMIR MHealth UHealth. 2015;3(1):e12.
14. Govender K, Beckett S, Masebo W, et al. Effects of a Short Message Service (SMS) Intervention on Reduction of HIV Risk Behaviours and Improving HIV Testing Rates Among Populations located near Roadside Wellness Clinics: A Cluster Randomised Controlled Trial in South Africa, Zimbabwe and Mozambique. AIDS Behav. 2019:1–10.
15. Mayer JE, Fontelo P. Meta-analysis on the effect of text message reminders for HIV-related compliance. AIDS care. 2017;29(4):409–417.
16. Albarracín D, Gillette JC, Earl AN, et al. A test of major assumptions about behavior change: a comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychol Bull. 2005;131(6):856–897.
17. Lucero RJ, Frimpong JA, Fehlberg EA, et al. The relationship between individual characteristics and interest in using a mobile phone app for HIV self-management: observational cohort study of people living with HIV. JMIR MHealth UHealth. 2017;5(7):e100.
18. Pop-Eleches C, Thirumurthy H, Habyarimana JP, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS. 2011;25(6):825–834.
19. Ammassari A, Trotta MP, Shalev N, et al. Timed short messaging service improves adherence and virological outcomes in HIV-1–infected patients with suboptimal adherence to antiretroviral therapy. J Acquir Immune Defic Syndr. 2011;58(4):e113-e115.
20. Prochaska JO, Velicer WF. The transtheoretical model of health behavior change. American journal of health promotion. 1997;12(1):38–48.
21. Medley A, Kennedy C, O'Reilly K, et al. Effectiveness of peer education interventions for HIV prevention in developing countries: a systematic review and meta-analysis. AIDS Educ Prev. 2009;21(3):181–206.
22. Rogers EM, Shoemaker FF. Communication of Innovations; A Cross-Cultural Approach. New York: Free Press. 1971.
23. Rao A, Tobin K, Davey-Rothwell M, et al. Social desirability bias and prevalence of sexual HIV risk behaviors among people who use drugs in Baltimore, Maryland: implications for identifying individuals prone to underreporting sexual risk behaviors. AIDS Behav. 2017;21(7):2207-2214.