Submit manuscript...
Journal of
eISSN: 2373-4426

Pediatrics & Neonatal Care

Conceptual Paper Volume 6 Issue 5

 A Framework for Developing and Evaluating Health and Mental Health Interventions

Edward De Vos EdD

Associate Vice President for Research, William James College, USA

Correspondence: dward De Vos, EdD, Associate Vice President for Research, William James College, USA, Tel 617-564-9407, Fax 617-327-4447

Received: August 11, 2016 | Published: April 27, 2017

Citation: De Vos E (2017) A Framework for Developing and Evaluating Health and Mental Health Interventions. J Pediatr Neonatal Care 5(5): 00260. DOI: 10.15406/jpnc.2017.06.00260

Download PDF

Abstract

Interventions designed to address the social and environmental determinants of health frequently rely on collaboration between healthcare and other professionals. To persuade healthcare providers that their patients’ unmet needs should be their concern, either through direct intervention or referral, requires evidence of the relationship between that need and health outcomes. While there are simple causal linkages between, for example, substandard housing conditions and acute cases of asthma, for most medical conditions the relationship can be far more complex. This paper describes a framework for thinking about these connections in an effort to develop and evaluate interventions that better target social and environmental determinants of health.

Keywords: Medical pathways; Health; Mental health; Program development; Program evaluation; Intervention; Research; Social determinant; Environmental determinant; Framework; Model; Health disparity

Introduction

The Medical-Legal Partnerships (MLP) developed in recognition that the greatest threats to the health and well-being of children living in the U.S. today are socially and environmentally determined [1,2]. Unlike infectious diseases and malignancies of earlier generations in America, today the greatest causes of death and morbidity are to be found in preventable social and environmental factors [3]. Medicine in general and pediatrics in particular, increasingly has accepted this relationship and with this acceptance has effected changes in practice to more directly address these issues.

Increased attention to social and environmental determinants has also promoted more collaborative relationships between healthcare and other groups of professionals to better meet the needs of patients. Many factors that can adversely impact child health (including mental health) are associated with poverty, frequently contributing to health disparities [4-6]. Legislation has sought to protect economically and socially disadvantaged groups so they will be able to meet their basic needs-to adequate shelter, to food, to healthcare -- for example, SSI (Supplemental Security Income), SNAP (Supplemental Nutrition Assistance Program), TANF (Temporary Assistance for Needy Families), WIC (Special Supplementary Program for Women, Infants, and Children), and Medicaid. Numerous government programs have been designed to provide a safety net against unmet needs for those who might otherwise fall between the cracks. However, ensuring the right to basic needs and gaining access to safety net programs may require some level of legal involvement, whether it is basic advice and advocacy, or active representation.

In 1993, Boston Medical Center’s (BMC’s) Department of Pediatrics, through the efforts of Dr. Barry Zuckerman and colleagues, concluded that the health interests of impoverished populations such as those receiving care at BMC, would be better served through medicine and legal advocacy than through medicine alone. Through the combined efforts of medical and legal champions, the program model was developed [7-9]. Combining medicine and law in the interest of disadvantaged patients is a powerful concept with great potential for effecting change. On its face, while MLP is innovative, it appears to be a quite natural idea-relying on logic and common sense for its justification. However, in an era justifiably demanding accountability and with increasing attention to evidence-based practice, MLP deserves and needs to be evaluated. This paper grows directly out of that need, yet also has wider implications for research, program development and strategic planning for interventions that target social-environmental determinants of health and mental health.  MLP will be used as an illustrative example throughout this paper, but readers are encouraged to use the framework to think about alternative intervention strategies as well.

Background: Before a Medical-Legal Partnership (MLP) is introduced into a new clinical setting or service, a Staff Assessment form is typically administered to determine priorities, to plan for training, and to serve as a potential baseline measure of staff knowledge, attitudes and behaviors prior to program implementation. Recently, in anticipation of expansion into Oncology at Boston Medical Center, the existing needs assessment instrument was reviewed and modified for the division. The old form assessed some critical areas of unmet legal needs, following the basic MLP I-HELP model to cover the Social/Environmental Determinants – factors typically addressed by MLP [10]. I-HELP is an acronym for simply summarizing the overarching areas within which MLP typically provides legal advocacy: Income (including money, food security, public benefits and health insurance), Housing, Education, Legal Status (issues related to immigration status), and Personal Safety, among other issue areas.

However, while reviewing the form it became apparent that something was missing --something essential if we were to persuade healthcare providers that just because a patient has a need in a particular area, that it is important for the provider to address it, either directly or through referral. Specifically, the form ignored the critical relationship between the specific social/environmental factor(s) and the health outcome(s) for their patients.

Developing a Model

Primary Care Model

Before proceeding to the model for secondary, condition-based (and frequently, service-based) health care, which is the core of this article, it is important to recognize that two models are likely to be required. Primary care, especially for child health providers, deals with illness and injury, but frequently concentrates on well-child and routine health maintenance visits [11]. Health promotion (and primary prevention) activities tend to follow a different course than the one to be described below and are intended to achieve a different set of outcomes. Specifically, health promotion is meant to help a child achieve his/her optimal developmental trajectory [12]. While optimization certainly may prevent certain conditions -- such as failure to thrive, becoming overweight, or bullying -- desired outcomes may be thought of more positively in terms of positive physical, social and emotional development. Certain aspects of the Medical Pathways model, which will be described below, may provide a useful framework for considering some of these dimensions as well; however, the introduction of a Medical-Legal Partnership into a primary care setting may resonate more naturally in the context of health promotion and optimal development, and may not require a more detailed sub-analysis to justify its introduction. Similarly, health promotion activities by their very nature broaden the traditional medical model and may be more open to collaborative, less linear, multi-factorial interventions.

Secondary, Tertiary, and Quaternary Care Model

As noted earlier, it was in the context of introducing the MLP program into a new service, Oncology to be specific, that the issue of salience arose. On reflection, this represented the expansion of the program model from what historically had been a primary care setting, pediatrics, into a new medical environment, one that tended to focus on a more clearly defined subset of cases and one that was steeped in a more traditional medical model. Further, the effects of social and environmental factors might be causal or contributing, mediating or moderating; they might vary by condition, and the point of impact might vary along a broad continuum of health/medicine-related processes and trajectories.

As a starting point, we needed a more nuanced framework within which to consider these potential impacts, the “Medical Pathways” model. “Medical Pathways” refers to various medical/health-related issues (or processes) that constitute a logical pathway from the development of a condition through to its prognosis and final endpoint. Specifically, the Medical Pathway consists of the following steps1 (or a similar variant):

T
I
M
E

1. Etiology of the condition [including predisposition as well as exposure]
2. Diagnosis
3. Treatment plan
4. Compliance w/ (adherence to) treatment plan
5. Prognosis
6. “Final” status
7. Recurrence

Using the pathways framework, the following might be a reasonable template for a critically important set of questions to be put to healthcare providers in a new departmental (or institutional) setting that is just beginning to consider MLP collaboration:

To what extent does [INSERT: Social/ Environmental Determinant] affect the [INSERT: Step along Medical Pathway] for your patient population?

For each of these steps, does the [Social/Environmental Determinant] have a direct effect, a mediating effect, and/or a moderating effect? [13,14] Further, from the standpoint of setting priorities, for each effect, is the presumed relationship small, medium, or large?

Independent of healthcare providers’ knowledge and attitudes, it is imperative that a literature review be conducted to identify what is known and where the gaps are. Such a review may be service and/or illness/medical condition specific. The latter is especially relevant for the most prevalent conditions seen in primary care settings, separate from the health promotion activities that are among the core services provided in those clinical settings. The following chart schematizes the framework as applied to MLP. The columns represent steps along the medical pathway while the rows represent the social and environmental factors that Medical-Legal Partnerships is prepared to address (Figure 1).

Figure 1: N=57; Epidemiological distribution of the pathological fractures, traumatic fractures, and nonunion.

Consider, for example, the “H” (Housing) row in the table above. For the purposes of this discussion, consider an Asthma Clinic (or, from the disease-specific perspective, Asthma, regardless of the clinical setting where it is seen-primary care, ER, etc.). Box [1] asks about the extent to which housing factors are involved in the development of asthma in the population served by the healthcare provider. We might code this on a 1-5 scale where:
1 = no relationship,
2 = small contributing factor,
3 = moderate contributing factor,
4 = large contributing factor,
5 = primary cause of the condition.

Proceeding across the row, we might consider the extent to which housing affects diagnosis, treatment planning, etc.Similarly, Box [2] (above) considers how family Income affects compliance with treatment, whether through the cost of prescriptions, correcting hazardous housing conditions, or attending follow-up appointments.

A more elaborate visual model for depicting this framework adapts a fishbone (or Ishikawa or “cause-and-effect”) diagram, popular in the quality improvement (QI) literature (Figure 2). While the diagram could be conceptualized in terms of specific health outcomes, here, the problem is posed in terms of health disparities. The primary steps along the Medical Pathway now define the framing elements at the end of each fishbone2. Along each bone, I have listed a number of potential factors that may contribute to the observed disparity. In addition, some factors have been italicized as they may be amenable to change through Medical-Legal Partnerships involvement. For example, exposure to known environmental hazards or risk factors, as well as limited access to or availability of protective factors may contribute to health disparities (e.g., consider differential rates of asthma). So-called targeted (or secondary) prevention that occurs outside of the healthcare setting is typically outside of the MLP direct service agenda. Yet these social-environmental factors may be addressed through legal means, and may respond to MLP advocacy. By contrast, genetic predisposition in and of itself, is not redressed through MLP involvement.

Figure 2: N=57; Epidemiological distribution of the pathological fractures, traumatic fractures, and nonunion.


1 It will be important to consider what does and does not belong along this medical pathway, and which elements are the most important and practical to include. For introduction to a new medical department or service, two important correlative factors: Access to Care and Quality of Care, were actively ignored. They were excluded given the emphasis on the biology of illness, treatment, and outcomes, and because the question arose in the context of surveying professionals within a particular service. It may not be the best strategy for relationship building to ask these types of questions within a small and closed system. However, for completeness, these steps along the path (as well as others that may have been overlooked) should be considered, even if they are later dropped from a more specific application of the model.

2 An alternative fishbone model dispenses with steps along the medical pathway, and instead substitutes a more traditional approach using the potential inputs: Patients, Clinical Professionals, Materials, Procedures and Policies

Implications

The Medical Pathways framework has numerous implications for developing, implementing, evaluating and disseminating the Medical Legal Partnerships model. This section addresses a few.

Research

If we are to argue persuasively for the introduction of the MLP model into other areas of practice, it will be necessary to map the knowledgebase that links social and environmental determinants (that lend themselves to legal redress) to the medical pathways (such as etiology, diagnosis, treatment, prognosis) where their effects are likely to be realized. As noted earlier, the mapping may be medical service specific as well as condition (i.e., disease or injury) specific. The map should identify relevant empirical literature as well as gaps to guide future research efforts.

Given the number of potential determinants and the number of steps along the medical pathway, this review will take resources for any single condition, let alone for any medical service area that typically treats numerous conditions. The first step is to set priorities on the areas and the sequence of reviews to be conducted. Within primary care, the priority may be determined based on the top three to five conditions seen within that practice setting; conditions that may have social and environmental determinants. Once priorities are set, the research literature must be systematically reviewed to identify the evidence for the association between social and environmental determinants and the medical pathways for the specific conditions and clinical areas. Since intervention research involving social and environmental influences frequently present challenges to more traditional Randomized Controlled Trials (RCTs), the types of studies considered for review should be more inclusive.15 The available literature needs to be synthesized and summarized as to the evidence of impact, the sufficiency of that evidence for making practice decisions, the strength of the observed relationship (weak to strong) and whether the presumed effect is direct or indirect (causal, mediating and/or moderating).

In addition to the empirical literature, we should consider surveys of practitioners working in these clinical areas to ascertain their baseline clinical impressions of the connections between the social and environmental determinants and the medical pathways for the specific conditions with which they are most familiar. The surveys will again look at the nature of the relationship (no relationship; small contributing factor; moderate contributing factor; large contributing factor); the form of the presumed effect (direct; mediating; and/or moderating); and the practitioner’s basis for her/his impression/belief (research; theory; and/or practice experience). By combining the results of the literature review and the surveys of practitioners, we will be able to identify gaps in the literature and set priorities for a social and environmental determinants research agenda. To advance the agenda, I would propose a series of research studies to be undertaken with grant support. The results of these reviews and practitioner/expert surveys will be submitted for publication and presentation in an effort to stimulate funding and promote research in the most critical areas of identified need.

Evaluation

The Medical Pathways framework has important implications for evaluation, especially within what I choose to call developmental evaluation. This approach places program evaluation in the context of a program’s own developmental trajectory and recognizes that the salient questions for key decision makers will vary over time. Appropriate evaluation activities depend on the program’s stage of development: from establishing the need for a program and planning it, through pilot implementation and adjustment, full implementation and efficacy, to effectiveness, dissemination and ongoing quality improvement.

The Medical Pathways framework has relevance at various points along this programmatic trajectory. As noted earlier, in identifying the need for a program such as MLP within a new clinical service area, it may be important to understand the perceptions of practitioners regarding the relationship between social and environmental determinants and the medical pathways for the conditions they treat. The Medical Pathways model may expand the clinicians’ frame of reference beyond a more traditional biologically based medical model. In addition to these practice-based observations, as noted above, it will be important to review the literature to determine which clinical opinions are grounded in evidence and which have yet to establish those linkages. In evaluating the need for training as well as the impact of training, it would be good to establish the extent to which practitioners buy into the model, and the Medical Pathways framework may provide a useful way of disentangling and articulating those connections.

Farther along in a program’s development, when outcome or impact evaluation are appropriate, the Medical Pathways can be of great use in the development of logic models to help guide the evaluation. The framework provides a guide for making explicit the underlying theory of change that is at the heart of the intervention. If, for example, the primary effect of MLP is through the reduction of stress on parents, which, it is hypothesized, interferes with their ability to comply effectively with a prescribed treatment plan, then a proximal indicator of impact ought to demonstrate a reduction in stress, coupled with better adherence to the treatment plan, and, subsequently, to better health outcomes for the patient. If the link between the treatment plan and the health outcome has been sufficiently well established in the literature, then demonstrating a process outcome such as treatment compliance should be sufficient to demonstrate the efficacy of the program.

Strategic Planning

The logic models which can be so useful for the development of the evaluation plan are also critically important for program development and implementation. By considering the Medical Pathways and the points at which an intervention may have an impact, those who develop and implement the program are pushed to assess the logical context of their work. This provides a structured opportunity to identify points of agreement as well as areas of potential disagreement, which may support the development of a truly collaborative approach to intervention.

Caveat

Finally, it is worth noting that this model may be a useful over simplification. Clearly, many medical conditions are multi-determined, and it may be a confluence of factors that is more important for understanding effects and impacts. Single factors operating additively and non-recursive linear relationships may be useful for conceptual purposes, but ultimately may fail to capture the dynamism of medical conditions or the systems that seek to address them.

Acknowledgement

This paper developed out of my involvement in the evaluation of Medical-Legal Partnerships Boston and the National Center for Medical-Legal Partnerships, work funded by Atlantic Philanthropies. My ideas have been shaped by many discussions and observations. The views expressed in this paper are my own, especially those with which others might not agree. For those elements that ring true, I have more people to thank than I can include in a brief acknowledgement. However, I would like to single out Barry Zuckerman, Ellen Lawton, Samantha Morton, and Robert Sege for their insights and offer a collective thank you to a wonderfully dedicated team of clinicians and researchers at Boston Medical Center for their ongoing commitment to children and families.

References

  1. (1991) American Academy of Pediatrics Task Force: Report on the Future Role of the Pediatrician in the Delivery of Health Care. Pediatrics 87(3): 401-409.
  2. Committee on Psychosocial Aspects of Child and Family Health (2001) American Academy of Pediatrics.. The New Morbidity Revisited: a renewed Commitment to the Psychosocial Aspects of Pediatric Care. Pediatric 108(5): 1227-1230.
  3. Centers for Disease Control and Prevention (CDC) / National Center for Injury Prevention and Control (NCIPC) / National Center for Health Statistics (NCHS). WISQARS Report: 10 Leading Causes of Death, United States, 2006, All Races, Both Sexes. 11/12/09 2006.
  4. Beal AC (2004) Policies to reduce racial and ethnic disparities in child health and health care. Health Aff (Millwood) 23(5):171-179.
  5. Williams DR, Costa MV, Odunlami AO, Mohammed SA, Costa MV, AO Odunlami, SA Mohammed (2008) Moving upstream: how interventions that address the social determinants of health can improve health and reduce disparities. J Public Health Manag Pract Suppl: S8-17.
  6. Shonkoff JP, Boyce WT, McEwen BS (2009) Neuroscience, Molecular Biology, and the Childhood Roots of Health Disparities: Building a New Framework for Health Promotion and Disease Prevention. JAMA 301(21):2252-2259.
  7. Zuckerman B, Sandel M, Smith L, Lawton E (2004) Why Pediatricians Need Lawyers to Keep Children Healthy. Pediatrics. 114(1):224-228.
  8. Zuckerman B, Lawton E, Morton S (2007) From principle to practice: moving from human rights to legal rights to ensure child health. Arch Dis Child 92(2): 100-101.
  9. Zuckerman B, Sandel M, Lawton E, Morton S (2008) Medical-legal partnerships: transforming health care. Lancet 372(9650): 1615-1617.
  10. Kenyon C, Sandel M, Silverstein M, Shakir A, Zuckerman B (2007) Revisiting the Social History for Child Health. Pediatrics 120(3): e734-738.
  11. Hagan JF, Shaw JS, Duncan P (2008) Bright Futures: Guidelines for Health Supervision, (3rd edn) American Academy of Pediatrics, Elk Grove Village, USA p. 1-43.
  12. Halfon N, Hochstein M (2002) Life course health development: an integrated framework for developing health, policy, and research. Milbank Q 80(3): 433-479.
  13. Kraemer HC, Stice E, Kazdin A, Offord D, Kupfer D(2001) How Do Risk Factors Work Together? Mediators, Moderators, and Independent, Overlapping, and Proxy Risk Factors. Am J Psychiatry 158(6): 848-856.
  14. Kraemer HC, Wilson GT, Fairburn CG, Agras WS (2002) Mediators and Moderators of treatment Effects in randomized Clinical Trials. Arch Gen Psychiatry 59(10): 877-883.
  15. Sege RD, De Vos E (2008) Care for children and evidence-based medicine. Pediatric Annals 37(3): 168-172.
Creative Commons Attribution License

©2017 De. This is an open access article distributed under the terms of the, which permits unrestricted use, distribution, and build upon your work non-commercially.