This research paper examines the critical use of public health information systems in enhancing health outcomes research on populations. As a systematic review, the paper will explore five published articles on studies that address the ever-increasing application of public health information systems to public health outcomes research within the domain framework of population-level health quality measures, related health measures, and efficiency measures. The studies were conducted using different information systems as data sources such as the Nationwide Inpatient Sample (NIS), the Pediatric Health Information Systems Database, the Discharge Abstract Database (DAD) of the Canadian Institute for Health Information (CIHI), Geographic Information Systems (GIS), and CDC’s National Center for Health Statistics (NCHS). The systematic review found that those systems were successfully used in population-based studies in collecting, evaluating, interpreting, assessing, and investigating health data outcomes from which findings will later be used for improved public health planning.

O’Carroll et al.,¹ define public health information systems as information systems exclusively used in public health domain to collect, store, monitor, manage, and exchange public health data. Public health information systems can be used at the local level (county health systems), state or regional level (state health systems), and the national or federal level (national systems such as National Vital Statistics). Correspondingly, population health outcomes are defined as the health states of a population or community as a result of a public health intervention. Parrish² agrees that those outcomes are influenced by results from “a complex web of cultural, environmental, political, social, economic, behavioral, and genetic factors” as illustrated in Figure 1.

Figure 1 “Level and distribution of various factors influencing health outcomes in populations and interactions among them. Solid arrows represent potential causal relationships between factors, diseases, and outcomes. Dashed arrows represent potential feedback from outcomes and diseases on proximal and distal factors. Distal and proximal factors operate through both intermediate factors and directly on health outcomes”.⁹

The Agency for Healthcare Research and Quality (AHRQ)³ depicts outcomes research as a critical means in studying the effects of particular medical interventions and practices on individual patients and populations. The conduct of outcomes research, particularly at the population level, generally involves the utilization of information systems to collect data about population-based outcome metrics, evaluate those data, and report on the findings.⁴ Additionally, Parrish² suggests that “when reported, outcome metrics should present both the overall level of health of a population and the distribution of health among different geographic, economic, and demographic groups in the population”. Hence, population-level outcomes researches are deemed increasingly important and require very advanced methodological approaches and resources. Such resources can be national or state databases, geographic information systems (GIS), and online data repositories used with a focus on population-based outcomes measures. This paper performs a systematic review of various previously published studies that clearly demonstrate the critical impact of public health information systems on enabling population-level outcomes research.

Murray et al.,⁵ agree that governmental health agencies must collect vital information on population mortality outcomes data in order to implement an effective public health system. This need for information goes beyond amassing mortality data in populations as outcomes metrics. The World Health Organization (WHO) describes health as “the state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity”. Consequently, measuring population health outcomes must portray all facets of a population’s physical, mental, and social health states, life expectancy, and other negative indicators such as mortality, morbidity, and inability to function. Various experts and researchers propose approaches and when properly applied, can significantly aid in measuring population health outcomes. Indeed those outcomes measures are only valuable if they can be used to advance population health outcomes research. Therefore, the need for information on population health outcomes must be integrally linked to an overall vision of improving population health outcomes research, which can be made possible by using public health information systems.

The use of public health information systems for population-based health outcomes research is far from being a new practice. Numerous authors have written about how such systems are being utilized for the enhancement of population-level outcomes. In the same context, O’Carroll et al.,¹ argue: Improvement in population-level outcomes, such as improvement in the incidence and prevalence of asthma or HIV/AIDS in the state or county, is the ultimate result of public health improvement efforts. It is hypothesized that an enhanced public health infrastructure can contribute to widespread behavior change to improve outcomes. It is important to develop a set of public health indicators of goal attainment that are accurate, available, and sensitive to state and community-determined efforts. Improved data systems can help monitor process and outcome to provide useful, formative information to guide decisions in the work of public health improvement (p. 651-652). Moreover, the idea those public information systems play an important role in population outcomes research has been confirmed by other studies. Mendelson & Salinsky,⁶ in Health Information Systems and the Role of State Government, write: One major goal of improved information systems is to build state capacity as a knowledgeable purchaser of care. Proposed revisions to Medicaid management information systems (MMIS) promise to deliver basic clinical and financial data, pending approval from the Health Care Financing Administration (HCFA). For example, California’s Medi-Cal (Medicaid) is developing a management information system that contains all fee-for-service and managed care claims and encounters. The database is expected to assist in setting appropriate rates, profiling providers and beneficiaries, assessing patterns of treatment, and tracking health outcomes and costs. Other states, such as Maryland, have built their analytic capacity by replicating Medicaid data in a relational format.

Furthermore, Parrish² explains how key public information systems are used by national and state agencies as primary data sources for population health outcomes. He writes: The principal sources of data available for US population health outcomes are mortality data derived from death certificates and data on subjective health status, functional status, and experiential state derived from population health surveys. The National Vital Statistics System (NVSS) collects and compiles data on births and deaths from all registration districts (most commonly states) in the United States. The most commonly used surveys are NHIS, BRFSS, NHANES, and the National Survey on Drug Use and Health (NSDUH). Several states conduct city or county-level risk factor surveys by using BRFSS methods and questions, and an increasing number of cities and counties now conduct their own surveys based on or derived from BRFSS. A few states and local areas (Wisconsin and New York City, for example) conduct surveys based on NHIS or NHANES methods to provide state or local estimates of health outcomes and determinants.

Finally, Parrish² also shows that geographic information systems (GIS) can be helpful in the practice of population health outcomes research. He advances: Mortality data are available for states and counties. Some states geocode their vital statistics data and provide data-usually through a Web-based data query and mapping tool-for zip codes, census tracts, or locally defined areas. BRFSS provides state-level estimates and estimates for selected metropolitan statistical areas with 500 or more respondents. Several states, including Florida, North Dakota, Washington, and Wisconsin, conduct their own county-level BRFSS to produce estimates for at least some of their counties. NSDUH provides national and state estimates. NHIS and NHANES only provide national estimates.

In PubMed, we conducted a search for the terms “population health outcomes” and “public information systems” from which a total of 131,170 article results were retrieved. From those articles, we selected five studies that are examined for this systematic review based on the following inclusion criteria:

Date of publication

Since the field of outcomes research is increasingly changing, we only included studies that were published within the last three years. Thus, we excluded studies with a date of publication prior to October, 2009.

Source of publication

We only selected studies that were published in nationally recognized professional journals that publish peer-reviewed articles and studies available to the healthcare and academic community, researchers, and the general population in the field of medicine, outcomes research, and biomedical informatics.

Types of information systems

A critical criterion of inclusion was the type of information systems used to conduct the studies. We only selected studies that were conducted with the use of national or state databases and information systems, geographic information systems (GIS), and online public data repositories.

Types of methods used in studies

Another criterion for inclusion was the methodologies used by investigators to evaluate and measure population outcomes. Hence, studies were chosen depending on the approach for measurement which Parrish² described as “the aggregation of health outcome measurements made on people into summary statistics such as population averages or medians, the assessment of the distribution of individual health outcome measures in a population and among specific population subgroups, and the measurement of the function and well-being of the population or society itself, as opposed to individual members”.

Domain framework

The final criterion for inclusion was the domain framework to which the population health outcomes belong. To be part of our review, the studies have had to use public health information systems in evaluating population health measures classified by the National Quality Measures Clearinghouse (NQMC) as:

1. Population health quality measures: population process, population access, population outcome, population structure, and population experience
2. Related population health measures: population health state, population management, population use of services, population cost, population health knowledge, social determinants of health, and environment
3. Population efficiency measures: efficiency

Kane & Radosevich⁷ agree that administrative, national, and federal databases are widely used in population based outcomes research (p. 292). Along with those databases, other geographic information systems and online public data repositories and registries can serve public health organizations and independent researchers in improving public health outcomes assessment. The next section provides an overview of the methods used within those studies in conducting population health outcomes research.

In the study, Population-Based Outcomes Following Endovascular and Open Repair of Ruptured Abdominal Aortic Aneurysm, Giles et al.,⁸ use the Nationwide Inpatient Sample (NIS) database to study “outcomes following endovascular and open surgical repair of ruptured abdominal aortic aneurysms (rAAA)”. Giles et al.,⁸ portray the NIS as “a database maintained through the Healthcare Cost and Utilization Project that captures ~20% of non-federal hospitalizations from 38 states in a stratified sample that reflects ~90% of all hospitalizations within the US”. Also, data from the NIS has been used extensively in medical research to provide population outcome analyses in a variety of healthcare topics. In fact, NIS represents an all-payer sample and is one of the largest and most comprehensive datasets available. Contributing hospitals provide 100% of their discharges, which allows the NIS to be used for volume-outcome calculations as well as population comparisons.⁸ According to Giles et al.,⁸ the NIS database was used to identify patients with rAAA who had an endovascular repair and those who had an open procedure based on ICD-9 codes and to compare outcomes such as “in-hospital mortality, length of stay (LOS), complications, and hospitalization charge” using of combination of methodologies such as SAS for database queries, STATA statistical software for statistical analyses, Wilcoxon rank-sum tests, chi-square tests for categorical variables, Kruskal-Wallis rank, univariate logistic regression, and multivariate logistic regression analysis.

Likewise, in Corticosteroids and Outcome in Children Undergoing Congenital Heart Surgery: Analysis of the Pediatric Health Information Systems Database, Pasquali et al.,⁹ use the Pediatric Health Information Systems Database to “evaluate outcomes associated with corticosteroids in children (0-18y) undergoing congenital heart surgery at 38 American centers from 2003-2008”. Pasquali et al.,⁹ “constructed propensity scores to account for potential confounders: age, sex, race, prematurity, genetic syndrome, type of surgery [Risk Adjustment in Congenital Heart Surgery (RACHS-1) category], center, and center volume. Also, multivariable analysis, adjusting for propensity score, and individual covariates were performed to evaluate outcomes of in-hospital mortality, postoperative length of stay (LOS), duration of ventilation, infection, and use of insulin”.

In another study, Using Geographical Information Systems Mapping to Identify Areas Presenting High Risk for Traumatic Brain Injury, Colantonio et al.,¹⁰ show how “geographical information systems (GIS) can be used to track and compare hospitalization rates for traumatic brain injury (TBI) over time and across a large geographical area using population based data”. To do so, they used the Ontario Trauma Registry Minimum Data Set and Statistics Canada Census Geographic files in mapping and analyzing data on TBI hospitalizations as shown in Figure 2, and geographic and demographic variables using “various visualization techniques, exploratory data analysis and spatial analysis”. In addition, they studied “both the raw and standardized rates by age/gender of the geographical unit”.

Figure 2 GIS maps showing traumatic brain injury outcomes. (Source : Colantonio et al., 2011)

Furthermore, in the study National Variation in United States Sepsis Mortality: A Descriptive Study, Wang et al.,¹¹ analyze national vital statistics data for a period of 1999 to 2005 retrieved from the National Center for Health Statistics’ Compressed Mortality File (CMF), “which contains data on the age, race, sex, year and causes of all US deaths in calculating national and state age-adjusted sepsis-attributed deaths according to the ICD-10. Figure 3 shows the regional variation in sepsis mortality in the United States from 1999 to 2005”. To do so, they “used an analytical approach similar to prior studies of stroke death clusters. We used age-adjusted mortality rates provided by CMF, which adjusts relative to intercensal (1999), actual (2000) or postcensal (2001 to 2005) US Census population estimates”.¹¹

Figure 3 Variation of sepsis mortality outcome in the United States by region. (Source: Wang et al., 2010)

Finally, in Immigrants’ Duration of Residence and Adverse Birth Outcomes: A Population-based Study, Urquia et al.,¹² examine “preterm and small-for-gestational-age (SGA) births among immigrants by duration of residence to compare them with the Canadian-born population” using the Discharge Abstract Database (DAD) of the Canadian Institute for Health Information (CIHI) linked to the Landed Immigrant Data System (LIDS), which is the official immigration registry compiled by Citizenship and Immigration Canada (CIC)”. They are able to do so by measuring “logistic regression models to estimate the effects of duration of residence with adjusted odds ratios and 95% confidence intervals. In analyses restricted to immigrants only, hierarchical models are used to account for the clustering of births into maternal countries of birth”.¹²

Various authors and researchers have intensively concluded that the practice of population heath outcomes research entails the use of high-level information systems to collect, store, and manage relevant outcomes data. In fact, Embi et al.,⁴ agree and write “Indeed, the efficient conduct of outcomes research requires access to robust clinical and population-level data sets, as well as existing research and knowledge data sets, to evaluate relevant metrics such as procedural complications, days of hospitalization, health status, and mortality”. Accordingly, we will use this section of the paper to systematically review the results and findings of these population-based outcomes studies.

Earlier we mentioned how Giles et al.,⁸ in Population-Based Outcomes Following Endovascular and Open Repair of Ruptured Abdominal Aortic Aneurysm, use the NIS database and various statistical methodologies in conducting “population-based outcomes study following endovascular and open surgical repair of ruptured abdominal aortic aneurysms (rAAA)”. Nonetheless we must identify the results and findings that suggest the NIS plays an important role in making the outcomes research possible. In the final results of their study, Giles et al.,⁸ concur that endovascular is a better option compared to open repair for ruptured aneurysms because endovascular provides a lower mortality rate in national population as shown in Table 1. In general, they agree that based upon prior institutional studies and further supported by large database analyses such as this, it is reasonable for hospitals with adequate endovascular repair for ruptured AAA (rEVAR) experience to adopt a rEVAR-first strategy for ruptured aneurysms when conditions allow. Consequently, Giles et al.,⁸ validate the positive utilization of the NIS in completing this study and advance: “To further expand on this work, the current study utilized the Nationwide Inpatient Sample (NIS) in order to analyze national outcomes for in-hospital mortality rates after repair of rAAAs and to assess the impact of procedural volume specifically in the setting of aneurysm rupture”.

Moreover, in the study Corticosteroids and Outcome in Children Undergoing Congenital Heart Surgery: Analysis of the Pediatric Health Information Systems Database, Pasquali et al.,⁹ using methods previously described, utilize the Pediatric Health Information System (PHIS) database to conduct a population-based outcomes research on the impact of “corticosteroids on children undergoing congenital heart surgery”. As a result of that study, Pasquali et al.,⁹ write: “We were unable to demonstrate a significant benefit associated with corticosteroids, and found that corticosteroids may be associated with increased morbidity, particularly in lower risk patients” as shown in Table 2. Later, they agree that the PHIS did have a major impact in the successful completion of the study and declare: “Using the Pediatric Health Information Systems Database we evaluated outcomes associated with corticosteroids in a multi-center cohort of more than 40,000 children undergoing congenital heart surgery from 2003-2008”.

In the third study, Using Geographical Information Systems Mapping to Identify Areas Presenting High Risk for Traumatic Brain Injury, Colantonio et al.,¹⁰ demonstrate “how geographic information systems can be successfully used to examine hospitalization rates for traumatic brain injury using a range of tools and techniques. Findings can then be used for local planning of both injury prevention and post discharge services, including rehabilitation”. In this same context, Colantonio et al.,¹⁰ agree that: “GIS has been used by epidemiologists to investigate associations between environmental exposures to, and the spatial distribution of, infectious disease. GIS research in health and healthcare has primarily relied on government supported databases of vital statistics to visualize mortality and morbidity”. Simply put, they provide a solid argument that confirms how GIS can be a significant tool in population health outcomes research as illustrated in Figure 4.

Figure 4 Representations of maps used in calculaing morbidity outcomes for traumatic brain injury. (Source: Colantonio et al., 2011)

Furthermore, in the study National Variation in United States Sepsis Mortality: A Descriptive Study, Wang et al.,¹¹ using National Center for Health Statistics’ Compressed Mortality File (CMF) as a primary data source, come to the findings that “National age-adjusted sepsis mortality was 65.5 per 100,000 persons (95% CI: 65.8 - 66.0). State level sepsis mortality varied more than two-fold (range 41 to 88.6 per 100,000 persons; median 60.8 per 100,000, IQR 53.9-74.4 per 100,000)”. In all, they agree that “Sepsis mortality varies across the US. The states with highest sepsis mortality form a contiguous cluster in the Southeastern and mid-Atlantic US”.¹¹ That is, they convey a clear understanding on the critical role play by National Center for Health Statistics in conducting that population health outcomes study (as shown in Table 3) because “it represents the only data set aggregating US death incidence and geographic distribution for different disease groups”.¹¹

Lastly, Urquia et al.,¹² in Immigrants’ Duration of Residence and Adverse Birth Outcomes: A Population-based Study, come to the findings that “Recent immigrants (<5years) had a lower risk of preterm birth (PTB) (4.7%) than non-immigrants (6.2%), but those with ≥15years of stay were at higher risk (7.4%). Among immigrants, a 5-year increase in Canadian residence was associated with an increase in PTB (AOR 1.14, 95% CI 1.10–1.19), but not in small-for-gestational-age (SGA) birth (AOR 0.99, 95% CI 0.96–1.02)”. In conclusion, in Urquia et al.,¹² study, it was found that the amount of time since migration leads to “an increase of PTB but not an increase of SGA”. Notably, the Discharge Abstract Database (DAD) of the Canadian Institute for Health Information (CIHI) was influential in making that population-based outcomes study possible as demonstrated by the findings in Table 3 and confirmed by Urquia et al.,¹² statement that “The DAD is an excellent source for population-based estimates of perinatal outcomes” (Table 4).

Although our systematic review of the studies has confirmed the useful role of public health information systems in conducting successful population-based outcomes research, it is important to point out the various limitations of those information systems and outcome data, since the findings of such studies can be affected and their validity questioned. First, in Population-Based Outcomes Following Endovascular and Open Repair of Ruptured Abdominal Aortic Aneurysm, it is critical to focus on issues associated with using outcome data that derives from administrative databases. That it, such utilization can be influenced by the coding inconsistency among different hospitals which can diminish the evaluation of outcome metrics following rAAA such as comorbid conditions and complications.

Second, in Corticosteroids and Outcome in Children Undergoing Congenital Heart Surgery: Analysis of the Pediatric Health Information Systems Database, there are two vital limitations that relate to the PHIS database. The first one, according to Pasquali et al.,⁹ is the inability to assess “the impact of different dosing regimens or exact timing of corticosteroid administration in relation to surgery as this information is not collected in the database”. The second one is the potential inaccuracy of coding of congenital heart surgery in administrative datasets, as in the PHIS.

Third, in Using Geographical Information Systems Mapping to Identify Areas Presenting High Risk for Traumatic Brain Injury, there are common methodological and functional limitations inherent to geographic information systems that can thwart the findings of the study. These can include the inability to provide solutions that are absolute and appropriate for outcomes research, the difficulty to “visualize and explore multivariate data relationships, the inability to control the method of creating neighbor relationships and other parameters for aggregation and spatial clustering analysis, the inefficacity to compare patterns of spatial clustering over time, and the inability to do regression analysis incorporating a spatial component”.¹⁰

Fourth, in National Variation in United States Sepsis Mortality: A Descriptive Study, Wang et al.,¹¹ recognize the limitations of using public “mortality data from the National Center for Health Statistics (NCHS)” for their study because such data “are subject to classification or misattribution bias, which could affect our results”.

Finally, in Immigrants’ Duration of Residence and Adverse Birth Outcomes: A Population-based Study, limitations can be attributed to the fact that “data on some important predictors of the outcomes such as tobacco smoking, alcohol consumption, and maternal height, weight, or body mass index (BMI)” were not used.¹² Here again, because of those limitations, final results of this study can be influenced or biased.

In conclusion, based on what has been demonstrated earlier in this paper, it is obvious that the practice of using public information systems in conducting population-based outcomes research has been recognized and agreed upon by many writers and experts as a useful means for success and improvement in that field.¹³ Remarkably, despite some limitations related to the information systems and outcome metrics used in those studies, our review of methodologies and findings has also shown that public information systems can help improve and widen population health outcomes research into a very advanced, functional, and scientific practice.¹⁴ Notably, future expectations are for researchers and investigators in the field of population health outcomes to take measures that will diminish or abolish biases related to common limitations, which can negatively affect findings of those researches and/or question their validity all together.

None.

The author declares no conflict of interest.

1. O’Carroll PW, Yasnoff W, Ward ME, et al. Public Health Informatics and Information Systems. USA: Springer; 2003.
2. Parrish RG. Measuring Population Health Outcomes. Prev Chronic Dis. 2010;7(4):1545–1151.
3. AHRQ outcomes research fact sheet. 2012.
4. Embi PJ, Kaufman SE, Payne PR. Biomedical Informatics and Outcomes Research Enabling Knowledge–Driven Health Care. Circulation. 2009;120(23):2393–2399.
5. Murray CJL, Lopez AD, Barofsky JT, et al. Estimating Population Cause–Specific Mortality Fractions from in–Hospital Mortality: Validation of a New Method. PLoS Med. 2007;4(11):e326.
6. Mendelson DN, Salinsky EM. Health Information Systems and the Role of State Government. Health Aff (Millwood). 1997;16(3):106–119.
7. Kane RL, Radosevich DM. Conducting Health Outcomes Research. MA: Jones & Bartlett Learning; 2011.
8. Giles KA, Hamdan AD, Pomposelli FB, et al. Population–based Outcomes Following Endovascular and Open Repair of Ruptured Abdominal Aortic Aneurysms. J Endovasc Ther. 2009;16(5):554–564.
9. Pasquali SK, Hall M, Li JS, et al. Corticosteroids and Outcome in Children Undergoing Congenital Heart Surgery: Analysis of the Pediatric Health Information Systems database. Circulation. 2011;122(21):2123–2130.
10. Colantonio A, Moldofsky B, Escobar M, et al. Using Geographical Information Systems Mapping to Identify Areas Presenting High Risk for Traumatic Brain Injury. Emerg Themes Epidemiol. 2011:8(7).
11. Wang HE, Devereaux RS, Yealy MD, et al. National Variation in United States Sepsis Mortality: A Descriptive Study. Int J Health Geogr. 2010:9(9).
12. Urquia ML, Frank JW, Moineddin R, et al. Immigrants’ Duration of Residence and Adverse Birth Outcomes: A Population–Based Study. BJOG. 2010;117(5):591–601.
13. National Quality Measures Clearinghouse. NQMC Domain Framework. 2012.
14. World Health Organization. Preamble to the constitution of the World Health Organization. Geneva; 1946.