Background: The evaluation of the care pathway in oncologic patients is complex, as the components of the decision process are dependent and influenced by each other. The statistical methods proposed in the literature oversimplify this entangled problem.

Objective: The aim of the present study was to propose a new method to evaluate the adherence to guidelines of procedures performed from the diagnostic to the follow-up phase of breast cancer care pathway.

Methods:  On women with incident breast cancer occurring in 2007-09, we calculated 24 health indicators as the proportion of women not performing a particular diagnostic/therapeutic procedure. Indicators were calculated for the 23 hospitals with all data available. We developed a theoretical model organizing the indicators into 4 different domains of the care pathway (Diagnosis; Surgical treatment; Medical treatment; Follow-up), plus one domain for patients' characteristics and one for complications. After excluding highly correlated procedures, the model included 19 indicators. We used the Partial Least Squares Path Model (PLS-PM) approach to summarize the indicators into these domains and investigate the relationship among them.

Results: We found a negative correlation between the two domains including the indicators measuring lack of adherence to diagnostic and surgical guidelines, and a positive correlation between the domain measuring lack of adherence to surgical guidelines and the one including patients’ characteristics.

Conclusion: This is a unique example of PLS-PM application to the evaluation of the care pathway of an oncologic disease, overcoming the limits of the previously applied methods.

Keywords: quality indicators, health care, breast neoplasms, partial least squares path model, guideline adherence

PLS-PM,partial least squares (or projection to latent structures) path model; LHA, local health authorities; VIF, variance inflation factor

To evaluate if what extent the care delivered to oncologic patients − such as woman with breast cancer − is adherent to evidence-based guidelines, it is necessary to consider multiple aspects of the diagnostic, therapeutic and follow-up pathway.¹ In fact, single health indicators are informative variables which allow to concisely evaluate a single aspect of complex phenomena. However, single indicators do not give a complete view of the cure paths and their appropriateness or adherence to the guidelines. Coherent sets of indicators have been then developed from evidence-based guidelines to measure the different aspects of the clinical pathway suitable for a patient with specific characteristics, such as comorbidities. Nevertheless, it is difficult to simultaneously evaluate many different indicators and a single summary measure is deemed necessary, but obtaining a methodologically sound and easy to interpret measure represents a challenge.^2,3

In the literature, different approaches have been proposed.^4,5 Some are along the line of providing a simple summary indicator that can be easily understood from health professionals and patients, such as using the proportion of indicators met by each patient or the all-or-non approach.⁶ These approaches have advantages if we want to promote a widespread use of a quality assessment tool. On the other side, they oversimplify an entangled problem. The evaluation of the entire diagnostic and therapeutic pathway is complex, as it involves elements of the decision process that cannot be directly observable or measurable. In addition, the components of the decision process are dependent and influenced by each other. For these reasons, other statistical methods are needed. One proposed approach uses a hierarchy or selection of the most relevant indicators, with respect to the outcome, and a system of weights to combine them. Although this approach better reflects complexity, the assigned weights are fairly arbitrary.^5,7 Other proposed methods to summarize a set of indicators into a single or a few measures use latent variables.^8,9

The aim of the present study is to evaluate the appropriateness of the procedures performed in each step of the breast cancer care pathway (i.e., diagnosis, surgery, medical treatment, and short-term follow-up) through the estimation of a summary measure for each of them, and to investigate their relationships. For this purpose, we used a Partial Least Squares (or Projection to Latent Structures) path model (PLS-PM) approach.¹⁰ The cohort under study is constituted of incident breast cancer cases occurring between 2007 and 2009 in six local health authorities (LHA) of Lombardy, Northern Italy.

Population: We moved from all incident breast cancer identified by the cancer registries occurring, between 2007 and 2009, in the geographic area corresponding to six LHA of Lombardy, Northern Italy, including Milan, Milan 1, Milano 2, Monza and Brianza, Bergamo, and Cremona (5,320,272 inhabitants on 31 December 2012).¹¹ For this cohort, a set of 22 indicators, measuring adherence to breast cancer international guidelines, had been previously developed based on a literature review^11-15 and calculated from cancer registries and administrative health databases.¹¹ To those, using the same methodology, we added 2 indicators: the proportion of patients not undergoing breast surgery, and the proportion of patients undergoing surgery more than 30 days from mammography.¹⁶ The 24 indicators were re-organized into four different domains of the care pathway: Diagnosis, Surgical treatment, Medical treatment, and Follow-up. We also calculated the proportion of patients experiencing three type of complications: developing lymphedema within two years from breast surgery, experiencing side effects requiring hospitalization during chemotherapy, and experiencing haematological side effects requiring hospitalization during chemotherapy.^17,18 Each patient was assigned to the hospital where she performed primary surgery or medical treatment, even if subsequent care was performed elsewhere. To perform PLS-PM analysis on a per-health provider basis, we calculated, for each provider and for all indicators, the proportion of patients not performing the diagnostic or therapeutic procedure. Thus, we considered in the PLS-PM analysis only the patients assigned to the providers for which all indicators included in the final model were available, as not all health providers performed all procedures, e.g. not all health providers have a radiotherapy unit.

Statistical analysis: PLS-PM¹⁰ is a methodology meant to estimate a network of causal relationships defined according to a theoretical model and to represent the causal relationships through a graph, called path diagram. The complexity of the theoretical construct is studied taking into account the relationships among non-measurable concepts (latent variables or domains), represented by a set of observed variables (manifest variables or indicators).

As in a path model approach, variables are grouped in two classes: 1) those that are caused by one or more variables (endogenous or dependent variables), and 2) those that are not caused by any other variables in the diagram (exogenous or independent variables). Moreover, PLS-PM involves two types of models: 1) the structural or inner model specifies the relations between the latent variables, i.e. between each endogenous latent variable and the other latent variables; 2) the measurement or outer model takes into account the relations between a latent variable and its corresponding manifest variables. Different types of measurement models exist, depending on the kind of relation between manifest variables and latent variable: reflective (manifest variables are an effect of the the unique corresponding latent variable) and formative models (latent variable is considered as being caused by its observed variables).

The PLS-PM approach is characterized by an iterative procedure of ordinary least squares regressions taking into account the relations of the measurement and structural model, to calculate weights required to give final estimates of each latent variable. In detail, the iterative procedure in PLS-PM works on standardised manifest variables and begins with an initial approximation of each latent variable as weighted sum of its manifest variables by using arbitrary outer weights (measurement model). Then, the relations among latent variables (structural model) are considered in order to obtain a proxy of each latent variable calculated as a weighted sum of its connected latent variables by using a weighting scheme (e.g., centroid, factorial, path). Subsequently, the algorithm turns around to the approximation of the measurement model updating the arbitrary initial weights with new ones, as regressions coefficients, through different ways depending how the manifest variables are related to their latent variables, e.g., by mode A for reflective construct, mode B for formative. The first implies simple linear regressions (the manifest variables are considered as dependent variables and each latent variable as independent) whereas the second implies multiple linear regressions (each latent variable is considered as dependent variable and the manifest variables as independent ones). The algorithm goes ahead iteratively until convergence of the weights is reached. After convergence of the outer weights (interpreted as a proxy for the relevance of each manifest variable in the construction of the latent variable), the latent variables are estimated as weighted sum of its observed variables. Thus, the path coefficients are calculated by ordinary least squares regressions between latent variables and are interpreted as standard regression coefficients.¹⁹

In the present study, we performed a PLS-PM based on the aggregated data of hospitals. We constructed the model including all available indicators by means of clinical expert assessment and review of the literature, making a-priori assumptions on the relationships between the defined domains. We used a formative model, since the latent variable is considered as being caused by its manifest variables and is defined as a linear combination of the corresponding manifest variables in our model. We applied the centroid scheme proposed by Wold, which considers the sign of the correlation between two connected latent variables. Unlike the reflective measurement model, the formative model does not assume homogeneity nor uni-dimensionality of the block and therefore the traditional validity (e.g., composite reliability, average variance extracted) assessments do not apply to the formative model. However, we checked for highly correlated indicators in order to avoid collinearity related issue in the formative measurement model and we subsequently excluded 5 indicators as they measured highly correlated procedures within each domain. Moreover, we assessed the level of collinearity of the final formative measurement model performing the variance inflation factors (VIFs), i.e., the reciprocal of the tolerance, where the tolerance is $1-R_{x_1}^2$  and the $R_{x_1}^2$ is the proportion of variance of a manifest variable $x_1$  explained by the others of the same block. In order to obtain $R_{x_1}^2$  we regressed one manifest variable $x_1$  (as dependent variable) on all remaining manifest variables of the same domain (as independent variables). A VIF value ≥5 indicates a potential collinearity problem.²⁰ We evaluated the quality of the structural model through the assessment of collinearity among domains and the R² determination coefficients.²⁰ In order to obtain the R², we regressed the scores of one domain (as dependent variable) on all connected exogenous domains (as independent variables).²⁰ The R² statistic is interpreted as the amount of variance in the endogenous domain explained by its exogenous domains. The R² values range from 0 to 1 and the explanation of the variance is described as substantial, moderate or weak with reference to thresholds of R2 above to 0.67, 0.33, and 0.19, respectively.²¹

We performed means and corresponding 95% confidence intervals for weights, loadings, and path coefficients using bootstrap technique where 5000 subsamples with replacement from the original dataset were drawn.²⁰

The indicators were calculated with SAS 9.3, while PLS-PM analysis was conducted using the plspm package in R software.²²

Descriptive: The cohort included 9614 incident breast cancer women, not metastatic at diagnosis, who received surgery/primary medical treatment from 62 different health providers in the territory of the six LHA. After defining the model, we restricted the analysis to subjects (n=6435) who had primary surgery/medical treatment in one of the 23 hospitals for which all the 19 selected indicators were available. About 24% of cases were aged ≤50 years, while about 4% were aged ≥85 years. Less than one third of cases had a radical surgical treatment (i.e. mastectomy). About 4% of the cases were not treated with surgical procedures. Eighty-two percent of women were in TNM stage I or II. The distribution of age, stage, treatment type and comorbidities did not differ from the whole cohort (Supplementary Table 1).¹¹

PLS-PM: We considered a model splitting up the process indicators into 4 different domains, plus one domain for patients’ characteristics and one for complications. The domains included indicators measuring lack of adherence to a recommended procedure for primary non-metastatic breast-cancer in different phases of the diagnostic and therapeutic pathway: Diagnosis; Surgical treatment; Medical treatment; Follow-up. Moreover, the Disadvantage domain included patients’ age and comorbidities, characteristics related to a lower probability of adherence to diagnostic and therapeutic guidelines on primary breast cancer.²³ The last domain (Complications) included indicators measuring the proportion of patients experiencing a specific complication from surgical or medical treatment. After removing highly correlated indicators, the final model consisted of 19 process indicators measuring adherence to international guidelines (Diagnosis: n=4; Surgical treatment: n=7; Medical treatment: n=6; Follow‑up: n=2), 3 indicators for complications, and 2 indicators for patient characteristics. A detailed description of the model, including domains and indicators, is provided in Table 1.

We a-priori assumed that the domain Disadvantage was connected to all the other domains;^24-26 Diagnosis was joined to Surgical and Medical treatments, and Follow-up [27-29]; Surgical treatment was related to Medical treatment and Follow-up [28,30-32]; Medical treatment was connected to Complications and Follow-up.^32-34 A detailed description of the rational of the links between domains of the structural model is reported in Table 2.

Figure 1 represents the hypothesized structural model of care pathway for breast cancer. Each domain is identified by a name that appears inside of an ellipse; the relations between domains, defined a priori, are represented by arrows. The statistics given in Figure1 represent the correlation coefficients between domains. The correlation coefficients (r) with an absolute value > 0.5 were those between: Diagnosis and Surgical treatment (r=-0.72), meaning that hospitals with a low adherence to the measured diagnostic procedures for primary breast cancer have a high adherence to the measured surgical standards for primary breast cancer, and vice versa; Disadvantage and Surgical treatment (r=0.72), suggesting that hospitals with a high proportion of elderly patients and chronic cardiovascular diseases and/or diabetes at diagnosis have a lower adherence to the measured surgical recommended procedures for primary breast cancer; Disadvantage and Diagnosis (r=-0.68), indicating on the contrary that hospitals with a high proportion of patients with advanced age and/or comorbidities have a higher adherence to the measured recommended diagnostic procedures; Medical treatment and Complications (r=-0.61), suggesting that hospitals being less adherent to measured guidelines for medical treatment have lower complications rate, a finding that will need further investigation; Medical treatment and Follow-up (r=-0.52), implying that hospitals being less adherent to the measured guidelines for follow-up have a higher compliance to medical treatment ones.

In the structural model graph, the path coefficients are usually showed; however, since we found them easier to interpret, we showed the correlations between domains in the graph and reported the standardized path coefficients for direct and total effects among domains in Supplementary Table 2.

Table 3 shows the outer weights and loadings of the PLS-PM in the original dataset and their mean values and corresponding 95% confidence intervals performed through bootstrap procedure. The weight represents the relevance of each indicator in the construction of the corresponding domain (indicator’s relative contribution). Considering the bootstrap procedure results, comorbidity was the most important indicator to the Disadvantage domain definition, absence of screening mammography (D2) for the Diagnosis, not undergoing to surgical treatment (S1) for Surgical treatments, not undergoing radiotherapy after conservative surgery (M4) for Medical treatment, hematological side effects (C3) for complication, and inappropriate intensive investigations (F2) for follow-up. No weight differ significantly from zero, probably because in small sample sizes standard errors are generally larger due to sampling error.²⁰ The loading represents the correlation between each indicator and domain (indicator’s absolute contribution). In a formative measurement model, the loadings are meaningless, but they are useful for the decision-making process of keeping or deleting formative indicators: if the weight is not significant, you should analyze the loading, i.e., if the loading is ≥0.5 you should keep the indicator, if the loading is <0.5 you should consider to remove it, if the loading is significantly <0.5 you should delete it. Anyhow, since the theory-driven formulation of the domain supports retaining the indicators (i.e., from a clinical point of view), they were kept in our model.²⁰

Figure 1 Structural model of the care pathway of breast cancer patients. The values represent the correlation coefficients between the domains performed according to the Partial Least Squares Path Modelling approach.

Figure 2 represents the scores of the main domains of breast cancer care pathway for each of the included hospital (belonging to 6 LHA) against the average yearly breast surgical hospital volume, allowing to examine differences across hospitals (and within LHA) for each domain in a single plot, and for the entire clinical pathway in four plots. Differences between LHA was present for all domains, roughly having the same magnitude. From the graph, for the diagnostic domain high volume providers appear to have been less adherent to guidelines than small volume ones. On the contrary, for the surgical domain they showed greater adherence to standards. No relation was evident between medical treatment or follow-up and volume.

Concerning the quality of the formative measurement model, we checked for the collinearity among the indicators through VIF in the final model (Supplementary Table 3). The highest VIF value was 3.74 and was observed for the indicator S3 (proportion of patients not undergoing SLNB in the setting of breast conserving surgery for T1 tumors). Hence all VIF values were below the threshold of 5, indicating that collinearity did not reach critical levels in any of the formative construct. Concerning the quality of the structural model, we checked for the collinearity among the domains through VIF (Supplementary Table 4) and we found no significant collinearity, being all VIF values below 5. Moreover, the structural model was approximately satisfactory for all the endogenous domains (R²=0.61 for Surgical treatment, 0.52 for follow-up, 0.46 for Diagnosis, 0.37 for Complications, and 0.27 for Medical treatment).

Figure 2 Scores of the main domains (calculated through the Partial Least Squares Path Modelling approach) of breast cancer care pathway for 23 hospitals in 6 areas of Lombardy, Italy. Hospitals with more than 300 cases/year are all plotted at 300.

In the present study we evaluated the appropriateness of the procedures performed in each step of the breast cancer care pathway through indicators measured in a cohort of breast cancer cases in Northern Italy. In a previous study, we considered each of the developed indicators singularly, using multilevel regression models.¹¹ However, the global interpretation of a care pathway, constituted of diagnosis, treatment and follow-up procedures, is difficult when the number of indicators is high. Thus, in the present study, using the PLS-PM approach, we summarized a large set of indicators with a few numbers, one for each step/domain of the primary breast cancer pathway of care. To our knowledge, the present study is the first one using PLS-PM to summarize indicators of a clinical care path.

Summarizing a long list of indicators in only few numbers for each hospital allowed us to compare more easily the appropriateness of the performed procedures across them, without oversimplifying the comparison to a single number. In fact different steps of the care pathway, which depend on different sub-organizations, may be more or less adherent to guidelines within a provider. We were also able to explore the relations among the different domains. We found a strong positive correlation (r=0.72) between low adherence to the measured surgical guidelines and disadvantage, which may be due to the known tendency not to adhere to guidelines in older patients.²³

There was also a strong inverse correlation between lack of adherence to the measured guidelines for surgical procedures and lack of adherence to the included diagnostic procedures. These findings do not have to be interpreted as ‘hospital with surgeon respecting guidelines have radiologist that do not’ or vice-versa, but can be used to explore more in detail the associations between indicators in different domains, after having examined which are the indicators that contribute the most to the latent variable.

In the literature, there is a debate on the opportunity to use the so called ‘composite measures’ to summarize complex processes in healthcare, such as the pathway of care for a chronic condition. The importance of giving the possibility to clinicians and patients to have an easy picture of the relative level attained by different geographic areas or providers has to be weighed against the risk of introducing biases related to the method use to obtain the ‘composite measures’.^2,35-37 It has been shown that different methods lead to substantially different results, mainly because of selection of which indicators should be included in the composite measure, how weights are assigned to the different indicators, which rule is used to obtain the ‘composite indicator’ (e.g. sum, average, decision rule) especially because of the effect on variability.

The main advantage of the proposed method is that, once the model describing the relations between the different domains has been hypothesized based on clinical knowledge and guidelines, the weights attributed to the different aspects of the pathway of care (domains) are not assigned on an arbitrary basis, as in the current literature methods, but it is assigned on the basis of the observed data through the PLS algorithm. Moreover, the domain (calculated as the weighted sum of the corresponding indicators) is summarized and analysed in such a way to consider the relations with the other connected domains, thus considering the whole pathway of care.

We chose to use PLS-PM due to its flexibility since, on contrary of other structural equation models, PLS does not require specific distributional assumptions of the data, independence of observations, or large sample sizes.

The limit of this method – as well as of all the structural equation models – is the subjectivity in the definition of the domains and of the relations. From this point of view, the model herein considered is only one of the possible models. Also, PLS requires a certain degree of statistical knowledge, nevertheless it can be conveyed to the general public through graphical representation, for example of the relative level attained by different hospitals in each domain. The major limitation of our analysis is the small sample size, as we performed the PLS-PM analysis on aggregate data of 23 hospitals. This implies that the model may produce unstable estimates. However, our purpose was to show the feasibility and utility of this method to summarize a clinical pathway, more than the interpretion of the results.

In conclusion, we were able to summarize the distance between guidelines and clinical practice of the different steps of the care pathway of breast cancer patients and to explore the relationships among them. This approach helps to identify the single aspects of the care pathway that need to be subject to quality improvement initiatives for each provider, and how these are expected to influence other domains. It also allows an easier comparison among providers than on a ‘per indicator’ basis.

The authors thank Prof. Vincenzo Esposito Vinzi for helpful suggestions on PLS-PM analysis.

The authors declare that they have no conflict of interest.

Funding of this work was provided by the Italian Ministry of Health (project code WFR RF-2011-02348959). The sponsor had no role in the design, implementation, analysis, and interpretation of the data.

1. Kolfschoten NE, Gooiker GA, Bastiaannet E, et al. Combining process indicators to evaluate quality of care for surgical patients with colorectal cancer: are scores consistent with short‒term outcome? BMJ Qual Saf. 2012;21(6):481‒489.
2. Jacobs R, Goddard M, Smith PC. How robust are hospital ranks based on composite performance measures? Med Care. 2005;43(12):1177‒1184.
3. Smith P. Developing composite indicators for assessing health system efficiency. In Measuring Up Improving Health System Performance in OECD Countries: Improving Health System Performance in OECD Countries. Edition Paris: OECD. 2002;295‒313.
4. Reeves D, Campbell SM, Adams J, et al. Combining multiple indicators of clinical quality: an evaluation of different analytic approaches. Med Care. 2007;45(6):489‒496.
5. Traberg A, Jacobsen P, Duthiers NM. Advancing the use of performance evaluation in health care. J Health Organ Manag. 2014;28(3):422‒436.
6. Nolan T, Berwick DM. All‒or‒none measurement raises the bar on performance. JAMA. 2006;295(10):1168‒1170.
7. Chen LM, Staiger DO, Birkmeyer JD, et al. Composite quality measures for common inpatient medical conditions. Med Care. 2013;51(9):832‒837.
8. Grunkemeier GL, Wu Y. What are the odds? Ann Thorac Surg. 2007;83(4):1240‒1244.
9. Shwartz M, Ren J, Peköz EA, Wang X, et al. Estimating a composite measure of hospital quality from the Hospital Compare database: differences when using a Bayesian hierarchical latent variable model versus denominator‒based weights. Med Care. 2008;46(8):778‒785.
10. Wold H. Soft modeling: the basic design and some extensions. In: Jöreskog KG, Wold H, editors. Systems under Indirect Observation: Causality, Structure, Prediction. Edition North‒Holland: Amsterdam, Netherlands. 1982.
11. Andreano A, Anghinoni E, Autelitano M, et al. Indicators based on registers and administrative data for breast cancer: routine evaluation of oncologic care pathway can be implemented. J Eval Clin Pract Epub ahead of print. 2015.
12. Stordeur S, Vrijens F, Beirens K, et al. Quality indicators in oncology: breast cancer. 2010.
13. Eusoma. The requirements of a specialist breast unit. European Journal of Cancer. 2000;36(18):2288‒2293.
14. Aebi S, Davidson T, Gruber G, et al. Primary breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow‒up. Annals of Oncology. 2011;22(suppl 6):vi12‒vi24.
15. Burstein HJ, Prestrud AA, Seidenfeld J. American Society of Clinical Oncology clinical practice guideline: update on adjuvant endocrine therapy for women with hormone receptor‒positive breast cancer. J Clin Oncol. 2010;28:3784‒3796.
16. Ferrua M, Couralet M, Nitenberg G, et al. Development and feasibility of a set of quality indicators relative to the timeliness and organisation of care for new breast cancer patients undergoing surgery. BMC Health Serv Res. 2012;12:167.
17.  Hassett MJ, O'Malley AJ, Pakes JR, et al. Frequency and cost of chemotherapy‒related serious adverse effects in a population sample of women with breast cancer. J Natl Cancer Inst. 2006;98(16):1108‒1117.
18. Lee TS, Kilbreath SL, Refshauge KM, et al. Prognosis of the upper limb following surgery and radiation for breast cancer. Breast Cancer Res Treat. 2008;110(1):19‒37.
19. Vinzi VE, Chin WW, Henseler J, et al. Handbook of Partial Least Squares: Concepts, Methods and applications. Berlin: Springer Berlin Heidelberg. 2010.
20. Hair JFJ, Hult GTM, Ringle CM, et al. A Primer on Partial Least Squares Structural Equation Modeling (PLS‒SEM). Thousand Oaks: Sage. 2014.
21. Chin WW. The partial least squares approach to structural equation modeling. In: Marcoulides GA, editor. Modern Methods for Business Research. Edition Mahwah, NJ: Lawrence Erlbaum Associates. 1998.
22. Partial Least Squares Path Modeling (PLS-PM) analysis for both metric and non-metric data, as well as REBUS analysis.
23. van de Water W, Bastiaannet E, Dekkers OM, et al. Adherence to treatment guidelines and survival in patients with early‒stage breast cancer by age at diagnosis. Br J Surg. 2012;99:813‒820.
24. Weggelaar I, Aben KK, Warlé MC, et al. Declined Guideline Adherence in Older Breast Cancer Patients: A Population‒Based Study in The Netherlands. The Breast Journal. 2011;17:239‒245.
25. Hancke K, Denkinger MD, König J, et al. Standard treatment of female patients with breast cancer decreases substantially for women aged 70 years and older: a German clinical cohort study. Ann Oncol. 2010;21(4):748‒753.
26. van de Water W, Bastiaannet E, Dekkers OM, et al. Adherence to treatment guidelines and survival in patients with early‒stage breast cancer by age at diagnosis. British Journal of Surgery. 2012;99(6):813‒820.
27. Senkus E, Kyriakides S, Ohno S, et al. Primary breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow‒up. Ann Oncol. 2015;26(Suppl 5):v8‒v30.
28. Boughey JC, Suman VJ, Mittendorf EA, et al. Sentinel lymph node surgery after neoadjuvant chemotherapy in patients with node‒positive breast cancer: the ACOSOG Z1071 (Alliance) clinical trial. Jama. 2013;310(14):1455‒1461.
29. The benefits and harms of breast cancer screening: an independent review. Lancet.2012;380(9855):1778‒1786.
30. Surgical guidelines for the management of breast cancer. Eur J Surg Oncol. 2009;35(Suppl 1):1‒22.
31. Houssami N, Macaskill P, Marinovich ML, et al. Meta‒analysis of the impact of surgical margins on local recurrence in women with early‒stage invasive breast cancer treated with breast‒conserving therapy. Eur J Cancer. 2010;46(18):3219‒3232.
32. Cancer NCCf. Early and locally advanced breast cancer: diagnosis and treatment. In Edition NICE. 2009.
33. Lohrisch C, Paltiel C, Gelmon K, et al. Impact on survival of time from definitive surgery to initiation of adjuvant chemotherapy for early‒stage breast cancer. J Clin Oncol. 2006;24(30):4888‒4894.
34. Swisher SK, Vila J, Tucker SL, et al. Locoregional Control According to Breast Cancer Subtype and Response to Neoadjuvant Chemotherapy in Breast Cancer Patients Undergoing Breast‒conserving Therapy. Ann Surg Oncol. 2015;23(3):749‒56.
35. Chen TT, Lai MS, Lin I, et al. Exploring and comparing the characteristics of nonlatent and latent composite scores: implications for pay‒for‒performance incentive design. Med Decis Making. 2012;32(1):132‒144.
36. Nitenberg G, Couralet M, Le Vaillant M, et al. Precision of composite performance scores: weighting systems impact the results. Med Care. 2013;51(9):854 p.
37. Stausberg J, Halim A, Färber R. Concordance and robustness of quality indicator sets for hospitals: an analysis of routine data. BMC Health Serv Res. 2011;11:106 p.