The objective of this study was to determine adherence to the 2010 Consolidated Standards of Reporting Trials (CONSORT) standards for reported randomized controlled trials (RCTs) assessing immunosuppressive medications in the kidney transplant population between 2007 and 2013. The study period was divided into two arms, 2007-2010 and 2010-2013. RCTs were extracted from 3 major databases, and adherence to CONSORT 2010 was evaluated using the CONSORT score. Of the 540 RCTs extracted, 110 were selected for the study, comprising 55 RCTs for each study period. The median CONSORT score was 17. Significant differences between the scores were detected for the periods 2007-2010 and 2010-2013 (p < 0.02). No difference in scores was detected when comparing the RCTs involving induction and maintenance therapy (H statistic = 171.5 and p = 2.8). Regression analysis revealed that the year of publication exhibited a significant relationship with the CONSORT score, but no relationship was observed between the year of publication and the number of authors, the impact factor of the journal or the country of publication. Despite recent improvements, significant underreporting of CONSORT items for randomization, allocation concealment, blinding and maintenance, changes of trial protocol, trial registration and accessibility of the full trial protocol was detected.

Keywords: Kidney transplantation; Immunosuppressive medications; RCTs; CONSORT

CONSORT: Consolidated Standards of Reporting Trials; IRB: Institutional Review Board; KTx: Kidney Transplantation; MeSH: Medical Subject Headings; SOT: Solid Organ Transplantation

Clinical trials are conducted to provide safe and effective treatment, which can maximize benefit and minimize risk. Following systematic reviews, randomized controlled trials (RCTs) represent the pinnacle of the pyramid-of-evidence hierarchy [1]. The decision to use a particular immunosuppressive agent is based on the results of clinical trials. It is imperative that trials are reported in a systematic manner that reflects the study’s methodology and reveals the framework of the trial, which ultimately reflects the safety and efficacy of a treatment. Different formats for the reporting of clinical trials have been introduced. For RCTs, the CONSORT guidelines, last revised in 2010, have surged in popularity. Peto et al. [2] in 1995 found it difficult to assess the reporting quality of clinical trials and suggested a need of structured assessment tool. In 2001 Junni et al. [3] after the revision of CONSORT guidelines; they pointed out the Randomization, Blinding and Allocation concealment are topics that require attention as they are under reported.

Many investigators and reviewers in various fields of research have assessed its implementation. In 2006 Karri et al. [4] evaluated the reporting standards of RCTs in the field of Surgery. Karri’s results showed significant non-compliance to standards of reporting set by CONSORT. His results also highlighted lapses in reporting of some critical CONSORT Items particularly ‘Sample size calculation’ and ‘Randomization’ along with ‘Blinding’. Another Item that raised concerns by Momeni et al. [5] in 2009 was reporting of ‘Funding’. Study published by Saman et al. [6] in 2013 reported that the standards of reporting of RCTs along with other types of trials are poor and emphasized the need for better implementation of CONSORT when reporting RCTs. Fritsche et al. [7] from Germany evaluated the quality of reporting of RCTs involving KTx and Immunosuppressive medications in 2004, using Jadad scale. Their study showed relationship between the quality of reporting and the ’Impact factor’ of publishing journals. Pengel and Liu et al. [8,9] have assessed adherence to CONSORT guidelines of RCTs conducted on all types of SOTs.

Literature review was done to compare adherence to CONSORT 2010 guidelines between the periods 2007 – 2010 and 2010 – 2013 by using the CONSORT score, which is calculated by summing all of the items of CONSORT 2010 that demonstrated compliance to the guidelines. Limited data exist on the adherence to CONSORT guidelines in the field of kidney transplantation. The scarcity of data led to the design and execution of this extensive literature review regarding kidney transplantation and immunosuppressive medications. The study also evaluated the relationship between the type of immunosuppressive medication used in the trial, i.e., induction vs maintenance, and the adherence to CONSORT standards using the CONSORT score derived from each RCT. The study also explored whether other factors as suggested by previous reviews, i.e., the year of publication, the number of authors involved, the impact factor of the publishing journal and the country of publication of the study, demonstrated any relationship with the adherence to CONSORT guidelines. Individual CONSORT items were also compared between the two study periods to identify which items were reported more appropriately than others.

Study design

A literature review was conducted of all of the relevant published RCTs from established databases to create a retrospective cohort literature report.

Screening of RCTs from databases

RCT extraction was performed from three major web-based databases reporting RCTs: Excerpta Medica dataBASE (EMBASE), MEDLINE and the Cochrane central library for controlled clinical trials.

Search technique

After accessing the databases, the investigator used Medical Subject Headings (MeSH) terms to conduct an advanced search for the relevant articles that were included in the study. In the advanced search option of each database, the following MeSH terms were used: kidney transplant, kidney transplantation, immunosuppressive agents, immunosuppressive therapy, randomized trials and randomized controlled trials. Additionally, a specified time period was also entered in the search: January 2007 to December 2013 and the search was confined to articles published in the English language. RCTs conducted on pediatric to adult patients were included.

Selection of RCTs for the study

After the removal of duplicate articles among the databases, the investigator compiled a list of articles that required individual screening according to the study inclusion and exclusion criteria. Screening led to the exclusion of many RCTs, most of which involved double-organ transplants. Others were excluded because they were cohort, non-randomized or cross-sectional studies. The final list of RCTs was then divided into two arms: one arm constituting studies published from the beginning of 2007 until the end of June 2010 and the other arm containing studies published from July 2010 until the end of 2013 (Figure 1).

Figure 1: Screening of RCTs.

Abstraction and scoring of RCTs

The data were collected from the selected articles by the investigator. RCTs were thoroughly reviewed, and the demographic data from each article were then recorded. The following information was collected from each article: Article name, Number of authors, Year of publication, Publishing journal, Impact factor of the journal, Country of origin of the article, Type of immunosuppression used (induction/maintenance), Single- or multi-centered. Each RCT was then scored based on the 25 items of the revised 2010 CONSORT guidelines. Of the contained 25 items in the guidelines, items 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17 and 18 are subdivided into ‘a’ and ‘b’. Each item in the CONSORT guideline was dichotomously rated (described or not described) to generate a CONSORT score. The items were scored as 1 if the requirement was met and 0 if it was not. However, the subdivided items were scored as 0.5 if the requirement was met and 0 if it was not. Thus, for subdivided items, if the requirements of both a and b were met, the item was scored as 1. Therefore, each study was scored out of a total possible score of 25.

Two independent reviewers cross-checked the scoring of articles and any discrepancy between the two was resolved by the investigator.

Data analysis

Descriptive statistics were used to report the demographic characteristics of the RCTs selected for the study. The CONSORT score was calculated by summing all of the items for each study. The median CONSORT score was calculated for the two periods (2007-2010 and 2010-2013) and compared by applying the Mann-Whitney test. The Kruskal-Wallis test was used to compare the CONSORT score and to report the difference in the types of immunosuppressive used, i.e., induction vs maintenance. Regression analysis was used to probe any association between the CONSORT score and the number of authors involved, the impact factor of the journal, the year of publication of the articles and the country publishing the articles. The proportions of each CONSORT item reported were also individually compared for the two study periods using the chi-square test. Statistical significance was defined as p ≤ 0.05, and the data were analyzed using the statistical software STATA v11.1.

Of the initial 540 articles, 259, 161 and 120 articles were extracted from EMBASE, MEDLINE and the Cochrane library, respectively. Duplications accounted for the exclusion of 68.5% of the articles. Of the remaining 170 articles, 6% were not accessible, leaving 161 articles that were subjected to screening according to the eligibility criteria. Of these 161 articles, 110 satisfied the eligibility criteria, mandating the exclusion of 51 articles. The PRISMA flow diagram in (Figure 2) elaborates the study selection and exclusion process. Baseline characteristics of the included articles are represented graphically in (Figure 3-9). The medians of the two periods were compared using the Mann-Whitney test. The results yielded an adjusted variance of 27,874.15 with a statistically significant p value of 0.02. The bar graphs comparing the two study periods are presented in (Figure 10).

Figure 2: Prisma flow diagram of the study.

Figure 3: Pie Graph showing Percentage of RCTs published yearly.

Figure 4: Bar graph showing Types of Immunosuppressive tested in RCTs yearly.

Figure 5: Stacked bar graph showing centers involved in RCTs Yearly.

Figure 6: Bar graph showing nations publishing RCTs Yearly.

Figure 7: Bar graph showing Number of Authors involved in RCTs yearly.

Figure 8: Line graph showing relationship of Impact factor of journals publishing RCTs yearly.

Figure 9: Bar graph showing the major journals publishing RCTs yearly.

Figure 10: Bar graphs showing CONSORT SCORES of RCTs conducted from 2007-2010 and 2010-2013 with Line representing Mean of CONSORT SCORES.

Kruskal Wallis test showed H statistic was 171.89 with an insignificant p value suggesting no association between the therapy types in the reporting RCT and their respective CONSORT SCORES. Regression analysis was also conducted to ascertain any association between 4 factors (the year of publication, the number of authors involved, the impact factor of the reporting journals and the country) and the CONSORT score. The P value for the year of publication was 0.02 (95% CI 0.05-0.70), whereas the p value for the number of authors involved was 0.19 (95% CI -0.36-1.76). For the impact factor of the publishing journals, the p value was 0.07 (95% CI -0.014-0.31), and for the country of the RCT, the p value was 0.27 (95% CI -1.45-0.41). Secondary analysis only revealed an association between the year of publication and the CONSORT score.

Each item of the CONSORT 2010 guideline was compared individually between the selected periods (2007-2010 and 2010-2013). The following items were found to differ significantly between the selected periods: item 6b, item 10, item 11a, item 23 and item 24. For item 10, 16% of the RCTs in 2007-2010 exhibited adherence vs 0% in 2010-2013 (p value 0.001; 95% CI 0.06-0.28). Similarly, for item 11, 91% of the 2007-2010 RCTs exhibited adherence to the CONSORT 2010 guideline vs 0% for 2010-2013 (p value <0.002; 95% CI 0.31-0.58). In contrast, for item 6b the adherence was 0% and 9% for 2007-2010 and 2010-2013, respectively (p value <0.0002; 95% CI 0.31-0.58) and similarly, for item 23 the adherence was 29% and 67%, respectively (p value <0.0002; 95% CI 0.19-0.53), and for item 24, the adherence was 9% and 33%, respectively (p value 0.002; 95% CI 0.08-0.37). Overall, adherence to 75.1%-100% of the items was 48.6% in both periods, whereas adherence to 50.1%-75% of the items was 16% vs 24% for the periods 2007-2010 and 2010-2013, respectively. For the respective periods, an adherence level of 25.1%-50% was observed for 13% vs 10% of the trials, with an adherence level of 0-25% for 21% vs 16% of the trials in the two periods, respectively.

The results of this study may have several implications, particularly regarding RCTs conducted in the field of kidney transplantation and immunosuppressive medications. Although there has been an improvement in the adherence to CONSORT 2010 standards of reporting of RCTs, many gaps also exist that require attention. To generate quality reports, authors must comply with the guidelines and report as many items as possible. This action will assist experts and practitioners in extracting evidence from RCTs without the need to repeat RCTs and waste resources. This study compared RCTs before and after the CONSORT 2010 revision with the objective of establishing adherence to the CONSORT 2010 guidelines. Furthermore, this study compared the level of adherence between the 2007-2010 and 2010-2013 periods. These objectives were achieved by comparing the individual items of CONSORT 2010 and by comparing the CONSORT scores of the RCTs before and after CONSORT revision.

The demographics of the included RCTs highlighted several important issues. First, we observed a steady increase in the number of published RCTs, most of which were conducted in the US and European countries. The majority of the RCTs was performed to test maintenance therapies and was multi-center trials. The RCTs were predominantly published in two journals, Transplantation and the American Journal of Kidney Transplantation. A considerable increase in the number of authors involved in the RCTs was also evident, a trend that may continue. It was also important to compare each item of the guideline individually and to observe the differences and lapses in the reporting of RCTs. This comparison provided crucial information about the standards of reporting. Considerable differences in the reporting of several items were detected, which included items 6b, 10, 11a, 23 and 24, whereas the remainder of the items was comparable. As a result of these differences between items, a significant difference was demonstrated between the CONSORT scores of the two study periods.

This improvement in the reporting standards of the more recently conducted RCTs may be explained by more than one reason. First, awareness of the CONSORT guidelines has certainly improved greatly. Investigators are now better trained than before about various aspects of clinical trials, including manuscript writing and the reporting of clinical trials. Some of this training has even extended to undergraduate and postgraduate programs, cultivating an environment for research, which might explain the difference between the reporting standards of the two study periods. More experienced and trained mentors now guide junior authors before manuscripts are submitted for publication, potentially explaining these results. Additionally, many of the publishing journals endorse the CONSORT guidelines; thus, acceptance or rejection of an article in the journal may depend on the degree of compliance to the guidelines.

However, when analyzing the scores, it is also evident that although a difference is noted between the medians of the CONSORT scores for the two study periods, the scores were not as high as expected. The medians of the CONSORT score were 17 out of a possible 25, revealing gaps in reporting standards. With respect to other items, despite the lack of significant difference observed between the two study periods, careful analysis reveals several important issues. Although many of the items are reported poorly, certain items were reported well regardless of the study period, although to a much lesser extent. Item 3, trial design, along with item 7, details of sample size calculation, are poorly reported throughout. Additionally, issues of randomization (item 8) are not adequately elaborated. Item 20, study limitations, is absent in as many as 50% of the reports.

Without such information, when clinical trials are critically appraised, it may be very difficult to evaluate the evidence, necessitating the need to repeat RCTs on a larger or more controlled scale. The repetition of clinical trials is costly and requires significant resources. To avoid such repetition, it is imperative that these items be reported with great care and consistency. However, certain items were reported consistently by authors, including item 1 (title and abstract), item 5 (intervention details), item 12 (statistical plan), item 15 (baseline characteristics), item 17 (results), item 19 (harms), item 21 (generalization), item 22 (interpretability) and item 25 (funding).

We performed analyses of the predictors and their relationship with the standards of reporting, and only the year of publication was found to be statistically significant. Other predictors that were implicated by other investigators did not reveal any significant relationship with the standards of reporting. The fact that the year of publication was a significant predictor is a crucial finding may be explained by improvements in the reporting standards. Other predictors, i.e., the type of immunosuppressive therapy, the number of authors involved, and the impact factor of the publishing journal and the country of publication were expected to exhibit significant results, as has been reported previously by other investigators who evaluated similar outcomes in other fields. Despite the increase in the number of authors, the scores have remained low, possibly resulting from the addition of many authors who are inexperienced and unsupervised. There was no relationship between the CONSORT score and the impact factor of the publishing journal, signifying that although the journals endorse the guidelines, their implementation is not strictly supervised. Supervising editors and reviewers must analyze the RCTs and match the standards of reporting according to the CONSORT guidelines. These predictors may have important implications and cannot be ignored, although the associations detected in the present study failed to meet statistical significance.

The strengths of this study are as follows: a large cohort of studies was included in the study, which pertain to recently conducted clinical trials; RCTs were extracted from the largest and most well-known databases; most of the included RCTs were published in reputable medical journals; a time period (2007-2013) was chosen to account for all of the RCTs published on the topic of interest; and last, the two arms of the study had an equal number of studies. The key limitations of the study are as follows: only 3 databases were used for the purpose of RCT selection; and the selection was performed by a single investigator, which might have introduced selection bias. Additionally, the CONSORT scores of two different periods were compared based on CONSORT 2010, although RCTs were published before the revision.

Overall, this study revealed significant differences in the reporting standards of clinical trials in the field of kidney transplantation between 2007-2010 and 2010-2013, with results indicative of an improvement in reporting standards. However, items reporting randomization, allocation concealment, blinding, sample size calculations and study limitations remain under-reported. Analyses also revealed relationships between the years of publication and reporting standards, suggesting progressive improvements in reporting standards as set by CONSORT. However, these results underline the need to continue such surveillance studies to identify the gaps in the standards of reporting of RCTs in the coming years.