| Japanese Journal of Clinical Oncology | Pages |
An Analysis of Institutional Effects in a Multicenter Cancer Clinical Trial: is it also Plausible from the Clinicians' Point of View?
Acknowledgments
References
An Analysis of Institutional Effects in a Multicenter Cancer Clinical Trial: is it also Plausible from the Clinicians' Point of View?
The institutional effect is a somewhat delicate issue in randomized clinical trials. Despite eager attempts to standardize the methods of diagnosis and treatment among participating institutions, it is still difficult to assert that such criteria could be applied equally not only in specialized cancer centers and also in local community hospitals.
It is a general consensus that the random treatment assignment should be balanced within an institution. Randomization of patients to treatments provides a valid statistical basis for testing the null hypothesis of no treatment effect without assuming any particular population model or taking account of all possibly important covariates. Thus, the conventional design based analysis ignoring institutional effects, e.g. the log-rank test, leads to a valid test of the null hypothesis, as long as the comparability of the treatment groups is maintained through randomization. This holds true even in traditional Japanese cancer clinical trials in which there are many participating institutions with a few patients in each institution.
A large institutional variation, however, causes a loss of power owing to the increase in random error. Furthermore, in such a traditional trial, it is difficult to investigate the generalizability of the results through stratified analysis by institution.
In the September 1998 issue of Statistics in Medicine (1), we published an analytical methodology of the institutional effects in multicenter cancer clinical trials where the end-point is time-to-event (recurrence or survival). In that paper, we analyzed the data from a multicenter cancer clinical trial which confirmed the efficacy of immunochemotherapy as an adjuvant treatment after curative resection of gastric cancer (SIP) (2) and proposed a Bayesian hierarchical survival model to investigate institutional effects on the efficacy of treatment (interaction between treatment and institutions) and on the baseline risk. The marginal posterior distributions are estimated by a computer-intensive Markov Chain Monte Carlo method, i.e. Gibbs sampling (3).
Application of the above methodology to that immunochemotherapy trial demonstrated a substantial variation in the baseline risk (0.32-3.14) (Fig. 1), but little difference in the treatment effects across institutions (Fig. 2) (1). These results, in fact, seem to reconfirm the diagnostic and therapeutic differences among institutions. On the other hand, the difference in treatment effect among institutions was proved to be minimal, indicating that immunochemotherapy treatment has generally been effective at each participating institution in that particular SIP trial.
Figure 1. Posterior distribution of the baseline hazard in each institution. Figure 2. Posterior distribution of the treatment effects in each institution. Two major questions should probably be raised from the clinician's side: are the results of this analysis really plausible and agreeable from the clinicians' point of view?; and how could we interpret and exploit this information in an actual and in a forthcoming clinical trial? First, it has to be stressed that the difference in the baseline risk that we estimated does not necessarily imply a direct difference in the therapeutic abilities of each institution. Other important factors (i.e. pathological diagnosis, stage migration, etc.) are also presumed to affect the baseline risk of each institution. Taking these assumption into consideration, we first examined the correlation between the accuracy of macroscopic diagnosis and the estimated baseline risk. In this analysis, the accuracy of diagnosis was defined by the absolute discrepancy between clinical and pathological TNM stage in each patient. The results indicated that institutions with a high baseline risk tend to have larger discrepancies between diagnoses, whereas institutions with a low baseline risk show a smaller discrepancy (p = 0.10) (Fig. 3). Figure 3. Correlation between stage discrepancy and estimated baseline risk. TNM stage grouping by the UICC was used. The vertical axis is the simple mean of the absolute discrepancy between clinical and pathological stage in each institution. To investigate further the reason for the variance in baseline risk, an inquiry was sent to three main investigators of the SIP trial (study chairman, study coordinators) to rank the participating institutions of the SIP trial according to their subjective impression (A, good; B, fair; C, poor) without informing the results of the baseline risk analysis. The scatter plot between the specialists' opinion and the baseline risk in each institution indicated again that institutions with a high baseline risk appear to be ranked as C, whereas institutions with a low baseline risk tend to be ranked as A (p < 0.0001) (Fig. 4). The results of these two analyses indicate that the baseline risk estimated from our analysis for the institutional effects correlate fairly well with the objective discrepancy of diagnosis and with the rating of the main investigators who are closely involved in the trial. Figure 4. Correlation plot between specialists' opinion and estimated baseline risk. Second, in this Bayesian hierarchical survival model, institutional effects were decomposed into two separate factors: difference in baseline and difference in treatment effect in each institution. Since a multi-center clinical trial is also conducted in order to investigate the generalizability of the observed treatment effects, it is necessary to examine the homogeneity of the treatment effects across institutions, that is, treatment-by-institution interaction. If the homogeneity of the observed treatment effect across institutions is confirmed as in the case of our immunochemotherapy clinical trial, a single summary measure such as risk ratio is adequate to describe the trial results and the conclusions involving treatment effect can be generalized to a broader patient population. If, on the other hand, a substantial institutional effect is detected in an actual clinical trial, exploratory analysis to examine the cause of such heterogeneity and its possible effect on the study conclusions should be necessary and also valuable to ascertain the generalizability of that trial. Another possible use of this analysis involves the planning of a forthcoming multi-center clinical trial. If substantial variations of the baseline risk and/or treatment effect between institutions were shown in the former trial, efforts to reduce those variations should be carried out in the planning of the next trial. These efforts consist of environmental improvements surrounding the clinical trial at the planning stage, i.e. standardization of the diagnostic and therapeutic methodology between the participating institutions prior to the start of the trial or selection of the qualified institutions and target populations for an elaborated clinical trial. In conclusion, the Bayesian hierarchical survival model we proposed could be plausible from the clinician's point of view and also be exploitable for the planning of forthcoming clinical studies. We thank Dr Hiroaki Nakazato, Mr Takashi Fujii and Mr Satoshi Teramukai for support and helpful discussions.
Acknowledgments
References
This article has been cited by other articles:
This page is run by Oxford University Press, Great Clarendon Street, Oxford OX2 6DP, as part of the OUP Journals
Comments and feedback: jnl.info{at}oup.co.uk
Last modification: 23 Aug 1999
Copyright© 1999 Foundation for Promotion of Cancer Research.
CiteULike 
Connotea 
Del.icio.us What's this?
P Del Bianco and R Borgoni
Handling dropout and clustering in longitudinal multicentre clinical trials
Statistical Modeling,
July 1, 2006;
6(2):
141 - 157.
[Abstract]
[PDF]
T Yamaguchi, Y Ohashi, and Y Matsuyama
Proportional hazards models with random effects to examine centre effects in multicentre cancer clinical trials
Statistical Methods in Medical Research,
June 1, 2002;
11(3):
221 - 236.
[Abstract]
[PDF]
This Article 
Extract
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (2)
Request Permissions
Google Scholar
Articles by Sakamoto, J
Articles by Ohashi, Y
Search for Related Content
PubMed
PubMed Citation
Articles by Sakamoto, J
Articles by Ohashi, Y
Social Bookmarking
What's this?