Japanese Journal of Clinical Oncology 32:19-26 (2002)
© 2002 Foundation for Promotion of Cancer Research
The Influence of Handling Censored Data on Estimating Progression-free Survival in Cancer Clinical Trials (JCOG9913-A)
1Japan Clinical Oncology Group Data Center, Cancer Information and Epidemiology Division, National Cancer Center Research Institute, Tokyo and 2Department of Urology, Institute of Clinical Medicine, University of Tsukuba, Ibaraki, Japan
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
Background: Progression-free survival (PFS) is a common endpoint in cancer clinical trials. This study was undertaken to assess the impact of data errors and data handling on the statistical estimation of PFS.
Methods: Data from four trials conducted by the Japan Clinical Oncology Group were examined. Three types of data handling methods were defined: (1) data handling method A (METHOD-A), the collected event data are used as much as possible, (2) METHOD-C, only reliable data with firm evidence are used, and (3) METHOD-B is intermediate between METHOD-A and METHOD-C. To assess the impact of each of the three methods, Kaplan–Meier survival curves, median PFS, proportion of PFS, log-rank p values and hazard ratios were estimated.
Results: In three trials that collected PFS data periodically, no remarkable differences in median PFS and the proportion of PFS were observed. In one trial with non-periodic data cleaning, however, the ratio of median PFS by METHOD-C to that by METHOD-B was 0.85, the maximum difference of proportion of PFS between METHOD-C and METHOD-B was 12.0% and the largest spread in PFS curves amongst the three methods was observed in this trial. In all trials, log-rank p values and hazard ratios for between arm comparisons did not differ between the three methods.
Conclusions: Periodic data management can reduce errors in comparisons of PFS and is a critical requirement when using PFS as a major endpoint. Furthermore, proper data handling is essential in the estimation of patient benefit and caution is needed when making clinical decisions based on PFS.
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
Time-to-event endpoints are commonly used as the major endpoints in cancer clinical trials. Such endpoints include overall survival (OS), time to progression (TTP) (also referred to as progression-free survival, PFS), disease-free survival (DFS) (also referred to as relapse-free survival, RFS), time to treatment failure (TTF) and so on. PFS is generally defined as the time from registration to the first observation of disease progression or death. As such, it is important as a surrogate of OS. However, since it could reflect patient benefit by prolonging the interval to disease progression, it may also have importance as a true endpoint in and of itself. It is in this sense that PFS is used as the primary endpoint in phase III trials in advanced or metastatic cancer. PFS has also recently come to be used as the primary endpoint in phase II trials for cytostatic drugs (1).
Time-to-event endpoints have received much attention in the statistical literature (2–7) and various cancer cooperative groups have addressed practical issues by establishing standard data collection procedures for time-to-event data (8,9). However, the impact of data management issues, especially data handling and follow-up of data collection errors, on the outcome of PFS or other time-to-event analyses has received no attention in the literature.
In oncology phase III trials, long-term follow-up is generally essential, except for studies in patients with extremely poor prognoses. Continued follow-up of each patient, even after accrual is closed, until the endpoint of interest is achieved in all patients is therefore crucial, although admittedly resource-intensive (10,11). The importance of preventing loss-to-follow-up and persuading patients to return to active follow-up has often been emphasized (10,11); however, detailed descriptions of how different data management approaches may impact on the estimation of time-to-event endpoints, based on analyses of actual clinical data, have not previously been published.
In the real-world conduct of clinical trials, failure to collect data on progression or relapse can easily occur for various reasons, such as the patient’s inability or unwillingness to return to clinic (dropouts or lost-to-follow-up) or investigator non-compliance with the protocol-specified follow-up schedule(s). Even in such cases, the estimation of OS (in which death from any cause is counted as an event) may not be substantially affected, since only the patient’s date of death or, if still alive, the date of last contact is required for this endpoint. Such information can easily be confirmed by telephone calls or (in Japan) inspection of the family register. In contrast, in the case of PFS estimation, progression can only be detected within the intervals between patient visits. Therefore, the frequency of clinical visits may affect determination of the date of progression. Similarly, non-periodic data collection and missing data may affect estimation of PFS. In the extreme case, the data may show the notice of death due to primary disease without an identifiable date of progression. Existing post hoc data processing or other statistical analyses cannot adequately compensate for such biased data collection
Our primary interests in conducting this study were (1) to determine the extent to which this type of data errors may affect the statistical estimation of PFS and (2) to investigate whether or not such biases could be reduced by post hoc data processing.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
Data Source
Selection criteria for inclusion in this study were as follows: candidate studies had to (1) be multi-center randomized phase II or III trials conducted by the Japan Clinical Oncology Group (JCOG; a cooperative oncology group which is supported by Grants-in-Aid for Cancer Research from the Ministry of Health, Labour and Welfare) (12); (2) have completed patient registration before July 1998; (3) have patient registration, data management and follow-up performed by the JCOG Data Center; (4) have final follow-up for the primary analysis completed between April 1998 and January 2001; and (5) have completed the primary analysis and published at least the major results either in scientific meetings or in medical journals. As a result, data from four trials [JCOG9104 (13), JCOG9114 (14), JCOG9205 (15) and JCOG9505 (16)] were used. We obtained permission to include the trial data in the current study from both the respective study chairs and the JCOG Data and Safety Monitoring Committee. Characteristics of trials are shown in Table 1.
|
Data Management in JCOG Data Center
The JCOG Data Center has routinely performed semi-annual systematic follow-up surveys for all active trials since its establishment. The current follow-up survey procedure includes the data elements used for calculating OS and PFS. These consist of; current status (dead or alive), date of death (if applicable), cause of death (if applicable), date of last contact for surviving patients, current disease status (progressive disease or progression-free), date of progression (if applicable), date on which the patient was confirmed to be progression-free and sites of progression (progression is replaced by relapse in studies of adjuvant therapy). However, prior to 1997, the date on which the patient had been confirmed to be progression-free had neither been included in routine follow-up forms nor systematically stored in the JCOG database and periodic updates of PFS had also not been regularly available until then, although date of progression had been included. If only the date on which progression occurred is reported and not the date on which the patient was known to be progression-free, then the date of progression must be inferred. In this situation, the last date on which the patient was known to be progression-free is sometimes used. Depending on the time interval between this date and the date on which the patient was confirmed to have progressed, this practice may result in an important bias in calculating progression-free survival. As a consequence, data management and data quality for PFS varied widely across JCOG trials. In 1998, the JCOG Data Center became aware of this problem and redesigned both the databases and follow-up forms and amended the data definitions in order to make the periodic updates available semi-annually since 1998.
At least part of the follow-up surveys for JCOG9104, -9114 and -9505 were performed in accordance with the 1998 changes. The final follow-up survey for JCOG9205 was performed without having the periodic updates of the date of PFS. The patient who died of primary cancer without the date of progression was censored for PFS at the most reliable date, which can be confirmed to be free from progression, e.g. at the date of the last negative CT scan even though it was in the early period of protocol treatment.
Methods of ‘Data Handling’
In this study, three types of data handling were defined to determine the PFS event date and the date of censoring (Table 2). These methods differed not in the definitions of events or of censoring, but rather in (a) what rules were used to adopt a particular ‘event date’ among various candidates event dates and censoring and (b) what rules were used to decide either that an event had occurred at a given time point or whether censoring was more appropriate. The three data handling methods are detailed as follows: (1) in data handling method A (METHOD-A), as much event data as possible from the collected data were used, regardless of reliability; (2) data handling method C (METHOD-C) was based on the idea that only reliable data should be adopted and used data with firm evidence only; and (3) data handling method B (METHOD-B) or the post hoc method was intermediate between METHOD-C and METHOD-A. This approach is the one that clinical investigators are most likely to adopt.
|
PFS was defined as the time from randomization to disease progression or death, whichever came first. In cases where the date of progression was available, that date was used as the PFS event date in all analyses. In the case of patients who died of primary disease without a recorded date of progression, PFS was censored at the last time point at which the patient was considered to be progression-free. The degree of certainty required for this assignment differed in METHOD-C (firm evidence required to support this assignment) and METHOD-A (date of death used as the PFS event date). The assignment of the PFS event date in METHOD-B depended on cause of death, i.e. the date of death was used as the PFS event date if the patient died of primary disease and PFS was censored at the date of death if the patient died of other causes. Regardless of the data handling method employed, the decision of which date to use as the last date of progression-free survival was made by the study of each trial coordinator, following completion of the data cleaning process.
Statistical Methods
To assess the impact on survival estimates of the three methods of data handling, Kaplan–Meier survival curves of PFS, median PFS, proportion of PFS at 0.5 and 1 year and maximum differences in the proportions of PFS among three types of data handling were determined. Ratios of median PFS were calculated to compare the median PFS by three data handling methods. Standard errors (SE) of the proportion of PFS were used as a measure of precision for estimation. To evaluate the difference in median PFS among two arms, two-sided log-rank p values and hazard ratios based on the proportional hazard model were used. All calculations were performed using the SAS v. 6.12 statistical software package (SAS, Tokyo, Japan).
The estimates shown in this study may differ from those in the original publications because the data handling methods used in the current study were defined independently from those used in the original primary analysis of each trial.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
Table 3 shows the median PFS, ratio to median PFS by METHOD-B, proportions of PFS at 0.5 and 1 year and their respective SEs, according to the method of data handling. In JCOG9104, -9114 and -9505, there was no remarkable difference in median PFS and proportion of PFS among the three methods. In JCOG9205, however, the median PFS and the 0.5- and 1-year PFS estimates in arm B by METHOD-C were 123 days (0.85 times METHOD-B), 26.3% and 12.8%, which differed from those obtained by the other data handling methods, METHOD-A and METHOD-B.
|
The largest difference in PFS curves was observed in JCOG9205; the maximum absolute differences of the proportion of PFS between METHOD-C and METHOD-B were 6.2% in arm A and 12.0% in arm B (Fig. 1, Table 4).
|
|
The SEs of the proportions of patients surviving progression-free showed no remarkable difference among the methods of data handling in JCOG9104, -9114 and -9505. However, the SE in arm B as determined by METHOD-C differed from those determined using the other methods in JCOG9205 (Table 3). METHOD-C showed the worst precision of the estimation of the three methods.
In all trials, log-rank p values and hazard ratio between the treatment arms did not change according to the method of data handling (Table 5). However, the differences in median PFS between the treatment arms did vary depending on the method of data handling. The ratio between the maximum and minimum differences of median PFS ranged from 1.63 (JCOG9104) to 1.35 (JCOG9205).
|
The importance of having reliable information regarding the confirmed data for PFS is highlighted by considering how the three data handling methods differ in deciding how to regard a given observation. Table 6 shows how information regarding PFS and overall survival was integrated into seven different patterns for classification purposes, and also the respective numbers of each case pattern in each of the four studies considered in this paper. In case patterns 2, 4 and 7, since the date of progression can be determined to have occurred at time point X, all three data handling methods agree in considering that an event occurred at that time point. However, in all other instances, for any given time point, the data handling methods differ in deciding whether to censor an observation or to consider an event to have occurred, depending on the ultimate clinical outcome. The case of JCOG9205 was particularly notable for the extent of missing data in this regard: 35 cases (16.7%) were lacking any information regarding progression, although these patients died of their primary disease.
|
The effect of the three different data handling methods is summarized in Table 7, which shows how the number of events and the median follow-up time of each trial differed according to the data handling method.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
OS is generally accepted as a hard and true endpoint for clinical trials in many disease fields. In contrast, other time-to-event endpoints including PFS are not regarded as hard endpoints because completely objective measurement of progression is presumed not to be possible. Clinical researchers have been making ongoing efforts to increase objectivity or reliability in the measurement of progression, e.g. by standardizing descriptions in protocols, continuous education of research staffs, etc. However, many sources of bias and error that may potentially reduce the reliability of time-to-event endpoints (except for OS) still remain. In this study, we chose to focus on the issues of data handling and data reliability, since they have received little attention so far in the literature and are of great importance to clinical trials organizations.
In this study, we demonstrated that PFS estimates in three trials with periodic data collection and cleaning (JCOG9104, -9114, -9505) were not affected by the data handling method. However, the choice of methodology did influence these estimates in the one trial whose data had not been managed periodically during the trial period (JCOG9205). The fact that the point estimates of PFS were lower with METHOD-C in JCOG9205 suggest that non-METHOD-C may result in overestimation of PFS, presumably due to informative censoring. The larger standard errors with METHOD-C are probably the logical consequence of discarding information that other data handling methods would have included. In statistical theory, censoring must be independent of future event history to obtain unbiased estimates, that is, subjects censored do not tend to die or live longer compared with those not censored. In Table 6, the definition of ‘case’ is the same regardless of data handling method for case patterns 2, 4, 6 and 7. However, this is not the case for pattern 1, in which the date of progression is missing in spite of the patient’s having died due to primary disease. Although the patient had clearly progressed between the last date on which he/she was confirmed to have been progression-free and the time of death, the point within that interval at which progression truly occurred is unknown. Although there is less certainty that the patient’s disease had progressed in case patterns 3 and 5, it is nevertheless still unclear that the patient’s disease had been truly progression-free. Looked at another way, in METHOD-C, case patterns 1, 3 and 5 are uniformly considered censored cases at time point X and determination of an observation is unrelated to future information. METHOD-A and METHOD-B, on the other hand, do rely on future information to define an event. Therefore, METHOD-C gives an unbiased estimate since future information does not affect the handling of observations. Conversely, in METHOD-C, the loss of information due to the decrease in events and shortening of follow-up time results in low precision in case patterns 1, 3 and 5 (especially pattern 1) (Table 7). Therefore, METHOD-C gives unbiased estimates but lower precision, whereas METHOD-A and METHOD-B give higher precision but are prone to bias. The choice of which method to use in a given study depends on the purpose or aim of the study. METHOD-C appears to be in accord with recent regulatory agency actions, such as those taken by the US Food and Drug Administration, whereas either METHOD-A or METHOD-B may be adopted in investigational studies.
It is important to note that use of METHOD-C leads to informative censoring and consequently to potentially biased estimation, if a large proportion of cases that progressed later were censored early on. Therefore, in these cases, it may be preferable to apply statistical analyses that are able to handle interval censoring, such as actuarial method.
Regardless of the data handling method employed, it is important to improve data quality. Method-dependent differences in outcome are particularly likely to occur in trials without periodic data collection and cleaning, such as JCOG9205. On the other hand, we found no method-dependent differences in results in the three trials with periodic data collection and cleaning. This is a noteworthy fact that demonstrates the importance of continuous quality monitoring and assurance practices.
Log-rank p values and hazard ratios were not affected by the choice of data handling method (Table 5). This confirms the statistical property that the alpha error remains the nominal level under the assumption that the censoring pattern is common over both treatment arms. To assume this property, the same data handling method must be used in both arms.
In contrast, the variation in the difference of median PFS observed even in the three trials with periodic data collection and cleaning is notable. This difference is considered to reflect the magnitude of benefit, which means the expected ‘gain’ for patients receiving a new treatment. In general, when making decisions based on a clinical trial’s outcome, clinicians take into account not only the statistical significance of the trial’s findings, but also the foreseeable magnitude of benefit compared with the anticipated risks. Since the methodology of progression data handling may affect the overall benefit/risk assessment, careful assessment is needed when the absolute difference in PFS between treatment arms is taken into account as an expected gain from a new treatment.
Another point to be noted is that METHOD-A does not always give ‘overestimates’ of PFS. In some cases, the differences in median PFS calculated by METHOD-A in the trials with periodic data collection were smaller than those calculated using the other data handling methods. These findings suggest that METHOD-A does not lead to liberal statistical decision making. Therefore, periodic data collection and regular monitoring of data quality are an important issue and must be conducted even when a trial is well designed and has clearly defined endpoints. Piantadosi (17) explains that survival time and time to disease progression are well-known examples of ‘definitive’ event times, because the outcome is usually determined with minimal error. However, the results of the present study show that the degree of ‘definitiveness’ of PFS seems to depend on the degree of data quality. We are currently investigating other issues for PFS, such as possible decision criteria for determining progressive disease in the absence of radiographic evidence and the influence on PFS estimates of patient visit frequency.
The major weakness and limitation of this study are its generalizability. Only one study, JCOG9205, was available as an example of a study with non-periodic data cleaning. It is difficult to obtain other such trials because the new standard procedures requiring periodic data collection for progression and relapse have been in force since 1998. Therefore, further investigations will have to be based on simulations and not on actual data.
In conclusion, periodic data management can avoid a substantial influence on the comparison of PFS and is a critical requirement when a trial adopts PFS as a major endpoint. Furthermore, proper data handling is essential in estimation of patient benefit and caution is needed when making clinical decisions based on PFS.
|
Acknowledgments |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
This study was supported in part by Grants-in-Aid for Cancer Research (11S-4) and the Second Term Comprehensive 10-year Strategy for Cancer Control (H12-Gan-012) from the Ministry of Health, Labor and Welfare. One of the authors (S.Y.) was supported in part by the Foundation of Cancer Research in Japan. The authors are grateful to Drs M. Shimoyama, N. Saijo, K. Tobinai, T. Watanabe, Y. Shimada, T. Tamura (National Cancer Center Hospital, Tokyo), K. Itoh (National Cancer Center Hospital East), K. Kobayashi (US Food and Drug Administration), data managers in the JCOG Data Center and the patients who were enrolled in the clinical studies.
|
FOOTNOTES |
|---|
+ For reprints and all correspondence: Miyuki Niimi, Japan Clinical Oncology Group Data Center, Cancer Information and Epidemiology Division, National Cancer Center Research Institute, 1–1 Tsukiji 5-chome, Chuo-ku, Tokyo 104-0045, Japan. E-mail: mniimi@gan2.ncc.go.jp
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONAcknowledgmentsREFERENCES |
|---|
1 Mick R, Crowley J, Carroll R. Phase II clinical trial design for noncytotoxic anticancer agents for which time to disease progression is the primary endpoint. Control Clin Trials 2000;21:343–59.[Web of Science][Medline]
2
Rubin DB. Inference and missing data. Biometrika 1976;63:581–92.
3 Lagakos SW. General right censoring and its impact on the analysis of survival data. Biometrics 1979;139–56.
4 Link WA. A model for informative censoring. J Am Stat Assoc 1989;84: 749–52.
5
Rotnitzky A, Robins JM. Semiparametric regression estimation in the presence of dependent censoring. Biometrika 1995;82:805–20.
6
Lin DY, Robins JM, Wei LJ. Comparing two failure time distributions in the presence of dependent censoring. Biometrika 1996;83:381–93.
7 Traxel AB, Lipstz SR, Brennan TA. Weighted estimating equations with nonignorably missing response data. Biometrics 1997;53:857–69.[Web of Science][Medline]
8 Green S, Benedetti J, Crowley J. Clinical Trials in Oncology. London: Chapman and Hall 1997;39–42, 101–4.
9 Buyse ME, Staquet MJ, Sylvester RJ. Cancer Clinical Trials. Oxford: Oxford Medical Publications 1988;51–3.
10 Meinert C, Tonascia S. Clinical Trials, Design, Conduct and Analysis. Oxford: Oxford University Press 1986;159–65.
11 McFadden E. Management of Data in Clinical Trials. New York: Wiley 1998;173–81.
12
Shimoyama M, Fukuda H, Saijo N, Yamaguchi N, Members of the Committees of the Japan Clinical Oncology Group (JCOG). Jpn J Clin Oncol 1998;28:158–62.
13 Goto K, Nishiwaki Y, Takada M, Fukuoka M, Kawahara M, Sugiura T, et al. Final results of a phase III study of concurrent versus sequential thoracic radiotherapy (TRT) in combination with cisplatin (P) and etoposide (E) for limited-stage small cell lung cancer (LD-SCLC): the Japan Clinical Oncology Group (JCOG) Study. Proc Am Soc Clin Oncol 1999;18:468.
14 Katsumata N, Watanabe T, Sano M, Tabei T, Aoyama H, Kitaya T, et al. Evaluation of addition of medroxyprogesterone acetate (MPA) at two doses to the chemoendocrine therapy with doxorubicin (A), cyclophosphamide (C) and tamoxifen (TAM) in metastatic breast cancer. Proc Am Soc Clin Oncol 2000;19:155.
15 Shimada Y, Shirao K, Ohtsu A, Hyodo I, Saito H, Yamamichi S, et al. Phase III study of UFT + MMC vs 5-FU + CDDP vs 5-FU alone in patients with advanced gastric cancer: JCOG Study 9205. Proc Am Soc Clin Oncol 1999;18:272.
16 Itoh K, Ohtsu T, Sasaki Y, Ogura M, Morishima Y, Chou T, et al. Randomized phase II study of biweekly CHOP and dose-escalated CHOP with prophylactic use of lenograstim (glycosylated G-CSF) in aggressive non-Hodgkin’s lymphoma (NHL): Japan Clinical Oncology Group (JCOG) Study 9505. Proc Am Soc Clin Oncol 2001;20:289.
17 Piantadosi S. Clinical Trials, a Methodologic Perspective. New York: Wiley 1997;133–7.
Received July 4, 2001; accepted October 1, 2001.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. Sakamoto and S. Teramukai Data Handling in Cancer Clinical Trials--How We Can Minimize Potential Biases Jpn. J. Clin. Oncol., January 1, 2002; 32(1): 1 - 2. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||