Japanese Journal of Clinical Oncology Advance Access published online on August 18, 2007
Japanese Journal of Clinical Oncology, doi:10.1093/jjco/hym064
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© 2007 Foundation for Promotion of Cancer Research
Inter-observer Variations in FDG-PET Interpretation for Cancer Screening
1 Department of Radiology, School of Medicine, Yokohama City University, Yokohama
2 Department of Diagnostic Radiology, Kyoto Graduate University School of Medicine, Kyoto
3 Research Center for Cancer Prevention and Screening, Cancer Screening Division, National Cancer Center, Tokyo
4 Diagnostic Imaging Center, Radiology, Yuai Clinic, Yokohama
5 Department of Radiology, PET/RI center, Okayama Kyokuto Hospital, Okayama
6 HIMEDIC Imaging Center at Lake Yamanaka, Yamanashi
7 Utsunomiya Central Clinic PET Center, Utsunomiya
8 Department of Radiology, School of Medicine, Yokohama City University, Yokohama
9 Institute of Biomedical Research and Innovation, Kobe
10 Nishidai Clinic Diagnostic Imaging Center, Tokyo, Japan
For reprints and all correspondence: Akiko Suzuki, Department of Radiology, School of Medicine, Yokohama City University, 3–9 Fukuura, Kanazawa-ku, Yokohama, Kanagawa, 236-0004, Japan. E-mail: akiko225{at}yokohama-cu.ac.jp
Received October 16, 2006; accepted March 20, 2007
 |
Abstract |
|---|
 TOP Abstract INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
Background: Diagnostic guidelines for the use of 2-(fluorine 18) fluoro-2 deoxy-D-glucose (FDG)-positron emission tomography (PET) in cancer screening have yet to be established. We assessed inter-observer variability in screening FDG-PET.
Methods: Subjects comprised 40 individuals who underwent FDG-PET and computed tomography (CT) for cancer screening. To assess various patterns of FDG uptakes, three subsets of the cases were selected: ‘Cancer’, 15 cases with cancer; ‘Not malignant’, 15 cases with suspected cancer by FDG-PET who were confirmed as cancer-free; and ‘Normal’, 10 cases without remarkable FDG uptake who were confirmed as cancer-free. A total of 68 lesions made up of malignancy (n = 18), benign (n = 21), and physiological FDG uptake (n = 29) were interpreted by six physicians. Each observer reviewed each case three times. Step 1 involved interpretation of PET images alone, Step 2 involved side-by-side reading of PET and CT images, and Step 3 involved re-evaluation of findings with the results of other screening tests. We assessed inter-observer agreement for each step.
Results: Inter-observer agreement for all lesions at each step was moderate, compared to fair agreement for ‘Normal’ subjects. Inter-observer agreement of ‘Cancer’ and ‘Not malignant’ subjects in Step 1 were better than those in Step 2 and 3; however, the differences were not statistically significant.
Conclusion: The interpretation of FDG-PET is adequately reproducible, while that of ‘Normal’ subjects is less reproducible. Improvement of inter-observer variability in assessing physiological FDG uptakes requires universal reporting criteria in FDG-PET. Correlative interpretation of PET, CT and other information may require standardization in subjects with suspected cancer by FDG-PET.
Key Words: radiology • PET • radiology • CT/MRI • cancer screening
|
INTRODUCTION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
2-(fluorine 18) fluoro-2 deoxy-D-glucose (FDG)-positron emission tomography (PET) plays an important role in the detection of malignant tumors, although the effectiveness of whole-body FDG-PET imaging in cancer screening remains uncertain (1,2). FDG-PET scans have been performed for cancer screening in Japan on many asymptomatic individuals who had previously never been diagnosed with cancer. FDG-PET is considered to be useful for whole-body survey because it can detect cancers of various organs that any single conventional organ-specific screening test cannot cover. High detection rates for a wide variety of cancers in cancer-screening FDG-PET has been reported by Yasuda (3) and Chen (4); however, FDG-PET cannot be an alternative to other conventional screening tests such as physical examination, laboratory studies, mammography and thoracic computed tomography (CT), because FDG-PET analysis has obvious limitations in detecting urological cancers, cancers of low cell density, small cancers and hypometabolic or FDG-negative cancers (1,3,5). Diagnostic guidelines for the use of whole-body FDG-PET imaging in cancer screening have yet to be established. Furthermore, FDG-PET may be better interpreted with reference to CT images in cancer screening, as the determination of the precise location of FDG-avid lesions using PET alone can be challenging (6). The role of whole-body FDG-PET and CT in cancer screening is yet to be evaluated.
We surveyed a large number of cancer-screening centers in Japan in January 2005 to investigate the actual situation of cancer screening by FDG-PET. Thirty cancer-screening centers answered the questionnaire and the results were reported (data not published). The recall rate (i.e. the rate recommending diagnostic work-up due to positive findings suggesting possible cancer) varied widely from 1 to 44% betweem the centers. We hypothesized that inter-observer variation in FDG-PET interpretation for cancer screening affected clinical decisions to recommend either close examination or follow-up and caused variability in the recall rate. Inter-observer variation in FDG-PET for cancer screening is of particular interest because of the challenge involved in detecting suspected lesions, of which the incidence is very low, out of numerous cases of equivocal FDG uptake; some radiologists tend to over-diagnose FDG uptake to avoid potential false negative outcomes. Numerous investigations have shown that considerable variability exists among radiologists in the interpretation of screening tests such as mammography and thoracic CT without training or computer-aided diagnosis (7–9). This variability affects the diagnostic accuracy of screening studies and clinical decisions to recommend either close examination or follow-up. Herder et al. reported that inter-observer agreement of FDG-PET between clinical and final stage was good in patients with suspected lung cancer (10). Inter-observer agreement of interpretation in relevant focal pulmonary abnormality of FDG-PET was also reported to be good by Joshi et al. (11). To the best of our knowledge, variability in radiologists' interpretations of whole-body FDG-PET for cancer screening has yet to be examined. The purpose of the present study is to assess inter-observer variations in screening FDG-PET.
|
MATERIALS AND METHODS |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
Case Materials
FDG-PET and CT data of 40 subjects (21 male, 19 female, median age 57 years) were collected from seven cancer-screening centers in Japan and were used in this study. The scanning took place between April 2004 and March 2005; all subjects were symptom-free and underwent FDG-PET and CT on the same day for cancer screening together with other physical and laboratory tests. The 40 subjects consisted of Group 1 ‘Cancer’, 15 true positive cases with suspected cancer by FDG-PET who were confirmed to have cancer by biopsy; Group 2 ‘Not malignant’, 15 false positive cases with suspected cancer by FDG-PET and recommended for close examination who were confirmed as cancer-free on follow-up or biopsy; and Group 3 ‘Normal’, 10 true negative cases who had no suspected lesion detected in FDG-PET and were confirmed to be cancer-free at 1-year follow-up. It should be noted that the number of cases in each group does not reflect the fractional occurrence of each outcome in cancer screening using FDG-PET. This study did not deal with false negative cases on PET because this is peripheral to the primary aim of assessing inter-observer variation in the interpretation of malignant lesions without remarkable FDG uptakes. Among the 15 ‘Cancer’ subjects, primary disease involved the lung (n = 3), thyroid (n = 3), colon (n = 2), breast (n = 2), stomach (n = 2), pancreas (n = 1) and malignant lymphoma (n = 2). These cancers are commonly detected in FDG-PET and CT during cancer screening (1,4).
On reviewing the 40 subjects together with all the reference data, a total of 103 lesions were identified, presenting varying intensities of FDG uptake. Thirty-five of 103 lesions that were considered true negative lesions in ‘Cancer’ and ‘Not malignant’ cases were excluded because of the absence of confirmed reference data. A final total of 68 lesions were diagnosed as malignant (n = 18), benign (n = 21), or physiological FDG uptake (n = 29). Among the 15 ‘Cancer’ subjects, 18 malignant lesions were detected and enrolled as true positive lesions. In the 15 ‘Not malignant’ subjects, 13 benign lesions and five physiological uptakes were enrolled as false positive lesions. In the 10 ‘Normal’ subjects, eight benign lesions and 24 physiological uptakes were enrolled as true negative lesions. All malignant lesions and seven benign lesions were confirmed on biopsy. At 1 year after FDG-PET, 14 benign lesions and 29 physiological uptakes were confirmed as stable or diminished in uptake. In the present study, we examine the interpretations of these 68 lesions made by each physician.
Scanning of Whole-Body PET and CT Study
FDG-PET and CT of the 40 cases were performed in seven different institutions between April 2004 and March 2005. All PET images were obtained using a standardized protocol in accordance with the FDG-PET guidelines for cancer screening issued in 2004 by the Japanese Society of Nuclear Medicine. Patients fasted for at least 5 h prior to scanning. We obtained a whole-body PET image from the head to the thigh using a PET or PET/CT scanner at 50–60 min following the injection of 300–450 MBq of FDG. Transmission images were obtained to correct for photon attenuation using a germanium-68 line source. For PET/CT, PET attenuation correction factors were calculated from the CT images. We reconstructed image datasets using the ordered-subsets expectation maximization algorithm. We acquired a whole-body CT image from the head to the pelvis without intravenous contrast agent using a CT scanner or PET/CT scanner. The CT scanners and technical parameters were as follows: (i) Robusto (Hitachi Medico, Tokyo, Japan) multi-detector four row CT, 120 kVp, 100–160 mAs, beam pitch 1.75 and 10 mm thickness; (ii) Light Speed Ultra (GE Medical Systems, Tokyo, Japan) multi-detector eight row CT, 120 kVp, 175 mAs maximum with automatic exposure control system, 1.35 pitch and 5 mm thickness; (iii) Brilliance 16 (Philips Electronics Japan, Tokyo, Japan) multi-detector 16 row CT, 120 kVp, 150 mAs maximum with automatic exposure control system, 0.9 pitch and 5 mm thickness; (iv) Biograph LSO (Siemens-Asahi Medical Technologies, Tokyo, Japan) PET/CT with multi-detector two row CT, 130 kVp, 80 mAs maximum with automatic exposure control system, 1.15 pitch and 4 mm thickness; (v) Biograph LSO (Siemens-Asahi Medical Technologies), PET/CT with multi-detector two row CT, 130 kVp, 60 mAs, 1.5 pitch and 3 mm thickness; (vi) Eminence-SOPHIA (Shimadzu Corporation, Kyoto, Japan) helical CT, 120 kVp, 187.5 mAs, 1.4 pitch and 7 mm thickness; (vii) CT-Turbo (Hitachi Medico) helical CT, 120 kVp, 100–120 mAs, and 5–10 mm thickness. The mAs settings were selected to optimize spatial and contrast resolution. Tube current modulation was used to minimize the radiation dose to the individuals.
The PET/CT images were divided into PET and CT images, which were interpreted by side-by-side reading.
Observing Radiologists
FDG-PET and CT data were interpreted by six physicians with experience in both FDG-PET and CT, but who had not previously seen the study cases. The six observers were based at the six different cancer-screening centers with various recall rates from 1 to 44% [1, 10, 12, 23, 35, 44] that were mentioned in the introduction; each observer had between 4 and 10 years experience in reading screening FDG-PET.
Image Interpretation
Each observer reviewed each case in three steps. Step 1 involved interpretation of PET images alone, Step 2 involved side-by-side reading of PET and CT, and Step 3 involved re-evaluation of findings with reference to past history, smoking and drinking habits, and the results of other screening tests performed at the same time such as blood tests, fecal occult blood inspection and other imaging modalities that included magnetic resonance (MR) imaging for lower abdomen assessment and ultrasonography (US) for upper abdomen and thyroid gland assessment. In Step 2, each observer interpreted PET and CT by side-by-side reading without using fusion images. Findings of FDG uptake in each step were recorded as site and score depending on the likelihood of malignancy (1–5 points: 1, definitely not malignant; 2, probably not malignant; 3, equivocal; 4, probably malignant; 5, definitely malignant).
Each observer gave a score rating for every lesion that he/she considered to represent remarkable FDG uptake in each case. We analyzed each observer's interpretation of the 68 lesions that had been identified in advance as having confirmed reference data. Any of the 68 lesions that were not recognized by the observer as remarkable FDG uptake were given a score of 1 (definitely not malignant). Any lesions that were detected other than the 68 previously mentioned were excluded from analysis because of the absence of confirmed reference data.
All images were viewed with the same software using Synapse, medical imaging and information management network system, housed at the Fujifilm's demonstration showroom in Ginza, Tokyo, Japan.
Data Analyses
We assessed observer accuracy and variation using the scores given by each observer based on the relevant lesion. Sensitivity, specificity and positive predictive value (PPV) data were calculated from the interpretations of the six observers for each step. Receiver-operating characteristic (ROC) analysis was also performed as the standard method for evaluating observer accuracy, as sensitivity and specificity offer an incomplete description of accuracy and depend on the decision threshold selected by the observer to define positive diagnoses. The area under the ROC curve (Az) was used as a summary index of accuracy. Sensitivity for malignancy was calculated as the proportion of malignancies given a rating of 3–5. Specificity was defined as the fraction of benign lesions or physiological FDG uptakes for which a rating of 1–2 was reported. Inter-observer agreement in 68 lesions on the likelihood of malignancy (1–5 points) for each step was also assessed using the 
statistic.
Statistical Analysis
The Wilcoxon matched pairs signed rank sum test was applied to the sensitivity, specificity, and PPV means for each step to test for significant differences. Values of P < 0.05 were considered indicative of statistically significant differences. We calculated weighted values to describe concordance in reporting as ‘slight’ (0.00–0.20), ‘fair’ (0.21–0.40), ‘moderate’ (0.41–0.60), ‘substantial’ (0.61–0.80), or ‘almost perfect’ (0.81–1.00) (12,13). We conducted all analyses using MedCalc for Windows, version 7.6.0.0
[EC]
(MedCalc Software, Mariakerke, Belgium), except for ROC analysis, which was performed using the software ROCKIT (C. Metz, University of Chicago, Chicago, IL, USA). ROC software was used to fit a binormal ROC curve to the data from each observer and to compare Az for each pair using a univariate z-score test (14).
|
RESULTS |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
Diagnostic Accuracy of Each Observer
A summary of ROC curves obtained from interpretation in Step 1 using PET alone is shown in Fig. 1. Az values did not differ significantly between the six observers for each step. The means and dispersion of sensitivity, specificity, PPV and Az are shown in Table 1. Although Az values did not differ significantly, sensitivity, specificity and PPV varied widely between the observers.
|
|
Effect of Reference to CT on Diagnostic Accuracy
The mean specificity increased significantly when observers referred to CT (P < 0.05), although sensitivity, PPV and Az did not change significantly (Table 1).
Variability in Interpretation
The mean value and the strength of inter-observer agreement for each step are shown in Table 2. Inter-observer agreement for all lesions in each step was moderate ( = 0.58 for Step 1; = 0.55 for Step 2; and = 0.53 for Step 3).
|
Inter-observer agreement was higher for ‘Cancer’ and ‘Not malignant’ lesions than for ‘Normal’ lesions for each step (moderate versus fair).
Although the values for each group in Step 1 were higher than those in Steps 2 and 3, the differences were not statistically significant. The values of ‘Cancer’ and ‘Not malignant’ lesions in Step 2 were higher than those in Step 3; however, the differences were not statistically significant.
Patterns of FDG Uptake with Poor Agreement
In assessing inter-observer agreement for each organ, we determined the organs that were interpreted with difficultly by PET alone. The numbers of sites presenting poor or good agreement in each organ for the 68 lesions in Step 1 using PET alone are shown in Tables 3–5. Lesions for which fewer than five observers agreed in diagnosis were considered as poor agreement, while lesions for which five or six observers agreed in diagnosis were considered as good agreement.
|
|
FDG uptakes in the ‘Normal’ 32 true negative lesions that presented poor agreement included nine physiological uptakes in the larynx, mediastinum, intestine and ovary, and four benign lesions in the thyroid, neck, lung and uterus (Table 3). The case with physiological FDG uptake in the ascending colon is shown in Fig. 2.
|
FDG uptakes in the 18 ‘Cancer’ true positive lesions that presented poor agreement included 10 malignant lesions in the thyroid, hilum, breast, colon and stomach (Table 4).
|
FDG uptakes in the 18 ‘Not malignant’ false negative lesions that presented poor agreement included eight benign lesions in the thyroid, lung, colon and joint, and four physiological uptakes in the hilum, intestine and ovary (Table 5).
|
DISCUSSION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
Diagnostic Accuracy of Each Observer
We assessed observer accuracy using the scores representing a rating on the likelihood of malignancy for 68 lesions by each observer. No significant differences were identified for any step in Az of the six observers. Wide variation in sensitivity, specificity and PPV detected between observers was caused by differing decision thresholds during interpretation. Selection of different thresholds does not cause Az to vary, as an ROC curve depicts all of the tradeoffs available as the threshold is varied. Therefore, variability of the decision threshold between observers exists where no significant differences were identified in diagnostic accuracy as quantified with Az. That is, the scores given for each lesion could vary between the six observers even though there were no significant differences in diagnostic accuracy indicated by Az.
Effect of Reference to CT on Diagnostic Accuracy
We defined sensitivity as the proportion of malignancies given a rating of 3–5 and specificity as the fraction of benign lesions or physiological FDG uptakes given a rating of 1–2. This classification has critical meaning, therefore, Az has less value when comparing test performance between Steps 1 to 3. The mean specificity increased significantly in this study when observers referred to CT. Chen et al. also reported that additional CT for localization and lesion characterization showed an increased specificity of PET for cancer screening in asymptomatic individuals (4). In the present study, however, mean sensitivity and PPV did not change significantly when observers referred to CT. Several investigators report that the combination of FDG-PET and CT significantly improves diagnostic accuracy in the diagnosis of malignancy (15–17). The fact that the present results demonstrate no improvement in sensitivity and PPV may be due to selection bias: the present study did not include false negative lesions of PET that are recognized in CT. Sensitivity on general FDG-PET screening may be improved when observers refer to CT, given the inclusion of CT positive lesions that are without remarkable FDG uptakes such as bronchioloalveolar lung carcinoma (18).
Variability in Interpretation
We assessed inter-observer agreement on the scores for likelihood of malignancy (1–5 points) in 68 lesions. The 68 lesions that presented varying intensities of FDG uptake were founded in whole-body FDG-PET performed for cancer screening of 40 asymptomatic individuals.
Inter-observer agreement for all lesions in each step was moderate. Berg et al. reported that inter-observer agreement among radiologists on mammogram screening after training in Breast Imaging Reporting and Data System was moderate (9). Our results suggest that interpretation of FDG-PET in cancer screening is adequately reproducible as a whole.
Inter-observer agreement for all lesions at each step was moderate, compared to fair agreement for ‘Normal’ subjects. The higher prevalence of malignant lesions (18/68) means that the set used in this study was not strictly representative of FDG-PET within the general screening population. Inter-observer agreement on general FDG-PET screening might normally be lower, given the inclusion of a larger number of normal healthy subjects. Low inter-observer agreement may cause the marked variability in recall rate among the institutions that perform screening FDG-PET.
Inter-observer agreement was lower for ‘Normal’ lesions than for ‘Cancer’ and ‘Not malignant’ lesions for each step (fair versus moderate). Since sensitivity are calculated based on the data for ‘Cancer’ and specificity are based on those for ‘Not malignant’ and ‘Normal’, inter-observer variation observed for ‘Cancer’ and ‘Not malignant and Normal’ are corresponding to variability in sensitivity and specificity between observers (58.0–74.0, 72.2–83.3 in Step 1, respectively).
Inter-observer agreement decreased when observers referred to CT, however, the differences were not statistically significant. Although side-by-side reading of PET and CT improves lesion localization and supports lesion characterization, correlative interpretation of PET and CT differed between observers. Metser et al. concluded that in-line PET/CT offers better lesion localization relative to visual fusion of PET and CT, especially for small lymph nodes, lesions adjacent to mobile organs and lesions adjacent to the chest or abdominal wall (19). Syed et al. reported that PET/CT increases inter-observer agreement and confidence in disease localization of FDG-avid lesions in patients with head and neck cancers (20). By precisely localizing FDG uptakes, interpretation of image fusion by integrated PET/CT might offer higher inter-observer agreement in comparison to interpretation of PET images alone or side-by-side interpretation of PET and CT images.
Although inter-observer agreement of ‘Cancer’ and ‘Not malignant’ lesions decreased when observers referred to other information, the differences were not statistically significant. In cancer screening, positive lesions are eventually recommended for diagnostic work-up or observation with close follow-up. The clinical recommendation is determined by evaluation of the PET and CT findings with reference to past history, smoking and drinking habits, and results of other screening tests. Correlative interpretation of PET, CT and other information may need to be standardized to achieve greater agreement in subjects with suspected cancer by FDG-PET.
Patterns of FDG Uptake with Poor Agreement
We evaluated the organs that were most difficult to interpret by PET alone. Some observers tended to over-diagnose FDG uptake to avoid potential false negative outcome, while others did not pick up suspected lesions in ‘Normal’ subjects. Physiological FDG uptake is recognized at various sites in various degrees. A focal intense uptake in intestine mimics FDG uptake of colon tumor as shown in Fig. 1. Reporting criteria for various patterns of FDG uptakes in intestine differed between observers. For higher agreement on results for the ‘Normal’ 32 true negative lesions in FDG-PET, interpretation of physiological FDG uptake for the larynx, mediastinum, intestine and ovary should be standardized.
Various FDG uptakes in goiter, pneumonia, colon adenoma and arthritis confound image interpretation of FDG-PET and act to reduce inter-observer agreement. Interpretation of FDG-avid lesions in the thyroid, lung, hilum, breast, colon, stomach and ovary may require standardization for higher agreement in ‘Cancer’ and ‘Not malignant’ subjects.
|
CONCLUSIONS |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
Our results suggest that interpretation of FDG-PET in cancer screening is adequately reproducible, whereas interpretation of physiological FDG uptake in ‘Normal’ subjects is less reproducible. Improvement of inter-observer variability in assessing physiological FDG uptakes requires universal reporting criteria in FDG-PET. Furthermore, correlative interpretation of PET, CT and other information may require standardization in subjects with suspected cancer by FDG-PET.
|
Acknowledgments |
|---|
This research was supported in part by a Grant from Foundation for Promotion of Cancer Research and by a Health and Labor Sciences Research Grant for the project titled ‘Third Term Comprehensive Control Research for Cancer’ from the Ministry of Health, Labor, and Welfare, Japan.
Conflict of interest statement
None declared.
|
References |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSReferences |
|---|
1 Yasuda S, Ide M. PET and cancer screening. Ann Nucl Med (2005) 19:167–77.[Web of Science][Medline]
2 Kubota K. From tumor biology to clinical PET: a review of positron emission tomography (PET) in oncology. Ann Nucl Med (2001) 15:471–86.[Web of Science][Medline]
3 Yasuda S, Ide M, Fujii H, Nakahara T, Mochizuki Y, Takahashi W, et al. Application of positron emission tomography imaging to cancer screening. Br J Cancer (2000) 83:1607–11.[CrossRef][Web of Science][Medline]
4 Chen YK, Ding HJ, Su CT, Shen YY, Chen LK, Liao AC, et al. Application of PET and PET/CT imaging for cancer screening. Anticancer Res (2004) 24:4103–8.
5 Nakai K, Yasuda S, Ide M, Kawada S, Shohtsu A. Cancer screening with 18F FDG-PET. In: PET and PET-CT in Oncology—Oeher P, Biersack HJ, Coleman RE, eds. (2004) Heidelberg: Springer. 309–20.
6 Ak I, Stokkel MP, Pauwels EK. Positron emission tomography with 2-(18F) fluoro-2-deoxy-D-glucose in oncology. Part II. The clinical value in detecting and staging primary tumours. J Cancer Res Clin Oncol (2000) 126:560–74.[CrossRef][Web of Science][Medline]
7 Jiang Y, Nishikawa RM, Schmidt RA, Toledano AY, Doi K. Potential of computer-aided diagnosis to reduce variability in radiologists' interpretations of mammograms depicting microcalcifications. Radiology (2001) 220:787–94.
8 Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology (2005) 234:274–83.
9 Berg WA, D'Orsi CJ, Jackson VP, Bassett LW, Beam CA, Lewis RS, et al. Does training in the Breast Imaging Reporting and Data System (BI-RADS) improve biopsy recommendations or feature analysis agreement with experienced breast imagers at mammography? Radiology (2002) 224:871–80.
10 Herder GJ, Kramer H, Hoekstra OS, Smit EF, Pruim J, van Tinteren H, et al. Traditional versus up-front (18F) fluorodeoxyglucose-positron emission tomography staging of non-small-cell lung cancer: a Dutch cooperative randomized study. J Clin Oncol (2006) 24:1800–6.
11 Joshi U, Raijmakers PG, van Lingen A, Comans EF, Pijpers R, Teule GJ, et al. Evaluation of pulmonary nodules: comparison of a prototype dual crystal (LSO/NAI) dual head coincidence camera and full ring positron emission tomography (PET). Eur J Radiol (2005) 55:250–4.[CrossRef][Web of Science][Medline]
12 Kundel HL, Polansky M. Measurement of observer agreement. Radiology (2003) 228:303–8.
13 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics (1977) 33:159–74.[CrossRef][Web of Science][Medline]
14 Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol (1989) 24:234–45.[Web of Science][Medline]
15 Eubank WB, Mankoff DA, Schmiedl UP, Winter TC III, Fisher ER, Olshen AB, et al. Imaging of oncologic patients: benefit of combined CT and FDG PET in the diagnosis of malignancy. Am J Roentgenol (1998) 171:1103–10.
16 Aquino SL, Asmuth JC, Alpert NM, Halpern EF, Fischman AJ. Improved radiologic staging of lung cancer with 2-(18F)-fluoro-2-deoxy-D-glucose-positron emission tomography and computed tomography registration. J Comput Assist Tomogr (2003) 27:479–84.[CrossRef][Web of Science][Medline]
17 Yoshida Y, Kurokawa T, Kawahara K, Tsuchida T, Okazawa H, Fujibayashi Y, et al. Incremental benefits of FDG positron emission tomography over CT alone for the preoperative staging of ovarian cancer. Am J Roentgenol (2004) 182:227–33.
18 Higashi K, Ueda Y, Seki H, Yuasa K, Oguchi M, Noguchi T, et al. Fluorine-18-FDG PET imaging is negative in bronchioloalveolar lung carcinoma. J Nucl Med (1998) 39:1016–20.
19 Metser U, Golan O, Levine CD, Even-Sapir E. Tumor lesion detection: when is integrated positron emission tomography/computed tomography more accurate than side-by-side interpretation of positron emission tomography and computed tomography? J Comput Assist Tomogr (2005) 29:554–9.[CrossRef][Web of Science][Medline]
20 Syed R, Bomanji JB, Nagabhushan N, Hughes S, Kayani I, Groves A, et al. Impact of combined (18)F-FDG PET/CT in head and neck tumours. Br J Cancer (2005) 92:1046–50.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  Y. Nakamoto, K. Togashi, T. Kaneta, H. Fukuda, K. Nakajima, K. Kitajima, K. Murakami, H. Fujii, M. Satake, U. Tateishi, et al. Clinical Value of Whole-body FDG-PET for Recurrent Gastric Cancer: A Multicenter Study Jpn. J. Clin. Oncol., May 1, 2009; 39(5): 297 - 302. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||