| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Department of Radiology, Cincinnati Children's Hospital, Cincinnati, Ohio
Correspondence and requests for reprints should be addressed to Alan S. Brody, M.D., Cincinnati Children's Hospital, 3333 Burnet Avenue, Cincinnati, OH 45229-3039. E-mail: alan.brody{at}cchmc.org
ABSTRACT
Over the last 15 years, several clinical trials in subjects with cystic fibrosis (CF) have employed computed tomography (CT) scanning as an outcome surrogate. These research trials have increased our knowledge about the appearance of the lungs in patients with CF, and the relationship between the CT appearance, pulmonary function tests, and clinical outcomes. In addition, practical information has been gained on the use of CT scanning in CF research trials. In this article, practical issues on the use of CT scanning in CF clinical trials are presented and specific lessons from three of these trials are discussed.
Key Words: clinical trial • surrogate endpoint • diagnostic imaging • high-resolution computed tomography
COMPUTED TOMOGRAPHY SCANNING IN CYSTIC FIBROSIS RESEARCH TRIALS
Using computed tomography (CT) scanning in a research trial presents challenges that are very different from those found with more commonly used outcome surrogates, such as pulmonary function testing. In the case of pulmonary function testing, a standard, well-understood methodology is available that can be applied to both clinical and research studies (1). These guidelines provide the pulmonary function testing laboratory with the necessary information to perform the study as needed for the research trial. In addition, the personnel in the pulmonary function laboratory are usually familiar with conducting research trials. Working with research coordinators and accommodating the needs of research subjects is a well developed process at most sites. The situation is very different for CT scanning. Incorporating CT scanning into a trial requires the development of an infrastructure that may exist in a limited fashion, or not at all, at most centers.
In most institutions, CT scanning is a clinical test that is rarely used in the research setting. The radiologist performing the clinical study usually determines the CT protocol. This CT protocol may be different for each physician, and is almost always different at each institution. Because research studies are performed according to a specific study protocol rather than the clinical protocol routinely used by the site, it is important to have participation by both radiologists and radiology technologists in the CT area in order to assure that the required protocol is followed. Radiologists have little or no time to devote to this process at most sites, and there are no research coordinators in the radiology departments. The radiologist is expected to fill the roles of patient coordinator, quality assurance overseer, and data manager.
In the United States, radiology departments are revenue centers for their hospitals, and radiologist salaries are high. The three areas in the hospital that are known to make the most money for the hospital are the operating rooms, laboratory services, and the radiology department. Hospitals count on this income to support the many other areas that do not generate a profit. Research studies compete with clinical studies for time and equipment, and the radiology department is held accountable for this decrease in income (2). CT scans performed as part of a research protocol are often reimbursed at a lower rate than clinical tests. In addition, radiologists' time is expensive; salary support at the National Institutes of Health's highest level pays about two thirds of the salary and benefits of a new assistant professor of radiology. Thus, whereas most radiology departments expect their radiologists' revenue to exceed their salaries, radiologist participation in research trials represents a cost to radiology departments.
The use of CT scanning in research trials also raises ethical and liability issues. The most obvious concern is that, when a CT scan is performed, a thoracic neoplasm or other clinically significant finding could be identified. Ethically, any such findings must be communicated to the subject's physician so that the subject can receive the appropriate care. In addition, concern has been raised by both sponsors and investigators that there could be legal liability if such a finding is not communicated. Institutional review boards (IRBs) have requested specific methods for review of the CT scans and communication of important findings to subjects and/or their physicians.
CT scanning requires ionizing radiation. The risk of cancer from this radiation is an area of active discussion (3). Careful analysis and presentation of the radiation dose from the CT scanning protocol is required for IRB application and when obtaining subject consent. The discussion of radiation risk by Huda in this symposium (pp. 316–320) provides a helpful example of the information that is needed.
CT scans provide graphic data in the form of a series of images. The image data must be converted to numeric data to allow statistical analysis. The most common way to perform this conversion is to use expert observers who provide scores for the appearances seen on the CT scans. This requires specific training and analysis of intraobserver and interobserver variability. Computerized scoring systems have been used to assess air trapping. In one intervention study, air trapping analysis showed an improvement not detected by expert reader score (4). Computer analysis cannot yet include all the observations made by expert observers. Some operator involvement may be required to select and prepare images for analysis. In addition, issues, including data compatibility and the need to transfer large amounts of data, may increase the time required for computerized scoring.
CT AND RHDNASE
The Pulmozyme Early Intervention Trial was a 96-week, randomized, double-blind, placebo-controlled trial involving 49 cystic fibrosis (CF) centers and 474 subjects. Subjects were 6–10 years old, with an FVC of 85% predicted or greater who were treated with dornase 
2.5 mg or placebo once daily (5). The U.S. Cystic Fibrosis Foundation funded a supplemental study evaluating CT scanning and systemic markers of inflammation, in which CT scans were performed at the beginning, middle, and end of the trial at 13 sites in the United States.
This study provided important new information, both about CF lung disease in young children and about the use of CT scanning as an outcome measure. The morphologic abnormalities of these children were described, and the correlation between morphology and pulmonary function testing was assessed (6). Longitudinal evaluation allowed demonstration of correlation between the outcome measure (CT scanning) and a true patient outcome (respiratory tract exacerbations) (7). This study demonstrated the dissociation between morphologic changes and pulmonary function in young children, and showed that FEV1 is insensitive to the lung disease in young patients with CF. Other studies have shown that these limitations of pulmonary function tests are not limited to young children (8, 9).
Important practical information was gained as well. This study demonstrated that CT scanning could be used in a multicenter trial of young children. A central coordinating center was established that provided oversight for the study centers and coordination for scoring and data tabulation. A high-resolution CT (HRCT) protocol was developed that was used at all participating centers with no CT scans excluded due to inadequate quality.
Although all studies were adequate for interpretation, there was marked variability between different sites in terms of the quality of the CT scans. Some sites did not comply with all requirements of the CT protocol when the study was initiated at their site. Initially, a detailed CT protocol was provided. At many sites, this protocol was ignored in favor of the standard clinical protocol, requiring additional imaging in some cases. A second CT protocol was provided, which included a check sheet followed by a detailed description of the protocol. The CT scans that resulted, and discussion with the site personnel, suggested that the check sheet was used without reference to the protocol. A final protocol that included the detailed protocol filling three quarters of the page, with a series of checkboxes with summary statements in a column occupying one quarter of the page, provided the best compliance with the study protocol.
The CT scans in this study were scored by expert readers using a newly developed scoring system. An additional practical consideration was the limited availability of radiologists to serve as expert readers of these scans, and the effort required to perform these readings. In the study, formal readings were completed on only the initial and final CT scans due to these constraints. In addition, the study group was decreased to the subset in which the CT scans were completed before the study drug was administered, again to decrease the amount of time required of the expert readers. Interobserver and intraobserver variability have been evaluated using the scoring system that was employed in this study (10). Although good interobserver and intraobserver reproducibility was demonstrated, reader scoring varied. It was recognized that it is important to maintain a constant group of readers throughout the study in order to minimize these effects.
Additional practical issues included delays in obtaining IRB approval and financial issues. IRB approval usually requires the additional step of radiation safety board evaluation, which delayed IRB approval by several months in some cases. There were financial issues in determining the cost of research CT scans and in providing support for the necessary radiology personnel.
ADENO-ASSOCIATED VIRUS SEROTYPE 2 GENE TRANSFER STUDY
The Targeted Genetics adeno-associated virus (AAV) gene transfer study used CT scanning to assess safety and as an exploratory endpoint for effectiveness (11). This study was performed at eight sites in the United States. Inclusion criteria included age greater than 11 years and FEV1 of 60% predicted or higher. HRCT was performed at baseline and at Day 90; 35 subjects were studied at baseline, and 31 at Day 90. The age range was 12–54 years.
No evidence of inflammation or other pulmonary toxicity was seen. CT scores at 90 days were unchanged relative to baseline in the control and treatment groups. This study demonstrated that the dissociation between structure and function seen in young children is also seen in older patients. In this cohort with broad variation in pulmonary function as well as disease severity by CT, there was fair to good overall correlation between lung function and CT score. However, there were many subjects with discordant results (Figure 1). In addition, there was a broad range of lung disease, as observed on CT in patients with normal pulmonary function (Figure 2). When comparing subjects with FEV1 greater than 90% predicted to those with FEV1 less than 80% predicted, there was overlap of the CT scores, with bronchiectasis and mucous plugging seen frequently in both groups (Table 1).
|
|
|
In response to concerns about medical liability and appropriate care, this and other recent trials have had the CT scan read at the originating site as though it were a routine clinical study. A clinical report is generated, which goes into the patient's chart, providing a full reading of the CT scan, and the CT scan is maintained at the institution in the same manner as all clinically ordered CT scans. This plan, though minimizing liability, adds significantly to the cost of CT scanning performed as part of research protocols.
All sites agreed to participate in the study, although concerns about the lack of funding for a site radiologist were raised at numerous sites. A responsible radiologist was identified at each site, but these radiologists frequently referred quality assurance and protocol concerns to the central coordinating site rather than addressing them directly, citing a lack of time due to the lack of funding.
DENUFOSOL TETRASODIUM INHALATION SOLUTION IN CF
The Inspire 08-103 trial was a 28-day study of denufosol tetrasodium, an inhaled P2Y2 agonist, in subjects with FEV1 of 75% predicted or higher. The study was performed at 14 sites in North America, with CT scans obtained at the baseline and end of the study. A total of 83 subjects participated, age 8–45 years.
Expert reader CT scoring was performed on all CT scans, and computer analysis of air trapping was conducted in a subset of 28 subjects. No significant changes in either the CT scores or air trapping were found. There was a significant difference in FEV1 between the treatment and control groups.
This study clearly showed the limitation of the HRCT technique. HRCT is an interval technique, with one 1-ml slice obtained every 10 ml, so that in serial studies it is possible to have as much as 5 ml difference in the location of the 1-mm sections. Figure 3 shows serial images in two subjects. In Figures 3A and 3B, the same level was imaged by chance, and accurate comparison is possible. In Figures 3C and 3D, slices were not obtained at an identical level. On the first CT (Figure 3C), a bronchus is seen that can not be seen on the second CT (Figure 3D), so no evaluation of interval change can be made. Although the sampling technique of HRCT is useful in many circumstances, it limits our ability to identify specific areas and assess them for interval change, decreasing the ability to detect a therapeutic effect. Volumetric scanning that images the entire lung markedly improves the ability to identify and compare specific structures on serial CT scans.
|
Image quality was specifically evaluated in this trial. A total of 95% of the inspiratory images were judged good or excellent, and 84% of the expiratory images were judged good or excellent. The most common limitations in the expiratory images were incomplete expiration and motion. In addition, inspiratory lung density of normal-appearing areas of lung was measured to assess the level and reproducibility of voluntary inspiration. Lung density showed an overall variance of approximately 7% in all subjects participating in this study compared to 3% in subjects studied at a single site using spirometer control of lung volume.
Mucous plugging was a finding that was felt likely to improve in response to denufosol. Review of the subjects in the Targeted Genetics AAV2 trial showed mucous plugging by CT scan in 40% of subjects with an FEV1 of 90% predicted or higher, and in 73% of subjects with an FEV1 between 60 and 80% predicted. Hence, it was surprising to find that only 20% of the lobes in the denufosol patients had mucus plugging at baseline as recorded by either reader. Evaluation of lobes rather than subjects among the AAV2 trial subjects revealed mucous plugging in only12% of lobes. Evaluating lobes rather than subjects is more appropriate because the scoring system assesses each lobe. Mucous plugging must be present in order to detect an improvement, and detecting a change is dependent on the number of lobes that have mucous plugging. When using previous studies to provide power estimates for future trials, very careful assessment will be necessary to determine whether the comparison group, the CT technique, and the proposed intervention are similar enough to allow the previous results to be applied to the new trial.
DISCUSSION
The use of a standardized protocol for pulmonary function tests makes it easy to include them in research trials. The sentence in many methods sections: "pulmonary function tests were performed according to ATS guidelines" provides a complete description for the pulmonary function laboratory. With CT scanning, no such widely accepted guidelines exist. In many ways, performing a research CT scan is more like performing pulmonary function tests than like the clinical CT scans performed in the radiology department. CT scans done for research require careful patient coaching and adherence to an externally specified protocol and acceptance criteria, rather than a protocol determined by individual preferences. To move forward, it will be necessary to develop detailed standard operating procedures and to document the ability of the radiology department to perform CT scans according to that protocol before beginning a research trial. For research studies where subtle changes need to be detected over time, a contiguous thin slice protocol is recommended. Spirometric gating or other measures to assure reproducible lung volumes will need to be considered (13).
Continued development of expert scoring and a computerized analysis system is needed. In addition to validating the scoring system, evaluation of expert readers should be performed before beginning a trial in order to establish the specific scoring behavior of each individual. Ideally, computer analysis systems should allow more rapid analysis with near perfect intratest reproducibility. Minimizing such expert observer tasks as matching images and identifying regions of interest will be important steps in reaching that goal. Systems that simplify entry of Digital Imaging and Communications in Medicine data should decrease both turnaround time and the likelihood of errors.
Increasing interest in using imaging outcomes for many other disease processes will allow radiology departments to support the development of this new role for medical imaging. There is widespread support for the research role of radiology departments (14–16). Financial considerations will have to be addressed and a research infrastructure will need to be developed. Investigators in CF have the opportunity to aid in this development as new studies using CT scanning in the evaluation of CF lung disease are planned.
CONCLUSIONS
The three trials reviewed in this section have provided valuable lessons in the use of CT scans as surrogate endpoints in CF clinical trials. All sites successfully performed the CT scans with acceptable image quality. A central coordinating center and centralized scoring were used successfully. In an era of increasing concern about radiation exposure, CT scanning was approved by the IRBs at all sites. No sites reported that potential subjects refused to participate because of the radiation risk. No sites were unable to participate due to financial considerations. Moving forward with CT scanning as an endpoint for research trials will require improvements in CT technique, in the interpretation of the CT scans, and in the research infrastructure in radiology departments.
FOOTNOTES
Conflict of Interest Statement: A.S.B. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript.
(Received in original form November 30, 2006; accepted in final form March 8, 2007)
REFERENCES
Related articles in Proceedings of the American Thoracic Society:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
