3-D Image Guidance for Minimally Invasive Robotic Coronary Artery Bypass

(#2000-9732 ... June 8, 2000)

Adeline M. Chiu, BASc¹, Damini Dey, PhD¹, Maria Drangova, PhD¹, W. Douglas Boyd, MD² , Terence M. Peters, PhD¹

¹The John P. Robarts Research Institute
²The London Health Science Centre University of Western Ontario, London, Ontario, Canada

ABSTRACT

Background: The introduction of a robot-assisted microsurgical system has made endoscopic coronary artery bypass grafting (ECABG) possible. Despite the success of this approach, surgeons still require better visualization tools for pre-surgical planning and intra-operative image guidance. Such visualization tools could, for example, assist in the placement of thoracic ports to acquire optimum access to the target vessels. In this paper we discuss the essential steps toward image-guided completely endoscopic coronary bypass surgery with robot assistance, and we present our preliminary efforts toward the development of a three-dimensional (3-D) virtual cardiac surgical planning platform (VCSP) for ECABG.

Methods: Preoperative 3-D images of the thorax acquired with computed tomography and electrocardiogram-gated magnetic resonance imaging are imported into VCSP. Using VCSP, a user may interactively visualize and manipulate the simulated thoracic ports in 3-D within the reconstructed thoracic region. We have also implemented a virtual endoscope to simulate the endoscopic view observed by the surgeon during the operation. Once the port placements for optimal access to the target vessels are determined, the positions of the simulated tools can be recorded and marked on the patient to specify the positions for port incisions.

Results: A static thorax phantom was used to verify the port placements obtained from VCSP simulations. The angles and the distances between the ports, the endoscope and the markers that were placed on the surface of the phantom were measured, and the results were compared with those obtained from simulation. The physical measured distances and angles agreed with the simulated results with average errors of 4 mm and 2 degrees, respectively.

Conclusions: The VCSP image-guided surgical system allows a surgeon to visualize a patient's thorax in a 3-D interactive environment for planning surgical procedures, and to determine the optimum port placement based on preoperative 3-D images. However, during an operation, the positions and orientation of the heart and the coronary arteries are changed from their corresponding locations in the preoperative images due to carbon-dioxide insufflation, lung deflation, and dynamic motions of the beating heart. One of our future goals of this project is the use of mathematical models that correct for these changes so that our system could be applied to intra-operative image guidance.

INTRODUCTION

Robot-assisted microsurgical systems have recently been introduced to minimally invasive direct coronary artery bypass (MIDCAB) procedures and are rapidly gaining acceptance. The port-access robotic tools greatly increase the precision in MIDCAB and make completely endoscopic anastomosis possible [Boehm 1999, Falk 1999, Loulmet 1999, Reichenspurner 1999, Boyd 2000, Damiano 2000]. Despite its recent success, robot-assisted MIDCAB has several technical limitations. These problems must be overcome to further reduce trauma and risk to patients, which would lead to shorter hospital stays and lower health care expenses.

One of the common technical limitations is that often only conventional 2-D angiograms and chest X-rays are available to the surgeons for preoperative planning. Without the visual perception of depth, it is difficult to locate the left internal mammary artery (LIMA) and the left anterior descending artery (LAD), and to estimate the sizes and shapes of the intercostal spaces. Moreover, some patients might have a small LAD or a small internal mammary artery (IMA) or an enlarged left ventricle, which leads to subsequent lateral positioning of the LAD [Gulbins 1998]. If all of this information could be provided to the surgeons ahead of time, the procedures could be modified, if necessary, before the actual operations, leading to shorter operating times. Preoperative three-dimensional (3-D) reconstruction of electron beam tomography (EBT) images [Achenbach 1997, Gulbins 1998] has been shown to be capable of identifying patients with potential technical difficulties and has proven to be a very useful tool for planning MIDCAB procedures. However, since EBT is not available for all the MIDCAB sites, we propose to use other imaging modalities to acquire the topology of the thorax. We are implementing a multimodality approach where images from electrocardiogram (ECG)-gated magnetic resonance imaging (MRI) and from conventional computed tomography (CT) are fused together. ECG-triggered MRI is used to locate the IMA and LAD, along with CT images, to obtain information about the rib cage. These images are then imported into our 3-D visualization tools that allow users to interactively examine the thoracic topology of a patient for planning surgical procedures.

Another challenge of MIDCAB is improper thoracic port placement, which limits the access to the target vessels [Loulmet 1999, Boyd 2000]. Also, improper placement may prolong the MIDCAB procedure, or even result in it being abandoned in favor of a conventional open-chest approach, affecting the outcome of surgery. Our 3-D visualization tool is equipped with simulated thoracic ports and a virtual endoscope that provides assistance in the planning of the port placements.

In addition to the two challenges mentioned above, there is a lack of relative context between the surgical tools and their surroundings due to the limited field of view of the endoscope and the confined working space in minimally invasive surgery. This poses a great challenge to IMA harvesting, identification of the LAD, and tracking of the tools and the distal ends of the vessels during anastomosis. Even the use of an auxiliary endoscope often proves to be inadequate. Ideally, surgeons require a larger field of view of the target site than is currently provided by an endoscope. These problems are similar to those that have been identified over the years for neurosurgery. The image-guided surgery laboratory in our institution has been active in the development of tools that fuse intra-operative endoscopic frames to preoperative 3-D MRI/CT brain images in order to address these problems. This technique is being adapted for cardiac surgery to provide relative context among the tools, the targets and the surrounding area.

MATERIALS AND METHODS

In order to achieve 3-D image guidance for robot-assisted totally endoscopic coronary artery bypass surgery, the problems discussed above should be resolved. First, 3-D CT and MRI images of the patient's thorax are acquired and imported into our software surgical planning platforms to reconstruct a 3-D model of the patient's thorax. Port placement simulation is achieved with the aid of simulated surgical tools and a virtual endoscope.

3-D Thorax Data Acquisition

In our preliminary studies, human data were used to demonstrate the feasibility of developing a 3-D image-guidance surgical planning platform for minimally invasive robotic coronary artery bypass. A 1.5T cardiac ECG-gated MR scanner (Signa CVMR, General Electric Medical Systems, Milwaukee, WI, USA) was used to image the heart and thorax of a 23-year-old male volunteer. The ECG-gated fast-gradient echo sequence was used, without MR contrast, to acquire 31 images, each 5 mm thick, spanning the range between the thoracic inlet and the apex of the heart. For each axial position, 20 images equally spaced in time were acquired during each cardiac cycle. Images at each axial position were acquired during a breath-hold lasting approximately 18 seconds. The volunteer was instructed to hold his breath at the same level during each breath-holding interval. The acquisition matrix was 256 x 256, and the field of view was 32 x 32 cm², giving a pixel size of 1.25 x 1.25 mm².

In the MRI images, the rib cage has low contrast to its surroundings tissues and thus cannot be differentiated accurately. We therefore propose the use of a conventional CT system to acquire information regarding the ribs. Integration of the CT images and the MRI volumes will enable the reconstruction of the entire thoracic cavity. The registration of MRI data set to the CT data, or vice versa, is achieved by manually adjusting the origins and the spacings of the data sets or by using a software registration tool, such as Register (McConnell Brain Imaging Centre, Montreal, Canada) [MacDonald 1993]. The volunteer for the MRI images was not imaged using CT due to the radiation dose associated with CT imaging. Instead, to demonstrate the proposed role of CT in our project, the CT data set of a male cadaver from the Visible Human project [National Library of Medicine 1994] was obtained. This data set consists of 40 slices, each 3 mm thick, reconstructed on a 512x512 matrix with a pixel size of 0.9x0.9 mm².

Segmentation of the Data Sets

The IMAs, left coronary artery and the heart were extracted from the MRI images, and the ribs and the chest wall were extracted from the CT images. The segmentation was performed semi-automatically using thresholding, 3-D region growing, and manual masking of regions with the Multimodality software (Nuclear Diagnostics, Stockholm, Sweden) [Slomka 1995].

3-D Reconstruction

The thorax data was imported into the surgical planning platforms and visualized in a 3-D user-interactive environment. The software and hardware required to operate the programs are discussed below. Two surgical planning platforms are also described.

3-D Graphics Software and Hardware

We all have experienced the astounding 3-D graphics of today's video games. Our image-guided surgery laboratory takes advantage of the OpenGL-based 3-D graphics libraries, used by video game programmers, to develop tools for visualizing medical data. The Visualization Toolkit (VTK) [Schroeder 1997] is an object-oriented library of C++ classes that allows efficient development of software to "see into" 3-D data sets. VTK is a collection of image processing algorithms and visualization libraries based on OpenGL. It allows programmers to build applications with several computer languages, and we have chosen to use Python, a scripting language, to build our surgical planning platforms using the VTK libraries. The resulting products are multi-platform and can be run without recompilation on Linux, Unix, Windows and Mac operating systems. A typical computer configuration required to run the surgical planning platforms is Pentium III 650 MHz, 128MB RAM, and an 8MB video card with hardware 3-D acceleration for OpenGL. Displaying the Original Data Sets Using Atamai Surgical Planning

The MRI and CT images are imported into Atamai Surgical Planning platform (ASP) [Atamai 2000], a multi-modality 3-D image visualization and manipulation system originally developed for neurosurgical image visualization, allowing image-to-image and image-to-patient registration, as well as the tracking and simulation of surgical tools. In addition to displaying images in orthogonal planes, ASP also allows the interactive display and manipulation of four-dimensional ECG-triggered MRI cardiac data in a cine loop, in order to achieve a real-time rendering of the dynamic beating heart.

Rendering 3-D Surfaces of Extracted Objects Using Virtual Cardiac Surgical Planning Platform

We are extending ASP's capabilities and adapting it for cardiac surgery to create a similar platform customized for robot-assisted MIDCAB. We have named this system the virtual cardiac surgical planning platform (VCSP). The extracted surfaces of IMAs, LAD, ribs, chest wall, and the heart are imported into VCSP and visualized in a 3-D interactive environment. Within VCSP, a user can zoom in and out of the 3-D rendered objects and view them from any angle in order to examine the topology of the thorax. He can also displace and rotate individual objects, as well as peel off the skin and expose the target for planning thoracic port placement. VCSP has a 3-D ruler option that allows a user to measure the distance between any two points in space, for example, the distance between the LAD and the IMA.

Simulated Tools

VCSP has two simulated instrument ports. The positions of these simulated ports can be manipulated with a mouse or a free-hand tracking system, whose position is tracked by a magnetic-field based tracking device (Flock of Birds; Ascension Technologies, Burlington, VT) [see Figure 1 :38:]. A magnetic sensor is attached to each dummy port that is in turn mounted on a flexible arm. While a user is adjusting the ports in real space, Flock of Birds tracks the positions of the dummy ports. The tracking system then sends the positions of the dummy ports to VCSP, which automatically updates their positions on the computer screen in real time.

Virtual Endoscope

VCSP has a virtual endoscope that simulates the actual endoscopic view during an operation. The user may simultaneously manipulate the simulated ports through the virtual endoscopic view and visualize a perspective view of the entire thoracic cavity. The movement of the virtual endoscope is controlled with a mouse or with customized buttons that are associated with the commands of the actual voice-controlled endoscope.

Determining the Optimum Port Placement Configuration

Tabaie et al. has described an ideal instrument and endoscope port configuration [Tabaie 1999]. In general, the instrument ports and the endoscope form a "magic pyramid" [see Figure 2 :39:]. The parameters of this pyramid are: 1) the distances among the instrument ports and the endoscope, 2) the angle between the instrument ports, and 3) the angle between the endoscope and the plane formed by the ports. VCSP automatically calculates the angles. The distances can be measured using the 3-D ruler. In addition to the magic pyramid, it is suggested that three markers be placed on the surface of the chest. The distances from each incision point to all the markers are then measured to define the location of the incision point on the surface of the chest.

Automatic Fusion of Intra-Operative Endoscopic Images to Preoperative Images

The process of fusing endoscopic and 3-D MRI/CT images has been adapted from other work in our laboratory relating to neuro-endoscopy, as described by Dey et al. [Dey 2000]. In the neurosurgical application, the state of the brain is monitored using an endoscope tracked with an optical tracking system. Endoscopic video images of the brain (physical space) are digitized and registered to the MRI/CT images (preoperative space) using markers that could be identified in both spaces. The endoscopic images are then "painted" onto the larger and more informative preoperative images via a novel texture-mapping algorithm. This automatic fusion technique is being adapted for our application in cardiac surgery.

RESULTS

Visualization of Raw Data in ASP

The ECG-triggered MRI images were imported into ASP. Figure 3 shows an axial slice about 4 cm below the aortic branch to the heart, including the right ventricle, left ventricle, LAD, and IMA [see Figure 3 :40:]. The 3-D volume data set is displayed in orthogonal planes [see Figure 4 :41:]. ASP is also capable of displaying a dynamic beating heart (see Movie). Visualization of Segmented Objects in VCSP

The rib cage and the chest wall were segmented from CT visible human data. The IMAs, left coronary artery, and the heart were segmented from the ECG-triggered images. The segmented objects were surface rendered and imported into VCSP, [see Figure 5 :42:]. The topology of the thorax was examined interactively from different angles and magnification scales (see Movie). Distances measurements (for example, the sizes of the intercostal spaces) can be made within this environment.

Verification of Port Placement

Because it was not feasible to insert the actual thoracic ports into the volunteer, a static thorax phantom model was used to verify the port placement simulation. This thorax phantom [see Figure 6a :43:] is routinely used for robot-assisted endoscopic anastomosis training. First, the target inside the thorax was located with the endoscope. Then, the dummy instrument ports were imported. The endoscopic images were recorded for subsequent verification of the virtual endoscope views. Three markers were placed on the surface of the phantom, and CT images of the phantom were obtained with the ports physically inserted. Surfaces corresponding to the ribs, target and the ports were extracted and imported into VCSP [see Figure 6b :44:]. The distances from the markers to each of the incision points of the tools were measured, and measurements of the magic pyramid were also made as tabulated in Table 1 and Table 2 [see Table 1 :47:, Table 2 :48:]. The surfaces corresponding to the instrument ports and the endoscope were then removed from the reconstructed 3-D thorax phantom. The VCSP virtual endoscope and simulated tools were inserted into the resulting data set [see Figure 7 :45:]. Several port placement configurations were attempted, including the left chest approach and the subxiphoid approach. Measurements of the simulated instrument configuration that was closest to the actual configuration were recorded and tabulated in Tables 1 and 2 [see Table 1 :47:, Table 2 :48:]. The virtual endoscopic views were then compared with those actually recorded [see Figure 8 :46:], and the actual measurements of the instrument configuration were compared with the simulated results. The average errors of measured distances and angles are 4 mm and 2 degrees, respectively.

DISCUSSION

Recently, in MIDCAB, port-access robotic tools have made completely endoscopic anatomosis possible. However, several technical difficulties have been identified. We discussed the essential steps that are needed to overcome these challenges and presented our preliminary efforts toward the development of an image-guided approach to completely endoscopic robot-assisted anastomosis.

The first problem in MIDCAB that is discussed relates to the surgeon's reliance only on conventional 2-D angiograms and chest X-rays to plan surgical procedures. Without the visual perception of depth, it is difficult to locate the IMAs and LAD during the operation. We presented a 3-D multi-modality imaging approach and demonstrated the capability of CT and ECG-triggered MRI to locate IMAs, LAD, ribs and other thorax structures. The disadvantage of the multi-modality imaging approach compared with EBT is that the former requires an extra step to fuse MRI images to CT images. However, the general lack of availability of EBT scanners makes it unavailable at most MIDCAB sites. We note, however, that there is growing evidence that new multi-slice helical CT scanners that are becoming available may be able to acquire sufficient high-quality CT data to reconstruct the dynamic 3-D heart [Freiherr 2000]. Such a multi-slice CT scanner will be available to our laboratory in the near future, and the need for integrating the CT images with ECG-gated MRI will then be re-assessed.

The acquired MRI and CT data sets were visualized in two display formats. 1) The ECG-triggered MRI cardiac frames were imported into our neurosurgical planning platform, ASP, which allows visualization of the dynamic motion of the beating heart in orthogonal planes-display format. 2) The extracted surfaces of the structures of interests, such as IMA, LAD and ribs, were imported into VCSP to create a 3-D interactive thorax model to provide assistance in planning surgical procedures. It should be noted that the MRI images were acquired with neither contrast agent nor respiratory gating. The 3-D reconstructed model showed only the location of the LAD and could not identify calcifications in the coronary arteries of a patient. Also, this model would not be sufficient to plan the location of the anastomosis, which would require either magnetic resonance or conventional X-ray angiography to visualize it.

The second challenge in MIDCAB procedures is proper port placement. We demonstrated that VCSP is capable of determining optimal port placement with the use of its user-interactive virtual thoracic ports and its virtual endoscope. In the preliminary error analysis of the port placement simulation, the errors are associated with identification of centers of the ports at the incision points, segmentation algorithms, size of the target, VCSP user's experience, and the accuracy of the magnetic tracking device in the case of using a free-hand tracking system.

It should be noted that the positions and orientations of the heart and arteries change dramatically between imaging and surgery due to the carbon dioxide insufflation of the chest cavity, lung deflation and dynamic motions of the beating heart. Thus the port simulation results from the VCSP cannot be applied directly to intra-operative image guidance. One of our goals for the future is the use of finite-element modeling techniques to predict and correct for the changes in positions of the heart and arteries, and the application of these techniques to VCSP.

The third challenge in MIDCAB is the limited field of view of the endoscope in the confined thoracic working spaces. We have demonstrated real-time merging of multiple tracked endoscopic views of brain tissue to the corresponding 3-D preoperative surface via texture-mapping for neurosurgery. In addition to enhancing visualization, this provides the necessary 3-D relative context. We are adapting this technique for cardiac surgery to assist surgeons in locating tools and targets during surgery. However, this method does not account for tissue shifts during an operation. A significant difference between the neurosurgical and cardiac situations is the dramatic changes in positions of the heart and arteries in cardiac surgery. Again, we plan to employ mathematical modeling techniques and local re-registration approaches to account for these changes.

We conclude that VCSP is a valuable tool that allows surgeons to examine the topology of a patient's thorax for planning MIDCAB surgical procedures. VCSP is capable of determining the optimum port placement based on preoperative 3-D images and of providing virtual endoscopic views. We recognize that the work presented here is merely the first step toward providing a comprehensive thoracic cardiovascular surgical planning system. Such a system must correct for position shifts of the heart and the coronary arteries in preoperative images before our existing techniques can be applied to intra-operative image guidance.

AUTHOR/ARTICLE INFORMATION

Presented at the Third Annual Meeting of the International Society for Minimally Invasive Cardiac Surgery, Atlanta, Georgia, June 8-10, 2000.

Reprint requests to: Adeline Chiu, Imaging Research Laboratories, John P. Robarts Research Institute, University of Western Ontario, 100 Perth Drive, London, Ontario, Canada, N6A 5K8, Phone: (519) 663-5777, ext. 34213, Fax: (519) 663-3403, Email: achiu@irus.rri.on.ca

Submitted on: This manuscript was peer reviewed and accepted for presentation at the Third Annual Meeting of the International Society for Minimally Invasive Cardiac Surgery, Atlanta, Georgia, June 8-10, 2000.

REFERENCES

1. Achenbach S, Moshage W, Ropers D, Nossen J, Bachmann K. Noninvasive, three-dimensional visualization of coronary artery bypass grafts by electron beam tomography. Am J Cardiol 79:856-61, 1997. :9104894:

2. Atamai Surgical Planning, London, Ontario, Canada. Available at: http://www.atamai.com, 2000

3. Boehm DH, Reichenspurner H, Gulbins H, Detter C, Meiser B, Brenner P, et al. Early experience with the robotic technology for coronary artery surgery. Ann Thorac Surg 68:1542-6, 1999. :10543565:

4. Boyd WD, Desai ND, Kiaii B, Rayman R, Menkis AH, McKenzie FN, et al. A comparison of robot-assisted versus manually constructed endoscopic coronary anastomosis. Ann Thorac Surg, 2000. In press.

5. Boyd WD, Rayman R, Desai ND, Menkis AH, Dobkowski W, Ganapathy S, et al. Closed-chest coronary artery bypass grafting on the beating heart using a computer-enhanced surgical robotic system. J Thorac Cardiovasc Surg, 2000. In press.

6. Dey D, Gobbi DG, Slomka PJ, Surry KJM, Peters TM. Mapping of Endoscopic Images to Object Surfaces via Ray-traced Texture Mapping for Image Guidance in Neurosurgery. Proceedings of SPIE, The International Society for Optical Engineering, 2000. In press.

7. Damiano RJ Jr, Ehrman WJ, Ducko CT, Tabaie HA, Stephenson ER, Kingsley CP, et al. Initial United States clinical trial of robotically assisted endoscopic coronary artery bypass grafting. J Thorac Cardiovasc Surg 119:77-82, 2000. :10612764:

8. Falk V, Diegeler A, Walther T, Löscher N, Vogel B, Ulmann C, et al. Endoscopic coronary artery bypass grafting on the beating heart using a computer enhanced telemanipulation system. Heart Surg Forum #1999-85922 2:199-205, 1999. :427:

9. Freiherr G. Clinical Needs and Technology Drive Medical Imaging to 4-D. Medical Device and Diagnostic Industry Web site. Available at: http://www.devicelink.com/mddi/archive/99/10/004.html Accessed June 28, 2000

10. Gulbins H, Reichenspurner H, Becker C, Boehm DH, Knez A, Schmitz C, et al. Preoperative 3D-reconstructions of ultrafast-CT images for the planning of minimally invasive direct coronary artery bypass operation (MIDCAB). Heart Surg Forum #1998-4092 1:111-5, 1998. :798:

11. Loulmet D, Carpentier A, d'Attellis N, Berrebi A, Cardon C, Ponzio O, et al. Endoscopic coronary artery bypass grafting with the aid of robotic assisted instruments. J Thorac Cardiovasc Surg 118:4-10, 1999. :10384177:

12. MacDonald D. Register of McConnell Brain Imaging Centre, Montréal Neurological Institute, McGill University, Montreal, Canada. Available at: http://www.bic.mni.mcgill.ca/software/register/register.html, 1993.

13. National Library of Medicine. The Visible Human Project. Bethesda, Maryland: National Institutes of Health; 1994. Available at: http://www.nlm.nih.gov/research/visible/visible_human.html

14. Reichenspurner H, Damiano R, Mack M, Boehm DH, Gulbins H, Detter C, et al. Use of the voice-controlled and computer-assisted surgical system Zeus for endoscopic coronary artery bypass grafting. J Thorac Cardiovasc Surg 118:11-6, 1999. :10384178:

15. Schroeder W, Martin K, Lorensen B. The Visualization Toolkit: An Object Oriented Approach to 3D Graphics. 2nd ed. (Upper Saddle River, NJ, Prentice Hall PTR, 1997).

16. Slomka PJ, Hurwitz GA, Stephenson J, et al. A volume-based image registration toolkit for automated comparison of paired nuclear medicine images. Med Phys 22:1017, 1995.

17. Tabaie HA, Reinbolt JA, Graper WP, Kelly TF, Connor MA. Endoscopic coronary artery bypass graft (ECABG) procedure with robotic assistance. Heart Surg Forum #1999-0552, 2:310-317, 1999. :336: