Image display in endoscopic surgery

Christopher S. Muratore Abstract — Advances in the technology of optical displays have changed the way surgeons are able
Beth A. Ryder to manage different illnesses. Minimally invasive surgery encompasses a wide range of endoscopic
Francois I. Luks procedures, whereby the body cavity (abdomen, thorax, gastrointestinal tract, and joint spaces) is
accessed through small incisions and the use of telescopes and fine, long instruments. These tech-
niques have rapidly gained in popularity during the last decades, as patients are experiencing less
discomfort after surgery. Visualization of the operative field requires optimal image capture, process-
ing, and display. The introduction of charge-coupled devices has enabled surgeons to view the
operative field on a video monitor, allowing ever-more-complex operations to be performed
endoscopically. However, limitations include loss of 3-D perception and tactile sense, poor ergonom-
ics, often suboptimal positioning of image display and image quality that is too dependent on outside
influences. These limitations, and possible solutions, are addressed, as is the “ideal” display system
for endoscopic surgery.

Keywords — Endoscopy, laparoscopy, medical imaging, head-mounted display, surgery, surgical

navigation, medical simulation.

1 Introduction 2 History of surgical endoscopy

The practice of surgery is largely based on diagnosis of a
disease process followed by visual inspection and removal or
repair of the affected organ. Advances in the technology of
optical display have changed the way surgeons are able to
manage different illnesses. Beginning in the 19th century,
surgeons used a variety of lighted tubes to inspect body cavi-
ties, including the bladder, rectum, and stomach. A major
limitation of these devices was their inability to produce an
image for more than one viewer at a time. This was finally
overcome in the early 1980s with the development of mini-
aturized video cameras with immediate display capabilities.
By the 1990s, the technique was expanded to more complex
procedures and operations inside the abdominal and chest
cavities, and the field of minimally invasive surgery was
born. This approach rapidly gained popularity within the
surgical community as patients experienced less discomfort
postoperatively, since their incisions were smaller. Many
procedures can now be performed on an outpatient basis
with a rapid return to a normal level of activity.1
Endoscopy and endoscopic surgery are the general
terms for a wide range of diagnostic and surgical procedures
within natural and virtual body cavities, from the abdomen
and the chest to joints and blood vessels. The ability to look
and work inside the body without the need for exposure
through a wide incision (“open” surgery) was made possible
by a large number of technological developments of the last
century.2
Physicians since the Middle Ages have wanted to utilize the
natural orifices of the human body for diagnostic informa-
tion, but were limited by the scarcity of natural points of
entry and the unavailability of adequate light sources. A
rigid tube with an unsophisticated lens system and illumina-
tion provided by candle or heated platinum wire provided
little opportunity for meaningful exploration. Endoscopes
improved over the last 150 years, with the transition from
conventional lenses to rod lenses (Hopkins) and the intro-
duction of fiber optics.3 The refinement of fiber-optics-
based imaging in medicine, initiated at the University of
Michigan by gastroenterologist Basil Hirschowitz and optical
physicist C. Wilbur Peters, led the path to flexible endoscopy,
starting with colonoscopy in the 1960s, pioneered by Ber-
gein Overholt. During that period, optical clarity improved
as did the maneuverability of the instrument. Flexible
endoscopy of the colon and stomach transitioned from a
diagnostic tool to a therapeutic modality with the introduc-
tion of snare polypectomy (the removal of potentially malig-
nant polyps from the bowel by fine and flexible instruments
alongside the telescope’s optic fibers).4,5
Other limited procedures could be performed: in addi-
tion to polypectomy, gynecologists could perform tubaliga-
tions for sterilization6 and pulmonologists could take
biopsies from the trachea and bronchi. However, it was not
until the miniaturization of the video camera that the
surgeon no longer had to peer down the eyepiece of the
telescope, freeing his hands to perform actual surgical
operations from the outside. Two- or three-handed opera-

C. S. Muratore and F. I. Luks are with the Division of Pediatric Surgery and the Department of Surgery, Brown Medical School, 2 Dudley St., Suite
180, Providence, RI 02905; telephone 401/228-0556, fax 401/444-6603, e-mail: Francois_Luks@brown.edu.
B. A. Ryder is with the Department of Surgery, Brown Medical School, Providence, RI.
© Copyright 2007 Society for Information Display 1071-0922/07/1506-0349$1.00

Journal of the SID 15/6, 2007 349

tions required the presence of one or more assistants, which
in turn required the endoscopic image to be projected for
all to see.7
In 1989, the first laparoscopic cholecystectomy was
reported: surgical removal of the gallbladder through four
holes in the abdominal wall, each barely 1 cm wide.8 This
represented a turning point in endoscopic, or “keyhole” surgery.
(Laparoscopy refers to endoscopic surgery in the abdominal
cavity while thoracoscopy is endoscopic chest surgery.) In
the last 15 years, more and more types of operations have FIGURE 1 — Left: Traditional set-up for laparoscopic surgery, with image
been found to lend themselves to this minimally invasive display at eye level or higher, away from the operative field. Right:
Head-down image display allows the surgeon’s hands to be in the same
form of surgery, and instruments have become ever more visual field as the laparoscopic image, which improves ergonomics and
sophisticated and ever more miniaturized.9,10 accelerates surgical performance.
Display of the endoscopic image first utilized a cath-
ode-ray-tube (CRT) monitor, placed at a distance from the
operative field. In addition to the large and heavy CRT
The alterations in hand-eye coordination occur in two
monitor, endoscopic surgery required a camera control unit
ways. Mislocation of the displayed image occurs when the
(CCU), a gas insufflator to expand the abdominal cavity, a
surgeon gazes forward at eye level, rather than at his hands
cold light source, and a video recorder or video printer. It
(Fig. 1). Misorientation of the image is caused by a differ-
initially made sense to concentrate this cumbersome equip-
ence in point of view between the endoscope’s lens and the
ment into one cart, the laparoscopic “tower.” The CRT was
surgeon’s eye when looking directly into the patient. As a
placed on top of the tower, high enough to be visible by
result, spatial relationships between objects are altered, and
all.11,12
the instrument on the monitor moves in a different direc-
The relatively bulky charge-coupled device (CCD)
tion than expected (Fig. 2). This contributes to operator
camera used two decades ago has made way for miniature
confusion and delay, and requires practice to override the
single-and three-chip CCD cameras, and image processing
visual “anti-cues.”11
and recording have become digital. The displaying of the
To emulate the traditional setting of open surgery, the
endoscopic image has undergone substantial change as well,
ideal set-up places the telescope (the surgeon’s eyes) at the
although many limitations remain.
apex of a diamond, the target organ at the other end, and
both instruments on either side of the long axis.11 Unlike in
open surgery, however, there is a disconnect between the
3 Principles and limitations of endoscopic surgeon’s eye (the telescope) and his hands (Fig. 3). Devia-
surgery tions from this diamond set-up or unsteady camera work
harm the quality of the image and the efficiency and safety
The transition from “open” operations to endoscopic sur- of the surgical intervention. The worst set-up has the tele-
gery imposes certain limitations.13 There is a loss of tactile
sense and a two-, rather than three-dimensional image to
work from. In addition, the use of endoscopic ports into the
abdominal or thoracic cavity (rather than a wide-open inci-
sion) limits the degrees of freedom of each operating instru-
ment. Most of all, the efficiency of endoscopic surgery relies
primarily on the quality of the captured image and that of its
display.
Generations of surgeons were trained to make large
incisions to palpate and retract organs, control bleeding by
finger pressure, and suture tissues with just the right
amount of tension. We take for granted the development of
fine motor skills, haptic cues, and hand-eye coordination
necessary for these delicate tasks. In open surgery, the sur-
geon directly observes and manipulates his hands and in-
struments, relying on immediate visual and tactile feedback.
Laparoscopic surgery feels unnatural due to the indirect
FIGURE 2 — Misorientation and paradoxical motion is experienced
method of observing and manipulating through a surrogate when the surgeon’s eyes and hands are not in line with the endoscope’s
display system.13 This leads to confusing signals between lens. Left, image captured from the natural point of view of the surgeon.
the visual-motor transformation and the real situation, resulting Motion (here, tip of the instrument moves from left to right) is faithfully
reproduced on screen. Right: endoscope’s point-of-view is opposite to
in altered or strained hand-eye coordination. This is the crux the surgeon’s, resulting in paradoxical motion (from left to right in reality,
of the limitation in endoscopic surgery. but right to left on screen).

350 Muratore et al. / Image display in endoscopic surgery


FIGURE 3 — Differences in image display between traditional and
endoscopic surgery. Left: The surgeon (2) maintains a continuous and
direct relationship between his eyes (1), his hands, his instruments and
his surgical target (3). Right: In endoscopic surgery, the surgeon (2) relies
on an image generated by an assistant to perform a surgical task. This
requires perfect coordination between the surgeon’s “surrogate” eyes (1)
and his hands (3). An inexperienced camera holder can make an
operation more difficult by not showing the surgeon the correct field of
vision at the appropriate times.
scope pointing toward, rather than away from the surgeon.
This creates paradoxical motion (a mirrored image) and
requires a high degree of cerebral disconnection between
hand motion, visual perception, and mental image (Fig. 2).
Single-chip CCD cameras have long been replaced by
three-chip technology, allowing a better separation of the
red, green, and blue signals, and most cameras currently in
use are digital. Paralleling advances in other fields of image
capture and display, high-definition imaging is likely to be
introduced in the surgical arena. However, the ultimate
quality of the perceived surgical image may be limited by
other aspects, including the stability of the camera work, the
imperfect internal environment (water vapor, fat particle
aerosolization, cautery-generated smoke, blood droplets),
and other factors that tend to decrease the clarity of the
image or frankly obstruct the lens of the telescope. More-
over, it is unclear whether an improved endoscopic image of
an already greatly magnified operative field would improve
the accuracy and efficiency of the surgical skills beyond
what is currently available.11
The actual display of the endoscopic image has, until
recently, remained anchored in tradition, with all required
endoscopic or laparoscopic hardware on one rack. Although
bundling all laparoscopic equipment into one tower saves
space and directs cables and tubing toward a single source,
the rack has to be placed at a certain distance from the
operating table and the surgeons. As a result, some of the
magnification advantages of laparoscopy are neutralized by
the relatively long distance between the surgeon’s eyes and
the CRT monitor.11,14 In addition, the monitor may not
always be wheeled into the optimal viewing spot due to the
tower’s size; and no viewing spot is ideal for all participants
(surgeon, assistants, and scrub nurses). The most commonly
used solution to that problem has been the use of two or
more CRT monitors, each on its own rack. The resulting
clutter of major equipment in an already busy operating
room is an obvious disadvantage of this approach.12,15,16
In addition to the disadvantage of their size, classic
CRT monitors produce a suboptimal image quality. This is
mainly a result of glare,11 which in a busy operating room
can be worsened by the presence of multiple bright light
sources: overhead operating lights, radiograph viewing box,
anesthesiologist’s monitoring displays, and reading light.
Most CRT monitors offer suboptimal luminosity as well,
requiring the operating room to be darkened.17,18 This
increases the risk of accidents from tripping over equipment
and cables for all operating room personnel, especially
nurses and anesthesiology staff who need to move around
the room while the lights are out. Flat-screen CRT moni-
tors, although they eliminate most of the glare, are not prac-
tical in the surgical setting, as they are even bulkier than
traditional monitors.
More recently, flat-panel liquid-crystal-display (LCD)
monitors have provided an improved image quality17 at a
reduced price, and many operating rooms are now replacing
CRT monitors with flat screens. Their light weight, versatil-
ity, and small footprint allow the use of multiple screens
placed near the operating table. While LCD monitors pro-
duce the best image and have a greater portability than CRT
monitors, the positioning of the screen is still limited by a
number of factors, including the need to keep the immedi-
ate surroundings of the operative field sterile and the pres-
ence of a large numbers of instruments, instrument tables,
cables, and other equipment that add to the clutter of the
endoscopic operating room (Fig. 4).
The ideal position of the image display should repro-
duce the natural relationship between the eyes, the hands,
and the target. This includes a viewing angle that is approxi-
mately 15–20° below the horizontal plane through both eyes
FIGURE 4 — Typical laparoscopic operating room set-up. Note the
multitude of image displays: Two flat-screen laparoscopic monitors (A
and D, off the laparoscopic “tower”), vital signs and anesthesia
parameters display (B), patient data entry and display system (C), and
drug-information display (E).
Journal of the SID 15/6, 2007 351
and a distance from the eyes to the display of at least
70–100 cm to avoid diplopia-induced strain, but no more
than 200–250 cm.19 In practice, these rules are rarely observed
and are difficult to apply for all members of the surgical
team. One available solution is the use of a sterile screen
placed within the operative field (Fig. 1, View-Site systems,
Storz Endoscopy, Culver City, CA). Several authors have
demonstrated that this is not only more ergonomic for the
surgeon, but actually improves task performance, both in a
laboratory setting20 and in clinical situations.21 Unfortu-
nately, current overhead video projection systems have
reduced brightness and resolution compared with LCD
monitors.22
The most visible difference between traditional and
endoscopic surgery is the loss of stereological vision in an
endoscopic world. The shortcomings of two-dimensional
image capture and display can be a handicap in certain
highly specialized operations. The search for the ideal three-
dimensional endoscopic system is as old as endoscopic surgery
itself, and several techniques have been proposed.23 Early
prototypes consisted of a dual-camera set-up: the three-dimen-
sional image was generated by two telescopes, stereologi-
cally placed to imitate human binocular vision.24 Ideally,
stereoscopic image capture requires an “interpupillary” dis-
tance of at least 60 mm. To limit the diameter of the dual
telescope, the lenses in most workable 3-D systems are only
separated by approximately 10 mm. As a result, the depth of
field is reduced and true 3-D perception is only possible in
the center of the image.23 In addition, the limitation on
overall scope diameter requires each of the two optical
channels to be substantially smaller (and therefore offering
lower brightness and resolution) than a comparable 2-D sin-
gle lens system.11
The display of a three-dimensional image can occur in
one of several ways. The best-studied techniques are the
synchronized shutter and the alternation of polarized
images.24–29 The first requires individual head-mounted
displays equipped with alternating shutters for left- and
right-eye viewing. With polarized images, individual goggles
consist of differently polarized glasses for each eye and are
therefore lighter and less bulky. In both systems, however,
each eye only captures half the images, and motion fluidity
relies on persistence of the retinal after-image. As a result,
image flicker and an overall reduced brightness are inevita-
ble. Newer types of 3-D telescopes have an “eyelid” shutter
at the tip of a single telescope,30 rather than a dual lens
system. This improves image brightness somewhat, but
image splitting and a short interpupillary distance still limit
the overall image resolution and 3-D perception.
While 3-D imaging would appear to be beneficial to
the performance of endoscopic procedures,25,27,29 in vitro
studies have yielded mixed results. Chan et al. 23 showed
that 3-D imaging did not significantly improve task effi-
ciency, regardless of the surgeon’s expertise (P = 0.6 for
experienced surgeons, P = 0.9 for novices). While two-thirds
of the surgeons subjectively felt that 3-D perception was
352 Muratore et al. / Image display in endoscopic surgery
enhanced, 40% reported decreased image resolution and
brightness. In surgery, the quality and detail of the image
appear to matter more than the faithful three-dimensional
reproduction of the target organ. In addition, the 3-D effect
is corrupted by distracting visual clues, such as the frame of
the monitor, the relatively short depth of field, and the image
distortion when viewed at an angle.
Most surgeons have learned to adapt to a 2-D display
system by utilizing shadows and differential motion speed to
judge the depth of field.31 The shadow effect is generally
suboptimal, since a light source in line with the telescope
provides an almost shadowless field of vision in close-up. At
least one manufacturer of endoscopic instruments offers a
telescope with a primary (in-line) and secondary light
source (Lapalux shadow telescope, MGB Endoscopy, Seoul,
Korea), which enhances the shadows cast on the operative
field.32
Although CCD cameras have been gradually miniatur-
ized and digitized, the image quality remains limited by the
resolution of the display, which most often is standard
NTSC. High-definition monitors are not commonly avail-
able, and many operating rooms still use composite image
signals, rather than RGB or Y/C standards.11 In addition,
image archiving is still often analog, whether through a
video printer for still images or through videorecording.
More recently, digital image capture has gained in popular-
ity, particularly with the growing capacity of modern storage
devices.
4 The ideal image display in endoscopic
surgery
The ideal image display in an operating room does not yet
exist, as it is limited by current technology and often contra-
dictory requirements. Some of the above-mentioned prob-
lems with image capture will likely be improved upon in the
future. Already, the availability of an Automated Endoscopic
System for Optimal Positioning (AESOP, Computer Motion,
Goleta, CA) offers a stable image that changes only with
verbal commands from the surgeon. This system removes
some of the human factors responsible for poor image qual-
ity: assistant fatigue, lack of experience, and independent
thought.33–35 Users of this system have learned that surpris-
ingly few changes in camera angle and close-up are neces-
sary once a target organ has been identified. The most
common commands have to do with zooming in on the tar-
get during the delicate parts of the operation and panning
out to a panoramic view when instruments are being
changed.
A natural extension of the robotic camera arm is the
widely popularized (and still exorbitantly expensive) laparo-
scopic robot (da Vinci, Intuitive Surgical, Sunny Valley CA
and Zeus, Computer Motion), which applies the principles
of virtual surgery, tremor cancellation, three-dimensional
immersion and motion amplification.35–39 With this technol-
ogy, the lead surgeon sits at a remote console and observes
a three-dimensional target image through a binocular-
mounted display. The surgeon manipulates virtual instruments
with up to eight degrees of freedom from a comfortable,
seated position. While this technique is ideal for very deli-
cate surgical interventions requiring significantly magnified
images, it requires an entire crew of assistants to install and
man complex robotic arms for the multiple instruments. In
addition to the cost of purchase and maintenance of the
robot, set-up and breakdown times are significant, limiting
its use for routine operations, at least for the present time.
One very visible, albeit experimental, application of
this robot technology is Operation Lindbergh,40 the first-
ever performance of a telesurgical intervention on a patient
in Strasbourg, France, by a surgeon sitting at a console in
New York. In 2001, Dr. Jacques Marescaux performed a
laparoscopic cholecystectomy (surgical removal of the
gallbladder) after transatlantic assistants had inserted and
positioned the necessary robotic arms and articulated
laparoscopic instruments in the patient. This technological
tour de force illustrates the sophistication of today’s telesur-
gical capabilities and was made possible by solving the
problem of delayed transmission. The use of dedicated
asynchronous transfer mode telecommunication allowed
the real-time display of high-definition images (over a dis-
tance of more than 4000 miles) that is necessary for such
delicate operations. While the immediate application of this
form of telesurgery is limited, one of its future goals is the
possibility of surgery in space.41,42
The immersive visual environment offered by the
robot can be used in other ways. Three-dimensional
endoscopy is an area that may hold promise, provided that
the quality of the 3-D image and the miniaturization of the
camera/telescope unit improve substantially.25 The size of
CCD cameras has already decreased dramatically over the
last decade, and several manufacturers of endoscopic instru-
ments already offer telescopes with tip-mounted cam-
eras.11,43 With these telescopes, the CCD capture occurs at
the tip of the telescope itself without need for optic fibers
to transmit the image from the internal organs to an eye-
piece-mounted camera. This allows a more efficient image
capture with minimal loss of illumination. In addition, the
FIGURE 5 — Surgical navigation using magnetic resonance (MRI) or computed tomography (CT scan) images to create a virtual
3-D image of the operative field (here, a twin pregnancy requiring fetal surgery for twin-to-twin transfusion syndrome).46 The virtual
images can be used preoperatively to plan an operative approach, or intraoperatively to improve surgical accuracy by
superimposing the images on the field (augmented reality).
camera can potentially be articulated in many more direc-
tions than is currently feasible. This would allow the sur-
geons to literally “look around,” much like the situation with
open surgery, whereby the surgeon is able to quickly alter-
nate between central target and peripheral vision.
Modern surgical interventions often rely on the inte-
gration of a variety of information sources and the simulta-
neous viewing of multiple images, including preoperative
radiology images, external anatomic landmarks, the opera-
tive field, and even intraoperative ultrasound imaging.
Advanced operative imaging systems allow simultaneous
viewing of multiple images, resulting in “augmented reality”
surgery.44,45 Surgical navigation and virtual surgery systems
are already in use in neurosurgery, orthopedics, ENT, and
fetal surgery (Fig. 5).46 Preoperative planning and 3-D
imaging can be superimposed on the actual operative field
to facilitate access to a target organ, particularly in delicate
areas such as the brain, the skull base and the nasal sinuses.47–49
This technology is mostly applicable to anatomic areas and
body cavities that are relatively fixed, so that the preopera-
tive image (MRI or CT) is a faithful representation of the
target organ and is not subject to periodic deformation
(breathing, shifting of organs).26 By placing spatial markers
on fixed body landmarks (skull, hip bones), spatial reposi-
tioning of the actual instrument can recall the correspond-
ing 3-D simulation on the screen.50
5 Future of endoscopic surgery
In only two decades, minimally invasive surgery has trans-
formed the field of surgery. The operating room of the future
will likely incorporate many of the technological advances
described here, most of which are already available today.
The future of surgical image display is likely to evolve into
one of two avenues: the individual display system or the
transparent patient.
Journal of the SID 15/6, 2007 353
 an actual screen or monitor obviates the need for steriliza-
tion, allows pointing and teaching during the operation, and
renders the patient’s abdominal or thoracic wall virtually
transparent.

7 Personal display system

An alternative solution to the problem of an optimal image
for all is the head-mounted display (HMD). Currently avail-
able goggles are lightweight and offer excellent image reso-
lution with ample peripheral vision. Figure 7 shows an
HMD in use during a laparoscopic procedure. The surgeon
can look down in the direction of the operative field, keep-
ing his or her hands and instruments within her peripheral
FIGURE 6 — Triple Directional Viewing LCD, a display that controls the vision while viewing a high-quality endoscopic image of the
viewing angle so that the display can show different images from the left,
right, and center simultaneously (Sharp Electronics Corp., Mahwah, NJ). patient’s internal organs. The freedom from gazing at an
overhead monitor also greatly improves ergonomics and
reduces neck strain. If other members of the surgical team
6 The transparent patient are equally equipped, each has an unobstructed and ideally
positioned image display of the operative field, regardless of
Currently available systems of in-field image projection are
his position in the room or at the operating table.
crude. Image quality is limited by the overhead projection
In addition, images can be individualized: vital signs
system and the coating surface of the screen. The screen is
(blood pressure, heart rate) can be superimposed on the dis-
rigid and its footprint forces it to be out of the operative
played image, and this information can be offered to all
field. Because the image is produced by an overhead projec-
members, or to only one individual. Paradoxical motion can
tor, shadowing and partial obstruction of the image is a prob-
be compensated with image inversion for some members of
lem during surgery. Newer systems will need to be curved
the surgical team, and electronic image-enhancement tech-
and possibly flexible to conform to the patient’s abdominal
nology may even allow individual surgeons to zoom in or out
or thoracic contour. Back-projection onto “transflective”
independently of each other. If high-definition 3-D image
(both reflective and transparent) surfaces44 will offer true
capture becomes available, the use of light-weight HMD is
augmented-reality surgery and avoid shadow effects.
the ideal display system. Finally, high quality and brightness
Curved surfaces of laminar glass, containing a sandwiched
of the HMD goggles should allow endoscopic surgery in
layer of wedge-shaped PVB to allow undistorted image pro-
jection, are already in use as windshields in some vehicles,
providing head-up display of navigation and other informa-
tion.51 Advances in thin-film transistor (TFT) and flexible
displays could produce high-resolution, flexible, light-
weight display systems sturdy enough to be sterilized and
placed within the operative field.
Even image inversion, to counter paradoxical motion
when the surgeon faces the telescope (see Fig. 2), could be
adapted to a single display system. Earlier this year, Sharp52
introduced an LCD monitor capable of displaying three dif-
ferent images depending on the angle of viewing (Fig. 6).
With such a display, the surgeon who looks at the surgical
target from the viewpoint of the telescope and camera
would see the correct image, while his assistant on the opposite
side of the operating table would be able to see an inverted
image.
The ideal form of in-field display would not require a FIGURE 7 — Head-mounted display of the laparoscopic image allows
projection surface or monitor. Suspended image systems the surgeon to keep laparoscopic image and surgical instruments in the
same visual field. This reduces neck strain and guarantees optimal
produce an undistorted image in the three-dimensional viewing of the endoscopic field regardless of the surgeon’s position. The
space close to the actual operative field, using high-preci- more traditional display (background) requires the surgeon to look up
sion retroflector and beam-splitting technology.53 The use toward the monitor; in addition to strain on neck and loss of hand-eye
coordination, viewing is often impeded because of crowding around the
of multiple beam splitters and curved mirrors can dramati- operating table. (HMD: MSpecs eyewear, courtesy of Kowon Technology
cally enhance the brightness of the image.11 The absence of Co., Ltd., of Yongin, Korea.)

354 Muratore et al. / Image display in endoscopic surgery

daylight conditions, which would greatly improve the safety
and comfort for all operating room personnel.
The limiting factors that remain with either system are
the dependence on good camera work by the assistant, and
the problem of tunnel vision: the surgeon cannot see “behind”
the telescope, a common cause of inadvertent damage to
other organs during laparoscopy. The ultimate advance in
endoscopic image display will be the complete individuali-
zation of the surgical field, allowing each member of the
team to recall, zoom in or out, pan or focus on any particular
area of the endoscopic field independently. This will require
a “fixed” panoramic or fish-eye lens inside the patient and
individual processing of the image by each member of the
team, fully mimicking the real-life situation of traditional
surgery. Voice activation has been the most commonly used
command method, as it frees the surgeon’s hands. Other
systems could include foot pedals, sterile trackballs, touch
screens or, ideally, head-position or eye-gaze-tracking
devices54 that automatically respond to a surgeon “looking
at” a particular segment of the total image.55,56 Thus, the
surgeon would truly be able to view the best of both worlds
– the internal, endoscopic one and the “outside” world.
References
1 F I Luks, J Logan, C K Breuer, A G Kurkchubasche, C W Jr. Wessel-
hoeft, and T F Jr. Tracy, “Cost-effectiveness of laparoscopy in children,”
Arch Pediatr Adolesc Med 153, 965–968 (Sept, 1999).
2 A G Gordon and A L Magos, “The development of laparoscopic sur-
gery,” Baillieres Clin Obstet Gynaecol 3, 429–49 (Sept, 1989).
3 I M Modlin, M Kidd, and K D Lye, “From the lumen to the laparo-
scope,” Arch Surg 139, 1110–26 (Oct, 2004).
4 J P Christie and H Shinya, “Technique of colonoscopic polypectomy,”
Surg Clin North Am 62, 877–887 (Oct, 1982).
5 C K McSherry, M Cwern, H Ferstenberg, A Ghazi, D H Sekons, H
Shinya, and W W Wolff, “Interventional endoscopy,” Curr Probl Surg
22, 1–73 (July, 1985).
6 K Semm, “Endoscopic appendectomy,” Endoscopy 15, 59–64 (Mar,
1983).
7 A G Harrell and B T Heniford, “Minimally invasive abdominal surgery:
lux et veritas past, present, and future,” Am J Surg 190, 239–243 (Aug,
2005).
8 F Dubois, P Icard, G Berthelot, and H Levard, “Coelioscopic cholecys-
tectomy. Preliminary report of 36 cases,” Ann Surg 211, 60–62 (Jan,
1990).
9 J G Bussey, F Luks, S R Carr, M Plevyak, and T F Jr. Tracy, “Minimal-
access fetal surgery for twin-to-twin transfusion syndrome,” Surg Endosc
18, 83–86 (Jan, 2004).
10 J T Tomicic, F I Luks, L Shalon, and T F Tracy, “Laparoscopic gastros-
tomy in infants and children,” Eur J Pediatr Surg 12, 107–110 (Apr,
2002).
11 G Hanna and A Cuschieri, “Image display technology and image proc-
essing,” World J Surg 25, 1419–1427 (Nov, 2001).
12 S D Schwaitzberg, “Imaging systems in minimally invasive surgery,”
Semin Laparosc Surg 8, 3–11 (Mar, 2001).
13 J Heemskerk, R Zandbergen, J G Maessen, J W Greve, and N D Bouvy,
“Advantages of advanced laparoscopic systems,” Surg Endosc 20,
730–733 (May, 2006).
14 G El Shallaly and A Cuschieri, “Optimum view distance for laparo-
scopic surgery,” Surg Endosc, EPub (Oct 9, 2006).
15 A K Hemal, M Srinivas, and A R Charles, “Ergonomic problems
associated with laparoscopy,” J Endourol 15, 499–503 (June, 2001).
16 J A Ibbotson, C L MacKenzie, C G Cao, and A J Lomax, “Gaze patterns
in laparoscopic surgery,” Stud Health Technol Inform 62, 154–160
(1999).
17 S I Brown, C White, K Wipat, G B Hanna, T G Frank, and A Cuschieri,
“Characterizing the ‘gold standard’ image for laparoscopic surgery,”
Surg Endosc 18, 1192–1195 (Aug, 2004).
18 P V Patil, G B Hanna, and A Cuschieri, “Effect of the angle between
the optical axis of the endoscope and the instruments’ plane on monitor
image and surgical performance,” Surg Endosc 18, 111–1114 (Jan,
2004).
19 A Cuschieri, “Epistemology of visual imaging in endoscopic surgery,”
Surg Endosc 20, Suppl 2, S419–S424 (Apr, 2006).
20 A M Omar, N J Wade, S I Brown, and A Cuschieri, “Assessing the
benefits of ‘gaze-down’ display location in complex tasks,” Surg Endosc
19, 105–108 (Jan, 2005).
21 K Erfanian, F I Luks, A G Kurkchubasche, C W Jr. Wesselhoeft, and
T F Jr. Tracy, “In-line image projection accelerates task performance
in laparoscopic appendectomy,” J Pediatr Surg 38, 1059–1062 (July,
2003).
22 S I Brown, T G Frank, A Cuschieri, R Sharpe, and C Cartwright,
“Optimization of the projection screen in a display system for minimal
access surgery,” Surg Endosc 17, 1251–1255 (Aug, 2003).
23 A C Chan, S C Chung, A P Yim, J Y Lau, E K Ng, and A K Li,
“Comparison of two-dimensional vs. three-dimensional camera sys-
tems in laparoscopic surgery,” Surg Endosc 11, 438–440 (May, 1997).
24 D M Herron, J C Lantis 2nd, J Maykel, C Basu, S D Schwaitzberg,
“The 3-D monitor and head-mounted display. A quantitative evaluation
of advanced laparoscopic viewing technologies,” Surg Endosc 13,
751–755 (Aug, 1999).
25 M N Thomsen and R D Lang, “An experimental comparison of 3-di-
mensional and 2-dimensional endoscopic systems in a model,” Arthro-
scopy 20, 419–423 (Apr, 2004).
26 J D Ackerman, K Keller, and H Fuchs, “Real-time anatomical 3D
image extraction for laparoscopic surgery,” Stud Health Technol Inform
81, 18–22 (2001).
27 D H Birkett, L G Josephs, and J Este-McDonald, “A new 3-D laparo-
scope in gastrointestinal surgery,” Surg Endosc 8, 1448–1451 (Dec,
1994).
28 H T Tevaearai, X M Mueller, and L K von Segesser, “3-D vision
improves performance in a pelvic trainer,” Endoscopy 32, 464–468
(June, 2000).
29 P van Bergen, W Kunert, and G F Buess, “Three-dimensional (3-D)
video systems: bi-channel or single-channel optics?,” Endoscopy 31,
732–737 (Nov, 1999).
30 http://www.inneroptic.com/tech_3DL.htm.
31 G B Hanna, S M Shimi, and A Cuschieri, “Randomised study of
influence of two-dimensional versus three-dimensional imaging on
performance of laparoscopic cholecystectomy,” Lancet 351, 248–251
(Jan 24, 1998).
32 R K Mishra, G B Hanna, S I Brown, and A Cuschieri, “Optimum
shadow-casting illumination for endoscopic task performance,” Arch
Surg 139, 889–892 (Aug, 2004).
33 B M Kraft, C Jager, K Kraft, B J Leibl, and R Bittner, “The AESOP
robot system in laparoscopic surgery: increased risk or advantage for
surgeon and patient?,” Surg Endosc 18, 1216–1223 (Aug, 2004).
34 K M Russell, T J Broderick, E J Demaria, S N Kothari, and R C Merrell,
“Laparoscopic telescope with alpha port and aesop to view open surgi-
cal procedures,” J Laparoendosc Adv Surg Tech A 11, 213–218 (Aug,
2001).
35 A A Wagner, I M Varkarakis, R E Link, W Sullivan, and L M Su,
“Comparison of surgical performance during laparoscopic radical pro-
statectomy of two robotic camera holders, EndoAssist and AESOP: A
pilot study,” Urology 68, 70–74 (July, 2006).
36 A Hoznek, Y Menard, L Salomon, and C C Abbou, “Update on laparo-
scopic and robotic radical prostatectomy,” Curr Opin Urol 15, 173–180
(May, 2005).
37 E Caratozzolo, A Recordare, M Massani, L Bonariol, A Jelmoni, M
Antoniutti, and N Bassi, “Telerobotic-assisted laparoscopic cholecys-
tectomy: our experience on 29 patients,” J Hepatobiliary Pancreat Surg
12, 163–166 (2005).
38 M R Marohn and E J Hanly, “Twenty-first century surgery using
twenty-first century technology: surgical robotics,” Curr Surg 61,
466–473 (Sept 2004–Oct 31, 2004).
39 G Jacobsen, F Elli, and S Horgan, “Robotic surgery update,” Surg
Endosc 18, 1186–1191 (Aug, 2004).
40 J Marescaux, J Leroy, F Rubino, M Smith, M Vix, M Simone, and D
Mutter, “Transcontinental robot-assisted remote telesurgery: Feasibil-
ity and potential applications,” Ann Surg 235 487–492 (Apr, 2002).
Journal of the SID 15/6, 2007 355
41 R M Satava, “Looking forward,” Surg Endosc 20, Suppl 2, S503–S504
(Apr, 2006).
42 R M Satava, “Minimally invasive surgery and its role in space explora-
tion,” Surg Endosc 15, 1530 (Dec, 2001).
43 J M Perrone, C D Ames, Y Yan, and J Landman, “Evaluation of surgical
performance with standard rigid and flexible-tip laparoscopes,” Surg
Endosc 19, 1325–1328 (Oct, 2005).
44 O Bimber, L M Encarnacao, and D Schamlstieg, “Augmented reality
with back-projection systems using transflective surfaces,” Computer
Graphics Forum 3, 161–169 (2000).
45 J Marescaux, F Rubino, M Arenas, D Mutter, and L Soler, “Aug-
mented-reality-assisted laparoscopic adrenalectomy,” JAMA 292,
2214–2215 (Nov. 10, 2004).
46 F I Luks, S R Carr, B Ponte, J M Rogg, and T F Jr Tracy, “Preoperative
planning with magnetic resonance imaging and computerized volume
rendering in twin-to-twin transfusion syndrome,” Am J Obstet Gynecol
185, 216–219 (July 2001).
47 H W Schroeder, W Wagner, W Tschiltschke, and M R Gaab, “Frameless
neuronavigation in intracranial endoscopic neurosurgery,” J Neurosurg
94, 72–79 (Jan, 2001).
48 Caversaccio and Freysinger, “Computer assistance for intraoperative
navigation in ENT surgery,” Minim Invasive Ther Allied Technol 12,
36–51 (Mar, 2003).
49 A R Gunkel, M Vogele, A Martin, R J Bale, W F Thumfart, and W
Freysinger, “Computer-aided surgery in the petrous bone,” Laryngo-
scope 109, 1793–1799 (Nov, 1999).
50 G R Cosgrove, B R Buchbinder, and H Jiang, “Functional magnetic
resonance imaging for intracranial navigation,” Neurosurg Clin N Am
7, 313–22 (Apr, 1996).
51 http://www.solutiaautomotive.com/en/lgi/hud.aspx.
52 http://www.sharpsma.com.
53 J Holden, T G Frank, and A Cuschieri, “Developing Technology for
Suspended Imaging for Minimal Access Surgery,” Semin Laparosc
Surg 4, 74–79 (June, 1997).
54 S R Goodridge, “The environment understanding interface: detecting
and tracking human activity through multimedia sensors,” Proceedings
of the 1995 Conference of the Centre for Advanced Studies on Collabo-
rative Research, 24 (1995).
55 M Levoy and R Whitacker, “Gaze-directed volume rendering,” Sym-
posium on Interactive 3D Graphics, Snowbird, 217–223 (1990).
56 M Lucente, “Interactive three-dimensional holographic displays: see-
ing the future in depth,” ACM SIGGRAPH Computer Graphics 31,
63–67 (1997).
356 Muratore et al. / Image display in endoscopic surgery
Christopher S Muratore, M.D., is an Assistant
Professor of Surgery and Pediatrics at Brown
Medical School and an Attending Pediatric Sur-
geon at Hasbro Children’s Hospital in Provi-
dence, R.I. He obtained his M.D. degree from
Georgetown University and completed a resi-
dency in General Surgery at Beth Israel-Deacon-
ess Medical Center in Boston. He completed a
research fellowship and a Pediatric Surgery resi-
dency at Children’s Hospital, Boston. His interests
are minimally invasive surgery in infants and children, endoscopic fetal
surgery, and treatment and research of congenital diaphragmatic hernia.
Beth A. Ryder, M.D., is a Clinical Assistant Pro-
fessor of Surgery at Brown Medical School and an
Attending Surgeon at Rhode Island Hospital in
Providence, R.I. She obtained her M.D. degree
from Tufts School of Medicine and completed a
residency in General Surgery and a fellowship in
Minimally Invasive Surgery at Rhode Island Hos-
pital. She specializes in laparoscopic and other
minimally invasive operations, including bariatric
surgery.
Francois I. Luks, M.D., Ph.D., is a Professor of
Surgery and Pediatrics and Director of the Pro-
gram in Fetal Medicine at Brown Medical School,
and an Attending Pediatric Surgeon at Hasbro Chil-
dren’s Hospital in Providence, R.I. He obtained his
M.D. degree from the University of Antwerp and
his Ph.D. degree from the Catholic University of
Leuven, both in Belgium. He completed a resi-
dency in General Surgery at Catholic Medical
Center of Brooklyn & Queens (Cornell affiliate)
and a residency in Pediatric Surgery at Ste-Justine Hospital (University
of Montreal, Canada). His clinical interests include minimally invasive
surgery, fetal surgery, and fetal treatment. Research interests include
image-display technology in endoscopic surgery and the etiology of
fetal conditions, including twin-to-twin transfusion syndrome.