You are on page 1of 6

VISUALIZATION CORNER

Editor: Jim X. Chen, jchen@cs.gmu.edu

3D KNEE MODELING AND BIOMECHANICAL SIMULATION


Ying Zhu, Jim X. Chen, and Shide Xiao, Computer Graphics Laboratory, George Mason University Edward B. Mac Mahon, Knee Alignment Program of Greater Washington

KNEE JOINTS LONGEVITY DEPENDS ON THE PROPER INTEGRATION OF FIVE VARIABLES: SURFACE CONGRU-

ENCY, LOAD DISTRIBUTION, STRESS DURING LOADING, CONTACT AREA, AND LIGAMENT TENSION. IF SOME VARIABLES IN JUST ONE

small area are abnormal, this system fails, resulting in arthritis. The complexity of various failures has led orthopedic surgeons to select total knee replacement as their rst treatment choice. In many cases, this means sacricing much of an otherwise normal joint. Its like replacing the front tire of a car thats worn on one side because of bad alignment. Knee osteotomy is a kind of orthopedic surgery to realign the lower limb by opening or cutting a bone wedge from the leg. It is a better alternative than other types of knee-replacement surgeries, especially for young people. However, knee osteotomy requires understanding the imbalance of stresses at the knee joint, analyzing an abnormal gait cycle, and cutting the bone wedge precisely. Therefore, it is difficult and can cause further damage even though it is simply a bone cut. While some computer-based surgical simulation systems have been developed to help surgeons perform knee surgeries,1,2 the knee models used either are not patient-specific2 or lack kinematic and kinetic information.3,4 No efcient method exists for patientspecific knee-model reconstruction,2 contact-stress computation, or visualization.5 Furthermore, no system inte-

grates stress visualization, gait-cycle simulation, and virtual surgery. Here, we discuss how we use computer graphics, physics-based modeling, and interactive visualization to assist knee-surgery study and osteotomies. Our patient-specic 3D knee surface model helps us calculate the contact stresses at the knee joint, perform virtual surgery, and record data from surgery simulation.
Constructing a patients knee model in 3D

After receiving a set of MRI (magnetic resonance imaging) slices of the knee, our goal is to construct the patients knee surface model in 3D. This involves image segmentation, alignment, surface reconstruction, and deformation. Image segmentation. The purpose of image segmentation is to nd the contours of certain 2D objects within a 2D image. Many knee-reconstruction methods still employ manual segmentation (hand tracing), which is time-consuming. On the other hand, clinical knee MRI slices usually have a high noise-to-signal ratio and low resolution. Therefore, fully automatic segmentation is currently not possible, and certain human interven-

tions must be employed to set the initial conditions, guide the segmentation process, or correct errors. Some semiautomatic segmentation methods exist that use deformable templates, probabilistic searching, or region-based methods,6,7 and weve integrated these ideas into our implementation. In our algorithm, we use a graphical user interface to highlight a few control points on one MRI slice, employ a deformable template (snake)7 to nd the bone contour points, exploit the previous contour points to serve as starting control points for the current slice, and repeat these steps for the rest of the slices. We then use a contour editor to modify the algorithms output error. The modied points serve as input control points back to the algorithm, which thus generates a modied contour interactively. After this, an MRI model (a set of 2D knee contours) is constructed. Rough alignment and surface reconstruction. We have an existing computer-based reference modela high-fidelity cadaver knee surface modelwhich represents an ordinary knee. Despite small variances, ordinary human knees have almost the same appearance and features. Our method uses physics-based affine transformation to align the MRI model (a set of 2D knee contours) with the reference model to reconstruct the patients knee model automatically. Specically, our program denes corresponding feature control points on the reference and MRI models, builds

82

COMPUTING IN SCIENCE & ENGINEERING

Figure 1. Reconstructing a patients knee surface mode: (a) the contour of an MRI slice and (b) the model-based reconstruction.

up constraint forces according to the distances of the corresponding control points, and transforms the reference model to align with the MRI model as close as possible based on the constraint forces. The reference model goes through an afne transformation so that its shape will be preserved while its orientation, location, and size change (see Figure 1). Fine-grained alignment and deformation. To recover the lost features between MRI slices, we match the MRI models corresponding features with the reference model and then deform the reference model. First, each MRI models contour vertex is paired with a vertex on the reference model. At the paired vertices, we maximize the function

( r n) d ,
r

where r is the vector from the MRI models vertex to the reference models vertex; n is the normalized surface normal at the reference models vertex; and d is a factor to attenuate the emphasis on distance. When the function is maximized, the distance between the paired vertices (r) and the angle between r and n are minimized. After this, each of the reference models paired vertices is translated to the corresponding contour vertex. Vertices that are not paired are translated to their corresponding positions through bilinear interpolation according to the translations of the nearest paired vertices. After this stage, we have the patients 3D knee surface model.
Calculating the stresses

biomechanics of a human knee joint is important for understanding knee problems and for making surgery decisions. Our goal is to efciently calculate a patients tibia and femur contact forces and stresses at the knee joint and to visualize the stress changes during walking. Considerable research deals with calculating forces and (a) stresses in this area, spanning methods from simple hingejoint dynamics to nite-element analysis.3,4,8 However, there is no simultaneous visualization of 3D surface-model and pressure changes. Also, knee-joint mechanics are so complex that after years of research, numerous assumptions are still not satisfactory to many researchers. Our methods, which integrate results from existing work,4 involve complex coordinate transformations and nonlinear equations and their deductions. Here, we describe only the steps needed to achieve our goal. Assumptions and simplications. In our work, we Assume the tibia and femur bones are rigid-body objects: their stiffness is much higher than that of the ligaments, menisci, and capsule layer. Simplify the ligaments and capsule to be nonlinear springs (these are widely used assumptions). Ignore the friction between the tibia and femur contact surfaces and assume the coefficient of friction between cartilage surfaces, owing to the synovia uid, is very low.

(b)

Represent the tibial and femoral condyle surfaces with mathematical surface equations: this is necessary to calculate the normals, curvatures, and contact forces and stresses. Ignore the meniscus: it is ignored because the material characters of the meniscus are not fully understood. Its like a sponge pad. In the future, we will consider it as an array of distributed, uniform springs without mass, to produce better results. Assume the tibia and femur consist of isotropic elastic material. Employ a rigid two-point contact model. In the future, we can consider
Patella Femur Articular cartilage

Meniscus

Collateral ligaments

A knee joint has three surfaces, covered by articular cartilage, two menisci between the surfaces, and several ligaments that stabilize the joint (see Figure 2). The

Tibia

Figure 2. Knee components and structure.

JULY/AUGUST 1999

83

VISUALIZATION CORNER

a 3D distributed spring contact model using the nite-element method. Tibial and femoral condyle surface analytical descriptions. Given a patients posture and other related data (height and weight), we can find the corresponding contact points, surface normals, and curvatures according to the surface equations to decide the contact areas and stresses. The 3D surface model provides the coordinates (points or triangle patches) on the surfaces, but no equation represents the curved surfaces. Currently, we use the surface-fitting method to find the mathematical equations. In the future, well consider higher-order Bezier, Bspline, or NURBS (nonuniform rational B-spline) curved patches. We implement two steps to nd the 3D surface equations: Define global and local coordinate systems: develop the tibial and femoral local coordinate systems and their interchanges and relations to the global coordinate system. Find the surface equations with the surface-fitting method: ignore the tibia ridge and femur groove areas, leaving two elliptic curved surfaces. Nonlinear equation deductions. Because the normals to the tibia and femur surfaces at the two contact points are aligned on a straight line in opposite directions, we need to employ 10 independent nonlinear equations. According to the space force and moment equilibrium principal, the tibia force and moment will be in equilibrium as follows: Fr + Fb + Fp1 N1 + Fp2 N 2 +

M r ez + Fr Pr + Fb Pb + Fp1 N1 P1 + Fp2 N 2 P2 + where Fr is the ground reaction force, Fb is the body weight, Fp1 and Fp2 are the two contactpoint forces, N1 and N2 are the two contact-point normals, Fj is the ligament and capsule forces, Mr is the knee-joint driver moment along the local ez axis to achieve the bending angle, and (Pr, Pb, P1, P2, and Vj) are the position vectors where the corresponding forces are applied. At a larger exion angle of about 90, if we take into account the patellar effect, Fj and Vj must be modied accordingly. Together with the contact nonlinear equations mentioned earlier and other conditions, there are 16 equations with 16 unknowns, including Fp1 and Fp2, the two contact-point forces; Mr, the knee-joint driver moment; and P1 and P2, the two contact-point position vectors. We employ the complex optimal iteration algorithm to nd a numerical solution. Pressure distributions from Hertzian elastic contact-stress methods. From the surface equations and the two contact points, we can calculate the corresponding principle curvatures and principal directions at the contact points. Then, according to the bonematerial properties and Hertzian elastic contactstress theory, we can calculate the pressure distributions. According to the Hertzian contactstress theory, the contact areas are elliptic. A contact areas major and minor

diameters, a and b, are


(2)

Fj Vj = 0 ,

a = Ca 3 and b = Cb where

3Fp (h1 + h2 ) 4( B + A) (3) a , Ca

Fp is the contact force; Ca and Cb are coefcients calculated from (B A)/(B + A); B + A = 0.5(R1 + R2 + r1 + r2); (B A)2 = .25[(R1 R2)2(r1 r2)2 + 2(R1 R2)(r1 r2)cos(A1 a1)]; R1 and R2 are the tibia curvatures with direction A1; and r1 and r2 are the femur curvatures with direction a1. Also, in Equation 3, h1 and h2 are the material properties of the tibia and femur. Therefore, we can obtain the contact stress for the area:

Fp

ab

(4)

The stress distributions on the elliptic contact areas also vary based on the elliptic law. Here the method is really an approximation because we ignored the effect of the meniscus. Future methods will employ the finite-element method and include the effect of the meniscus.
Integrating gait cycle, knee kinematics, and pressure distributions

Fj = 0
j

(1)

After generating a patients 3D knee surface model and calculating the pressure distributions on the tibia and femur contact surfaces, our goal is to integrate the patients gait cycle, tibia and femur motions, and pressure distributions into one seamless simulation.

84

COMPUTING IN SCIENCE & ENGINEERING

Contact area (cm2)

(a)

30.0 25.0 20.0 15.0 10.0 5.0 0.0

Total area Left area Right area

(d)

20 10

0 10

30

50

70

90

Angle

(e)

(b)

Flexion angle:
MPa

0.0 1.4 2.8 4.3 5.7

45

(c)

90

(f)

Figure 3. The activities we hope to integrate into one simulation: (a) tracking leg motion and gait cycle; (b) two-leg motion simulation; (c) determination of the corresponding coordinates and stresses for the contact area (the femur model is lifted for better viewing); (d) contact-area measurement; (e) knee biomechanical analysis; and (f) osteotomy surgery simulation (side and front view).

Gait cycle and knee kinematics. To diagnose knee problems, doctors must observe knee behavior during the gait cycle. Gait analysis is widely used to measure a patients gait pattern. To collect gait-cycle kinematic data, we use a commercial motion-analysis system (Qualisys 3D, 60-Hz PC Reex System) that has three cameras connected to a computer. Twelve reective markers are attached to various positions on the patients leg so that the angle-positioned

cameras can capture the gait-cycle motion. The tracking software processes the information to identify and convert 2D camera data into 3D motion data at different time frames (see Figure 3a). We measure ground reaction forces, used to calculate the pressures, with a force platform (AMTI Biomechanics Force Platform #BP 2416-2000). Knee motion involves a combination of transformationsa series of three rotations (flexion-extension, abduc-

tion-adduction, and internal-external) and three translations (anterior-posterior, superior-inferior, and medial-lateral). The rotation angles and the translation distances can be calculated from the 3D motion data. If the rotation axes are properly located, the kinematics of the knee can be accurately simulated by simultaneous rotations. A recent study found that there is a single axis of exion and extension, the FE axis, which remains xed in the femur.9

JULY/AUGUST 1999

85

VISUALIZATION CORNER

The line passing through the two most prominent points on the femoral epicondylesthe TE lineis very close to the FE axis. Therefore, we define the flexion-extension axis as the FE axis,9 the internal-external rotation axis as the longitudinal axis of the calf, and abduction-adduction axis as perpendicular to the exion-extension and internal-external rotation axes passing through the middle of the knee joint. We can simulate the knee kinematics by animating the 3D knee model according to the captured gait-cycle motion data and the rotation axes (FE axis, longitudinal axis, and abductionadduction axis). By integrating the legmotion-capture system and the 3D knee-kinematics model, we can simulate the leg and 3D knee motions simultaneously. That is, a doctor can observe the leg and knee behaviors at the same time while the patient is walking (Figure 3b). Surface mapping, pressure distribution, and interactive interpolation. Current 3D knee models have no visible corresponding coordinate relations between the tibia and femur, making it difcult for the surgeons to identify and describe the exact deformed contact points without looking at the knee image. In our implementation, the tibia and femur surfaces (the cartilage and the meniscus) are tessellated with polygons (see Figure 3c). The tessellation for all human knees is the same, so surgeons can label the areas with different names and communicate with each other even without looking at the image or model. Given the posture of a given gait position and other patient data (such as weight, ground reaction forces, and material properties), we calculate the corresponding contact stresses as we discussed earlier (see Figure 3c). The light-colored circles are the contact

points between the tibia and meniscus, and the color intensities show the values of the stresses (Figure 3c does not show the pressure distributions on the contact areas around the contact points.) We plan to implement the nite-element method to nd more accurate pressure distributions. Unlike contact stresses, which are calculated by material dynamics, the contact areas between the femur, meniscus, and tibia can be determined by collision detection during animation (walking.) Figure 3d shows the femur contact area as a function of the exionextension angle. Figure 3e shows a sequence of animation angles. The contact-stress computation involves 16 nonlinear equations, which take a signicant amount of time to calculate. To integrate the activities as shown in Figure 3 into one seamless real-time simulation, we must nd creative solutions that synchronize timeconsuming computation and real-time simulation. Our following method will solve this problem so that we can have real-time simulation as well as pressuredistribution visualization on the y. In our method, the leg and knee animation will not wait for the corresponding contact-stress calculation results. Instead, we will invoke a parallel process (on a different CPU of a multiprocessor computer system such as an SGI Onyx2) to calculate the stress and pressure distributions. Well save the immediate pressure-distribution result and use it as a texture for the next corresponding posture. Well also save multiple pressure-distribution results and use them to interpolate all animation frames. Therefore, during the simulation, the pressures will come into the picture. (There is no simulation delay on a multiprocessor computer.) Once a pressure texture is generated, it can be reused; there is no need to recalculate unless a virtual surgery is performed,

which requires recalculating all the pressure distributions. This method can be generalized to many applications that need both timeconsuming computations to find certain key frame results and interactive visualization of the results in a realtime simulation. The restriction is that the key frames must be reused; that is, the corresponding simulation process is an animation cycle.
Virtual examinations and surgeries with interactive visualization

Surgery simulation can help surgeons improve surgery plans and practice surgery on a 3D knee model. Doctors can evaluate the simulated surgery results before surgery on a real patient.1,2 This is especially helpful in osteotomy surgeries. The surgery simulation displays the 3D knee model and a virtual cutting device. A simple plane with 3D control handles represents the virtual cutting device. Before surgery, several guiding points can be marked on the knees surface. A surgeon can use the mouse or keyboard to navigate the cutting device to cut through the 3D knee model, following the path indicated by the guiding points. The 3D knee model is split into two pieces along the intersected curve made by the cutting device (Figure 3f). Surgery consists of one or two cuts (to add or remove a wedge). The surgeon can move the model around while cutting to examine the incisions path. This process, which cannot be done on a real patient in the surgery, helps the surgeon correct the incision, gain experience, and therefore improve operation skills. The virtual surgery is based on a patient-specic model. So, when the real surgery occurs, the surgeon is already familiar with all the required operations. In the virtual surgery, the surgeon can grab and cut parts of the tibia or femur,

86

COMPUTING IN SCIENCE & ENGINEERING

and move them to the expected positions. The system will then recalculate the axes of the tibia and femur, the angle between the two axes, the tibia and femur contact areas, and other biomechanical parameters. The weight-bearing line will be redrawn. When a surgery simulation has been completed, the saved cutting angle and other surgery data can be visualized. The surgeon can compare the virtual surgery data with the real surgery plan, modify it, and repeat the simulation until he or she is satised with the surgery plan and familiar with the surgery process.

delity virtual-reality stereoscopic environment through 3D human-computerinteraction surgery tools.

Ying Zhu is a PhD candidate in the Department of Computer Science at George Mason University. His research interests are in graphical simulation and medical imaging. He received his BS and MS degrees in computer science from the

Acknowledgments
Thanks to Douglas Pew and Bobbie Pew for helping with the implementation of the knee reconstruction, and to Xiaodong Fu for implementing the surface mapping.

University of Electronic Science and Technology, China. He is a member of GMU Graphics Group. Contact him at yzhu1@gmu.edu. Jim X. Chen is an assistant professor and the director of the Graphics Lab in the Department of Computer Science at George Mason University. His research interests are in physics-based mod-

References
1. E.Y.S. Chao et al., Computer-Aided Preoperative Planning in Knee Osteotomy, The Iowa Orthopedic J., Vol. 15, No. 1, Jan. 1995, pp. 418. 2. S. Gibson et al., Simulating Arthroscopic Knee Surgery Using Volumetric Object Representations, Real-Time Volume Rendering and Haptic Feedback, First Joint Conf. Computer Vision, Virtual Reality, and Robotics in Medicine and Medical Robotics and Computer-Assisted Surgery, J. Troccaz, E. Grimson, and R. Mosges, eds., Springer-Verlag, Berlin, 1997, pp. 369378. 3. A. Garg et al., Prediction of Total Knee Motion Using a 3D Computer-Graphics Model, J. Biomechanics, Vol. 23, No. 1, Jan. 1990, pp. 4558. 4. J. Wismans et al., A 3D Mathematical Model of the Knee-Joint, J. Biomechanics, Vol. 13, No. 8, Aug. 1980, pp. 677685. 5. J.R. Steele et al., A Three-Dimensional Representation of an Athletic Female Knee Joint Using Magnetic Resonance Imaging, Medical Eng. and Physics, Vol. 16, No. 6, Sept. 1994, pp. 363369. 6. S.K. Wareld et al., Adaptive Template Moderated Spatially Varying Statistical Classication, Proc. First Intl Conf. Medical Image Computing and Computer-Assisted Intervention, Springer-Verlag, 1998, pp. 431438. 7. M. Kass et al., SnakesActive Contour Models, Intl J. Computer Vision, Vol. 1, No. 4, Mar. 1988, pp. 321331. 8. M. Nissan, Review of Some Basic Assumptions in Knee Biomechanics, J. Biomechanics, Vol. 13, No. 6, June 1980, pp. 375381. 9. D. Churchill et al., A Comparison of the Transepicondylar Line with the Optimal Flexion/Extension Axis of the Knee, Proc. 44th Ann. Meeting Orthopedic Research Soc., Orthopedic Research Soc., Rosemont, Ill., 1998.

eling, real-time simulation, interactive visualization, networked environments, and medical imaging. He received his BS and MS in computer science from Southwest Jiaotong University, China, and his PhD in computer science from the University of Central Florida. He is a member of the IEEE Computer Society and ACM. Contact him at the Dept. of Computer Science, George Mason Univ., MS 4A5, Fairfax, VA 22030-4444; jchen@cs.gmu.edu.; www.cs.gmu.edu/~jchen. Shide Xiao is an associate professor in the College of Mechanical Engineering at Southwest Jiaotong University, China. He joined the Graphics Lab at George Mason University as a visiting scholar in 1998. His research interests include CAD and visualization. He received his BS, MS, and PhD in mechanical engineering from China University of Mining and Technology. Contact him at sdxiao@ science.gmu.edu. Edward B. Mac Mahon is a clinical associate professor of orthopedic surgery at Georgetown University, Washington, DC. He is a former codirector of the Orthopedic Research Lab at Georgetown University. He is board-certied and a fellow of the American Academy of Orthopedic Surgeons. He is a fellow of the American College of Surgeons and a member of the Orthopedic Research Society. He practices with the Knee Alignment and Arthritis Group in Alexandria, VA. Contact him at (540) 253-5501 (phone); (540) 253-5163 (fax); shermacm@erols.com.

e plan to build comprehensive human-computer interfaces including 3D graphical user interfaces, virtual-reality power gloves, and virtualreality cutting devices with force feedback. This will allow surgeries on different platforms and with different levels of realism for low-cost manipulations as well as high-delity operations. Surgeons will be able to preview the postoperative behavior of a knee joint during the gait cycle. Because the geometry and biomechanical conditions change after the surgery, predicting and simulating knee motion under the new conditions are very complicated; we need further biomechanical analysis on the knee motion to achieve this goal. We also intend to improve the existing work in many perspectives, including using the nite-element method to calculate contact-surface pressures with more accuracy, distributed interactive simulation, and virtual reality. All animations will be displayed in a distributed interactive environment so that distant surgeons, learners, and other observers can engage in the same virtual examination and surgery simulation. The virtual surgery will be carried out in a high-

JULY/AUGUST 1999

87

You might also like