Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.

php

Bibliography

[1] R. A DAMS AND J. F OURNIER, Sobolev Spaces, 2nd ed., Academic Press, New York, 2003.
[2] D. A. A RULIAH , U. M. A SCHER , E. H ABER , AND D. O LDENBURG, A method for the for-
ward modelling of 3-D electromagnetic quasi-static problems, Math. Models Methods Appl.
Sci., 11 (2001), pp. 121.
[3] U. M. A SCHER, Numerical Methods for Evolutionary Differential Equations, SIAM, Philadel-
phia, 2008.
[4] U. M. A SCHER , R. M. M. M ATTHEIJ , AND R. D. RUSSELL, Numerical Solution of Boundary
Value Problems for Ordinary Differential Equations, SIAM, Philadelphia, 1995.
[5] U. M. A SCHER AND L. R. P ETZOLD, Computer Methods for Ordinary Differential Equations
and Differential-Algebraic Equations, SIAM, Philadelphia, 1998.
[6] M. B ENZI , G. H. G OLUB , AND J. L IESEN, Numerical solution of saddle point problems, Acta
Numerica, 14 (2005), pp. 1137.
[7] M. W. B ERRY AND M. B ROWNE, Understanding Search Engines: Mathematical Modeling
and Text Retrieval, 2nd ed., SIAM, Philadelphia, 2005.
[8] L. B IEGLER , O. G HATTAS , M. H EINKENSCHLOSS , AND B. VAN B LOEMEN WAANDERS,
eds., Large-Scale PDE-Constrained optimization, vol. 30 of Lecture Notes in Computational
Science and Engineering, Springer-Verlag, Berlin, 2003.
[9] L. B LUM , F. C UCKER , M. S HUB , AND S. S MALE, Complexity and Real Computation,
Springer-Verlag, Berlin, 2001.
[10] D. R. B RILLINGER, Time Series: Data Analysis and Theory, SIAM, Philadelphia, 2001.
[11] R. L. B URDEN AND J. D. FAIRES, Numerical Analysis, 8th ed., Brooks/Cole, Pacific Grove,
CA, 2004.
[12] W. C HENEY AND D. K INCAID, Numerical Mathematics and Computing, 6th ed.,
Brooks/Cole, Pacific Grove, CA, 2008.
[13] S. D. C ONTE AND C. DE B OOR, Elementary Numerical Analysis, an Algorithmic Approach,
McGrawHill, New York, 1972.
[14] T. C ORMEN , C. L EISERSON , AND R. R IVEST, Introduction to Algorithms, MIT Press, Cam-
bridge, MA, 1990.
[15] G. DAHLQUIST AND A. B JRCK, Numerical Methods, PrenticeHall, Englewood Cliffs, NJ,
1974.

539

540 Bibliography
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

[16] I. DAUBECHIES, Ten Lectures on Wavelets, SIAM, Philadelphia, 1992.

[17] P. DAVIS AND P. R ABINOWITZ, Methods of Numerical Integration, 2nd ed., Academic Press,
New York, 1985.

[18] P. J. DAVIS, Circulant Matrices, John Wiley & Sons, New York, 1979.

[19] T. A. DAVIS, Direct Methods for Sparse Linear Systems, SIAM, Philadelphia, 2006.

[20] C. DE B OOR, A Practical Guide to Splines, Springer-Verlag, New York, 1978.

[21] J. W. D EMMEL, Applied Numerical Linear Algebra, SIAM, Philadelphia, 1997.

[22] J. E. D ENNIS , J R ., AND R. B. S CHNABEL, Numerical Methods for Unconstrained Optimiza-

tion and Nonlinear Equations, SIAM, Philadelphia, 1996.

[23] H. C. E LMAN , D. J. S ILVESTER , AND A. J. WATHEN, Finite Elements and Fast Iterative
Solvers: With Applications in Incompressible Fluid Dynamics, Oxford University Press, New
York, 2005.

[24] H. W. E NGL , M. H ANKE , AND A. N EUBAUER, Regularization of Inverse Problems, Kluwer,

Dordrecht, Netherlands, 1996.

[25] R. F LETCHER, Practical Methods of Optimization, 2nd ed., John Wiley & Sons, New York,
1987.

[26] B. F ORNBERG, A Practical Guide to Pseudospectral Methods, Cambridge University Press,

London, 1998.

[27] W. G ANDER AND W. G AUTSCHI, Adaptive quadrature revisited, BIT, 40 (2000), pp. 84101.
[28] C. W. G EAR, Numerical Initial Value Problems in Ordinary Differential Equations, Prentice
Hall, Englewood Cliffs, NJ, 1973.

[29] P. E. G ILL , W. M URRAY, AND M. H. W RIGHT, Practical Optimization, Academic Press,

New York, 1981.

[30] G. H. G OLUB AND C. F. VAN L OAN, Matrix Computations, 3rd ed., Johns Hopkins Univer-
sity Press, Baltimore, MD, 1996.

[31] R. G RAHAM , D. K NUTH , AND O. PATASHNIK, Concrete Mathematics, 2nd ed., Addison
Wesley, Reading, MA, 1994.

[32] A. G REENBAUM, Iterative Methods for Solving Linear Systems, SIAM, Philadelphia, 1997.

[33] A. G RIEWANK, Evaluating Derivatives: Principles and Techniques of Algorithmic Differenti-

ation, SIAM, Philadelphia, 2000.
[34] I. G RIVA , S. G. NASH , AND A. S OFER, Linear and Nonlinear Optimization, 2nd ed., SIAM,
Philadelphia, 2009.

[35] E. H AIRER , C. L UBICH , AND G. WANNER, Geometric Numerical Integration, Springer-

Verlag, New York, 2002.

[36] E. H AIRER , S. P. N ORSETT, AND G. WANNER, Solving Ordinary Differential Equations I:

Nonstiff Problems, Springer-Verlag, Berln, 1993.

Bibliography 541
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

[37] E. H AIRER AND G. WANNER, Solving Ordinary Differential Equations II: Stiff and
Differential-Algebraic Problems, 2nd ed., Springer-Verlag, Berlin, 1996.
[38] T. H ASTIE , R. T IBSHIRANI , AND J. F RIEDMAN, The Elements of Statistical Learning,
Springer-Verlag, New York, 2001.
[39] M. T. H EATH, Scientific Computing: An Introductory Survey, 2nd ed., McGrawHill, New
York, 2002.
[40] N. J. H IGHAM, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia,
2002.
[41] F. S. H ILL, Computer Graphics Using Open GL, 2nd ed., PrenticeHall, Englewood Cliffs,
NJ, 2001.
[42] B. H ORN, Robot Vision, MIT Press, Cambridge, MA, 1986.
[43] H. H UANG, Efficient Reconstruction of 2D Images and 3D Surfaces, Ph.D. thesis, University
of British Columbia, Vancouver, Canada, 2008.
[44] H. H UANG AND U. A SCHER, Fast denoising of surface meshes with intrinsic texture, Inverse
Problems, 24 (2008), article 034003.
[45] H. H UANG , D. L I , R. Z HANG , U. A SCHER , AND D. C OHEN -O R, Consolidation of unorga-
nized point clouds for surface reconstruction, ACM Trans. Graphics (SIGGRAPH Asia), 28(5)
(2009).
[46] H. B. K ELLER, Numerical Solution of Two Point Boundary Value Problems, SIAM, Philadel-
phia, 1976.
[47] A. N. L ANGVILLE AND C. D. M EYER, Googles PageRank and Beyond: The Science of
Search Engine Rankings, Princeton University Press, Princeton, NJ, 2006.
[48] R. B. L EHOUCQ , D. C. S ORENSEN , AND C. YANG, ARPACK Users Guide, SIAM, Philadel-
phia, 1998.
[49] B. L EIMKUHLER AND S. R EICH, Simulating Hamiltonian Dynamics, Cambridge University
Press, London, 2004.
[50] R. J. L E V EQUE, Finite Difference Methods for Ordinary and Partial Differential Equations,
SIAM, Philadelphia, 2007.
[51] D. L UENBERGER, Introduction to Linear and Nonlinear Programming, AddisonWesley, New
York, 1973.
[52] S. M ALLAT, A Wavelet Tour of Signal Processing: The Sparse Way, 3rd ed., Academic Press,
New York, 2009.
[53] S. M EHROTRA, On the implementation of a primal-dual interior point method, SIAM J. Op-
tim., 2 (1992), pp. 575601.
[54] R. E. M OORE , R. B. K EARFOTT, AND M. J. C LOUD, Introduction to Interval Analysis,
SIAM, Philadelphia, 2009.
[55] N. M. NACHTIGAL , S. C. R EDDY, AND L. N. T REFETHEN, How fast are nonsymmetric
matrix iterations?, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 778795.

542 Bibliography
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

[56] F. NATTERER, The Mathematics of Computerized Tomography, SIAM, Philadelphia, 2001.

[57] J. N OCEDAL AND S. W RIGHT, Numerical Optimization, 2nd ed., Springer-Verlag, New York,
2006.
[58] M. L. OVERTON, Numerical Computing with IEEE Floating Point Arithmetic, SIAM,
Philadelphia, 2001.
[59] W. P RESS , S. T EUKOLSKY, W. V ETTERLING , AND B. F LANNERY, Numerical Recipes, 3rd
ed., Cambridge University Press, London, 2007.
[60] T. R IVLIN, An Introduction to the Approximation of Functions, Dover, New York, 1981.
[61] W. RUDIN, Functional Analysis, McGrawHill, New York, 1991.
[62] Y. S AAD, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, Philadelphia, 2003.
[63] G. W. S TEWART, Matrix Algorithms. Vol. II: Eigensystems, SIAM, Philadelphia, 2001.
[64] G. S TRANG, Introduction to Linear Algebra, 4th ed., SIAM, Philadelphia, 2009.
[65] S. H. S TROGATZ, Nonlinear Dynamics and Chaos, AddisonWesley, Reading, MA, 1994.
[66] A. M. S TUART AND A. R. H UMPHRIES, Dynamical Systems and Numerical Analysis, Cam-
bridge University Press, London, 1996.
[67] A. N. T IKHONOV AND V. Y. A RSENIN, Methods for Solving Ill-Posed Problems, John Wiley
& Sons, New York, 1977.
[68] L. N. T REFETHEN, Lax-stability vs. eigenvalue stability of spectral methods, in Numerical
Methods for Fluid Dynamics III, K. Morton and M. Bains, eds., Clarendon Press, Oxford, UK,
1988, pp. 237253.
[69] L. N. T REFETHEN, Spectral Methods in MATLAB, SIAM, Philadelphia, 2000.
[70] L. N. T REFETHEN AND D. BAU , III, Numerical Linear Algebra, SIAM, Philadelphia, 1997.
[71] U. T ROTTENBERG , C. O OSTERLEE , AND A. S CHULLER, Multigrid, Academic Press, New
York, 2001.
[72] H. VAN DER VORST, Iterative Krylov Methods for Large Linear Systems, Cambridge Univer-
sity Press, London, 2003.
[73] C. R. VOGEL, Computational Methods for Inverse Problem, SIAM, Philadelphia, 2002.
[74] D. WATKINS, Fundamentals of Matrix Computations, 2nd ed., John Wiley & Sons, New York,
2002.
[75] J. H. W ILKINSON, The Algebraic Eigenvalue Problem, Oxford University Press, New York,
1988.

Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Index
(Page numbers set in bold type indicate where the statement of the corresponding theorem or the
detailed account of the corresponding algorithm are provided.)

A-stability, 510 for systems, 254

absolute stability, 484, 491 for unconstrained minimization, 261
absolute stability region, 509 polynomial interpolation Newton form,
active set method, 275 321
Adams method, 501 power method, 222
AdamsBashforth method, 502 preconditioned conjugate gradient
AdamsMoulton method, 502 (PCG), 188
adaptive algorithm, 307, 463 predictor-corrector, 506
adaptive quadrature, 462 QR with shifts, 241
adiabatic invariant, 519 Rayleigh quotient iteration, 228
advection equation, 525 Romberg integration, 471
algorithm secant method, 53
Arnoldi, 194 algorithm property, 914
backward substitution, 95 adaptivity, 307
BFGS, 268 efficiency, 42
Cholesky decomposition, 116 AMD, see approximate minimum degree
conjugate gradient, 184 approximate inverse, 187
continuous GramSchmidt, 374 approximate minimum degree (AMD), 125
continuous least squares, 370 approximation, 295
cubic spline interpolation, 342 continuous least squares, 366
discrete cosine transform, 403 general, 365
discrete Fourier transform, 389 local vs. global, 316
fixed point iteration, 46 polynomial, 298, 363
forward substitution, 96 spline, 363
GaussNewton least squares, 269 Arnoldi algorithm, 192, 194
Gaussian elimination, 99 artificial time, 537
inverse iteration, 228 augmented Lagrangian method, 275
Lagrange polynomial interpolation, 305 augmented matrix, 97
least squares via normal equations, 145 automatic differentiation, 424, 439
least squares via QR, 156
least squares via SVD, 235 B-spline, 347, 349, 363
LU decomposition, 103 backtracking, 264
for banded matrices, 120 backward difference operator, 327, 410, 416
modified GramSchmidt, 159 backward differentiation formulas (BDF),
multigrid, 206 501
Newtons method, 51 backward error analysis, 130
for stiff ODEs, 513 backward Euler, 485, 492

543

544 Index
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

semi-implicit, 513 chopping, 22

backward substitution, 94 compact difference, 420
algorithm, 95 complementarity slackness, 277
bandwidth, 119 complex number
upper and lower, 119 conjugate, 71
barrier method, 275 imaginary part, 71
barycentric magnitude, 71
interpolation, 305 real part, 71
weight, 305 complexity theory, 37
basis composite
monomial, 297 Gaussian quadrature, 477
orthogonal, 192, 371 midpoint quadrature, 449, 453, 476
piecewise polynomial, 344 Simpson quadrature, 448, 449, 463
basis function, 297, 366, 527 trapezoidal quadrature, 448, 449
Bayesian method, 479 computational geometry, 357
BDF method, see backward differentiation computer aided geometric design (CAGD),
formulas 331, 347, 351, 357, 363
best approximation, 365 computer graphics, 351, 357, 363
best lower rank approximation theorem, 231 computer vision, 406
Bzier polynomial, 352, 362 condition number, 93, 128, 153
BFGS algorithm, 268 conditioning, 11, 129, 301, 304
BiCGSTAB, 202 ill-conditioned problem, 11, 12, 69, 301
bidiagonalization, 244 well-conditioned problem, 12, 55, 69
bisection algorithm, 43 conjugate gradient (CG), 167, 182, 184190,
bit, 18 330
BLAS, 113 algorithm, 184
boundary value problem, 256, 481, 484 convergence theorem, 187
finite difference, 522 for least squares (CGLS), 191
implicit RungeKutta, 522 preconditioned, see preconditioned con-
multiple shooting, 524 jugate gradient
shooting, 523 constrained minimization conditions theo-
Burgers equation, 530 rem, 274
Butcher tableau, 499 constrained optimization, 271276
constraint qualification, 273
CAGD, see computer aided geometric design continuation, 536
CG, see conjugate gradient continuous GramSchmidt algorithm, 374
CGLS, see conjugate gradient for least continuous GramSchmidt orthogonalization
squares theorem, 375
characteristic curve, 528 continuous least squares, 370
Chebyshev, 316 contraction, 47
collocation, 429 convection-diffusion equation, 199
differentiation matrix, 428 convergence, 46
extremum points, 328, 428 linear, 52
interpolation, 316, 394 order, 52
points, 316, 382, 394 quadratic, 52, 54
polynomial, 316, 330, 376378 rate, 10, 49, 52
Chebyshev min-max property theorem, 379 superlinear, 52, 54
checkerboard ordering, 176 convex function, 58
Cholesky decomposition, 115 convolution, 387, 401
algorithm, 116 corrected trapezoidal rule, 476

Index 545
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

CPLEX, 293 approximation, 5

CrankNicolson scheme, 511 cancellation, 20, 26
critical point, 56, 260 convergence, 5
cubic interpolation algorithm, 342 discretization, 5, 6
curve, 353 estimate, 415, 463, 472
Gaussian quadrature, 457
data compression, 404 global, 488, 517
data fitting, 83, 85, 141, 146, 295, 365 in input data, 5
DCT, see discrete cosine transform in model, 4
deblurring, 401 in problem, 4
Delaunay triangulation, 357 in quadrature, 443
descent direction, 262 local, 517
DFT, see discrete Fourier transform local truncation, 488, 501
differential equation, 9, 85, 256, 481 order, 335
differentiation matrix, 409 relative, 3, 19
Chebyshev, 428 roundoff, 5, 17, 55, 492
Fourier, 429 Simpson rule, 445
polynomial, 428 trapezoidal rule, 444
digital signal processing, 384 truncation, 410, 416
directional derivative, 259 error tolerance, 462
discrete cosine transform (DCT), 403 absolute, 44
algorithm, 403 relative, 44
fast, 404
Eulers method
inverse transform, 403
symplectic, 535
discrete Fourier transform (DFT), 388, 389
extrapolation, 296, 409, 413, 414, 470
algorithm, 389
discrete orthogonality, 388
fast Fourier transform (FFT), 396, 400, 406
divide-and-conquer, 399, 463
algorithm, 400
divided difference, 308, 312, 313, 320
formula, 308 fast matrix-vector product, 400, 428
table, 308, 323 feasible solution, 271
divided difference and derivative theorem, FEM, see finite element method
312 FFT, see fast Fourier transform
double precision, 19, 422 finite difference, 256
drop tolerance, 189 method, 168
duality, 276 finite element method (FEM), 168, 346, 382,
duality gap, 278 527
dynamical systems, 537 finite volume method, 356
fixed point, 45
eigenfunction, 242 theorem, 47, 49, 59
eigenvalue, 69, 219 fixed point iteration, 45, 52, 172, 506
algebraic vs. geometric multiplicity, 70 algorithm, 46
inverse iteration, 219, 227 floating point
power method, 219 double precision, 30
QR method, 237 guard digit, 19
eigenvector, 69 IEEE standard, 29, 36
energy norm, 90 long word, 30
error, 3 mantissa, 33
a posteriori estimate, 463 normalization, 22
a priori estimate, 463 representation, 18
absolute, 3, 6 representation error theorem, 24

546 Index
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

single precision, 30 gradient descent, 167, 182, 214, 263

flop, 10 GramSchmidt algorithm, 157, 158
flux function, 438 GramSchmidt orthogonalization, 374, 375
FOM, see full orthogonalization method greedy algorithm, 265
forward difference operator, 326, 410, 416, grid, 168, 463
485 guidepoint, 352
forward Euler, 485
convergence theorem, 489 Hamiltonian system, 535
forward substitution, 96, 307 hat basis function, 345, 346
algorithm, 96 heat equation, 510
Fourier transform, 383 Hermite polynomial, 376
continuous, 383 Hessian matrix, 259
cutoff frequency, 384 high frequency component, 204
discrete, 383, 388, 389, 450 Hilbert matrix, 368
discrete inverse, 389 Hopf bifurcation, 534
fast, see fast Fourier transform Horners rule, 10, 59
inverse, 387, 389, 396 Householder reflection, 154, 157, 159
inverse fast, see IFFT
IC, see incomplete Cholesky
series, 384, 387
IEEE floating point standard, 19, 29, 36
frequency, 384 IFFT, 401
full orthogonalization method (FOM), 197 ill-posed problem, 439
function minimization, 79 ILU, see incomplete LU
one variable, 40 image deblurring, 401
function norm, 365 image processing, 407
function space, 366 incomplete Cholesky (IC), 188
drop tolerance, 189
Gauss points, 376, 382, 455 incomplete factorization, 188
GaussJordan elimination, 134 incomplete LU (ILU), 203
GaussNewton method, 269 inexact Newton method, 267
algorithm, 269 initial value ODE, 481
GaussSeidel method, 175, 205 initial value problem, 481, 482
Gaussian elimination, 93, 94, 96, 301, 343 inner product, 110
algorithm, 99 integral
partial pivoting (GEPP), 106 definite, 441
Gaussian noise, 424 indefinite, 441
Gaussian quadrature, 441, 454456, 477 integration by parts, 368
generalized minimum residual (GMRES), interior point method, 275, 278
197 intermediate value theorem, 10, 46
algorithm, 198 interpolant
restarted, 199 construction, 298
geometric integration, 537 evaluation, 298
GEPP, see Gaussian elimination, partial piv- interpolation, 83, 295
oting bicubic, 354
Gibbs phenomenon, 395 bilinear, 206, 354
Givens rotation, 159, 198 break point, 332, 333
global minimization, 58 broken line, 333, 334
GMRES, see generalized minimum residual error, 335
golden section search algorithm, 64 Hermite, 320
Google, 220 Hermite piecewise cubic, 336, 344, 362
gradient, 79, 259 Lagrange form, 302

Index 547
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

linear form, 297 theorem, 144

parametric, 349
parametric Bzier polynomial, 362
parametric Hermite, 351, 362
patch, 354
piecewise constant, 335
piecewise cubic, 335
piecewise linear, 333, 335
piecewise polynomial, 298, 332
polynomial, 297, 299, 324, 415
rational, 363
trigonometric, 298, 388, 406
interval arithmetic, 37
inverse iteration, 219, 227
inverse iteration algorithm, 228
iterated integral, 473
iterated least squares, 292
iteration matrix, 179
iterative method, 5, 41, 172
Jacobis method, 174, 205
damped, 178, 206, 213
under-relaxed, 178
Jacobian matrix, 508
KarushKuhnTucker (KKT) conditions, 273
Kronecker product, 200
Krylov
orthogonal basis, 192
subspace, 186, 192
subspace method, 167, 191
L-BFGS, 267
L-stability, 513
lagged steepest descent method, 266
Lagrange multiplier, 273
Lagrange polynomial, 302, 304
Lagrange polynomial interpolation algo-
rithm, 305
Lagrangian, 273
Laguerre polynomial, 376
Lanczos algorithm, 195
latent semantic analysis, 231
latent semantic indexing, 231
law of large numbers, 475
leapfrog method, 529, 535
least squares, 191, 366
nonlinear, 268
normal equations, 143
pseudo-inverse, 144
QR, 153
via normal equations, 145
via QR, 156
via SVD, 235
Legendre polynomial, 371
level set, 272
lexicographic ordering, 176
limit cycle, 498, 534
limited memory method, 267
line search, 263
linear independence, 67
linear programming (LP), 151, 166, 276
center path, 278
dual form, 276
duality gap, 278
interior point method, 278
predictor-corrector algorithm, 279
primal form, 276
primal-dual method, 278
simplex, 277
linear system
direct method, 93
iterative method, 93
linearly independent functions, 297, 366
linearly independent vectors, 67
Lipschitz continuity, 489
Lipschitz continuous function, 60
Lobatto points, 461
local maximizer, 56
local minimizer, 56
local refinement, 463
local truncation error, 494, 501
long difference, 412
low pass filter, 385
LP, see linear programming
LU
decomposition, 93, 103, 168
algorithm, 103
for banded matrices algorithm, 120
incomplete, 203
machine epsilon, 19, 23
machine precision, 19, 23, 31, 421, 422
mantissa, 18
Maple, 438, 479
mathematical model, 1
mathematical software, 11, 463
MATLAB, 2
adaptive quadrature, 464
backslash, 84, 111, 150

548 Index
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

bisec, 45 inverse, 103

bisect, 44, 48, 59 lower triangular, 96, 307
circshift, 389 Markov, 246
cond, 129 norm, 73
dct, 403 nullspace, 68
delaunaytri, 357 orthogonal, 65, 80, 108, 131
eig, 73, 237, 509 permutation, 107
fft, 400, 430 principal minor, 79, 114
fft2, 401 projection, 245
fzero, 39, 523 pseudo-inverse, 235
ginput, 362 range space, 68
ifft, 401, 430 similar, 72
ldl, 136 singular, 67
myfft, 400 singular value, 77, 78, 81, 219, 229
norm, 75 singular value decomposition, 80, 229
ode23s, 514 skew-symmetric, 90, 216, 248
ode45, 517, 518, 523 sparse, 86, 93, 117
plot, 362 spectral radius, 77
polyfit, 150, 164 spectrum, 69
qr, 154 stochastic, 246
quad, 381, 462 symmetric, 78, 114
quad2d, 478 symmetric positive definite, 65, 78, 114,
quadl, 461 343
quads, 515 tridiagonal, 119, 343
rk4, 495, 497 unitary, 165
spline, 164, 339, 363 upper Hessenberg, 139
svd, 81 upper triangular, 94
symamd, 126 Vandermonde, 84, 148
symrcm, 126 mean value theorem, 10
triplot, 357 for integrals, 475
triscatteredinterp, 357 memory hierarchy, 113
matrix mesh, 463
approximate inverse, 173 method
banded, 119, 522 Chebyshev collocation, 429
Cholesky decomposition, 114, 115 finite element, 527
column stochastic, 224 nodal, 528
condition number, 93, 128, 153 pseudospectral, 429
defective, 226 simultaneous relaxation, 174
dense, 96 spectral, 430
determinant, 67, 104 midpoint rule, 443, 455, 476
diagonal, 94 min-max, 151, 316, 379, 382
diagonally dominant, 109, 175, 343 minimum residual, 197
eigenpair, 69 minimum residual (MINRES), 197, 198
eigenvalue, 69, 77, 219 modified GramSchmidt algorithm, 159
eigenvector, 69 molecular dynamics, 535
fill-in, 121, 168 monomial basis function, 297
graph, 124 Monte Carlo method, 474, 479
Hermitian, 137 multigrid method, 167, 204
Hessenberg, 139 algorithm, 206
Hilbert, 368 multiple root, 54, 55

Index 549
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

multiresolution, 173, 407 operation count

multistep method, 500 flop, 95
multistep method order theorem, 504 operator splitting, 432
optimization, 55, 258
NaN, 19, 31 active set, 272
nested form, 10, 298 active set method, 275
Newton and secant convergence theorem, 54 augmented Lagrangian method, 275
Newton backward difference, 327, 330 barrier method, 275
Newton forward difference, 327, 330 complementarity slackness, 277
Newtons method, 50, 522 constrained, 271, 293
algorithm, 51 continuous, 293
for stiff ODEs, 513 convexity, 58
for systems, 254 discrete, 293
for unconstrained minimization, 261 feasible solution, 271
geometric interpretation, 50 global, 58, 293
NewtonCotes, 446 interior point method, 275, 278
closed vs. open, 443 KKT, 273
nodal method, 346 Lagrange multiplier, 273
noise, 424, 425 Lagrangian, 273
filter, 424 linear programming, 276
nonlinear equation, 39
local, 56
contraction, 47
objective function, 55
root (zero), 39
penalty method, 275
nonlinear least squares, 166, 268
primal-dual form, 277
nonsingular matrix, 67
primal-dual method, 278
norm, 73
quadratic programming, 274
L 1 , L 2 and L , 366
saddle point, 260, 273, 289
1 and  , 151
simplex, 277
2 , 141
SQP, 276
function, 366
unconstrained, 258, 293
matrix, 75
sup vs. max, 366 order, 501
vector, 73 order notation, 7
normal direction, 88 order of accuracy, 449
normal equations, 143, 191, 367, 374 ordinary differential equation (ODE), 481
nullspace, 68 autonomous, 483
numerical algorithm, 1 boundary value problem, 484, 521
numerical analysis, 1 FPU, 519
numerical differentiation, 6, 337, 409, 418 initial condition, 482
backward, 410 initial value problem, 482, 521
centered, 411, 412 nonlinear, 483
compact, 420 stiff, 492, 506, 507, 510
forward, 410 terminal value problem, 484
general formula, 417 test equation, 491
long, 412 trajectory, 482
one-sided, 410 ordinary differential equation (ODE) method
numerical integration, 441 A-stable, 510, 513
multidimensional, 472, 479 absolute stability, 491, 505
absolute stability region, 509
objective function, 55 Adams method, 501
ODE, see ordinary differential equation AdamsBashforth, 502, 505

550 Index
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

AdamsMoulton, 502, 505 penalty method, 275

backward differentiation formulas pendulum, 482
(BDF), 505 periodic function, 383
backward Euler, 502, 510, 512 piecewise polynomial
classical Runge-Kutta, 495 basis, 344
explicit, 487, 493, 501 interpolation error theorem, 336
explicit midpoint, 494 pivoting, 93
explicit trapezoidal, 494 complete, 109
extrapolation, 531, 532 partial, 106
forward Euler, 484, 485, 492 scaled partial, 108
global error, 488 point cloud, 88
implicit, 487, 493, 501, 510 Poisson equation, 168
implicit midpoint, 494 polynomial
implicit trapezoidal, 493, 502 Bzier, 352
L-stable, 513 Chebyshev, 376, 378
leapfrog, 535 Hermite, 376, 479
local truncation error, 488, 494 Laguerre, 376, 479
midpoint, 510 Legendre, 371, 375, 455
multistep, 500 monic, 379
order, 501 nested form evaluation, 10, 59
order of accuracy, 488 orthogonal, 365, 374
predictor-corrector, 506 trigonometric, 373, 383
RungeKutta, 493 polynomial interpolation, 83, 299, 324, 409,
semi-implicit backward Euler, 513 442
trapezoidal, 510 equispaced, 326, 327, 330
verlet, 535 error, 313, 416
orthogonal error theorem, 314
basis, 371 existence and uniqueness theorem, 301
function, 366, 371 Hermite, 320
iteration, 239 Hermite cubic, 320
polynomial, 366, 370, 455 Lagrange form, 302, 305, 416, 442
projection, 368 Newton form, 306, 308
vectors, 144 Newton form algorithm, 321
orthogonal decomposition, 153 osculating, 319, 321
orthonormal basis, 373 parametric, 351
osculating interpolation, 319 trigonometric, 389
outer product, 110 uniqueness, 301
overflow, 20, 26, 49 power method, 219
algorithm, 222
PageRank, 220 precision, 18, 446, 449, 454
parametrization, 350 double, 19, 421
partial differential equation, 481 single, 19, 421
partial differential equation (PDE), 168, 484 preconditioned conjugate gradient (PCG),
elliptic, 525 187190
hyperbolic, 525 algorithm, 188
parabolic, 525 preconditioner, 167
partial differential equation (PDF), 199 predator-prey model, 497
PCA, see principal component analysis predictor-corrector algorithm, 506
PCG, see preconditioned conjugate gradient primal-dual method, 278
PDE, see partial differential equation principal component analysis (PCA), 87

Index 551
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

principal error term, 462 Ritz value, 196

problem solving environment, 2 Rolle theorem, 10
product notation, 301 Romberg integration, 441, 469, 471
projector, 245 root of unity, 398
pseudo-inverse, 144, 235 rotation, 159
pseudo-spectral method, 429 roughing, 453
rounding, 22
QMR, 202 exact, 19
QR rounding unit, 19, 23, 31, 33, 55, 69, 421,
eigenvalue algorithm, 240 424, 460
iteration, 241 roundoff error, 17, 492
QR decomposition, 153 Runge example, 316
quadratic form, 259 RungeKutta method, 493
quadratic programming, 274 RungeKutta tableau notation, 499
quadrature, 441
abscissa, 441 saddle point, 274
adaptive, 462
SAXPY, 113
basic rule, 441
scientific computing, 1
composite rule, 446
search algorithm, 220
degree of accuracy, 446
search direction, 182
error, 443
secant method, 53, 267
Gaussian, 454, 477
secant method algorithm, 53
Lobatto, 461
sensitivity, 11
midpoint rule, 443
sequential quadratic programming (SQP)
NewtonCotes, 443
method, 276
panel, 448
shift: explicit and implicit, 243
precision, 446
significant digits, 23, 31
Radau, 461
similarity transformation, 72
Romberg integration, 469
Simpson, 533 simplex, 277
Simpson rule, 443, 463 Simpson rule, 443
trapezoidal rule, 443 simultaneous iteration, 239
weights, 441, 442 sinc function, 62
quadrature errors theorem, 453 singular value decomposition (SVD), 80,
quasi-Newton method, 53, 267 153, 219, 229
algorithm, 268 economy size, 236
truncated, 231, 234
Radau points, 461 smoothing, 453
radial basis function (RBF), 358 smoothing factor, 217
range space, 68 smoothness, 335
rate of convergence, 10, 49, 180 Sobolev space, 382
Rayleigh quotient, 222 soliton, 433
Rayleigh quotient iteration, 228 SOR method, see successive over-relaxation
RCM, see reverse Cuthill McKee method
recursion, 44, 399 sparse solution, 285
red-black ordering, 176 spectral
reflection, 160 accuracy, 319, 328, 393, 394
regularization, 230, 402, 424 collocation, 330
relaxation method, 174, 205 method, 407, 430, 439
reverse Cuthill McKee (RCM), 125 spline interpolation, 164
Richardson extrapolation, 413, 470, 478 boundary conditions, 342

552 Index
Downloaded 02/02/17 to 128.84.125.143. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

clamped, 339, 343, 361 floating point representation error, 24

complete, 339, 362 forward Euler convergence, 489
cubic, 337, 339 intermediate value, 10
free boundary, 339 least squares, 144
natural, 339, 342, 344 mean value, 10
not-a-knot, 339, 343, 360 mean value for integrals, 475
splitting, 173 multistep method order, 504
splitting method, 432 Newton and secant convergence, 54
SQP method, see sequential quadratic pro- piecewise polynomial interpolation er-
gramming method ror, 336
SSOR, see symmetric SOR polynomial interpolant existence and
stability, 11, 12, 108, 507 uniqueness, 301
staggered mesh, 426 polynomial interpolation error, 314
stationary method convergence theorem, 180 quadrature errors, 453
steepest descent method, 167, 183, 214, 266 Rolle, 10
step size, 57, 182, 500 stationary method convergence, 180
stiff ODE, 461, 492 Taylor series, 5
stiff problem, 484, 507, 510 Taylor series for vector functions, 253
Stirling centered difference, 330 Taylor series in several variables, 259
stretching transformation, 532 unconstrained minimization conditions,
subnormal number, 25 260
subspace, 67 thin plate spline, 425
subspace iteration, 239 Thomas algorithm, 121
successive over-relaxation (SOR) method, time series data analysis, 406
177 tomography, 406
sufficiently smooth, 410 trajectory, 482
support trapezoidal rule, 443, 461
local, 346 trigonometric basis function, 383
of a function, 345 trigonometric polynomial, 373, 383
surface, 353 truncated SVD, 231, 234
surface mesh, 353 trust region, 263
SVD, see singular value decomposition
symbolic computing, 438, 479 unconstrained minimization conditions theo-
symmetric SOR (SSOR), 179, 188 rem, 260
underflow, 20, 26
tangent plane, 88 unit vector, 104
Taylor polynomial, 320 upstream method, 530
Taylor series, 5, 6, 50, 302, 320 upwind method, 530
for vector functions theorem, 253
in several variables theorem, 259 variational form, 527
theorem, 5 vector
tensor product, 354 orthogonal, 79, 144
theorem orthonormal, 79
best lower rank approximation, 231 Verlet method, 535
Chebyshev min-max property, 379 vertex degree, 125
conjugate gradient convergence, 187 wave number, 387
constrained minimization conditions, wave speed, 528
274 wavelet, 407
continuous GramSchmidt orthogonal- transform, 407
ization, 375 weak line search, 264
divided difference and derivative, 312 weighted least squares, 373
fixed point, 47