Chapter IV Solving Systems of Linear Equations Goal: to construct some general-purpose algorithms for solving systems of linear Equations
S4.4 Norms and the Analysis of Errors
S4.4 Norms and the Analysis of Errors ½ÂµeN : R n R + {0} vµ šk5 X 0, X = 0 X = 0 àg5 a R, ax = a X nøª X + Y X + Y K TN þ mr n þ «þ ê(vector norm).
S4.4 Norms and the Analysis of Errors ½ÂµeN : R n R + {0} vµ šk5 X 0, X = 0 X = 0 àg5 a R, ax = a X nøª X + Y X + Y K TN þ mr n þ «þ ê(vector norm). X = (x 1, x 2,, x n). x p ( n x i p ) p 1 þp ê. p ê ~ kµ 1 X 1 = n x i 1- ê i=1 n 2 X 2 = x 2 i 2- ê i=1 i=1 3 X = max{ x i } - ê
1 ~µož þx = 2 1 ê 2 ê ê" 1.5
1 ~µož þx = 2 1 ê 2 ê ê" 1.5 )µd þ ê½âµ X 1 =
1 ~µož þx = 2 1 ê 2 ê ê" 1.5 )µd þ ê½âµ X 1 = 3 x k = 1 + 2 + 1.5 = 4.5 k=1 X 2 =
1 ~µož þx = 2 1 ê 2 ê ê" 1.5 )µd þ ê½âµ X 1 = 3 x k = 1 + 2 + 1.5 = 4.5 k=1 X 2 = 3 xk 1 2 = + ( 2) 2 + (1.5) 2 = 7.25 = 2.69258 X = k=1
1 ~µož þx = 2 1 ê 2 ê ê" 1.5 )µd þ ê½âµ X 1 = 3 x k = 1 + 2 + 1.5 = 4.5 k=1 X 2 = 3 xk 1 2 = + ( 2) 2 + (1.5) 2 = 7.25 = 2.69258 k=1 X = max 1 k 3 { x k } = 2
Matrix Norm (Ý ê) ½Âµ ±n Cþ Š¼ê N v 1 šk5µ A 0 A = 0 =A = 0. 2 àg5µ αa = α A, α R 3 nøª: A + B A + B 4 ƒn5: AB A B K N A «Ý ê.
Theorem (Theorem on Subordinate Matrix Norm) If is any vector norm on R n, then the equation { A = sup Au : u R n} Au = sup u =1 u R n,u 0 u defines a norm on the linear space of all n n matrices.
Theorem (Theorem on Subordinate Matrix Norm) If is any vector norm on R n, then the equation { A = sup Au : u R n} Au = sup u =1 u R n,u 0 u defines a norm on the linear space of all n n matrices. Proof. See P188.
Theorem (Theorem on Infinity Matrix Norm) If the vector norm is defined by x = max 1 i n x i then its subordinate matrix norm is given by A = max 1 i n n a ij j=1
Proof.
Proof. A = sup u =1 Au = sup {max i } u =1 1 i n
Proof. A = sup u =1 Au = sup {max i } u =1 1 i n = max (Au) i } 1 i n u =1
Proof. A = sup u =1 Au = sup {max i } u =1 1 i n = max (Au) i } 1 i n u =1 = max { sup 1 i n u =1 n a ij u j } j=1
Proof. A = sup u =1 Au = sup {max i } u =1 1 i n = max (Au) i } 1 i n u =1 = max { sup 1 i n u =1 n = max a ij 1 i n j=1 n a ij u j } j=1
é3«~ þp ê k3«ƒap lá Ý ê
é3«~ þp ê k3«ƒap lá Ý ê n A 1 = max a ij Ú ŒŠ 1 j n i=1 A = max 1 i n j=1 n a ij 1Ú ŒŠ A 2 = ρ(a T A) Ì ê
é3«~ þp ê k3«ƒap lá Ý ê n A 1 = max a ij Ú ŒŠ 1 j n i=1 A = max 1 i n j=1 n a ij 1Ú ŒŠ A 2 = ρ(a T A) Ì ê ½Âµ {λ i } A kaš. ρ(a) max λ i L«A 1 i n AŠ AÌŒ». Œ
,ös ( ) 1 2 ~µožýa = 1 ê 2 ê êúeuclid ê" 3 5
~µožýa = )µ,ös ( ) 1 2 1 ê 2 ê êúeuclid ê" 3 5 A 1 = A 2 = A = n n A E = a ij 2 = i=1 j=1
n ~ µ A 1 = max a ij 1 j n i=1
n ~ µ A 1 = max a ij 1 j n i=1 Pf : A 1 = sup Ax 1 x 1 =1 = sup x 1 =1 = sup x 1 =1 sup x 1 =1 sup x 1 =1 (a 11 x 1 + + a 1n x n,, a n1 x 1 + + a nnx n) 1 ( a11 x 1 + + a 1n x n + + a n1 x 1 + + a nnx n ) ( ( a11 + + a n1 ) x 1 + + ( a 1n + + a nn ) x n ) ( max 1 j n 1 i n a ij ) ( x 1 + + x n ) = max a ij 1 j n 1 i n
n ~ µ A 1 = max a ij 1 j n i=1 Pf : A 1 = sup Ax 1 x 1 =1 = sup x 1 =1 = sup x 1 =1 sup x 1 =1 sup x 1 =1 (a 11 x 1 + + a 1n x n,, a n1 x 1 + + a nnx n) 1 ( a11 x 1 + + a 1n x n + + a n1 x 1 + + a nnx n ) ( ( a11 + + a n1 ) x 1 + + ( a 1n + + a nn ) x n ) ( max 1 j n 1 i n a ij ) ( x 1 + + x n ) = max 1 j n 1 i n a ij 1kÚ Œ x k = 1ÚÙ{x i = 0U Ò á.
n ~ µ A 1 = max a ij 1 j n i=1 Pf : A 1 = sup Ax 1 x 1 =1 = sup x 1 =1 = sup x 1 =1 sup x 1 =1 sup x 1 =1 (a 11 x 1 + + a 1n x n,, a n1 x 1 + + a nnx n) 1 ( a11 x 1 + + a 1n x n + + a n1 x 1 + + a nnx n ) ( ( a11 + + a n1 ) x 1 + + ( a 1n + + a nn ) x n ) ( max 1 j n 1 i n a ij ) ( x 1 + + x n ) = max 1 j n 1 i n a ij 1kÚ Œ x k = 1ÚÙ{x i = 0U Ò á. n ÓnŒyµ A = max a ij. 1 i n j=1
y²µé? ÝA, A 2 = ρ(a T A).
y²µé? ÝA, A 2 = ρ(a T A). PfµA T A é Œ½ Ùkλ 1,, λ n škaš ÙéAn ü A þu 1,, u nœ R n IOÄ. x R n, x 2 = 1 x = x 1 u 1 + + x nu n. Kµ Ax 2 2 = (Ax) T Ax = ( A(x 1 u 1 + + x nu n) )T A(x 1 u 1 + + x nu n) = (x 1 u 1 + + x nu n) T (A T Ax 1 u 1 + + A T Ax nu n) = (x 1 u 1 + + x nu n) T (λ 1 x 1 u 1 + + λ nx nu n) = λ 1 x1 2 + + λ nxn 2 max λ i x i 2 = max λ i = ρ(a T A) i i 1 i n
y²µé? ÝA, A 2 = ρ(a T A). PfµA T A é Œ½ Ùkλ 1,, λ n škaš ÙéAn ü A þu 1,, u nœ R n IOÄ. x R n, x 2 = 1 x = x 1 u 1 + + x nu n. Kµ Ax 2 2 = (Ax) T Ax = ( A(x 1 u 1 + + x nu n) )T A(x 1 u 1 + + x nu n) = (x 1 u 1 + + x nu n) T (A T Ax 1 u 1 + + A T Ax nu n) = (x 1 u 1 + + x nu n) T (λ 1 x 1 u 1 + + λ nx nu n) = λ 1 x1 2 + + λ nxn 2 max λ i x i 2 = max λ i = ρ(a T A) i i 1 i n ÒŒ±. A 2 = sup Ax 2 = ρ(a T A). x 2 =1
½nµλ AAŠ? pý ê. K λ A.
½nµλ AAŠ? pý ê. K λ A. Pfµ λéaa þ x KAx = λx. d A = éƒa þ êk Ax A x. Ax sup x R n,x 0 x = sup Ax x R n, x =1 λ x = λ x = A x A x = λ A d½nµ ρ(a) A.
½nµλ AAŠ? pý ê. K λ A. Pfµ λéaa þ x KAx = λx. d A = éƒa þ êk Ax A x. Ax sup x R n,x 0 x = sup Ax x R n, x =1 λ x = λ x = A x A x = λ A d½nµ ρ(a) A. 5µÒkŒU á ~XA é ž Œyρ(A) = A 2.
^ êú¾ý ½ÂµCond p(a) A p A 1 p p ê Âe^ ê.
^ êú¾ý ½ÂµCond p(a) A p A 1 p p ê Âe^ ê. ~X A é p = 2ž Cond 2 (A) = A 2 A 1 2 = λ 1 λ n Ù λ 1, λ n O A Œ AŠ. Ú\^ êvg Ì 5 ^ êœ éa 5 ¾.?Ø) ØCz ) ½5. `
Let x be an approximate solution to the system Ax = b. Theorem (Theorem on Bounds Involving Condition Number) In solving systems of equations Ax = b, the condition number κ(a), the residue vector r = b A x, and the error vector e = x x satisfy the following inequality: 1 r κ(a) b e x κ(a) r b
Proof. The inequality on the right can be written as which is true since e b A A 1 r x e b = A 1 r Ax A 1 r A x
Proof. The inequality on the right can be written as which is true since e b A A 1 r x e b = A 1 r Ax A 1 r A x The inequality on the left can be written as r x A A 1 b e and this follows at once from r x = Ae A 1 b A e A 1 b
) ØCz K µbkøδbž é Ax = b)kõœk º
) ØCz K µbkøδbž é Ax = b)kõœk º bkøδbž ) x + δx. Kµ { Ax = b A(x + δx) = b + δb = { A x b δx A 1 δb = δx A x 1 A δb b = Cond(A) δb b
) ØCz K µbkøδbž é Ax = b)kõœk º bkøδbž ) x + δx. Kµ { Ax = b A(x + δx) = b + δb = { A x b δx A 1 δb = δx A x 1 A δb b = Cond(A) δb b ùpcond(a) ã)ƒéø bƒéø ŒÇ.
~µa = 1 2 1 3 1 4 1 3 1 4 1 5 1 4 1 5 1 6, b = b 1 b 2 b 3 x 1 = 72b 1 240b 2 + 180b 3. Ax = b) µ x 2 = 240b 1 + 900b 2 720b 3 x 3 = 180b 1 720b 2 + 600b 3 Œ±wÑ)ébØ é a = ¾. džµ Cond 1 (A) = Cond (A) = 2015, Cond 2 (A) = 1353.28
) ØCz KµAkØδAž é Ax = b)kõœk º
) ØCz KµAkØδAž é Ax = b)kõœk º Yµ δx A 1 A δa A x 1 A 1 A δa A = Cond(A) δa A 1 Cond(A) δa A Cond(A) δa A