I. Vector and Matrix Manipulations

Start with "mat" All operations on matrices must start with the module name "mat" to distinguish it from other modules (calculus). A typical pattern for an operation on a matrix or vector \(A\) is "mat; f(A); A; matAarray", where "f(A)" represents a valid matrix function or operation, and the pair "A; matAarray" defines the \(m×n\) matrix \(A\) with entries as row vectors \(A=[(a_{11},a_{12},\cdots,a_{1n}),\cdots,(a_{m1},a_{m2}, \cdots,a_{mn})]\). The matrix \(A\) appear flat from the first to last rows, and thus the "matAarray" is required to be entered in exactly the same format as shown here, where each row is within a parenthesis, all rows are within a square bracket (or parenthesis), and all elements are separated by a comma (,).

It is a good practice to check the entries of a matrix after they are entered, and use "mat; A; A; matAarray" to display the entries of the matrix \(A\) for \(f(A)=A\). For example, "mat;A;A;[(1,2,3),(4,5,6),(a,b,c)]" displays the matrix \(A=\begin{bmatrix}1&2&3\\4&5&6\\a&b&c\end{bmatrix}\).

Function Names for Matrix Operations As a typical function name, the name of a matrix function typically has three letters that represent a certain matrix operation. For example, "elf(A)" reduces A to row echelon form, and the function "A*tsp(A)" calculates the product AA^T.

A function can taken two or more matrices and combine with other functions. Thus, the pattern "mat; f(A); A; matAarray" can be extended to two matrices as "mat; f(A, B); A; matAarray, B; matBarray", and to three matrices as "mat; f(A, B, C); A; matAarray, B; matBarray; C; matCarray". For example, "mat; inv(P)*A*P; A; matAarray; P; matParray" calculates the matrix product P^-1AP.

Attention A matrix function, either "f(A)", "f(A, B)", or "f(A, B, C)" and no matter how many matrices are involved, cannot contain any other unknown variables except the variables representing a matrix. For example, "mat;k*A;A;matAarray" does not work because the variable "k" is not defined and is unknown, but "mat;-2*A;A;[(2,3),(5,9)]" works. However, the entries of a matrix can be variables or expressions.

Row Vectors, Column Vectors, and Matrices For a vector v = (a, b, c), "mat; v; v; [(a,b,c)]" displays v as a 1 × 3 row vector, and "mat; v; v; [a, b, c]" or "mat; v; v; [a, b, c]" displays v as a 3 × 1 column vector. The elements of a row or column vector are separated by a comma (,), and they can be either a number, a letter representing a variable, or an expression. A row or column vector is a special form of a matrix. In particular, "mat; A; A; [c]" returns a 1 × 1 matrix with the only entry c.

A matrix consists of multiple rows or columns and can be entered in a similar fashion. It is more convenient to enter a matrix row by row in order. The array "mat; A; A; [(a, b), (c, d)]" displays a 2 × 2 matrix A with the first row "(a, b)" and the second row "(c, d)".

If you are concerned with the columns of a matrix A, enter the matrix row by row in order, and then use the transpose function "tsp(A)". For example, "mat; tsp(A); A; [(a, b, c), (x, y, z), (1, 2, 3)]" shows a 3 × 3 matrix with the first column (a, b, c)', second (x, y, z)', and third (1, 2, 3)'. Or enter the matrix directly by "mat;A;A;[(a,x,1),(b,y,2),(c,z,3)]".

Complex Matrices and Special Symbols The entries of a complex matrix involve complex numbers of the form a + bi. Use "I" for the imaginary number \(i\), for example, "mat;A;A;[1+2*I,2-3*I,-4*I]" displays a column vector (1+2i, 2-3i, -4i)\(^T\). Do not use the letters "I, E, N, O, S, Q" to represent a matrix or use it as an entry in a matrix, because they are reserved for other use.

In addition, if the entries of a matrix involve the irrational numbers \(e,π\), use "E" for \(e\) and "pi" for \( π\).

Q: mat;a;a;[(2,2/3,-5^(1/2),pi,c^2)] A: \(a=\begin{bmatrix}2&\frac{2}{3}&-\sqrt{5}&π&c^2\end{bmatrix}\)

Q: mat;b;b;[5,-3,7] A: \(b=\begin{bmatrix}5\\-3\\7\end{bmatrix}\)

Q: mat;A;A;[(2,3,5,2+3*I),(0,1/2^(1/2),-1,0),(2,-1,3,-5*I)] A: \(A=\begin{bmatrix}2&3&5&2+3i\\0&\frac{\sqrt{2}}{2}&-1&0\\2&-1&3&-5i\end{bmatrix}\)

Echelon Form and Reduced Row Echelon Form The solution of a linear system of equations can be obtained by Gaussian-elimination. The first step for finding a solution is to reduce the matrix of a system to row echelon form, identify pivot and free variables, determine whether the system has a unique solution, no solution, or more than one solution, and then further reduce the matrix to reduced row echelon form for the final solution.

Reduce a matrix A to echelon form by "mat; elf(A); A; matAarray", where "elf(A)" is a resulting row echelon form ('elf'). To obtain the reduced row echelon form ("rref") of A, use "mat; rref(A); A; matAarray".

Examples
(1) In Table 1.1.2, A is a 5 × 4 matrix. The result elf(A) is in row echelon form, which has 4 nonzero rows, so there are four pivots 1, -4, -6, -8 corresponding to the columns \(C_1,C_3,C_4,C_5\), and one non-pivot column \(C_2\).
(2) The result of rref(B) is in reduced row echelon form. It has one zero row and three nonzero rows with a leading 1's (pivot) at each nonzero row. Thus, the three columns are \(C_1,C_3,C_6\), and the four non-pivot columns are \(C_2,C_4,C_5,C_7\).
(3) The matrix C is nonsingular, so it is row equivalent to an identity matrix rref(C). It has three pivots and three nonzero rows.

Note that the echelon form of a matrix is not unique, but any matrix can reduce to a unique reduced row echelon form.

Q: mat;elf(A);A;[(1,2,2,3,3),(2,4,0,4,4),(1,2,3,5,5),(2,4,0,4,7)] A: elf(A) \(=\begin{bmatrix}1&2&2&3&3\\0&0&-4&-2&-1\\0&0&0&-6&-7\\0&0&0&0&-8\end{bmatrix}\)

Q: mat;rref(B);B;[(1,3,-2,0,2,0,0),(2,6,-5,-2,4,-3,-1),(0,0,5,10,0,15,5),(2,6,0,8,4,18,6)] A: rref(B) \(=\begin{bmatrix}1&2&0&4&2&0&0 \\0&0&1&2&0&0&0\\ 0&0&0&0&0&1&\frac{1}{3}\\0&0&0&0&0&0&0\end{bmatrix}\)

Q: mat;rref(C);C;[(1,2,2),(2,5,7),(3,6,8)] A: rref(C) \(=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\)

Augmented Matrices Matrices with the same number of rows can be merged by rows to form a new matrix, and matrices with the same number of columns can be merged by columns. This can be useful in some situations. For example, to solve the linear system \(A{\bf x}={\bf b}\), use "mat; rref(mrg(A, b)); A; matAarray; b; matbarray", where the function "mrg(A, b)" joins matrices A and \({\bf b}\) and forms the augmented matrix M = [A, \({\bf b}\)]. In another case, one can obtain the inverse of a matrix A by performing elementary row operations to the matrix [A, B] by "mat;rref(mrg(A,B));A;matAarray;B;matBarray", where B is an identity matrix. As a result, rref(mrg(A,B)) = [B, A^-1], which means the left side is an identity matrix and the right is the inverse of A.

The array "mat; mrg(A,B,1); A; matAarray; B; matBarray" places matrix B under A and forms a new matrix by columns.

Q: mat;mrg(A,b);A;[(2,3,-2),(1,0,5),(3,1,-2)];b;(a,b,c) A: \(=\begin{bmatrix}2&3&-2&a\\1&0&5&b\\3&1&-2&c\end{bmatrix}\)

Q: mat;mrg(A,B);A;[(1,2,2),(1,3,2),(0,1,1)];B;[(1,0,0),(0,1,0),(0,0,1)] A: \(=\begin{bmatrix}1&2&2&1&0&0\\1&3&2&0&1&0\\0&1&1&0&0&1\end{bmatrix}\)

Q: mat;mrg(A,B,1);A;[(2,1,3),(5,1,0)];B;[(a,b,c),(x,y,z)] A: \(=\begin{bmatrix}2&1&3\\5&1&0\\a&b&c\\x&y&z\end{bmatrix}\)

Rotation Matrices For a given angle θ measured in radian, the arrays "mat;rtm(θ)", "mat;rtm(θ, 2)", and "mat;rtm(θ, 3)" display the three 3 × 3 rotation matrices R_x, R_y, and R_y that correspond to the rotation about the x-axis, y-axis, and z-axis in R³, respectively. They are all orthogonal matrices with norm equal to 1, which can be verified by the fact AA^T = I, for instance, both "mat;A*tsp(A);A;rtm(1)" and "mat;norm(A);A;rtm(1.5)" yield a 3 × 3 identity matrix.

Q: mat;rtm(pi/4) A: Rx \(=\begin{bmatrix}1&0&0\\0&\frac{\sqrt{2}}{2}&\frac{\sqrt{2}}{2}\\0&-\frac{\sqrt{2}}{2}& \frac{\sqrt{2}}{2}\end{bmatrix}\)

Q: mat;A;A;rtm(pi/2,2) A: Ry \(=\begin{bmatrix}0&0&-1\\0&1&0\\1&0&0\end{bmatrix}\)

Q: mat;rtm(pi/3,3) A: Rz\(=\begin{bmatrix}\frac{1}{2}&\frac{\sqrt{3}}{2}&0\\-\frac{\sqrt{3}}{2}&\frac{1}{2}&0\\0&0&1\end{bmatrix}\)

Jacobian Matrices In vector calculus, the Jacobian matrix is the first partial derivatives of a vector-valued function F(x, y, z) in R³ taken with respect to the variables u, v, and w, for instance. The array "mat; jcb(a, b); a; [x(u,v,w), y(u,v,w), z(u,v,w)]; b; [u, v, w]" calculates the Jacobian matrix of F(x, y, z) = x(u, v, w)i + y(u, v, w)j + z(u, v, w)k" in the example, where a = [x(u, v, w), y(u, v, w), z(u, v, w)] is a vector consisting of the component functions of F(u, v, w), and b = (u, v, w) is a vector consisting of the variables u, v, and w. Moreover, the function "det(jcb(a,b))" calculates the determinant of the Jacobian matrix jcb(a,b).

Q: mat;jcb(a,b);a;[u-2*v,3*u+v];b;[u,v] A: \(=\begin{bmatrix}1&-2\\3&1\end{bmatrix}\)	Q: mat;jcb(a,b);a;[2*u-3*v, u*v];b;[u,v] A: \(=\begin{bmatrix}2&-3\\v&u\end{bmatrix}\)	Q: mat;jcb(u,v);u;[x^2*y,5*x+sin(y)];v;[x,y] A: \(=\begin{bmatrix}2xy&x^2\\5&\cos y\end{bmatrix}\)
Q: mat;det(jcb(a,b));a;[u-2*v,3*u+v];b;[u,v] A: \(=7\)	Q: mat;det(jcb(a,b));a;[2*u-3*v, u*v];b;[u,v] A: Jacobian \(=2u+3v\)	Q: mat;det(jcb(u,v));u;[x^2*y,5*x+sin(y)];v;[x,y] A: \(=2xy\cos y-5x^2\)
Q: mat;det(jcb(x,y));x;[r*cos(t),r*sin(t)];y;[r,t] A: Jacobian \(=r\)	Q: mat;det(jcb(u,v));u;[r*cos(t),r*sin(t),z];v;[r,t,z] A: Jacobian \(=r\)	Q: mat;det(jcb(u,v));u;[r*sin(s)*cos(t),r*sin(s)*sin(t),r*cos(s)];v;[r,s,t] A: Jacobian \(=r^2\sin(s)\)
Q: mat;jcb(x,y);x;[r*cos(t),r*sin(t)];y;[r,t] A: Jacobian \(=\begin{bmatrix}\cos t&-r\sin t\\\sin t&r\cos t\end{bmatrix}\)	Q: mat;jcb(u,v);u;[r*cos(t),r*sin(t),z];v;[r,t,z] A: Jacobian \(=\begin{bmatrix}\cos t&-r\sin t&0\\\sin t&r\cos t&0\\0&0&1\end{bmatrix}\)	Q: mat;jcb(u,v);u;[r*sin(s)*cos(t),r*sin(s)*sin(t),r*cos(s)];v;[r,s,t] A: Jacobian \(=\begin{bmatrix}\sin s\cos t&r\cos s\cos t&-r\sin s\sin t\\\sin s\sin t&r\sin t\cos s&r\sin s\cos t\\\cos s&-r\sin s&0\end{bmatrix}\)

Special Square Matrices Diagonal, symmetric and triangular matrices are important special square matrices. The array "mat;diag(1,2,3)" or "mat;A;A; diag(a,b,c)" displays a diagonal matrix with diagonal entries a, b, and c. Use the function "diag" to verify some important properties for diagonal matrices. For example, the inverse, sum, difference and product of diagonal matrices are still diagonal, and "mat;A^(-2);A;diag(a,b,c)" returns the inverse of \(A^2\), which is \(A^{-2}\) = diag\((\frac{1}{a^2},\frac{1}{b^2},\frac{1}{c^2})\).

The function "eye(m)" returns an m × m identity matrix, "ones(m)" returns an m × m matrix with the entries all ones, and "zeros(m)" returns an m × m matrix with the entries all zeros (the zero matrix). For example, mat;eye(3) || mat;ones(4) || mat;zeros(5) || mat;diag(2,3,4,5) ||.

Random m × n Matrices The function "rdm(m)" returns an m-square random matrix with entries between -10 and 10 by default. Use "rdm(m, n, type, seed, lower, upper)", where "n, type, lower, upper, seed" are optional. There are six types indicated from 0 to 5: 0 - random matrix, 1 - diagonal matrix, 2 - upper triangular, 3 - lower triangular, 4 - symmetric matrix, 5 - Jordan cell (block). To generate an m × n random matrix with seed and integer entries, use "mat;rdm(m,n,0,seed,lower,upper)". Thus, "mat;rdm(2)" returns a 2 × 2 random matrix, and "mat;rdm(3, 2, 0, 234, -4, 4)" returns a 3 × 2 matrix with integer entries within the interval [-4, 4] and a seed = 234.

The function "mat;rdm(3,3,1)" returns a 2 × 2 diagonal matrix, "mat;rdm(4,4,2)" returns a 4 × 4 upper triangular matrix, "mat;rdm(3,3,3)" returns a 3 × 3 lower triangular matrix, "mat;rdm(5,5,4)" returns a 5 × 5 symmetric matrix, "mat;rdm(4,4,5,1.5)" returns a 4 × 4 Jordan cell with eigenvalue equal to 1.5, and "mat;rdm(3,3,2,100,1,9)" returns a 3 × 3 random upper triangular matrix with seed = 100 and entries randomly selected from 1 to 9.

The function "rdm" can be used to verify some properties for symmetric and triangular matrices. The inverse, sum, difference and product of real symmetric matrices are still symmetric. In particular, the diagonal entries of the product of triangular matrices are the product of their corresponding diagonal entries, and the diagonal entries of the inverse of a triangular matrix are the reciprocal of its diagonal entries. For example, "mat;inv(rdm(3,3,2,100))", "mat;A*B-B*A;A;rdm(3,3,2,100);B;rdm(3,3,2,200)", and "mat;A^2-2*A*B+B^2;A;rdm(3,3,2,100);B;rdm(3,3,2,200)" show that the three resulting matrices are upper triangular.

Q: mat;diag(1,2,3) A: \(=\begin{bmatrix}1&0&0\\0&2&0\\0&0&3\end{bmatrix}\)	Q: mat;ones(3) A: \(=\begin{bmatrix}1&1&1\\1&1&1\\1&1&1\end{bmatrix}\)	Q: mat;eye(3) A: \(=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\)
Q: mat;A;A;rdm(2,3,0,345,0,9) A: \(=\begin{bmatrix}4&0&2\\6&1&3\end{bmatrix}\)	Q: mat;rdm(3,3,4,20) A: \(=\begin{bmatrix}-6&-2&10\\-2&-7&0\\10&0&8\end{bmatrix}\)	Q: mat;rdm(3,3,2,30) A: \(=\begin{bmatrix}-7&-1&9\\0&9&10\\0&0&-9\end{bmatrix}\)

II. Matrix and Vector Operations

Addition, Subtraction, Multiplication, and Power The operators +, -, * also apply to matrix addition, subtraction, scalar and matrix multiplication. For example, "A+3*B" gets the sum of A + 3B, and "A*B" and "B*A" finds the product AB and BA, respectively, provided they are defined. For the n power of a square matrix A, enter "A**n" or "A^n".

Transpose, Conjugate, and Conjugate Transpose: The function "tsp(A)" returns A^T, the transpose of A. The function "tsp" (transpose) can be used for all matrices and combined with other matrix operations. For complex matrices, "tsp(A)" returns the conjugate transpose of A, and "cjg(A)" just returns the conjugate of A.

Matrix Inverse A nonsingular square matrix has an inverse. Both functions "inv(A)" and "A^(-1)" return the inverse of A. If A is not square, "inv(A)" returns the Moore-Penrose pseudoinverse, but "A^(-1)" is not defined. The function "inv" (inverse) or negative integer power of a matrix can combine with other matrix operations. For example, "A*B**(-1)" gives the result of AB^-1, and "3*A*B**3+4*C**(-2)" calculates 3AB³ + 4C^-2.

Matrix Polynomials A matrix polynomial is a polynomial with square matrices as variables. Given an ordinary scalar-valued polynomial \(f(t)=a_0+a_1t+a_2t^2+\cdots+a_nt^n\), the matrix polynomial is \(f(A)=a_0I+a_1A+a_2A^2+\cdots+a_nA^n\). For example, "mat;A^2-3*A+2*eye(3);A;[(2,3,5),(7,1,3),(2,6,0)]" calculates the matrix polynomial \(f(A)=A^2-3A+2I_3\).

Examples
(1) There are several ways to obtain the dot product ax + by + cz of two real vectors, for instance, u = (a, b, c) and v = (x, y, z). Enter each vector as a column by "mat;tsp(u)*v;u;[a,b,c];v;[x,y,z]", enter each vector as a row by "mat;u*tsp(v);u;[(a,b,c)];v;[(x,y,z)]", or enter one as a row and the other as a column by "mat;u*v;u;[(a,b,c)];v;[x,y,z]".

(2) For complex matrices in Table 1.2 for example, "mat;tsp(v)*u;u;[1+I,2,3];v;[4-3*I,5*I,6+I]" returns the dot product ⟨u, u⟩, which is 19 - 6\(i\), because tsp(v) gives the conjugate transpose of the 3 × 1 complex matrix v (column vector). The same result can be obtained by "mat;cjg(v)*u;u;[1+I,2,3];v;[(4-3*I,5*I,6+I)]", where v is a row vector. Note that "mat;v*u;u;[1+I,2,3];v;[(4-3*I,5*I,6+I)]", "mat;u*v;u;[(1+I,2,3)];v;[4-3*I,5*I,6+I]", "mat;dot(u,v);u;[1+I,2,3];v;[4-3*I,5*I,6+I]" return a different result 25 + 14\(i\) (not shown in the table).

(3) The last example in the table shows that the matrix M is a root of the polynomial \(f(t)=(t-2)(t^2-4t+6)\), where B represents the 3 × 3 identity matrix I. The polynomial \(f(t)\) is the characteristic polynomial of M, which can be obtained by "mat;chp(M);M;[(2,-1,0),(1,2,1),(0,-1,2)]". Keep in mind do not use the letter "I" to represent a matrix because it is reserved for the complex number \(i\).

Q: mat;A*B;A;[(1,2,3),(4,5,6)];B;[3,-1,2] A: \(=\begin{bmatrix}7\\19\end{bmatrix}\)	Q: mat;F*tsp(G);F;[(2,-1,3),(-4,1,2),(0,-1,5)];G;[(1,0,1),(2,3,1)] A: \(=\begin{bmatrix}5&4\\-2&-3\\5&2\end{bmatrix}\)	Q: mat;A*B;A;[1,2,3];B;[(4,5,6)] A: \(=\begin{bmatrix}4&5&6\\8&10&12\\12&15&18\end{bmatrix}\)
Q: mat;tsp(v)*u;u;[1+I,2,3];v;[4-3*I,5*I,6+I] A: \(=19-6i\)	Q: mat;a*A*b;a;[(x,y,z)];A;[(2,-1,0),(1,2,1),(0,-1,2)];b;[x,y,z] A: \(=\begin{bmatrix}2x^2+2y^2+2z^2\end{bmatrix}\)	Q: mat;P*D*inv(P);P;[(1,1,-1),(1,1,0),(1,0,3)];D;diag(1,2,2) A: \(=\begin{bmatrix}-1&3&-1\\-3&5&-1\\-3&3&1\end{bmatrix}\)
Q: mat;A*tsp(A);A;[(1,2,3),(4,5,6)] A: \(=\begin{bmatrix}14&32\\32&77\end{bmatrix}\)	Q: mat;tsp(A)*A;A;[(1,2,3),(4,5,6)] A: \(=\begin{bmatrix}17&22&27\\22&29&36\\27&36&45\end{bmatrix}\)	Q: mat;tsp(A*B)-tsp(B)*tsp(A);A;[1,2,3];B;[(4,5,6)] A: \(=\begin{bmatrix}0&0&0\\0&0&0\\0&0&0\end{bmatrix}\)
Q: mat;C^2-5*C+6*B;C;[(1,-2),(1,4)];B;eye(2) A: \(=\begin{bmatrix}0&0\\0&0\end{bmatrix}\)	Q: mat;2*A+3*A**2;A;[(2,-1,0),(1,2,1),(0,-1,2)] A: \(=\begin{bmatrix}13&-14&-3\\14&10&14\\-3&-14&13\end{bmatrix}\)	Q: mat;(M-2*B)*(M^2-4*M+6*B);M;[(2,-1,0),(1,2,1),(0,-1,2)];B;eye(3) A: \(=\begin{bmatrix}0&0&0\\0&0&0\\0&0&0\end{bmatrix}\)

back to top

Linear System, Linear Combination, Linear Independence, and Coordinate Vector

Solve a Linear System Many problems in linear algebra reduce to finding the solution of a system of linear equations. A linear system of \(m\) equations in \(n\) unknowns has the standard form
\(a_{11}x_1+a_{12}x_2+\cdots+a_{1n}x_n=b_1\)
\(a_{21}x_1+a_{22}x_2+\cdots+a_{2n}x_n=b_2\)
\(\cdots\cdots\cdots\)
\(a_{m1}x_1+a_{m2}x_2+\cdots+a_{mn}x_n=b_m\).

The above system is usually written as a matrix equation \(A{\bf x}={\bf b}\), where \(A\) is the \(m×n\) coefficient matrix, \({\bf x}\) is the vector of \(n\) unknowns, and \({\bf b}\) is the vector of \(m\) constants, that is, \(A=\begin{bmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\a_{21}&a_{22}&\cdots&a_{2n}\\\cdots&\cdots&\cdots&\cdots\\a_{m1} &a_{m2}&\cdots&a_{mn}\end{bmatrix},{\bf x }=\begin{bmatrix}x_1\\x_2\\\cdots\\x_n\end{bmatrix},{\bf b}=\begin{bmatrix}b_1\\b_2\\\cdots\\b_m\end{bmatrix}\).

It is also helpful to write the system as \(x_1\begin{bmatrix}a_{11}\\a_{21}\\\cdots\\a_{m1}\end{bmatrix}+x_2\begin{bmatrix}a_{12}\\a_{22}\\\cdots\\a_{m2}\end{bmatrix}+\cdots+x_n\begin{bmatrix}a_{1n}\\a_{2n}\\\cdots\\a_{mn}\end{bmatrix}=\begin{bmatrix}b_1\\b_2\\\cdots\\b_m\end{bmatrix}\), in which the vector \({\bf b}\) is viewed as a linear combination of the column vectors of \(A\), and the unknowns of \({\bf x}\) are the coefficients. In this sense, the question of expressing a given vector \({\bf b}∈K^n\) as a linear combination of vectors \({\bf u}_1,{\bf u}_2,\cdots,{\bf u}_n\) in \(K^n\) is equivalent to solving the linear system \(A{\bf x}={\bf b}\), where \(A=[{\bf u}_1,{\bf u}_2,\cdots,{\bf u}_n]\) whose columns are the given vectors.

Homogeneous Linear System For an m × n homogeneous linear system Ax = 0, the constant vector b = 0. Use "mat;rref(A);A;matAarray" to determine the solution of the system, where A represents the m × n coefficients matrix of the system with m equations in n unknowns. The zero vector b is not needed for finding the solution. A homogeneous linear system always has the zero solution, so we only need to determine if the system has a nonzero solution. If rref(A) has a free variable, it has infinitely many nonzero solutions. If rref(A) has no free variables, it has the zero or trivial solution. A simple rule is that if a homogeneous system has more unknowns than equations (or n > m), it has a nonzero solution.

Example Solve the system of linear equations
\(x_1+3x_2-2x_3+2x_5=0\\2x_1+6x_2-5x_3-2x_4+4x_5=0\\x_1+3x_2+x_3+6x_4+2x_5=0\\x_1+3x_2+4x_4+2x_5=0\)
Form the coefficient matrix A, and reduce A to reduced row echelon form by "mat;rref(A);A;[(1,3,-2,0,2),(2,6,-5,-2,4),(1,3,1,6,2),(1,3,0,4,2)]". Observe that rref(A)=\(\begin{bmatrix}1&3&0&4&2\\0&0&1&2&0\\0&0&0&0&0\\0&0&0&0&0\end{bmatrix}\). There are two pivot variables \(x_1,x_3\) and three free variables \(x_2,x_4,x_5\). Thus, the general solution is \(x_1=-3x_2-4x_4-2x_5,x_3=-2x_4\), or \(x_1=-3r-4s-2t,x_2=r,x_3=-2s,x_4=s,x_5=t\) in parametric form for some arbitrary numbers \(r,s,t\). Using the general solution, we can determine the dimension and a basis of the solution space.

Nonhomogeneous Linear System To solve a nonhomogeneous system of linear equations Ax = b, write the augmented matrix M = [A, b], where "A" represents the coefficients matrix and "b" the column of constants, and use "rref(M)" to determine the solution. The same results can be obtained by "rref(mrg(A,b))", where M = mrg(A,b) gives the augmented matrix M = [A, b].

A nonhomogeneous system may have no solution, a unique solution, or infinitely many solutions. If rref(M) has a row "0 = 1", the system has no solution; if rref(M) has a free variable, it has infinitely many solution; if rref(M) has no free variables, it has a unique solution.

Examples
(1) In the last example (in the second row) of Table 2, rref(mrg(C,b)) has no free variables, so the last column corresponds to the unique solution, which is \(x_1=2,x_2=\frac{1}{2},x_3=\frac{-2}{3}\). The same pattern is observed for the three examples in the first low of Table 2.

(2) "0 = 1" appears in matrix rref(mrg(M,b)) in the first example (in the second row) of Table 2, so the system has no solution.

(3) The matrix rref(mrg(B,b)) in the second example (in the second row) of Table 2 has a free variable that corresponds to a non-pivot column (column 3). Thus, the general solution is \(x_1=\frac{2-x_3}{3},x_2=\frac{1+10x_3}{6}\), or the parametric form of the general solution is \(x_1=\frac{2-t}{3},x_2=\frac{1+10t}{6},x_3=t\) for an arbitrary number \(t\).

Q: mat;rref(mrg(A,b));A;[(2,1),(3,-2)];b;[5,4] A: Solution \(=[2,1]\)	Q: mat;rref(mrg(A,b));A;[(1,2,2),(2,5,7),(3,6,8)];b;[1,0,-2] A: Solution \(=\begin{bmatrix}-5&5.5&2.5\end{bmatrix}\)	Q: mat;rref(mrg(A,b));A;[(2,-1,3,5),(1,0,-1,2),(1,1,-2,3),(0,1,1,1)];b;[1,2,5,-3] A: \(=\begin{bmatrix}-13&-6&-3&6\end{bmatrix}\)
Q: mat;rref(mrg(M,b));M;[(1,3,-2,0),(2,6,-5,-2),(0,0,5,10)];b;[2,0,1] A: (no solution) rrref \(=\begin{bmatrix}1&3&0&4&0\\0&0&1&2&0\\0&0&0&0&1\end{bmatrix}\)	Q: mat;rref(mrg(B,b));B;[(1,2,-3),(3,0,1),(2,-2,4)];b;[1,2,1] A: (infinitely many solutions) rref \(=\begin{bmatrix}1&0&\frac{1}{3}&\frac{2}{3}\\0&1&\frac{-5}{3}&\frac{1}{6}\\0&0&0&0\end{bmatrix}\)	Q:  mat;rref(mrg(C,b));C;[(1,2,0),(2,-4,3),(1,2,3)];b;[3,0,1] A: (unique solution) rref \(=\begin{bmatrix}1&0&0&2\\0&1&0&\frac{1}{2}\\0&0&1&\frac{-2}{3}\end{bmatrix}\)

Example Solve the system of linear equations
\(x_1-2x_2+x_3-2x_4+3x_5=2\\x_3+5x_4-4x_5=1\\2x_1-4x_2+3x_3+x_4+2x_5=5\\3x_1-6x_2+4x_3-x_4+5x_5=7\\x_1-2x_1+3x_3+8x_4-5x_4=4\)
Use "mat;rref(mrg(A,b));A;[(1,-2,1,-2,3),(0,0,1,5,-4),(2,-4,3,1,2),(3,-6,4,-1,5),(1,-2,3,8,-5)];b;[2,1,5,7,4]" to determine the solution, where A is the coefficient matrix, b is the column vector of the constants. Observe that rref(mrg(A,b)) \(=\begin{bmatrix}1&-2&0&-7&7&1\\0&0&1&5&-4&1\\0&0&0&0&0&0 \\0&0&0&0&0&0\\0&0&0&0&0&0\end{bmatrix}\). There are two pivot variables \(x_1,x_3\) and three free variables \(x_2,x_4,x_5\). Thus, the general solution is \(x_1=1+2x_2+7x_4-7x_5,x_3=1-5x_4+4x_5\), or the general solution in parametric form is \(x_1=1+2r+7s-7t,x_2=r,x_3=1-5s+4t,x_4=s,x_5=t\) for some arbitrary numbers \(r,s,t\).

Linear Combination Given a vector \({\bf u}\) and a set of vectors \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_r\}\) in \(K^n\), to determine if \({\bf u}\) can be expressed as a linear combination of the vectors in \(S\) is equivalent to solve the system \(A{\bf x}={\bf u}\), where the columns of A are the given vectors in \(S\). Reduce the matrix [A, u] to reduced row echelon form by "rref(mrg(A,u))". If the system has a nonzero solution, \({\bf u}\) can be expressed as a linear combination of the vectors in \(S\).

Examples
(1) Express \({\bf u}\) = (2, 3, 5) as a linear combination of \({\bf v}_1\) = (2, 2, 3), \({\bf v}_2\) = (1, 0, 4), \({\bf v}_3\) = (3, 5, -1). Obtain the result rref(mrg(tsp(A),u)) from "mat;rref(mrg(tsp(A),u));A;[(2,2,3),(1,0,4),(3,5,-1)];u;[2,3,5]", and thus \({\bf u}=24{\bf v}_1-19{\bf v}_2-9{\bf v}_3\), where the coefficients of the linear combination are in the last column of rref(mrg(tsp(A),u)).

(2) Express the vector (4, 7, -1) as a linear combination of the vectors (1, 1, 2), (3, -1, 4), (2, -2, 2). Obtain the result from "mat;rref(mrg(tsp(A),u));A;[(1,1,2),(3,-1,4),(2,-2,2)];u;[4,7,-1]". The last row of the reduced row echelon form has "0 = 1", so the system has no solution, and the vector cannot be expressed as a linear combination of the three vectors.

(3) Express the polynomial \(p=5t^2-3t+2\) as a linear combination of \(p_1=2t^2-t+1,p_2=t^2-3t,p_3=-t+3\). Form a matrix A whose columns are the coefficients of each polynomial \(p_1,p_2,p_3\), form a column vector \({\bf u}=(5,-3,2)^T\) with the coefficients of \(p\), and solve the linear system \(A{\bf x}={\bf u}\) by "mat;rref(mrg(tsp(A),u));A;[(2,-1,1),(1,-3,0),(0,-1,3)];u;[5,-3,2]". Hence \(p=\frac{19}{8}p_1+\frac{1}{4}p_2- \frac{1}{8}p_3\), where the coefficients correspond to the last column of rref(mrg(tsp(A),u)).

(4) Express the matrix \(M=\begin{bmatrix}2&-3\\-8&4\end{bmatrix}\) as a linear combination of \(A=\begin{bmatrix}1&-1\\0&1\end{bmatrix},B=\begin{bmatrix}2&1\\2&3\end{bmatrix}, C=\begin{bmatrix}3&2\\-4&7\end{bmatrix}\). To find the coefficients \(a,b,c\) of \(M=aA+bB+cC\), form a matrix G whose columns are the entries in \(A,B,C,M\) (M at the last column), and row reduce G to reduced echelon form by "mat;rref(mrg(tsp(F),M));F;[(1,-1,0,1),(2,1,2,3),(3,2,-4,7)];M;[2,-3,-8,4]". Note that G = mrg(tsp(F),M), the coefficients \(a=3,b=-2,c=1\) are in the last column of rref(G). Thus, \(M=3A-2B+C\).

(5) Express the vector \({\bf u}\) = (3,2,3) as a linear combination of the vectors in S = { (2,-2,3),(1,1,0),(-3,3,4) } by "mat;rref(mrg(tsp(A),u));A;[(2,-2,3),(1,1,0),(-3,3,4)];u;[3,2,3]". The last column of rref(mrg(tsp(A),u)) corresponds to the coefficients of the linear combination. Note that each pair of vectors in S is orthogonal that can be verified by "mat;A*tsp(A);A;[(2,-2,3),(1,1,0),(-3,3,4)]", where the product AA^T is diagonal. Thus, \({\bf u}=c_1{\bf v}_1+c_2{\bf v}_2+c_3{\bf v}_3\), where \({\bf v}_i\) are the vectors in S, and \(c_i=\frac{⟨{\bf u},{\bf v}_i⟩}{⟨{\bf v}_i,{\bf v}_i⟩}\) for \(i\) = 1, 2, 3. Observe that \(⟨{\bf v}_i,{\bf v}_i⟩\) are the diagonal entries of AA^T, and \(c_i\) is the elements of A\({\bf u}\) by "mat;A*u;A;[(2,-2,3),(1,1,0),(-3,3,4)];u;[3,2,3]". It follows that \({\bf u}=\frac{11}{17}{\bf v}_1+\frac{5}{2}{\bf v}_2+\frac{9}{34}{\bf v}_3\).

Linear Independence Given a set of vectors \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_r\}\) in \(K^n\), if \(r > n\), the set is linearly dependent. To determine if \(S\) is linearly independent and find a subset of linearly independent vectors, form a matrix A = [\({\bf v}_1,{\bf v}_2,\cdots,{\bf v}_r\)] whose columns are the given vectors in \(S\), and solve the homogeneous system \(x_1{\bf v}_1+x_2{\bf v}_2+\cdots+x_n{\bf v}_r={\bf 0}\) or \(A{\bf x}={\bf 0}\) for \({\bf x}\) by "elf(A)", where elf(A) reduces A to echelon form, and the column vector \({\bf x}=(x_1,x_2,\cdots,x_r)^T\) is the coefficients of the linear combination. If the system has only the zero solution, the set is linearly independent. Otherwise, the set is linearly dependent, the columns of elf(A) with a pivot form a linearly independent subset, and any non-pivot column can be expressed as a linear combination of its previous columns.

In other words, if the echelon matrix elf(A) has a zero row, the set is linearly dependent, because the system has a nonzero solution. If elf(A) does not have zero rows, the set is linearly independent, because the system only has the zero solution.

Examples
(1) Determine if the vectors \({\bf v}_1\) = (2,-2,3,1), \({\bf v}_2\) = (1,-4,4,-3), \({\bf v}_3\) = (1,2,-1,4), \({\bf v}_4\) = (3,0,2,5) are linearly independent. Form a matrix \(A=[{\bf v}_1,{\bf v}_2,{\bf v}_3,{\bf v}_4]\) whose columns are the vectors and row reduce \(A\) to echelon form by "mat;elf(tsp(A));A;[(2,-2,3,1),(1,-4,4,-3), (1,2,-1,4),(3,0,2,5)]". Observe that the echelon matrix elf(tsp(A)) has two zero rows, and thus the four vectors are linearly dependent. In particularly, the columns \(C_1,C_2\) are linearly independent because they correspond to a pivot, and the columns \(C_3,C_4\) can be expressed as a linear combination of \(C_1,C_2\). Accordingly, the vectors \({\bf v}_1,{\bf v}_2\) form a linearly independent subset, and \({\bf v}_3=a{\bf v}_1+b{\bf }_2,{\bf v}_4=c{\bf v}_1+d{\bf v}_2\) for some constants \(a,b,c,d\).

(2) Determine if the matrices \(A=\begin{bmatrix}2&2&1\\3&0&4\end{bmatrix},B=\begin{bmatrix}3&5&4\\-2&1&3\end{bmatrix},C=\begin{bmatrix}1&7&8\\-18&3&-7\end{bmatrix}\) are linearly independent. Form a matrix M whose rows are the rows of each matrix, and reduce M to echelon form by " mat;elf(M);M;[(2,2,1,3,0,4),(3,5,4,-2,1,3), (1,7,8,-18,3,-7)]". Observe that the echelon matrix elf(M) has a zero row, and thus the three matrices \(A,B,C\) are linearly dependent.

(3) Determine if the polynomials \(p_1=t^4+2t^3+t^2-3t+1,p_2=t^4+t^3-3t^2+2t-4,p_3=t^4+4t^3+9t^2-13t+11\) are linearly independent. Form a matrix whose rows are the coefficients of the polynomials and row reduce A to echelon form by "mat;elf(A);A;[(1,2,1,-3,1),(1,1,-3,2,-4),(1,4,9,-13,11)]". Observe that the echelon matrix has a zero row. Thus, the polynomials are linearly dependent.

(4) Suppose the vectors \({\bf u},{\bf v},{\bf w}\) are linearly independent. Determine if the vectors \(S=\{{\bf u}+2{\bf v}-{\bf w},2{\bf u}-{\bf v}+3{\bf w},{\bf v}-4{\bf w}\}\) are linearly independent. The question is to determine if the linear system \(x({\bf u}+2{\bf v}-{\bf w})+y(2{\bf u}-{\bf v}+3{\bf w})+z({\bf v}-4{\bf w})={\bf 0}\) has a nonzero solution. Form a matrix whose columns are the coefficients of each vector and row reduce A to echelon form by "mat;elf(tsp(A));A;[(1,2,-1),(2,-1,3),(0,1,-4))]". Observe that elf(A) has three nonzero rows and thus the vectors in \(S\) are linearly independent.

Determine a Vector in a Span Given a spanning set \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\), to determine if a vector \({\bf u}\) is in the span\((S)\) is equivalent to expressing \({\bf u}\) as a linear combination of the vectors in \(S\), which is equivalent to solving the linear system \(A{\bf x}={\bf u}\), where \(A=[{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n]\) or the columns of \(A\) are the vectors in \(S\). If the system has a nonzero solution, the vector \({\bf u}\) is in the span\((S)\). If the system has no solution, the vector is not in the span. Accordingly, if "0 = 1" appears in the reduced row echelon matrix rref(mrg(A,u)), the vector \({\bf u}\) is not in the span. Otherwise, \({\bf u}\) is in span\((S)\), or it can be expressed as a linear combination of the vectors in \(S\).

Find Coordinate Vectors Given a basis \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) of a vector space V, to obtain the coordinate vector of a vector u ∈ V relative to the basis \(S\) is equivalent to finding the solution of the linear system \(A{\bf x}={\bf u}\) by "rref(mrg(A,u))", where the columns of \(A\) are the vectors in \(S\). The coordinate vector \([{\bf u}]_S\) is in the last column of rref(mrg(A,u)).

Examples
(1) Determine under what condition the vector u = (a, b, c) in \(R^3\) is in the span of the vectors (3, 5, 2), (1, -1, 6), and (3, 13, -14). The question is equivalent to expressing u as a linear combinations of the given vectors, or equivalent to solving the system \(A{\bf x}={\bf u}\). Form a matrix A whose columns are the given vectors, and reduce the matrix [A, u] to reduced row echelon form by "mat;elf(mrg(tsp(A),u));A;[(3,5,2),(1,-1,6),(3,13,-14)]; u;[a,b,c]". Observe that the system has a nonzero solution if and only if the last equation 96a - 48b - 24c = 0 holds. Thus, the vector u = (a, b, c) is in the span of the three vectors if and only if 4a - 2b - c = 0.

(2) Let u = (2,3,-1) and S = { (1,1,0),(0,1,1),(2,1,-3) } be a basis of \(R^3\). To find the coordinates of u relative to the basis S, form a matrix A whose columns are the vectors in S, use "mat;rref(mrg(A,u));A;[(1,0,2),(0,1,1),(0,1,3)];u;[2,3,-1]", and obtain that the coordinates [u]_S = (6,5,-2)^T correspond to the last column of rref(mrg(A,u)).

(3) Determine the coordinate vector of A = \(\begin{bmatrix}2&3\\-1&4\end{bmatrix}\) relative to the basis \(\begin{bmatrix}1&1\\0&1\end{bmatrix},\begin{bmatrix}1&-1\\1&0\end{bmatrix},\begin{bmatrix}1&1\\-1&0\end{bmatrix},\begin{bmatrix}0&1\\1&0\end{bmatrix}\). Form a matrix M whose columns are the entries of the basis matrices and the last column corresponds to the entries of A. Row reduce M to reduced row echelon form by "mat;rref(mrg(tsp(B),A));B;[(1,1,0,1),(1,-1,1,0),(1,1,-1,0),(0,1,1,0)];A;[2,3,-1,4]". Observe that the coordinates of A relative to the basis S in the last column of rref(mrg(tsp(B),A)) is [A]_S = (4,-1,-1,-1).

(4) Find the coordinate vector of \(p=3t^3-2t^2+t-4\) relative to the basis S = \(\{(t+1)^3,(t+1)^2,(t+1),1\}\) in the vector space \(P_3(t)\). Expand the basis polynomials and form a matrix M whose columns are the coefficients of the basis polynomials and the last column is the coefficients of \(p\). Row reduce M to reduced row echelon form by "mat;rref(mrg(tsp(A),b));A;[(1,3,3,1),(0,1,2,1),(0,0,1,1),(0,0,0,1)];b;[3,-2,1,4]". Thus, the coordinate vector relative to the basis S is [p]_S = (3,-11,14,-2), which is in the last column of rref(mrg(tsp(A),b)).

(5) Determine the coordinate vector of \(p=at^2+bt+c\) relative to the basis S = \(\{(t-1)^2,t-1,1\}\). Form a matrix M whose columns are the coefficients of the given polynomials and the coefficients of \(p\) are in the last column. Thus, the coordinate vector is [p]_S = (a, 2a + b, a + b + c) by "mat;rref(mrg(tsp(A),u));A;[(1,-2,1),(0,1,-1),(0,0,1)];u;[a,b,c]".

back to top

Vector Space, Subspace, Dimension, and Basis

Four Fundamental Subspaces Let \(A=[a_{ij}]\) be an \(m×n\) matrix with rows \(R_1,R_2,\cdots,R_m\) and columns \(C_1,C_2,\cdots,C_n\) over a field \(K\). The rows of \(A\) are viewed as vectors in \(K^n\), they span the row space of \(A\), denoted as \(R(A)\), and thus \(R(A)\) = span\((R_1,R_2,\cdots,R_m)\). Analogously, the columns of \(A\) are viewed as vectors in \(K^m\), they span the column space of \(A\), denoted as \(R(A)\), and \(C(A)\) = span\((C_1,C_2,\cdots,C_n)\). Hence \(C(A)=R(A^T)\) and \(R(A)=C(A^T)\). Note that the rows and columns of a matrix may span a different subspace with a different dimension.

Two matrices are row equivalent if one can be obtained from the other by a sequence of elementary row operations. Thus, row equivalent matrices have the same row space, and the row space of a matrix is the same as the space spanned by the nonzero rows from its echelon matrix. Two matrices have the same row space if and only if they have the same nonzero rows in their reduced row echelon forms.

To determine if two sets of vectors span the same space, form two separate matrices whose rows are the given sets of vectors, reduce them to reduced row echelon form, and check if the nonzero rows of the reduced echelon matrices are the same.

The function "rsp(A)" returns a list of vectors (row) that span the row space of A, and "rsp(tsp(A))" returns a list of vectors (row) that span the column space of A, where "rsp" (row space) finds the spanning set of the row space of A.

Example Given two sets \(S_1\) = { (1,-3,-2,5),(2,3,7,-1),(3,0,5,4) }, \(S_2\) = { (1,6,9,-6),(4,-3,3,9),(0,9,11,-11) } in \(R^4\), determine if span\((S_1)\) = span\((S_2)\). Form a matrix A whose rows are the given vectors in \(S_1\), and reduce A to reduced row echelon form. Form a matrix B whose rows are the given vectors in \(S_2\), and reduce B to reduced row echelon form. Check the results with "mat;rref(A);A;[(1,-3,-2,5),(2,3,7,-1),(3,0,5,4)]" and "mat;rref(B);B;[(1,6,9,-6),(4,-3,3,9),(0,9,11,-11)]". Observed that rref(A) and rref(B) have the same nonzero rows, and thus span\((S_1)\) = span\((S_2)\).

To determine the dimension and a basis of span\((S_1)\) or span\((S_2)\), use "mat;elf(A);A;[(1,-3,-2,5),(2,3,7,-1),(3,0,5,4)]" or "mat;rsp(A);A;[(1,-3,-2,5),(2,3,7,-1),(3,0,5,4)]", and the results of elf(A) and rsp(A) shows the dimension is 2 and a basis is (1, -3, -2, 5) and (0, 9, 11, -11).

Example Given a matrix A = [(2,1,3,1,2,-3), (4,2,4,1,-1,1),(2,1,-1,-1,-8,11),(4,2,6,2,3,1)], row reduce A to echelon form by elf(A). Observe that the echelon matrix elf(A) has three non zero rows and the pivot columns are \(C_1,C_3,C_5\). Thus, the three pivot columns \(C_1,C_3,C_5\) are linearly independent and form a column space of \(A\). The columns \(C_2,C_4,C_6\) are the linear combinations of their previous columns, where \(C_2=\frac{1}{2}C_1,C_4=aC_1+bC_3,C_6=rC_1+sC_3+tC_5\) for some constant \(a,b,r,s,t\). The ranks of sumbmatrices are rank\([C_1,C_2]\) = 1, rank\([C_1,C_2,C_3,C_4]\) = 2, and rank\([C_1,C_2,C_3,C_4,C_5,C_6]\) = rank(A) = 3.

Null Space An \(m×n\) matrix \(A\) can be viewed as a linear mapping, so the image of \(A\) is the column space of \(A\), and the kernel of \(A\) consists of all vectors \({\bf x}\) such that \(A{\bf x}={\bf 0}\). This implies the kernel (or null space) of \(A\) is the solution space of the homogeneous system \(A{\bf x}={\bf 0}\). The function "nsp(A)" returns a set of row vectors (basis) that span the null space of \(A\), and "nsp(tsp(A))" returns a list of row vectors (basis) that span the null space of \(A^T\) (left null space).

Examples
(1) Find the dimension and a basis of the solution space of the homogeneous linear system of equations
\(2x_1+x_2+2x_3+2x_4-3x_5=0\\2x_1+x_2-x_3-3x_4+2x_5=0\\2x_1+x_2+8x_3+12x_4-13x_5=0\)
Form a matrix A whose rows are the coefficients and reduce A to reduced row echelon form rref(A) by "mat;rref(A);A;[(2,1,2,2,-3),(2,1,-1,-3,2),(2,1,8,12,-13)]". Observe that rref(A) has two pivot variables \(x_1,x_3\) and three free variables \(x_2,x_4,x_5\). Thus the dimension of the solution space is 3, and the general solution is \(x_1=\frac{-x_2}{2}+\frac{2x_4}{3}+\frac{-x_5}{6},x_3=\frac{-5x_4}{3}+\frac{5x_5}{3}\). Setting \((x_2,x_4,x_5)=(1,0,0), (x_1,x_2,x_3,x_4,x_5)\)
\(=(\frac{-1}{2},1,0,0,0)\); setting \((x_2,x_4,x_5)=(0,1,0),(x_1,x_2,x_3,x_4,x_5)=(\frac{2}{3},0,\frac{-5}{3},1,0)\); setting \((x_2,x_4,x_5)=(0,0,1),(x_1,x_2,x_3,x_4,x_5)=(\frac{-1}{6},0,\frac{5}{3},0,1)\). The three particular solutions form a basis of the solution space of the homogeneous system.

The above process can be further simplified by "mat;nsp(A);A;[(2,1,2,2,-3),(2,1,-1,-3,2),(2,1,8,12,-13)]", which returns a basis (row vectors) of the solution space with dimension 3.

(2) Find the dimension and a basis of the solution space W of the linear system x + y - z = 0, 2x - 3y + 4z = 0, 3x - 2y + z = 0. Form a matrix A whose rows are the coefficients of the system and reduce A to row echelon form by "mat;elf(A);A;[(1,1,-1),(2,-3,4),(3,-2,1)]". Observe that the echelon matrix elf(A) has three nonzero rows and no free variables. Thus, the system has a zero (trivial) solution, the dimension of W is 0, and W has no basis. Verify the result by "mat;nsp(A);A;[(1,1,-1),(2,-3,4),(3,-2,1)]".

(3) Find a homogeneous system whose solution space is spanned by the vectors in S = { (2,1,3,2,1),(1,1,2,3,-2),(2,1,4,-3,2) } in \(R^5\). The question is equivalent to seek the orthogonal complement to the solution space, or equivalent to seek the set of vectors \({\bf x}=(x_1,x_2,x_3,x_4,x_5)\) orthogonal to each vector in S, or equivalent to solving the system \(A{\bf x}={\bf 0}\). The solution space of the system is the null space of \(A\) obtained by "mat;nsp(A);A;[(2,1,3,2,1),(1,1,2,3,-2),(2,1,4,-3,2)]", which returns the vectors { (-4,-9,5,1,0),(-2,6,-1,0,1) }. Thus, a homogeneous system can be \(4x_1+9x_2-5x_3-x_4=0,2x_2-6x_2+x_3-x_5=0\) (note that it is not unique).

Dimension and Basis of a Vector Space Given a set of vectors \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) in \(K^n\), to determine the dimension and a basis of the subspace \(W\) spanned by the set \(S\), form a matrix A whose rows are the given vectors in \(S\), reduce A to echelon form elf(A), and determine the dimension and a basis of \(W\) from the echelon matrix elf(A). The number of nonzero rows in echelon form is the dimension of \(W\), and the nonzero rows in elf(A) form a basis of \(W\).

Examples
(1) Determine whether the vectors { (1,1,2,3),(3,-7,4,-1),(3,-2,5,4),(2,-8,2,-4) } in \(R^4\) form a basis of \(R^4\). Form a matrix A whose rows are the given vectors and reduce A to row echelon form by "mat;elf(A);A;[(1,1,2,3),(3,-7,4,-1),(3,-2,5,4),(2,-8,2,-4)]". Observe that the result of elf(A) has two zero rows, and thus the vectors do not form a basis of \(R^4\). They span a subspace of dimension 2.

(2) Extend the set { (1,1,2,3),(3,-2,5,4) } of vectors to form a basis of \(R^4\). Form a matrix A whose rows are the given vectors and reduce A to echelon form by "mat;elf(A);A;[(1,1,2,3), (2,2,5,4)]". Observe that the echelon matrix elf(A) has two nonzero rows, and the two pivot columns are \(C_1,C_3\). Form a basis of \(R^4\) by replacing the two zero rows by two nonzero rows with pivots corresponding to columns \(C_2,C_4\), for example, adding the two vectors (0,1,0,0) and (0,0,0,1) to the set forms a basis of \(R^4\). The four vectors form a matrix in echelon form (check it by "mat;elf(M);M;[(1,1,2,3), (2,2,5,4),(0,1,0,0),(0,0,0,1)]"), and hence they are linearly independent.

(3) Suppose the subspaces \(U=\) span{ (1,2,2,3,-1),(1,-4,-8,1,7),(2,1,-1,5,2) } and \(W=\) span{ (0,3,5,1,-4),(1,-1,-3,2,3),(3,3,1,8,1) } of \(R^5\). Determine the dimension and a basis of the \(U+W\) and \(U∩W\). Given that \(U+W\) is a subspace spanned by the six vectors, form a matrix A whose rows are the spanning set of \(U\), form a matrix B whose rows are the spanning set of \(W\), merge A and B in a matrix of six rows, and reduce the matrix in echelon form by "mat;elf(mrg(A,B,1));A;[(1,2,2,3,-1),(1,-4,-8,1,7),(2,1,-1,5,2)];B;[(0,3,5,1,-4),(1,-1,-3,2,3),(3,3,1,8,1)]". Observe that there are two nonzero rows in the echelon matrix. Thus the dimension of \(U+W\) is 2, and a basis is { (1,2,2,3,-1), (0,-6,-10,-2,8) }.

A vector \({\bf v}\) is \(U∩W\) if and only if \({\bf v}\) can be expressed as a linear combination of the spanning sets in \(U\) and \(W\), or equivalently we seek a set of vectors orthogonal to both sets of vectors in \(U\) and \(W\). Obtain the null space of A by "mat;nsp(A);A;[(1,2,2,3,-1),(1,-4,-8,1,7), (2,1,-1,5,2)]" and the null space of B by "mat;nsp(B);B;[(0,3,5,1,-4),(1,-1,-3,2,3),(3,3,1,8,1)]", then form the matrix M whose rows are the six columns of nsp(A) and nsp(B), and get the null space of M by "mat;nsp(M);M;[(4,-5,3,0,0),(-7,-1,0,3,0),(-5,4,0,0,3),(4,-5,3,0,0),(-7,-1,0,3,0),(-5,4,0,0,3)]". Observe that the dimension of \(U∩W\) is 2, and a basis of \(U∩W\) is { (4,5,3,11,0),(1,-7,-13,0,11) }.

(4) Determine the dimension and a basis of the subspace W of \(P(t)\) spanned by \(\{t^3+3t^2-2t+4,2t^3-t+3t-2,7t^3+7t^2+8\}\). Form a matrix A whose rows are the coefficients of the polynomials, and row reduce A to echelon form by "mat;elf(A);A;[(1,3,-2,4),(2,-1,3,-2),(7,7,0,8)]". Observe that the echelon matrix elf(A) has one zero row. Thus, the dimension of W is 2, and a basis of W is \(\{t^3+3t^2-2t+4,-7t^2+7t+10\}\).

(5) Find the dimension a basis of the subspace W spanned by \(\begin{bmatrix}3&5\\2&4\end{bmatrix},\begin{bmatrix}1&2\\2&3\end{bmatrix}, \begin{bmatrix}5&7\\-2&0\end{bmatrix},\begin{bmatrix}3&4\\-2&-1\end{bmatrix}\) in a vector space of 2 × 2 matrices. Form a matrix A whose rows are the entries of these matrices, and row reduce A to echelon form by "mat;elf(A);A;[(3,5,2,4),(1,2,2,3),(5,7,-2,0),(3,4,-2,-1)]". The echelon matrix elf(A) has two zeros rows, the dimension of W is 2, and a basis of W is \(\begin{bmatrix}3&5\\2&4\end{bmatrix},\begin{bmatrix}0&1\\4&5\end{bmatrix}\).

(6) Given a vector \({\bf u}\) = (1,1,-3) in \(R^3\), find the dimension and a basis for the subspace \({\bf u}^⊥\). Let \({\bf v}=(x,y,z)∈{\bf u}^⊥\). Then \(⟨{\bf u},{\bf v}⟩\) = 0, or equivalently \(x+y-3z=0\). The equation has two free variables, and thus the subspace \({\bf u}^⊥\) has dimension 2. A basis of \({\bf u}^⊥\) is (1,-1,0) and (3,0,1) obtained by setting \((y,z)\) as (1,0) and (0,1), or use "mat;nsp(a);a;[(1,1,-3)]".

Q: mat;rsp(A);A;[(2,4,2),(1,2,1),(0,1,0)] A: \(=\begin{bmatrix}2&4&2\\0&1&0\end{bmatrix}\)	Q: mat;rsp(tsp(A));A;[(2,-1,2),(2,2,4),(2,-1,2)] A: \(=\begin{bmatrix}2&2&2\\0&6&0\end{bmatrix}\)	Q: mat;nsp(tsp(A));A;[(2,3,1),(3,-4,5),(5,-1,6)] A: \(=\begin{bmatrix}-1&-1&1\end{bmatrix}\)
Q: mat;rank(rsp(B));B;[(1,2,1,2),(0,3,1,-2),(-1,1,0,-4),(3,0,1,-1)] A: \(=3\)	Q: mat;rank(nsp(B));B;[(1,2,1,2),(0,3,1,-2),(-1,1,0,-4),(3,0,1,-1)] A: \(=1\)	Q: mat;rank(rsp(tsp(B)));B;[(1,2,1,2),(0,3,1,-2),(-1,1,0,-4),(3,0,1,-1)] A: \(=3\)

back to top

Linear Mappings, Matrix Representations, and Change of Basis

Linear Mapping Let \(U,V\) be vector spaces over the same field \(K\). A mapping \({\bf T}:U→V\) is linear if for any vectors \({\bf u},{\bf w}\) in \(U\), the two conditions \({\bf T}({\bf u}+{\bf w})={\bf T}({\bf u})+{\bf T}({\bf w})\) and \((k{\bf T})({\bf u})=k{\bf T}({\bf u})\) are satisfied for any scalar \(k∈K\). In general, a linear mapping has the basis property \({\bf T}(a_1{\bf u}_1+\cdots+a_n{\bf u}_n)=a_1{\bf T}({\bf u}_1)+\cdots+a_n{\bf T}({\bf u}_n)\) for \(a_i∈K\) and \({\bf u}_i∈U\).

Example
(1) Let \({\bf T}:R^3→R^3\) be a linear mapping and \({\bf T}\)(1,1,2) = (2,3,5), \({\bf T}\)(3,2,-2) = (1,0,3), and \({\bf T}\)(2,0,-5) = (3,-5,7). Note that the vectors in S = { (1,1,2), (3,2,-2) and (2,0,-5) } form a basis of \(R^3\) and \({\bf T}\) is determined by the values of the basis vectors. To find the formula for \({\bf T}\), let \({\bf v}=(a,b,c)∈R^3\) and form a matrix A whose columns are the vectors in S. Then the coordinates \([{\bf v}]_S=(x_1,x_2,x_3)\) of \({\bf v}\) relative to S is the solution to the system \(A{\bf x}={\bf v}\), which is last column of the matrix rref(mrg(tsp(A),v)) from "mat;rref(mrg(tsp(A),v));A;[(1,1,2),(3,2,-2),(2,0,-5)];v;[a,b,c]". Since \({\bf v}=x_1{\bf u}_1+x_2{\bf u}_2+x_3{\bf u}_3\) and \({\bf u}_1,{\bf u}_2,{\bf u}_3\) represent the basis vectors in S, \({\bf T}({\bf v})\) = \(x_1{\bf T}({\bf u}_1)+x_2{\bf T}({\bf u}_2)+x_3{\bf T}({\bf u}_3)\) = B\([{\bf v}]_S\), where the columns of B are the image of the basis vectors or B = \([{\bf T}({\bf u}_1),{\bf T}({\bf u}_2),{\bf T}({\bf u}_3)]\). Observe that the formula is \({\bf T}(a,b,c)=(\frac{33a-37b+9c}{7},b+c,11a-12b+3c)\) by "mat;tsp(B)*u;B;[(2,3,5),(1,0,3),(3,-5,7)];u; [(10*a-11*b+4*c)/7,(-5*a+9*b-2*c)/7,(6*a-8*b+c)/7]".

(2) Let \({\bf T}:R^2→R^2\) be a linear mapping defined by \({\bf T}(x,y)=(3x-4,2x+1)\). Find \({\bf T}^{-1}\). Form a matrix A whose rows are the coefficients of the components of \({\bf T}(x,y)\) and row reduce A to reduced echelon form by "mat;rref(A);A;[(3,-4),(2,1)]". Observe that A is nonsingular, and \({\bf T}^{-1}\) can be obtained by solving A\({\bf x}={\bf u}\) for \({\bf x}\) by "mat;rref(mrg(A,u));A;[(3,-4),(2,1)];u;[a,b]". Thus, \({\bf T}^{-1}(a,b)=(\frac{a+4b}{11},\frac{-2a+3b}{11})\).

(3) Let \({\bf T}:R^2→R^3\) be a linear mapping defined by \({\bf T}(x,y)=(x+y,2x-3y,4x-y)\). Solve the system \({\bf T}(x,y)\) = (0,0,0) by "mat;rref(A);A;[(1,1),(2,-3),(4,-1)]". Observe that the only solution is \(x=0,y=0\), thus \({\bf T}\) is nonsingular. However, \({\bf T}\) is not invertible because \(R^2\) and \(R^3\) have different dimension and \({\bf T}^{-1}\) does not exist.

(4) Let \({\bf T}:R^3→R^3\) be a linear mapping defined by \({\bf T}(x,y,z)=(x+y-z,x-2y+z,x-y)\). Find \({\bf T}^2,{\bf T}^{-1},{\bf T}^{-2}\). Form a matrix A whose rows are the coefficients of the components of \({\bf T}\). The matrix rref(A) is an identity matrix by "mat;rref(A);A;[(1,1,-1),(1,-2,1),(1,-1,0)]", which implies \({\bf T}\) is nonsingular and invertible. Get A\(^2\) by "mat;A^2;A;[(1,1,-1),(1,-2,1),(1,-1,0)]" and \({\bf T}(x,y,z)=(x,4y-3z,3y-2z)\). Obtain A\(^{-1}\) by "mat;inv(A);A;[(1,1,-1),(1,-2,1),(1,-1,0)]" and \({\bf T}^{-1}(x,y,z)=(x+y-z,x+y-2z,x+2y-3z)\). Get A\(^{-2}\) by "mat;A^(-2);A;[(1,1,-1),(1,-2,1),(1,-1,0)]" and \({\bf T}^{-2}(x,y,z)=(x,-2y+3z,-3y+4z)\).

(5) Find a matrix mapping A in \(R^2\) so A maps (2,-1) and (3,2) to (1,3) and (2,4), respectively. Note that S = { (2,-1),(3,2) } form a basis of \(R^2\). Form a matrix B whose columns are the basis vectors in S. Then AB = C, where the columns of C are the images of the basis vectors on A. Thus, A = CB\(^{-1}\) = \(\frac{1}{7}\begin{bmatrix}4&1\\10&-1\end{bmatrix}\) by "mat;C*inv(B);C;[(1,2),(3,4)];B;[(2,3),(-1,2)]".

Matrix as Linear Mapping An \(m×n\) matrix \(A\) determines a linear mapping \({\bf T}:K^n→K^m\) by \({\bf T}({\bf u})=A{\bf u}\). Thus, a matrix can be view as a linear mapping.

Matrix Representation of Linear Operator Given a linear operator \({\bf T}:V→V\) and a basis \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) of \(V\), the image of each basis vector \({\bf T}({\bf v}_i)\) can be expressed uniquely as a linear combination of the vectors in \(S\). The matrix representation \([{\bf T}]_S\) of \({\bf T}\) relative to the basis \(S\) consists of the columns of such coordinate vectors of \([{\bf T}({\bf v}_i)]_S\), and it can be obtained by solving each linear system \(A{\bf x}={\bf T}({\bf v}_i)\) by the function rref(mrg(A,b_i)), where the columns of \(A\) are the vectors in \(S\) and \({\bf b}_i={\bf T}({\bf v}_i)\) is the image of each basis vector \({\bf v}_i\) in \(S\). The solution \({\bf x}=[{\bf T}({\bf v}_i)]_S\) of each such system gives the coordinates of \({\bf T}({\bf v}_i)\) relative to the basis \(S\), which lies in the last column of rref(mrg(A,b_i)), and \([{\bf T}]_S=[[{\bf T}({\bf v}_1)]_S,\cdots,[{\bf T}({\bf v}_n)]_S]\) whose columns are the solutions \([{\bf T}({\bf v}_i)]_S\).

An more efficient method for finding the matrix representing \({\bf T}\) is to solve the linear systems at the same time by forming the matrix \(B=[{\bf T}({\bf v}_1),{\bf T}({\bf v}_2),\cdots,{\bf T}({\bf v}_n)]\) whose columns are the images of \({\bf T}\) on the basis vectors, and the right half of the matrix rref(mrg(A,B)) is \([{\bf T}]_S\), the matrix representation of \({\bf T}\) relative to basis \(S\).

Example
(1) Let \({\bf T}:R^3→R^3\) be a linear operator defined by \({\bf T}\)(x, y, z) = (2x - 3y + z, x + 2y, 4x - y + 2z). Find the matrix representation of \({\bf T}\) relative to the basis \(S\) = { \({\bf u}_1\) = (1,1,0), \({\bf u}_2\) = (2,1,-2), \({\bf u}_3\) = (1,0,3) }. Form a matrix A whose rows are the coefficients of the components of \({\bf T}\)(x,y,z). Note that A is the matrix representation of \({\bf T}\) relative to the standard basis, and \({\bf T}({\bf u})=A{\bf u}\) for \(A=\begin{bmatrix}2&-3&1\\1&2&0\\4&-1&2\end{bmatrix}\). Thus, the images are \({\bf T}({\bf u}_1)=A{\bf u}_1\) = (-1,3,3) by "mat;A*u;A;[(2,-3,1),(1,2,0),(4,-1,2)];u;[1,1,0]", \({\bf T}({\bf u}_2)=A{\bf u}_2\) = (-1,4,3) by "mat;A*u;A;[(2,-3,1),(1,2,0),(4,-1,2)];u;[2,1,-2]", and \({\bf T}({\bf u}_3)=A{\bf u}_3\) = (5,1,10) by "mat;A*u;A;[(2,-3,1),(1,2,0),(4,-1,2)];u;[1,0,3]". Form a matrix \(B=[{\bf u}_1,{\bf u}_2,{\bf u}_3]\) whose columns are the basis vectors, and form a matrix \(M=[{\bf T}({\bf u}_1),{\bf T}({\bf u}_2),{\bf T}({\bf u}_3)]\) whose columns are the images. Then "mat;rref(mrg(B,M));B;[(1,2,1),(1,1,0),(0,-2,3)];M;[(-1,-1,5),(3,4,1),(3,3,10)]" obtains rref(mrg(B,M)) \(=\begin{bmatrix}1&0&0&6&\frac{38}{5}&\frac{3}{5} \\0&1&0&-3&\frac{-18}{5}&\frac{2}{5}\\0&0&1&-1&\frac{-7}{5}&\frac{18}{5}\end{bmatrix}\). The matrix representation of \({\bf T}\) relative the basis \(S\) is \([{\bf T}]_S=\begin{bmatrix}6&\frac{38}{5}&\frac{3}{5}\\-3& \frac{-18}{5}&\frac{2}{5}\\-1&\frac{-7}{5}&\frac{18}{5}\end{bmatrix}\), the right half of rref(mrg(B,M)).

The above procedure can be further simplified. Instead of finding the image of each basis vector one at a time, observe that the images are the product \(AB=[A{\bf u}_1,A{\bf u}_2,\cdots,A{\bf u}_n]=[{\bf T}({\bf u}_1),{\bf T}({\bf u}_2),{\bf T}({\bf u}_3)]\). Thus, "mat;rref(mrg(B,A*B));B;[(1,2,1),(1,1,0),(0,-2,3)]; A;[(2,-3,1),(1,2,0),(4,-1,2)]" returns the same result \([{\bf T}]_S\) as shown above.

Note that \(A\) is the matrix representation of \({\bf T}\) relative to the standard basis of \(R^3\). Verify that the right half of rref(mrg(B,A*B)) from "mat;rref(mrg(B,A*B));B;[(1,0,0),(0,1,0),(0,0,1)]; A;[(2,-3,1),(1,2,0),(4,-1,2)]" is exactly \(A\). If the basis \(S_1\) is { (1,1,0), (2,1,5),(1,0,2) }, then the matrix representation of \({\bf T}\) relative to \(S_1\) can be obtained by "mat;rref(mrg(B,A*B));B;[(1,1,0), (2,1,5),(1,0,2)]; A;[(2,-3,1),(1,2,0),(4,-1,2)]".

(2) Let \(G:R^2→R^2\) be a linear operator defined by \(G(x,y)=(x-2y,3x+4y)\) and S = { (1,1), (2,1) } be a basis of \(R^2\). The matrix representation of \(G\) relative to S is \([G]_S=\begin{bmatrix}15&20\\-8&-10\end{bmatrix}\) by "mat;rref(mrg(B,A*B));A;[(1,-2),(3,4)];B;[(1,2),(1,1)]". To verify \([G({\bf v})]_S=[G]_S[{\bf v}_S]\), find the coordinate vector \([{\bf v}]_S\) of \({\bf v}=(a,b)\) in \(R^2\) relative to S. It is the solution of the system B\({\bf x}={\bf v}\) by "mat;rref(mrg(B,v));B;[(1,2),(1,1)];v;[a,b]", where the columns of B are the given basis vectors. Observe that \([{\bf v}]_S=(-a+2b,a-b)\). Thus, \(G({\bf v})=(a-2b,3a+4b),[G({\bf v})]_S=(5a+10b,-2a-6b)\) and \([G]_S[{\bf v}]_S=(5a+10b,-2a-6b)\). If \({\bf v}\) = (3,2), \([{\bf v}]_S\) = (1,1), \(G({\bf v})\) = (-1,17), \([G({\bf v})]_S\) = (35,-18) and \([G]_S[{\bf v}]_S\) = (35,-18).

Matrix Representation of Matrix Mapping Any \(m×n\) matrix \(A\) determines a linear mapping \({\bf T}:K^n→K^m\) by \({\bf T}({\bf u})=A{\bf u}\). If a matrix mapping \(A\) is given relative to the standard basis, the matrix representation \([A]_S\) of \(A\) relative to another basis \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) can be obtained by rref(mrg(B,A*B)), where the columns of B are the given basis vectors. We emphasize that the function rref(mrg(B,A*B)) gets the coordinate vectors (columns) of the images AB relative to \(S\) by solving all linear systems \(B{\bf x}=A{\bf v}_i\) at the same time.

Example If \(A\) = [(2,5,4),(0,1,2),(3,0,-4)] is considered as a linear operator on \(R^3\), and \(S\) = { (1,4,2),(2,-3,7),(3,1,0) } is a basis of \(R^3\), the matrix representation \([A]_S\) of \(A\) relative to \(S\) can be obtained by "mat;rref(B,A*B);A;[(2,5,4), (0,1,2),(3,0,-4)];B;[(1,4,2),(2,-3,7),(3,1,0)]".

Kernel and Image of Linear or Matrix Mapping Since an \(m×n\) matrix \(A\) can be viewed as a linear mapping, the image of the mapping is the column space of \(A\) obtained by "rsp(tsp(A))", and the kernel of the mapping is the null space of \(A\) obtained by "nsp(A)".

Examples
(1) Let F: R\(^4→\) R\(^3\) be a linear mapping defined by F(x,y,z,w) = (x+2y-z+3w, 2x+y-4z+2w, 3x-3y+z-w). Form a matrix \(A\) whose rows are the coefficients of the components of F(x,y,z,w). Then the image of F is the column space of A, which can be obtained by "mat;elf(tsp(A));A;[(1,2,-1,3), (2,1,-4,2),(1,-3,1,-1)]". The echelon matrix elf(tsp(A)) has three nonzero rows (1,2,1), (0,-3,-5), (0,0,-16) that form a basis of the image of F, and thus the dimension of the image of F is 3 (or rank(F) = 3). The kernel of F is the null space of \(A\), or the solution space of the system \(A{\bf x}={\bf 0}\), and a basis of the null space can be obtained by "mat;nsp(A);A;[(1,2,-1,3), (2,1,-4,2),(1,-3,1,-1)]". Observe that a basis of the null space nsp(A) is (3,2,1,-2), the dimension the kernel of F is 1, and dim(ker(F)) + dim(img(F)) = dim(R\(^4\)).

(2) Given a matrix mapping \(A:R^4→R^3\) for \(A\) = [(2,2,-3,4),(1,1,-2,3),(2,2,-1,0)], the image of \(A\) is the column space of \(A\). The echelon matrix elf(tsp(A)) by "mat;elf(tsp(A));A;[(2,2,-3,4),(1,1,-2,3),(2,2,-1,0)]" has two nonzero rows (2,1,2) and (0,-1,4) that form a basis of the image of \(A\). The kernel of \(A\) is the null space of \(A\), and a basis of the null space is (1,-1,0,0) and (1,0,2,1) by "mat;nsp(A);A;[(2,2,-3,4), (1,1,-2,3),(2,2,-1,0)]". Thus, dim(R\(^4\)) = dim(img(\(A\))) + dim(ker(\(A\))) = 4.

(3) Let F: R\(^3→\) R\(^3\) be a linear mapping defined by F(x, y, z) = (x - y + z, y - z, x + 2y - z) and S is a set consisting of the points on the unit sphere. Find F(S) and F\(^{-1}\)(S). If (a,b,c) ∈ F(S), there is a point (x,y,z) ∈ \(R^3\) such that F(x, y, z) = (a, b, c) and x² + y² + z² = 1. Solve the linear system for (x, y, z) = (a + b, -a - 2b + c,-a - 3b + c) by "mat;rref(A);A;[(1,-1,1,a),(0,1,-1,b),(1,2,-1,c)]" and obtain F(S) that satisfies 3a² + 14b² + 2c² +12ab - 4ac - 10bc = 1.

Let (x,y,z) ∈ F\(^{-1}\)(S) be the preimage of F(S). Then F(x,y,z) ∈ S and (x-y+z)² + (y-z)² + (x+2y-z)² = 1 or 2x² + 6y² + 3z² + 2xy - 8yz = 1.

(4) Find a linear mapping F: R\(^3→\) R\(^3\) whose image is spanned by (2, 3, -2), (1, -4, 5), (0, 1, 1). Form a matrix B whose columns are the given vectors. The column space of B is the image of F. Thus, F(x, y, z) = B(x, y, z)^T = (2x + y, 3x - 4y + z, -2x + 5y + z) by "mat;tsp(B)*v;B;[(2,3,-2),(1,-4,5),(0,1,1)];v;[x,y,z]".

(5) Find a linear mapping F: R\(^4→\) R\(^3\) whose kernel is spanned by (2,3,2,1) and (1,-2,2,3). Form a matrix A whose rows are the given vectors and obtain the null space of A by "mat;nsp(A);A;[(2,3,2,1),(1,-2,2,3)]". Thus, F(x, y, z, w) = (10x - 2y - 7z, 11x - 5y - 7w).

(6) Let F: R\(^3→\) R\(^3\) be a linear operator defined by F(x, y, z) = (2x - y + z, x - 2y - z, x + y - 2z) and f(t) = t² - 2t + 3. Find f(F). Form a matrix whose rows are the coefficients of the components of F(x,y,z). Then f(F) = F(A), which can be obtained by "mat;A^2-2*A+3*eye(3);A;[(2,-1,2),(1,-2,-1),(1,1,-2)]", where "eye(3)" gives an 3 × 3 identity matrix. Thus, f(F)(x, y, z) = (4x + 4y - 3z, -3x + 9y + 8z, -z - 7y + 12z).

Rank and Nullity of Linear or Matrix Mapping If A is the matrix representation of a linear mapping \({\bf T}:V→U\), the rank of the mapping is the dimension of the image of \({\bf T}\) (or the column space of A), and the nullity of the mapping is the dimension of kernel of \({\bf T}\) (or the null space of A). Reduces the matrix A to echelon form by "elf(A)", the number of pivots or nonzero rows is the rank of \({\bf T}\), the number of non-pivot columns is the nullity of \({\bf T}\), and the sum of the rank and nullity is the dimension of \(V\). The rank of A can be obtained directly by the function rank(A), which is the same as the number of nonzero rows in elf(A).

Change of Basis Matrix Given two different bases \(S_1=\{{\bf u}_1,\cdots,{\bf u}_n\}\) (old) and \(S_2=\{{\bf v}_1,\cdots,{\bf v}_n\}\) (new) of a vector space \(V,P=[[{\bf v}_1]_{S_1},\cdots,[{\bf v}_n]_{S_1}]\) is the change of basis matrix from \(S_1\) to \(S_2\) or from old to new, where the columns \([{\bf v}_i]_{S_1}\) are the coordinate vectors of new basis vector \({\bf v}_i\) relative to the old basis \(S_1\) for each \(i\). Note that \([{\bf v}_i]_{S_1}\) is the solution of the linear system \(A{\bf x}={\bf v}_i\) and can be obtained by the array "rref(mrg(A,b)", where the columns of matrix \(A\) consists of the old basis vectors in \(S_1\) and \({\bf b }={\bf v}_i\) is a new basis vector (column). If the old basis \(S_1\) is the standard basis of \(V\), then \(P=[{\bf v}_1,\cdots,{\bf v}_n]\), whose columns are the new basis vectors in \(S_2\). Thus, for any vector \({\bf w}∈V,P[{\bf w}]_{S_2}=[{\bf w}]_{S_1}\) and \([{\bf w}]_{S_2}=P^{-1}[{\bf w}]_{S_1}\).

The change of matrix from the bases \(S_2\) to \(S_1\) (or from new to old) is \(Q=P^{-1}=[[{\bf u}_1]_{S_2},\cdots,[{\bf u}_n]_{S_2}]\). Follow the above method to obtain the matrix \(Q\), or find the inverse of \(P\) by "inv(P)".

From computing perspective, instead of solving each system \(A{\bf x}={\bf v}_i\) for the change of basis matrix \(P\), find a formula for the coordinate vector \([{\bf w}]_{S_1}\) for any vector \({\bf w}=(w_1,\cdots,w_n)∈V\) relative to the old basis \(S_1\) by "rref(mrg(A,w))". Form a matrix \(F\) whose rows are the coefficients of the components of \([{\bf w}]_{S_1}\), and then calculate \(P=FB\), where the columns of \(B\) are the new basis vectors in \(S_2\).

Effects of Change of Basis Matrix on Linear Operator If \({\bf T}\) is any linear operator on \(V\), the matrix representation of \({\bf T}\) relative to bases \(S_1,S_2\) have the relation \([{\bf T}]_{S_2}=P^{-1}[{\bf T}]_{S_1}P\). In particular, if the old basis \(S_1\) is the standard basis of \(R^n\), then the rows of the matrix \([{\bf T}]_{S_1}\) representing \({\bf T}\) relative to the standard basis are the coefficients of the components of \({\bf T}\), and the columns of the change of basis matrix \(P\) are the vectors in \(S_2\).

Examples
(1) Suppose S = { (2,5), (1,3) } and S' = { (2,4), (3,5) } are two bases of \(R^2\), the change of basis matrix P can be obtained by two steps. First find the formula for coordinate vector for a vector \({\bf u}\) = (a, b) relative to S by "mat;rref(mrg(tsp(A),u));A;[(2,5),(1,3)];u;[a,b]". Observe that \([{\bf u}]_S\) = (3a - b, -5a + 2b). Form a matrix F = \(\begin{bmatrix}3&-1\\-5&2\end{bmatrix}\) whose rows are the coefficients of \([{\bf u}]_S\). Then calculate the change of basis matrix P = FB = \(\begin{bmatrix}2&4\\-2&-5\end{bmatrix}\) from S to S' by "mat;F*B;F;[(3,-1),(-5,2)];B;[(2,3),(4,5)]", where the columns of B are the vectors in S'. The change of basis matrix from S' to S is Q = P\(^{-1}\) = \(\begin{bmatrix}2.5&2\\-1&-1\end{bmatrix}\) by "mat;inv(P);P;[(2,4),(-2,-5)]".

To verify \([{\bf v}]_{S'}=P^{-1}[{\bf v}]_S\), let \({\bf v}\) = (6,8). Then the coordinate vector of \({\bf v}\) relative to S is \([{\bf v}]_S\) = (10, -14) by "mat;rref(M);M;[(2,1,6),(5,3,8)]", \([{\bf v}]_{S'}\) = (-3, 4) by "mat;rref(M);M;[(2,3,6),(4,5,8)]", and \(P^{-1}[{\bf v}]_S\) = (-3,4) by "mat;P*v;P;[(2.5,2),(-1,-1)];v;[10,-14]".

(2) Let S = { (1,1,1),(2,1,2),(1,-1,2) } be a basis of \(R^3\) and \({\bf T}\) be a linear operator on \(R^3\) defined by \({\bf T}\)(x, y, z) = (2x - 3y + z, y - 2z, 3x - 5y). Find the matrix representation of \({\bf T}\) relative to the basis S. Form a matrix \(A\) whose rows are the coefficients of the components of \({\bf T}\)(x,y,z). Then \(A\) is the matrix representation \({\bf T}\) relative to the standard basis E. The the columns of the change of basis matrix \(P\) from E to S are the vectors in S. Thus, the matrix representation of \({\bf T}\) relative to S is \([{\bf T}]_S=P^{-1}AP\),which can be obtained by "mat;inv(P)*A*P;P;[(1,2,1),(1,1,-1),(1,2,2)];A;[(2,-3,1),(0,1,-2),(3,-5,0)]".

Verify the above result by another method using "mat;rref(mrg(B,A*B));[(1,2,1),(1,1,-1),(1,2,2)]; A;[(2,-3,1),(0,1,-2),(3,-5,0)]".

(3) Given that S = { (1,2,3), (0,1,2), (1,1,0) } is a basis of \(R^3\), the change of basis matrix from the standard basis E to S is the matrix P whose columns are the given basis vectors in S, and the change of basis matrix from S to E is P\(^{-1}\) obtained by "mat;inv(P);P;[(1,2,3),(0,1,2),(1,1,0)]". For any vector \({\bf v}\) = (a, b, c) in \(R^3\), the coordinate vector of \({\bf v}\) relative to the basis S is P\(^{-1}{\bf v}\) that can be obtained by "mat;inv(P)*v;P;[(1,2,3),(0,1,2),(1,1,0)];v;[a,b,c]", or by "mat;rref(mrg(P,v));P;[(1,2,3),(0,1,2),(1,1,0)];v;[a,b,c]". The results from the two methods for \([\bf v]_S\) are the same.

(4) Let \(F:R^3→R^2\) be a linear mapping defined by \(F\)(x, y, z) = (3x - 2y + z, x - 2y + 3z) and S = { (1,1,2), (2,1,2), (2,0,1) } and S' = { (1,1), (0,1) } be the bases for \(R^3\) and \(R^2\), respectively. Find \([F]_{S,S'}\), the matrix representation of F in the bases S and S'. Form a matrix A whose rows are the coefficients of \(F\)(x, y, z), form a matrix B whose columns are the given basis vectors in S, and form a matrix C whose columns are the vectors in S'. Then \([F]_{S,S'}\) are the right half of the matrix rref(mrg(B,A*C)) by "mat;rref(mrg(B,A*C));A;[(3,-2,1),(1,-2,3)];B;[(1,0),(1,1)];C;[(1,2,2),(1,1,0),(2,2,1)]".

To verify \([F]_{S,S'}[{\bf v}]_S=[F({\bf v})]_{S'}\), let \({\bf v}\) = (a,b,c) in \(R^3\). The coordinate vector \([{\bf v}]_S\) is by "mat;rref(mrg(C,v));C;[(1,2,2),(1,1,0),(2,2,1)];v;[a,b,c]", \([F({\bf v})]_{S'}\) is by "mat;rref(mrg(B,A*v));A;[(3,-2,1),(1,-2,3)];B;[(1,0),(1,1)];v;[a,b,c]", and \([F]_{S,S'}[{\bf v}]_S\) is by "mat;F*v;F;[(3,6,7),(2,0,-2)];v;[-a-2*b+2*c,a+3*b-2*c,-2*b+c]". The results match.

(5) Given a linear mapping \(F\)(x, y) = (2x + 3y, x - y) on \(R^2\) and the standard basis B and a basis S = { (2, 1), (1,2) } of \(R^2\). Form a matrix A whose rows are the coefficients of the components of \(F\)(x, y). The matrix representation of F in the bases B and S can be obtained by "mat;rref(mrg(M,A*B));A;[(2,3),(1,-1)]; M;[(2,1),(1,2)];B;[(1,0),(0,1)]", and the matrix representation of F in the basis S and B can be obtained by "mat;rref(mrg(B,A*M));A;[(2,3),(1,-1)];M;[(2,1),(1,2)]; B;[(1,0),(0,1)]".

The above matrices \([F]_{B,S}\) and \([F]_{S,B}\) are related by \([F]_{S,B}=Q^{-1}[F]_{B,S}P\), where \(P\) is the change of basis matrix from B to S, Q is the change of matrix from S to B, Q\(^{-1}\) = P. Verify the result by "mat;P*H*P;P;[(2,1),(1,2)];H;[(1,7/3),(0,-5/3)]".

(6) Let F(x, y) = (3x + 5y, 2x - y) be a linear operator on \(R^2\) and S = { (1,2),(1,3) } and S' = { (3,4),(4,2) } be two basis of \(R^2\). The matrix A representation of F is by "mat;rref(mrg(B,M*B));M;[(3,5),(2,-1)];B;[(1,1),(2,3)]" (right half of rref(mrg(B,M*B))), the matrix B representation of F is by "mat;rref(mrg(B,M*B));M;[(3,5),(2,-1)];B;[(3,4),(4,2)]" (right half of rref(mrg(B,M*B))), and change of basis matrix P = \(\begin{bmatrix}5&10\\-2&-6\end{bmatrix}\) from S to S' is by "mat;rref(mrg(B,v));B;[(1,1),(2,3)];v;[a,b]" and "mat;M*C;M;[(3,-1),(-2,1)];C;[(3,4),(4,2)]". Verify that B = P\(^{-1}\)AP = \(\begin{bmatrix}-5&-2\\11&7\end{bmatrix}\) by "mat;inv(P)*A*P;A;[(39,55),(-26,-37)];P;[(5,10),(-2,-6)]". Note that A and B are similar matrices, and they represent F relative to different bases.

back to top

Inner Product Space

Inner Product Space on \(R^n\) Given each pair of vectors \({\bf u},{\bf v}\) in a vector space V over \(R\), the function \(⟨{\bf u},{\bf v}⟩\) assigns a real value and is called an inner product if it satisfies (1) linear property \(⟨a{\bf u}_1+b{\bf u}_2,{\bf v}⟩=a⟨{\bf u}_1,{\bf v}⟩+b⟨{\bf u}_2,{\bf v}⟩\), (2) symmetry property \(⟨{\bf u},{\bf v}⟩=⟨{\bf v},{\bf u}⟩\), and (3) positive definite property \(⟨{\bf u},{\bf v}⟩≥0\) and \(⟨{\bf u},{\bf u}⟩=0\) if and only if \({\bf u}={\bf 0}\).

Inner Product Space on \(C^n\) If V is a vector space over \(C\), the complex inner product \(⟨{\bf u},{\bf v}⟩\) is linear in the first position and conjugate linear in the second position. Thus, \(⟨{\bf u},a{\bf v}_1+b{\bf v}_2⟩=\overline{⟨a{\bf v}_1+b{\bf v}_2,{\bf u}⟩}=\bar{a}\overline{⟨{\bf v}_1,{\bf u}⟩}+\bar{b}\overline{⟨{\bf v}_2,{\bf u}⟩}=\bar{a}⟨{\bf u},{\bf v}_1⟩+\bar{b}⟨{\bf u},{\bf v}_2⟩\).

An inner product on a vector space M of m × n matrices is defined by ⟨A, B⟩ = trace(B^TA). The inner product on a function space \(C[a,b]\) and polynomial space \(P(t)\) is defined by \(⟨f,g⟩=∫_a^bf(t)g(t)dt\).

Normal on \(R^n,C^n\) The nonnegative number \(||{\bf u}||=\sqrt{⟨{\bf u},{\bf u}⟩}\) is the norm or length of \({\bf u}\). Multiplying a vector \({\bf u}\) by the reciprocal of its length is to normalize \({\bf u}\) such that the vector \(\frac{1}{||{\bf u}||}{\bf u}\) has length 1 (called unit vector).

The norm \(||(a_1,a_2,\cdots,a_n)||_∞=\text{max}(|a_i|)\) is called infinity norm, \(||(a_1,a_2,\cdots,a_n)||_1=|a_1|+|a_2|+\cdots+|a_n|\) is one-norm, and \(||(a_1,a_2,\cdots,a_n)||_2=\sqrt{a_1^2+a_2^2+\cdots+a_n^2}\) is two-norm.

Computing Usual Inner Product and Norm For vectors u and v, the function "dot(u,v)" returns the usual inner (dot) product ⟨u, v⟩. To normalize a vector, use "u/norm(u)". Note that dot(u,v) = dot(v,u) and "dot(u,u)" returns ||u||\(^2\) = ⟨u, u⟩.

Attention For complex inner product, the order matters because of the property of conjugate linear in the second position. Use the transpose function "tsp" for conjugate transpose, or manually enter the conjugate transpose of a vector in the second position. The function "dot(u,tsp(v))" returns ⟨u, v⟩, and "dot(u,tsp(u))" returns ||u||\(^2\) = ⟨u, u⟩. Note that dot(u, tsp(v)) = dot(tsp(u), v).

Examples
(1) Calculate the inner product of (2,13,5,-4,11) and (3,1,24,17,7) by "mat;dot(u,v);u;[(2,13,5,-4,11)];v;[3,1,24,17,7]"; calculate the inner product ⟨A, B⟩ for A = \(\begin{bmatrix}2&3&9\\12&25&-4\end{bmatrix}\) and B = \(\begin{bmatrix}10&7&-8\\21&8&-6\end{bmatrix}\) by "mat;trace(tsp(B)*A);A;[(2,3,9),(12,25,-4)];B;[(10,7,-8),(21,8,-6)]"; calculate ⟨u, v⟩ = 68 + 7i for u = (1+2i,-i,3-4i,9) and v = (-3i,5+7i,9,6-5i) by "mat;dot(tsp(v),u);u;[1+2*I,-I,3-4*I,9];v;[-3*I,5+7*I,9,6-5*I]" and ⟨v, u⟩ = 68 - 7i by "mat;dot(tsp(u),v);u;[1+2*I,-I,3-4*I,9];v;[-3*I,5+7*I,9,6-5*I]".

(2) Consider the inner product \(⟨f,g⟩=∫_0^1f(t)g(t)dt\). Calculate \(⟨f,g⟩\) for \(f(t)=3t^2-2t,g(t)=2t^2-3t+4\) by "itg;(3*t^2-2*t)*(2*t^2-3*t+4);t;0;1", and calculate \(||f||^2\) by "itg;(3*t^2-2*t)^2;t;0;1", which implies \(||f||=\sqrt{\frac{2}{15}}\).

(3) Given \({\bf u}\) = (2,13,5,1,-4). Get \(||{\bf u}||_1\) = 25 by "mat;norm(u,1);u;[2,13,5,1,-4]"; get \(||{\bf u}||_2=\sqrt{215}\) by "mat;norm(u,2);u;[2,13,5,1,-4]"; get \(||{\bf u}||_∞\) = 13 by "mat;norm(u,oo);u;[2,13,5,1,-4]". Normalize the vector (1,2,2) by "mat;u/norm(u);u;[1,2,2]", and normalize the vector (1, 1+i, 1-2i) by "mat;u/norm(u);u;[1,1+I,1-2*I]".

(4) Given \({\bf u}\) = (2,5,3,7) and \({\bf v}\) = (1,6,-3,4). Get \(d_1({\bf u}-{\bf v})=||{\bf u}-{\bf v}||_1\) = 11 by "mat;norm(u-v,1);u;[2,5,3,7];v;[1,6,-3,4]"; get \(d_2({\bf u}-{\bf v})=||{\bf u}-{\bf v}||_2=\sqrt{47}\) by "mat;norm(u-v,2);u;[2,5,3,7];v;[1,6,-3,4]"; get \(d_∞({\bf u}-{\bf v})=||{\bf u}-{\bf v}||_∞\) = 6 by "mat;norm(u-v,oo);u;[2,5,3,7];v;[1,6,-3,4]".

(5) Given \({\bf u}\) = (2i,2+i,3) and \({\bf v}\) = (1,2-3i,3+5i). Get \(d_1({\bf u}-{\bf v})=||{\bf u}-{\bf v}||_1\) = 11 by "mat;norm(u-v,1);u;[2*I,2+I,3];v;[1,2-3*I,3+5*I]"; get \(d_2({\bf u}-{\bf v})=||{\bf u}-{\bf v}||_2\) by "mat;norm(u-v,2);u;[2*I,2+I,3];v;[1,2-3*I,3+5*I]"; get \(d_∞({\bf u}-{\bf v})=||{\bf u}-{\bf v}||_∞\) = 6 by "mat;norm(u-v,oo);u;[2*I,2+I,3];v;[1,2-3*I,3+5*I]".

Orthogonal Complements The orthogonal complement S\(^⊥\) of a subset S consists of the vectors \({\bf v}\) in an inner product space V such that \(⟨{\bf v},{\bf u}⟩=0\) for \({\bf u}\) ∈ S.

Example
(1) Let \(W\) be a subspace of \(R^5\) spanned by (2,3,-1,0,4) and (2,4,7,-6,3). If a vector \((x_1,x_2,x_3,x_4,x_5)∈W^⊥\), then the unknowns satisfy \(2x_1+3x_2-x_3+4x_5=0,2x_1+4x_2+7x_3-6x_4+3x_5=0\). To find a basis of the orthogonal complement \(W^⊥\) is equivalent to find a basis of the solution space of the system by "mat;nsp(A);A;[(2,3,-1,0,4),(2,4,2,-6,3)]".

(2) Find an orthogonal basis of \({\bf u}^⊥\) in \(C^3\) where \({\bf u}\) = (1,-2i,1+2i). A vector \({\bf v}\) = (x, y, z) in \({\bf u}^⊥\) satisfies the equation \(⟨{\bf v},{\bf u}⟩\) = x - 2iy + (1+zi)z = 0. A basis of the solution space of this equation can be obtained by "mat;nsp(A);A;[(1,-2*I,1+2*I)]". Orthogonalize this basis by "mat;grs(nsp(A));A;[(1,-2*I,1+2*I)]".

Matrix Representation of an Inner Product Let V be a real inner product space with a basis S = \(\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\). Then the matrix A = \([⟨{\bf v}_i,{\bf v}_j⟩]\) represents the inner product on V relative to the basis S. Clearly, A is symmetric. For any vectors u and v in V, ⟨u, v⟩ = [u]^TA[v].

Examples
(1) Find a matrix representation of the usual inner product on \(R^2\) relative to the basis { (1,2),(3,5) }. Form a matrix A whose columns are the given basis vectors. Then AA^T is the matrix representation by "mat;A*tsp(A);A;[(1,3),(2,5)]".

(2) The matrix representation of the usual inner product on \(R^3\) relative to a basis { (1,2,2),(2,-5,4),(6,0,-3) } can be obtained by "mat;a*tsp(a);a;[(1,2,2),(2,-5,4),(6,0,-3)]". Since the basis set is orthogonal, the resulting matrix is diagonal.

(3) The matrix representation of the usual inner product on \(C^3\) relative to a basis { (1,i,1+i),(i,1-i,2),(1+i,0,-3i) } can be obtained by "mat;a*tsp(a);a;[(1,I,1+I),(I,1-I,2),(1+I,0,-3*I)]". The resulting matrix is Hermitian.

Projections The projection of the vectors \({\bf v}\) along \({\bf u}\) in a inner product space is proj\(({\bf v},{\bf u})=c{\bf u}=\frac{⟨{\bf v},{\bf u}⟩}{⟨{\bf u},{\bf u}⟩}{\bf u}\).

The projection of \({\bf v}\) along a subspace \(W=\text{span}\{{\bf w}_1,{\bf w}_2,\cdots,{\bf w}_r\}\) of \(R^n\) is proj\(({\bf v},W)=A(A^TA)^{-1}A^T{\bf v}\), where the columns of \(A\) are the vectors \({\bf w}_1,{\bf w}_2,\cdots,{\bf w}_r\). In other words, to find the projection of \({\bf v}\) along a subspace \(W\) is equivalent to find \({\bf w}∈W\) such that \(||{\bf v}-{\bf w}||\) is minimized. In particular, if the spanning set is orthogonal, proj\(({\bf v},W)=c_1{\bf w}_1+\cdots+c_r{\bf w}_r\).

Attention Although the dot product of two (row or column) vectors u and v can be directly calculated by "u*v" or "v*u", the result is a one-entry matrix (instead of a scalar), and thus it cannot be combined with other operations. The function "dot(u,v)" or "dot(v,u)" calculates the dot product of the two vectors and returns a scalar that can be combined with other operations. For example, "dot(u,v)/dot(v,v)" returns the Fourier coefficient of u with respect to v.

Examples
(1) Find the projection of (1,2,3,4) along (2,2,-1,3) by "mat;(dot(u,v)/dot(v,v))*v;u;[1,2,3,4];v;[2,2,-1,3]".

(2) Find cos(θ) by "mat;dot(u,v)/(norm(u)*norm(v));u;[2,2,-1,3];v;[1,1,2,4]", where θ is an angle between the vectors (2,2,-1,3) and (1,1,2,4).

(3) Find the projection of the vector (2,4,6,8) onto the subspace spanned by { (1,2,2,-1),(2,3,1,2) } by "mat;A*inv(tsp(A)*A)*tsp(A)*v;v;[2,4,6,8];A;[(1,2),(2,3),(2,1),(-1,2)]".

(4) Find the projection of the vector (1,2,-3,5) onto the subspace spanned by { (1,0,1,1),(0,1,1,-1) } by "mat;u*dot(v,u)/dot(u,u)+w*dot(v,w)/dot(w,w);v;[1,2,-3,5];u;[1,0,1,1];w;[0,1,1,-1]" or "mat;A*inv(tsp(A)*A)*tsp(A)*v;v;[1,2,-3,5];A;[(1,0),(0,1),(1,1),(1,-1)]".

Gram-Schmidt Orthogonalization Process To obtain an orthonormal basis from a given basis \(\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) of an inner product space V, set \({\bf w}_1={\bf v}_1\\{\bf w}_2={\bf v}_2-\frac{⟨{\bf v}_2,{\bf w}_1⟩}{⟨{\bf w}_1,{\bf w}_1⟩}{\bf w}_1\\{\bf w}_3={\bf v}_3-\frac{⟨{\bf v}_3,{\bf w}_1⟩}{⟨{\bf w}_1,{\bf w}_1⟩}{\bf w}_1-\frac{⟨{\bf v}_3,{\bf w}_2⟩}{⟨{\bf w}_2,{\bf w}_2⟩}{\bf w}_2\\\cdots\\{\bf w}_n={\bf v}_n-\frac{⟨{\bf v}_n,{\bf w}_1⟩}{⟨{\bf w}_1,{\bf w}_1⟩}{\bf w}_1-\frac{⟨{\bf v}_n,{\bf w}_2⟩}{⟨{\bf w}_2,{\bf w}_2⟩}{\bf w}_2-\cdots-\frac{⟨{\bf v}_n,{\bf w}_{n-1}⟩}{⟨{\bf w}_{n-1},{\bf w}_{n-1}⟩}{\bf w}_{n-1}\).
The set \(\{{\bf w}_1,{\bf w}_2,\cdots,{\bf w}_n\}\) form an orthogonal basis for V. Normalizing each vector in the set yields an orthonormal basis for V.

Given a basis \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\), the function "grs(A)" returns a set of orthogonal vectors from \(S\), where the rows of \(A\) consists the vectors in \(S\) and the rows of grs(A) are orthogonal vectors corresponding to the vectors in \(S\). To normalize each vector of the orthogonal set, use "grs(A,1)", and an orthonormal set is contained in the rows of grs(A,1).

Examples
(1) Find an orthogonal basis of W\(^⊥\) for W = span(2,2,4,3) by "mat;grs(nsp(A));A;[(2,2,4,3)]". Note that "nsp(A)" returns a basis (row) of the null space of A with dimension 3, and the function grs(nsp(A)) orthogonalizes the basis by the Gram-Schmidt process.

(2) A subspace W of \(R^4\) is spanned by (1,2,2,1), (3,-2,1,-1), (1,0,2,1). Find an orthogonal basis of W by "mat;grs(A);A;[(1,2,2,1),(3,-2,1,-1),(1,0,2,1)]" and an orthonormal basis of W by "mat;grs(A,1);A;[(1,2,2,1),(3,-2,1,-1),(1,0,2,1)]".

(3) Find an orthogonal matrix whose first row is (2,1,2). Extend the vector to form a basis of \(R^3\) by adding two vectors (1,2,1) and (1,1,4), and then use "mat;grs(A,1);A;[(2,1,2),(1,2,1),(1,1,4)]" to get an orthonormal set of the basis. It also returns an orthogonal matrix. Note that this matrix is not unique.

(4) Find an orthogonal basis of the subspace of \(C^3\) spanned by (i,2,3-4i) and (1,2-i,3+2i). "mat;grs(A);A;[(I,2,3-4*I),(1,2-I,3+2*I)]" gives an orthogonal basis of W, and "mat;grs(A);A;[(I,2,3-4*I),(1,2-I,3+2*I)]" yields an orthonormal basis of W.

Verify Orthogonal Matrix A matrix \(P\) is orthogonal if \(P\) is nonsingular and \(P^{-1}=P^T\) or \(PP^T=P^TP=I\). If \(P\) is orthogonal, the rows or columns form an orthonormal set.

Check if "inv(P)*P" is an identity matrix by "inv(P)*P". If it is an identity matrix, \(P\) is an orthogonal matrix. To verify if \(P\) is an orthogonal matrix is to verify that each row and each column has unit length and that each pair of rows and columns is orthogonal.

Example Check if A = [(3,2,1),(1,-2,1),(4,-2,-8)] is an orthogonal matrix by "mat;A*tsp(A);A;[(3,2,1),(1,-2,1),(4,-2,-8)]", which returns a diagonal matrix (but is not an identity matrix). Thus, A is not an orthogonal matrix. Observe that each pair of rows of A is orthogonal, and the Gram-Schmidt process returns the same rows of A by "mat;grs(A);A;[(3,2,1),(1,-2,1),(4,-2,-8)]". Normalize each row by "mat;grs(A,1);A;[(3,2,1),(1,-2,1),(4,-2,-8)]", and the resulting matrix grs(A,1) is orthogonal. Check it by "mat;grs(A,1)*tsp(grs(A,1));A;[(3,2,1),(1,-2,1),(4,-2,-8)]".

Positive Definite Matrices Let \(A\) be a real symmetric matrix \((A^T=A)\). If for every nonzero vector \({\bf u}∈R^n,⟨{\bf u},A{\bf u}⟩={\bf u}^TA{\bf u}>0\), then \(A\) is positive definite. If \(A\) is a real positive definite matrix, then \(⟨{\bf u},{\bf v}⟩={\bf u}^TA{\bf v}\) defines an inner product. Use the function "tof(A)" to determine if a matrix A is positive definite.

Example Verify \(⟨{\bf u},{\bf v}⟩=2x_1y_1-3x_1y_2-3x_2y_1+5y_1y_2\) defines an inner product on \(R^2\). Write \(⟨{\bf u},{\bf v}⟩=[x_1,x_2]\begin{bmatrix}2&-3\\-3&5\end{bmatrix}\begin{bmatrix}y_1\\y_2\end{bmatrix}\). Now need to verify if the matrix A in the middle is positive definite. Since A is real symmetric, each diagonal entry is positive, and the determinant |A| = 1 > 0. Thus, A is positive definite, and "mat;tof(A);A;[(2,-3),(-3,5)]" returns "True".

Q: mat;B*A;A;[1,2,1+I];B;[(2-3*I,2*I,3+2*I)] A: \(=[3+6i]\)	Q: mat;cjg(B)*A;A;[1,2,1+I];B;[(2-3*I,2*I,3+2*I)] A: \(=[7]\)	Q:  mat;tsp(a)*b;a;[1+I,2-3*I,4*I];b;[8*I,-4*I,-5*I] A: \(=[0]\)
Q: mat;grs(A);A;[(1,2,-1),(0,-1,0),(1,1,1)] A: \(=\begin{bmatrix}1&2&-1\\\frac{1}{3}&\frac{-1}{3}&\frac{1}{3}\\1&0&1\end{bmatrix}\)	Q: mat;grs(A,1);A;[(2,1,2),(0,-1,2),(2,1,1)] A: \(=\frac{1}{3}\begin{bmatrix}2&1&2\\-1&-2&2\\2&-2&-1\end{bmatrix}\)	Q: mat;grs(A,1)*tsp(grs(A,1));A;[(1,2,3),(0,1,0),(1,1,1)] A: \(=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\)

back to top

I. Determinant, Minor and Cofactor, Volume

Determinant Each n-square matrix is assigned a value of determinant. If A = [c], det(A) = c; if A = \(\begin{bmatrix}a&b\\c&d\end{bmatrix}\), det(A) = ad - bc; if A = \(\begin{bmatrix}a_{11}&a_{12}&a_{13}\\a_{21}&a_{22}&a_{23}\\a_{31}&a_{32}&a_{33}\end{bmatrix}\), det(A) = \(a_{11}(a_{22}a_{33}-a_{23}a_{32})-a_{12}(a_{21}a_{33}-a_{23}a_{31})+a_{13}(a_{21}a_{32}-a_{22}a_{31})\). A determinant of order n is equal to the linear combination of the determinants of order n - 1, and determinant can be calculate by cofactor expansion by any row or any column.

Minors, Cofactors, Adjugate Matrices Given an n-square matrix \(A=[a_{ij}]\), the minor \(A_{ij}\) is the determinant of a submatrix obtained by deleting the \(i\)th row and \(j\)th column of \(A\), and the value \((-1)^{i+j}A_{ij}\) is called cofactor corresponding to the entry \((i,j)\). The transpose of the cofactor matrix is called the adjugate (classic adjoint) matrix of \(A\). Note that \(A^{-1}=\frac{1}{\text{det}(A)}\text{adj}(A)\).

Determinants of Linear Operators The determinant of a linear operator F on a finite dimensional vector space V is equal to the determinant of the matrix A representation of F relative to some basis S of V, that is, det(F) = det(A).

The function "det(A)" returns the determinant of a matrix A. For the submatrix A_ij obtained by deleting the i-th row and j-th column of A, the function "mno(A,i,j)" returns the minor (determinant) of A_ij corresponding to the entry (i, j) of A, and "cfm(A)" returns the cofactor (signed minor) matrix of A (for all entries). The transpose of a cofactor matrix by tsp(cfm(A)) is called ajugate of A. Note that minor corresponds to the entry (1, 1) of A is obtained by mno(A,0,0), rather than mno(A,1,1).

Examples
(1) Given A = [(2,5,3),(3,6,7),(4,-1,2)], get the determinant by "mat;det(A);A;[(2,5,3),(3,6,7),(4,-1,2)]"; get the minor corresponding to the entry (2,3) by "mat;mno(A,1,2);A;[(2,5,3),(3,6,7),(4,-1,2)]"; get the cofactor matrix by "mat;cfm(A);A;[(2,5,3),(3,6,7),(4,-1,2)]"; verify the equations \(A^{-1}=\frac{1}{\text{det}(A)}\text{adj}(A)\) by "mat;inv(A)-tsp(cfm(A))/det(A);A;[(2,5,3),(3,6,7),(4,-1,2)]" (check if the resulting matrix is a zero matrix).

(2) Find the determinant of \(\begin{bmatrix}2&0&0\\15&6&0\\23&14&-3\end{bmatrix}\). The determinant of this upper triangular matrix is -36, the product of its diagonal entries. Verify the result by "mat;det(A);A;[(2,0,0),(15,6,0),(23,14,-3)]". Find the determinant of the matrix \(\begin{bmatrix}2&3&12&-15\\5&6&21&32\\0&0&7&-8\\0&0&2&6\end{bmatrix}\). The matrix is a lower triangular block matrix, and its determinant is the product of the determinant of its diagonal block matrices by "mat;det(A)*det(B);A;[(2,3),(5,6)];B;[(7,8),(2,6)]". Get the same result by "mat;det(C);C;[(2,3,12,-15),(5,6,21,32),(0,0,7,8),(0,0,2,6)]".

(3) Find the volume of the parallelopiped determined by the three vectors (2,3,5), (1,4,0), (5,3,-6) by "mat;abs(det(A));A;[(2,3,5),(1,4,0),(5,3,-6)]". The volume is |det(A)|, which is 115.

(4)The signed volume of the parallelopiped determined by "mat;det(A);A;[(x,0,0),(y,1,0),(z,0,1)]" is x. Similarly, "mat;det(A);A;[(1,x,0),(0,y,0),(0,z,1)]" gives y, and "mat;det(A);A;[(1,0,x),(0,1,y),(0,0,z)]" gives z.

(5) Calculates the cross product of (1,2,3) and (4,5,6) by "mat;cross(a,b);a;[1,2,3];b;[4,5,6]", where the length of the cross product by "mat;norm(cro(a,b));a;[1,2,3];b;[4,5,6]" is the area of the parallelogram determined by these two vectors, and the direction of the cross product obeys the right-hand rule. Cross product only applies to 3-dimensional vectors, and the formula of cross product can be got by "mat;cross(u,v);u;[x,y,z];v;[a,b,c]".

(6) Verify that similar matrices have the same determinant. Let A = [(5,6,1),(4,-9,2),(3,0,5)] and P = [(1,2,3),(7,2,4),(4,2,0)]. Verify that P is nonsingular by "mat;det(P);P;[(1,2,3),(7,2,4),(4,2,0)]", which shows det(P) ≠ 0. The matrix B = P\(^{-1}\)AP is similar to A and they have the same determinant. The results from "mat;det(A);A;[(5,6,1),(4,-9,2),(3,0,5)]" and "mat;det(inv(P)*A*P);P;[(1,2,3),(7,2,4),(4,2,0)];A;[(5,6,1),(4,-9,2),(3,0,5)]" are the same.

Q: mat;norm(A);A;[(1,1,1),(1,2,2),(1,2,3)] A:  = \(\sqrt{26}\)	Q: mat;rank(B);B;[(1,2,1,2,2),(0,3,1,-2,2),(4,-2,0,1,1),(3,0,1,-1,4)] A:  = 4	Q: mat;det(A);A;[(2,-1,0),(1,2,1),(0,-1,2)] A:  = 12
Q: mat;cnd(A);A;[(1.85,2.36),(-3.71,5.4)] A:  = 2.34264621295597	Q: mat;norm(B,1);B;[(1,2,1,2),(0,3,1,-2),(4,-2,0,1),(3,0,1,-1)] A:  = 8	Q: mat;norm(A,oo);A;[(2,-1,0),(1,2,1),(0,-1,2)] A:  = 4

II. Definiteness, Diagonalizable, Nilpotent

True or False ("tof") Function By default, the function "tof(A)" (True or False) tells whether a matrix is positive definite. Options can be added to the function as "tof(A, type)", and there are seven types indicated by integers from 0 to 6: 0 - diagonalizable, 1 - positive definite; 2 - negative definite; 3 - positive semidefinite; 4 - negative semidefinite; 5 - indefinite; 6 - nilpotent.

Q: mat;tof(A,0);A;[(2,-1,0),(1,2,1),(0,-1,2)] A:  = True

Q: mat;tof(B);B;[(1,2,1,2),(0,3,1,-2),(4,-2,0,1),(3,0,1,-1)] A:  = False

Q: mat;tof(A,6);A;[(5,-3,2),(15,-9,6),(10,-6,4)] A:  = True

Q: mat;mno(C,1,2);C;[(1,6,-1),(2,5,3),(2,1,4)] A: \(=-11\)

Q: mat;cfm(A);A;[(2,-1,0),(1,2,1),(0,-1,2)] A: \(=\begin{bmatrix}5&-2&-1\\2&4&2\\-1&-24&5\end{bmatrix}\)

Q: mat;tsp(cfm(A));A;[(2,3,1),(1,0,-4),(5,-1,6)] A: \(=\begin{bmatrix}-4&-19&-12\\-26&7&9\\-1&17&-3\end{bmatrix}\)

back to top

I. Eigenvalues and Eigenvectors

An n-square matrix \(A\) is diagonalizable if there exists a nonsingular matrix \(P\) such that \(B=P^{-1}AP\) is diagonal. A linear operator \({\bf T}\) on V with dimension n is diagonalizable if there exists a basis S such that the matrix representing \({\bf T}\) relative to S is diagonal. Diagonalizing \(A\) and \({\bf T}\) is essentially the same, for \(A\) may be viewed as a linear operator \({\bf T}\) defined by \({\bf T}({\bf u})=A{\bf u}\). In this case, \(A\) represents \({\bf T}\) relative to the usual basis, \(B\) represents \({\bf T}\) (or \(A\)) relative to S, and \(P\), whose columns are the basis vectors in S, is the change of basis matrix from the standard basis of V to S. Now we need a process to determine if such a basis S exists and how to find it (provided it exists).

Eigenvalues and Eigenvectors for Matrices An n-square matrix \(A\) can be represented by a diagonal matrix \(D=\) diag\((λ_1,λ_2,\cdots,λ_n)\) if and only if there exists a basis S consisting of vectors \(\{{\bf u}_1,{\bf u}_2,\cdots,{\bf u}_n\}\) such \(A{\bf u}_1=λ_1{\bf u}_1,A{\bf u}_2=λ_2{\bf u}_2,\cdots,A{\bf u}_n=λ_n{\bf u}_n\). In such a case, \(A\) is diagonalizable, and \(D=P^{-1}AP\), where the columns of \(P\) are the vectors \(\{{\bf u}_1,\cdots,{\bf u}_n\}\). A scalar \(λ\) is an eigenvalue of \(A\) if there exists a nonzero vector \({\bf u}\) such that \(A{\bf u}=λ{\bf u}\), and any vector satisfying this relation is an eigenvector of \(A\) belonging to \(λ\). An n-matrix \(A\) can be diagonalized if and only if \(A\) has n linearly independent eigenvectors.

Eigenvalues and Eigenvectors for Linear Operators A linear operator \({\bf T}\) on V with dimension n is diagonalizable if and only if there exists a basis S = \(\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) of V for which \({\bf T}({\bf v}_1)=λ_1{\bf v}_1,{\bf T}({\bf v}_2)=λ_2{\bf v}_2,\cdots,{\bf T}({\bf v}_n)=λ_n{\bf v}_n\). In such a case, \({\bf T}\) is represented by a diagonal matrix \(D=\) diag\((λ_1,λ_2,\cdots,λ_n)\) relative to the basis S. A scalar \(λ\) is an eigenvalue of \({\bf T}\) if there exists a nonzero vector \({\bf v}\) such that \({\bf T}({\bf v})=λ{\bf v}\), and any vector satisfying this relation is an eigenvector of \({\bf T}\) belonging to \(λ\). A linear operator \({\bf T}\) can be diagonalized if and only if there exists a basis S of V consisting of eigenvectors of \({\bf T}\), or \({\bf T}\) can be diagonalized if and only if the matrix representation of \({\bf T}\) is diagonalizable.

Characteristic Polynomials For an n-square matrix \(A\), the characteristic polynomial of \(A\) is \(Δ(t)=\text{det}(tI_n-A)\), the determinant of the matrix \(tI_n-A\). The characteristic polynomial of a linear operator \({\bf T}\) is the characteristic polynomial of any matrix representation of \({\bf T}\). Note that similar matrices have the same characteristic polynomial, have the same determinant, and have the save eigenvalues. Also observe that for an n-square A, the following states are equivalent. (1) λ is an eigenvalue of A. (2) The matrix A - λI is singular. (3) λ is a root of the characteristic polynomial of A.

If \(A\) is a 2 × 2 matrix, \(Δ(t)=t^2-\text{trace}(A)t+\text{det}A\). In general, the characteristic polynomial of an n-square matrix is \(Δ(t)=t^n-S_1t^{n-1}+S_2t^{n-1}+\cdots+(-1)^nS_n\), where \(S_i\) is the sum of the principal minors of order \(i\).

Computing Eigenvalues, Eigenvectors, and Characteristic Polynomials The function "eig(A)" returns the eigenvalues and eigenvectors of the matrix \(A\), which include all eigenvalues, the algebraic multiplicity of each eigenvalue, and the corresponding eigenvectors.The function "chp(A)" returns the characteristic polynomial \(p(λ)\) of the matrix \(A\).

Procedure for Diagonalizing a Matrix or Linear Operator For a given n-square matrix A, (1) find the characteristic polynomial Δ(t) of A. (2) Find the roots of Δ(t) (or the eigenvalues of A). (3) Find the eigenvectors of each eigenvalue by solving the homogeneous system (A - λI)x = 0. (4) If n linearly independent eigenvectors are obtained, A is diagonalizable. Otherwise, A is not diagonalizable. Directly obtain the matrix P and D by the function "dgl(A)".

Examples
(1) If \(f(t)=t^2-8t+7\) and \(A\) = [(3,2),(4,5)], calculate \(f(A)\) by "mat;A^2-8*A+7*eye(2);A;[(3,2),(4,5)]". Observe that \(f(A)\) is a zero matrix, because \(f(t)\) is the characteristic polynomial of \(A\) for which \(A\) is a root. Use "chp(A)" to verify it.

(2) Find a matrix A whose eigenvalues are 1, 2, 5 and the corresponding eigenvectors are (2,3,-1),(1,1,2),(3,0,4). Let D = diag(1,2,5) and P is the matrix whose columns are the eigenvectors. Then A = PDP\(^{-1}\) by "mat;P*D*inv(P);P;[(2,1,3),(3,1,0),(-1,2,4)];D;diag(1,2,3)".

(3) Diagonalize A = \(\begin{bmatrix}3&2\\3&4\end{bmatrix}\). The characteristic polynomial is p(λ) = λ\(^2\) - 7λ + 6 = (λ-1)(λ-6). So the eigenvalues are λ = 1 and λ = 6. Substitute λ = 1 and solve the system (A - I)x = 0 for a solution (1, -1). Substitute λ = 6 and solve (A - 6I)x = 0 for a solution (2, 3). Then D = P\(^{-1}\)AP = diag(1,6) by "mat;P^(-1)*A*P;P;[(1,2),(-1,3)];A;[(3,2),(3,4)]", where the columns of P are the two eigenvectors. Given that A and D are similar matrices, their determinant are 6 and their characteristic polynomial are the same.

Verify eigenvalues and eigenvectors by "mat;eig(A);A;[(3,2),(3,4)]", and verify diagonalizing A directly by "mat;dgl(A);A;[(3,2),(3,4)]". Verify A = PAP\(^{-1}\) by "mat;P*diag(1,6)*inv(P);P;[(1,2),(-1,3)]". Calculate A\(^5\) by "mat;P*diag(1,6)^5*inv(P);P;[(1,2),(-1,3)]" or "mat;A^5;P;[(1,2),(-1,3)];A;[(3,2),(3,4)]". If \(f(t)=t^3-6t^2+3t-4,f(1)=-6,f(6)=14\), calculate \(f(A)\) by "mat;P*diag(-6,14)*inv(P);P;[(1,2),(-1,3)]". Verify B\(^2\) = A by "mat;(P*D*inv(P))^2;P;[(1,2),(-1,3)];D;diag(1,6^(1/2))".

(4) Find the characteristic polynomial by "mat;chp(A);A;[(3,2,2),(3,1,-4),(2,1,-3)]". Show that the characteristic polynomial of a block triangular matrix is the product of the characteristic polynomials of their diagonal blocks by the example "mat;chp(A);A;[(2,3,-5,8),(5,4,6,3),(0,0,9,7),(0,0,1,4)]" and "mat;chp(B)*chp(C);B;[(2,3),(5,4)];C;[(9,7),(1,4)]".

(5) Let \({\bf T}\) be a linear operator on \(R^3\) defined by \({\bf T}\)(x,y,z) = (-x+3y-z, -3x+5y-z, -3x+3y+z). To diagonalize \({\bf T}\) is to diagonalize the matrix A representation of \({\bf T}\) relative to the usual basis. The characteristic polynomial of A is by "mat;chp(A);A;[(-1,3,-1), (-3,5,-1),(-3,3,1)]". Observe that there are two eigenvalues λ = 1 and λ = 2. Solve the system (A - I)x = 0 by "mat;rref(A-eye(3));A;[(-1,3,-1),(-3,5,-1),(-3,3,1)]" for an eigenvector (1,1,1). Solve the system (A - 2I)x = 0 by "mat;rref(A-2*eye(3));A;[(-1,3,-1),(-3,5,-1),(-3,3,1)]" for two eigenvectors (1,1,0) and (-1,0,3). Or directly obtain a basis of the solution space to each system by "mat;nsp(A-eye(3));A;[(-1,3,-1),(-3,5,-1),(-3,3,1)]" and "mat;nsp(A-2*eye(3));A;[(-1,3,-1),(-3,5,-1),(-3,3,1)]". Calculate D = P\(^{-1}\)AP by "mat;inv(P)*A*P;A;[(-1,3,-1),(-3,5,-1),(-3,3,1)];P;[(1,1,-1),(1,1,0),(1,0,3)]", where the columns of P are the eigenvectors. Thus, choosing a basis { (1,1,1), (1,1,0), (-1,0,3) } of \(R^3\), \({\bf T}\) is represented by a diagonal matrix D = diag(1,2,2).

Verify the above results by "mat;dgl(A);A;[(-1,3,-1),(-3,5,-1),(-3,3,1)]". Verify the eigenvalues and eigenvectors by "mat;eig(A);A;[(-1,3,-1),(-3,5,-1),(-3,3,1)]".

(6) Find a matrix A such that B = A\(^3\) = \(\begin{bmatrix}27&54&0\\0&27&81\\0&0&27\end{bmatrix}\). Use "mat;A^3;A;[(3,a,b),(0,3,c),(0,0,3)]". Observe that a = 2, b = -2, c= 3.

Q: mat;chp(A);A;[(0,1,1),(1,0,0),(1,1,1)] A:  \(=λ(λ-2)(λ+1)\)	Q: mat;A*tsp(A)-tsp(A)*A;A;[(2,3*I),(3*I,2)] A:  \(=\begin{bmatrix}0&0\\0&0\end{bmatrix}\)
Q: mat;eig(A);A;[(0,1,1),(1,0,0),(1,1,1)] A:  \(=\left[\left((-1, 1),\ \left[\left[\begin{matrix}-1\\1\\0\end{matrix}\right]\right]\right),\ \left((0, 1),\ \left[\left[\begin{matrix}0\\-1\\1\end{matrix}\right]\right]\right),\ \left((2, 1),\ \left[ \left[\begin{matrix}\frac{2}{3}\\\frac{1}{3}\\1\end{matrix}\right]\right]\right)\right]\)	Q: mat;eig(A);A;[(1,2),(2,1)] A:  \(=\left[ \left((-1,1),\ \left[ \left[\begin{matrix}-1\\1\end{matrix}\right]\right]\right), \ \left((3, 1), \ \left[ \left[\begin{matrix}1\\1\end{matrix}\right]\right]\right)\right]\)

II. Diagonalizing Real Symmetric Matrices

Many real matrices are not diagonalizable. Real symmetric matrices are diagonalizable. They have real eigenvalues, and eigenvectors belonging to distinct eigenvalues are orthogonal. Thus, for a real symmetric matrix, there exists an orthogonal matrix P such that D = P\(^{-1}\)AP is diagonal.

The procedure for orthogonally diagonalizing real symmetric matrices is similar to general diagonalizing procedure. However, when diagonalizing a real symmetric matrix, normalize the orthogonal eigenvectors, and orthogonalize eigenvectors that belong to the same eigenvalue (eigenvectors are automatically orthogonal for distinct eigenvalues).

Refer to Section 8 for diagonalizing symmetric bilinear forms in which a simple method for finding nonsingular matrices to diagonalize real symmetric matrices is described.

Example Diagonalize a symmetric matrix A = [(7,3),(3,-1)]. The characteristic polynomial of A is p(λ) = (λ+2)(λ-8), so the eigenvalues are λ = -2 and λ = 8. Solve the system (A + 2I)x = 0 and obtain an eigenvector (1,-3). Solve (A - 8I)x = 0 and obtain an eigenvector (3, 1). The two eigenvectors are orthogonal. Normalize each eigenvector and form a matrix P whose columns are the normalized eigenvectors. Thus, A is similar to D = diag(-2,8) = P\(^{-1}\)AP by "mat;inv(P)*A*P;A;[(7,3),(3,-1)];P;[(1/10^(1/2),3/10^(1/2)),(-3/10^(1/2),1/10^(1/2))]".

III. Quadratic Forms

Diagonalizing Quadratic Forms The real polynomial \(q(x_1,x_2,\cdots,x_n)=\sum_i^nc_ix_i^2+\sum_{i< j}d_{ij}x_ix_j\) is a quadratic form. It determines a real symmetric matrix \(A=[a_{ij}]\), where \(a_{ii}=c_i,a_{ij}=a_{ji}=\frac{1}{2}d_{ij}\) so that \(q\) can be written in matrix form \(q({\bf x})={\bf x}^TA{\bf x}\) for \({\bf x}=(x_1,x_2,\cdots,x_n)^T\). Let \({\bf x}=P{\bf y}\). Then \(q({\bf y})={\bf y}^T(P^TAP){\bf y}\). Diagonalizing a quadratic form is to seek an orthogonal matrix \(P\) such that the orthogonal substitution \({\bf x}=P{\bf y}\) yields a diagonal quadratic form. Refer to Section 8 for more information on diagonalizing quadratic forms.

Example Given \(q(x,y,z)=(3x-2y-3z,-2x+2y-3z,-3x-2y+3z),q(x,y,z)\) determines a real symmetric matrix A. To diagonalize \(q(x,y,z)\) is to diagonalize A by "mat;dgl(A,1);A;[(3,-2,-3),(-2,2,-2),(-3,-2,3)]", where dgl(A,1) returns orthonormal eigenvectors. Substitute \({\bf x}=P{\bf y}\) for \(x=\frac{2\sqrt{3}r+\sqrt{6}s-3\sqrt{2}t}{6},y=\frac{2\sqrt{3}r-2\sqrt{6}s}{6},z=\frac{2\sqrt{3}r+\sqrt{6}s+3\sqrt{2}t}{6}\) by "mat;dgl(A,1,1)*y;A;[(3,-2,-3),(-2,2,-2),(-3,-2,3)];y;[r,s,t]" (dgl(A,1,1) returns matrix \(P\)) where \({\bf y}=(r,s,t)\). Obtain \(q({\bf y})=-2r^2+4s^2+6t^2\) by "mat;tsp(dgl(A,1,1)*y)*A*(dgl(A,1,1)*y);A;[(3,-2,-3),(-2,2,-2),(-3,-2,3)];y;[r,s,t]".

back to top

I. Linear Operators on Inner Product Spaces

Adjoint Operators Let \({\bf T}\) be a linear operator on a finite-dimensional inner product space V. Then \({\bf T}^*\) is an adjoint operator on V if \(⟨{\bf T}({\bf u}),{\bf v}⟩=⟨{\bf u},{\bf T}^*({\bf v})⟩\) for every \({\bf u},{\bf v}\) in V. A real n-square matrix \(A\) is viewed as a linear operator, so \(A^T\) is the adjoint of \(A\). If \(B\) is n-square complex matrix, \(B^*\) is the adjoint of \(B\).

If \(A\) is the matrix representation of \({\bf T}\) relative to any orthonormal basis S of V, then \(A^*\) is the matrix representation of \({\bf T}^*\) relative to S.

Special Linear Operators (1) If \({\bf T}^*={\bf T}^{-1},{\bf T}\) is an orthogonal or unitary operator. If \({\bf T}\) is unitary, its eigenvalues are 1 or -1. (2) If \({\bf T}^*={\bf T},{\bf T}\) is a self-adjoint (symmetric or Hermitian) operator. If \({\bf T}\) is self-adjoint, its eigenvalues are real. (3) If \({\bf T}^*=-{\bf T}^{-1},{\bf T}\) is a skew-adjoint (skew-symmetric or skew-Hermitian) operator. If \({\bf T}\) is skew-adjoint, its eigenvalues are pure imaginary. (4) If \({\bf T}={\bf S}^*{\bf S},{\bf T}\) is a positive definite operator, where \({\bf S}\) is nonsingular. If \({\bf T}\) is positive definite, its eigenvalues are positive.

For symmetric or Hermitian operators, its eigenvectors that correspond to distinct eigenvalues are orthogonal. Unitary Operators preserve inner products, lengths, and distances. A complex matrix \(A\) represents a unitary operator relative to an orthonormal basis if and only if \(A\) is unitary (\(A^*=A^{-1})\). A real matrix represents an orthogonal operator if and only if \(A\) is orthogonal (\(A^{-1}=A^T)\). If a matrix is unitary, its rows or columns form an orthogonal set, and the converse is also true.

Normal Operators and Normal Matrices An operator that commutes with its adjoint is called a normal operator, that is \({\bf T}^*{\bf T}={\bf T}{\bf T}^*\). Unitary, self-adjoint, skew-adjoint, and positive definite operators are all normal operators, and each commutes with its adjoint.

Similarly, a matrix that commutes with its conjugate transpose is called a normal matrix, that is \(AA^*=A^*A\). Note that the matrix \(A^*A\) is Hermitian and has nonnegative eigenvalues. The square roots of the eigenvalues of \(A^*A\) are called the singular values of \(A\).

Positive Definite Operators A linear operator \({\bf T}\) on an inner product space V is positive definite if \({\bf T}={\bf S}^*{\bf S}\), where \({\bf S}\) is a nonsingular operator, and \({\bf T}\) is semidefinite or nonnegative for some operator \(S\). A matrix \(A\) is viewed as a linear operator. If \(A\) positive semidefinite, there is a unique positive semidefinite \(B\) such that \(A=B^*B=BB\).

The following statements on a positive definite operator \({\bf T}\) are equivalent. (1) \({\bf T}\) is positive definite. (2) \({\bf T}={\bf S}^2\) for some nonsingular self-adjoint operator \({\bf S}\). (3) \({\bf T}\) is self-adjoint and \(⟨{\bf T}({\bf u}),{\bf u}⟩>0\) for every \({\bf u}\) in V.

Computing Adjoint Operators Given an linear operator on an inner product space, to find its adjoint operator, first determine its matrix representation A relative to the usual basis, and then obtain the adjoint by "mat;tsp(A)*v;A;A;matAarray;v;[x,y,z]" (for instance).

Verify Normal Matrices To verify if a square matrix is normal \(AA^{T}=A^{T}A\), check if A*tsp(A) - tsp(A)*A is a zero matrix by "mat;A*tsp(A)-tsp(A)*A; A; matAarray".

Test Positive Definite The function "tof(A)" or "tof(A,1)" tests if a square matrix is positive definite, and it returns "True" or "False". In addition, "tof(A,2)" tests if A is negative definite, "tof(A,3)" tests if A is positive semidefinite, "tof(A,4)" tests if A is negative semidefinite, and "tof(A,5)" tests if A is indefinite.

Examples
(1) Find the adjoint of \({\bf T}\) on \(R^3\) defined by \({\bf T}(x,y,z)=(2x-3y+4z,5x+y-2z,x-2y+z)\) by "mat;tsp(A)*u;A;[(2,-3,4),(5,1,-2),(1,-2,1)];u;[x,y,z]". Thus, \({\bf T}^*(x,y,z)=(2x+5y+z,-3x+y-2z,4x-2y+z)\).

(2) Find the adjoint of \({\bf T}\) on \(C^3\) defined by \({\bf T}(x,y,z)=(3x-iy+(2+3i)z,(2-5i)x+4y+7iz,6x-(5+4i)y-9iz)\). It is \({\bf T}^*(x,y,z)=(3x+(2+5i)y+6z,ix+6y-(5-4i)z,(2-3i)x-7iy+9iz)\) by "mat;tsp(A)*u;A;[(3,-I,2+3*I),(2-5*I,4,7*I),(6,-5-4*I,-9*I)];u;[x,y,z]".

(3) Determine if the matrix A = [(1,3),(2,4)] is normal by "mat;tsp(A)*A-A*tsp(A);A;[(1,3),(2,4)]" (not normal). Determine if A = [(3,5),(-5,3)] is normal by "mat;tsp(A)*A-A*tsp(A);A;[(3,5),(-5,3)]" (normal). Determine if the matrix A = [(1+2i,3i),(3,2+3i)] is normal by "mat;tsp(A)*A-A*tsp(A);A;[(1+2*I,3*I),(3,2+3*I)]" (normal).

(4) Calculate the singular values of A = [(2,2),(0,1),(1,0)] by "mat;eig(tsp(A)*A);A;[(2,2),(0,1),(1,0)]". The eigenvalues of A^TA are 9 and 1, and thus the singular values of A are 3 and 1.

II. Diagonalization in Real and Complex Inner Product Spaces

Real Inner Product Space For any self-adjoint (symmetric) operator \({\bf T}\) on a finite dimensional real inner product space V, there exists an orthonormal basis S of V such that \({\bf T}\) can be represented by a diagonal matrix relative to S. For any real symmetric matrix \(A\), there exists an orthogonal matrix \(P\) such that \(A\) is similar to a diagonal matrix \(D=P^{-1}AP\).

For any orthogonal operator \({\bf T}\) on a finite dimensional real inner product space V, there exists an orthonormal basis S of V such that \({\bf T}\) can be represented by a block diagonal matrix relative to S.

Complex Inner Product Space For any normal operator on a complex finite dimensional vector space V, there exists an orthonormal basis of V consisting of the eigenvectors of \({\bf T}\) such that \({\bf T}\) can be represented by a diagonal matrix relative to S.

For an arbitrary operator \({\bf T}\) on a complex finite dimensional vector space V, \({\bf T}\) can be represented by a triangular matrix relative to an orthonormal basis of V.

For any normal matrix \(A\), there exists an unitary matrix \(P\) such that \(D=P^*AP\) is diagonal. Normal matrices are orthogonally diagonalizable, and their eigenvalues are nonnegative. For an arbitrary n-square complex matrix \(A\), there exists an unitary matrix \(P\) such that \(A\) is congruent to a triangular matrix \(B=P^*AP\).

Test Diagonalizable All normal matrices are diagonalizable. Normal matrices include symmetric, Hermitian, orthogonal and unitary matrices. It is easy to tell if a matrix is symmetric or Hermitian. Use "tsp(A)*A" to test if a matrix is orthogonal or unitary (check if the result is identity), and use "tsp(A)*A-A*tsp(A)" to test if a matrix is normal (check if the result is the zero matrix).

For a given linear operator \({\bf T}\) on a complex finite-dimensional inner product space, find its matrix representation A relative to the usual basis, and test if A is normal. If A is normal,\({\bf T}\) can be represented by a diagonal matrix. Directly test if a matrix or linear operator is diagonalizable by the function "tof(A,0)", and it returns "True" or "False".

Computing Diagonalization The function "dgl(A)" diagonalizes the matrix \(A\) and returns two matrices \(P\) and \(D\) such that \(D=P^{-1}AP\), where \(D\) is diagonal, and \(P\) contains the eigenvectors of \(A\). The function "dgl" (diagonalization) has two additional parameters dgl(A, normalize, PD). By default, the function returns two matrices \(P,D\) and the columns of \(P\) are not normalized (normalize = 0). If normalize = 1, the columns of \(P\) are normalized. If \(A\) is normal, then \(P\) is an orthogonal or unitary matrix. The function returns the matrix \(P\) if PD = 1, and returns the matrix \(D\) if PD = 2. For example, "inv(dgl(A,0,1))*A*dgl(A,0,1)" verifies the relation \(P^{-1}AP=D\).

III. Jordan Canonical Form

A non-normal operator \({\bf T}\) on a complex finite-dimensional inner product space V has a relatively simple form—Jordan canonical form, or \({\bf T}\) can be represented by an upper triangular matrix relative to an orthonormal basis of V.

If \(A\) is an arbitrary n-square complex matrix, there exists a unitary matrix \(P\) such that \(A\) is similar to a Jordan canonical form \(J=P^*AP\), where \(J\) is an upper triangular matrix with the eigenvalues of \(A\) on the diagonal. Note that if \(A\) has n linearly independent eigenvectors, \(A\) is diagonalizable, and \(J\) is diagonal.

Computing Jordan Canonical Form The function "jcf(A)" returns matrices P and J (Jordan canonical form) for which J = P^-1AP.

Examples mat;jcf(A);A;[(2,3),(3,2)] || mat;jcf(A);A;[[6,5,-2,-3],[-3,-1,3,3],[2,1,-2,-3],[-1,1,5,5]] || mat;jcf(A);A;[(5,4,2,1),(0,1,-1,-1),(-1,-1,3,0),(1,1,-1,2)]

IV. Linear Forms (Functionals), Dual Space, and Dual Basis

Linear Forms The scalar field \(K\) itself is viewed as a vector space. A mapping \(ϕ:V→K\) is a linear form if \(ϕ(a{\bf u}+b{\bf v})=aϕ({\bf u})+bϕ({\bf v})\).

Dual Space The set of linear forms on a vector space V over the field \(K\) is a vector space over \(K\), and it is called the dual space of V, denoted by V\(^*\). If V = \(K^n\), any linear form in V\(^*\) has the representation \(ϕ(x_1,x_2,\cdots,x_n)=a_1x_1+a_2x_2+\cdots+a_nx_n\).

Dual Basis If the dimension of \(V\) is n, then the dimension the dual space \(V^*\) is also n. Each basis of \(V\) determines a basis of \(V^*\). Suppose \(\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) is a basis of \(V\) and \(ϕ_1,ϕ_2,\cdots,ϕ_n∈V^*\). Define \(ϕ_i({\bf v}_i)\) = 1 if \(i=j\) and 0 otherwise. Then \(\{ϕ_1,ϕ_2,\cdots,ϕ_n\}\) forms a basis of \(V^*\). Thus, to determine the dual basis \(\{ϕ_1,ϕ_2,\cdots,ϕ_n\}\), solve the n linear systems one by one.

Computing Dual Basis For a given basis of V, to find a basis of the dual space V\(^*\), form a matrix A whose columns are the given basis vectors, use "rref(mrg(A,eye(n))" to obtain the solutions at the same time for the n linear systems.

Examples
(1) Given that { (2,-5), (1,3) } is a basis of \(R^2\), find the dual basis \(\{ϕ_1,ϕ_2\}\). By definition, we seek linear forms \(ϕ_1(x,y)=ax+by\),
\(ϕ_2(x,y)=cx+dy\) such that the images of the basis vector on \(ϕ_1,ϕ_2\) are 1 and 0's. Solve the two linear system (one for \(ϕ_1\) and the other for \(ϕ_2\)) by "mat;rref(mrg(tsp(A),eye(2)));A;[(2,-5),(1,3)]", where the columns of A are the given basis vectors. Thus, \(ϕ_1(x,y)=\frac{1}{11}(3x-y),ϕ_2(x,y)=\frac{1}{11}(5x+2y)\) form the dual basis.

(2) Find the basis \(\{ϕ_1,ϕ_2,ϕ_3\}\) that is dual to a basis { (2,-1,1),(1,3,2),(2,4,5) } of \(R^3\). Form a matrix A whose columns are the given basis vectors, and solve the three systems by "mat;rref(mrg(tsp(A),eye(3)));A;[(2,-1,1),(1,3,2),(2,4,5)]". Thus, \(ϕ_1(x,y,z)=\frac{1}{13}(7x-y-2z)\),
\(ϕ_2(x,y,z)=\frac{1}{13}(9x+8y-10z),ϕ_3(x,y,z)=\frac{1}{13}(-5x-3y+7z)\) form the dual basis.

V. Bilinear, Quadratic, and Hermitian Forms

Bilinear Forms A mapping \(f:V×V→K\) is a bilinear form on a finite dimensional vector space \(V\) over a field \(K\) if \(f(a{\bf u}_1+b{\bf u}_2,{\bf v})=af({\bf u}_1,{\bf v})+bf({\bf u}_2,{\bf v})\) and \(f({\bf u},a{\bf v}_1+b{\bf v}_2,)=af({\bf u},{\bf v}_1)+bf({\bf u},{\bf v}_2)\) for \(a,b∈K\) and \({\bf u}_i,{\bf v}_i∈V\).

The dot product on \(R^n\) is a bilinear form \(f({\bf u},{\bf v})={\bf v}^T{\bf u}\). If \(g,h\) are linear forms on V, then \(f({\bf u},{\bf v})=g({\bf u})h({\bf v})\) defines a bilinear form. In general, a bilinear has the form \(f({\bf x},{\bf y})={\bf x}^TA{\bf y}=\sum_{i,j}a_{ij}x_iy_j\) for \(A=[a_{ij}]\).

Let \(S=\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) be a basis of V and \({\bf u},{\bf w}∈\) V. Then \({\bf u}=a_1{\bf v}_1+\cdots,a_n{\bf v}_n,{\bf w}=b_1{\bf v}_1+\cdots,b_n{\bf w}_n\), and \(f({\bf u},{\bf w})=f(a_1{\bf v}_1+\cdots+a_n{\bf v}_n,b_1{\bf v}_1+\cdots+b_n{\bf v}_n)=\sum_{i,j}a_ib_jf({\bf v}_i,{\bf v}_j)=[{\bf u}]_S^TA[{\bf v}]_S\), where \(A=f({\bf v}_i,{\bf v}_j)\) is the matrix representation of \(f\) relative to \(S\) and \([{\bf u}]_S=(a_1,\cdots,a_n),[{\bf v}]_S=(b_1,\cdots,b_n)\).

Symmetric Bilinear and Quadratic Forms A bilinear form \(f\) on V is symmetric if \(f({\bf u},{\bf v})=f({\bf v},{\bf u})\) for every \({\bf u},{\bf v}∈\) V. If \(f\) is symmetric on V, there exists a basis \(\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) of V in which \(f\) can be represented by a diagonal matrix, where \(f({\bf v}_i,{\bf v}_j)=0\) for \(i≠j\). Alternatively, for a symmetric matrix \(A\), there exists a nonsingular matrix \(P\) such that \(A\) is congruent to a diagonal matrix \(D=P^TAP\).

Note that every other diagonal matrix representation of a bilinear form \(f\) has the same number of positive entires and the same number of negative entries. Also note that rank\((f)=p+n\) and the signature of \(f\) is \(p-n\), where \(p\) is the number of positive entries and \(n\) the number of negative entries of a diagonal matrix representation of \(f\).

A quadratic form is a symmetric bilinear form \(q({\bf x})=f({\bf x},{\bf x})={\bf x}^TA{\bf x}\). If \(q\) is positive definite, \(q({\bf x})>0\) for every \({\bf x}≠{\bf 0}\), or the diagonal entries of the diagonal representation of \(q\) are all positive.

Hermitian Forms A mapping \(f:V×V→C\) is a Hermitian form if it is linear in the first variable and conjugate linear in the second variable. That is, \(f({\bf u},{\bf v})=\overline{f({\bf v},{\bf u})}\), and \(f({\bf u},a{\bf v}_1+b{\bf v}_2)=\overline{f(a{\bf v}_1+b{\bf v}_2,{\bf u})}=\bar{a}f({\bf u},{\bf v}_1)+\bar{b}f({\bf u},{\bf v}_2)\) for \(a,b∈C, {\bf u},{\bf v}_i∈V\). The matrix representation of a Hermitian form \(f\) relative to a basis \(\{{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_n\}\) of \(V\) is a Hermitian matrix \(H=[h_{ij}]=[f({\bf v}_i,{\bf v}_j)]\) with real diagonal entries.

Diagonalizing Bilinear (Symmetric), Quadratic and Hermitian forms To diagonalize a symmetric bilinear form \(f\), find the matrix representation \(A\) of \(f\) in a basis, and then use the following procedure to determine the nonsingular matrix \(P\). Form a matrix M = [A, I], apply a sequence of elementary row operations to M, and then apply the corresponding column operators to A (the left of M). This is equivalent to pre-multiplying \(A\) by \(Q\) and post-multiplying \(A\) by \(Q^*\), and thus \(D=QAQ^*\) or \(D=P^*AP\), where \(P=Q^*\). Note that a nonsingular matrix \(Q\) is a product of elementary matrices.

The above procedure for diagonalizing a symmetric matrix is much simpler than orthogonally diagonalizing a symmetric matrix. It does not involving computing eigenvalues and eigenvectors.

Examples
(1) Express the bilinear form \(f({\bf u},{\bf v})=2x_1y_1-5x_1y_2+4x_1y_3+6x_2y_2+x_2y_1-7x_2y_3+9x_3y_3\) in matrix equation by "mat;tsp(u)*A*v;A;[(2,-5,4),(1,6,-7),(0,0,9)];u;[a,b,c];v;[r,s,t]". Note that \(f({\bf u},{\bf v})={\bf u}^TA{\bf v}\) for A = [(2,-5,4),(1,6,-7),(0,0,9)].

One the other hand, if A = [(2,3,-5),(3,1,6),(-5,6,4)], the quadratic form u\(^T\)Au determined by the symmetric matrix A can be obtained by "mat;tsp(u)*A*u;A;[(2,3,-5),(3,1,6),(-5,6,4)];u;[x,y,z]".

(2) Given a bilinear form \(f({\bf u},{\bf v})=3x_1y_1+2x_1y_2-6x_1y_3+4x_2y_2-x_2y_1+8x_2y_3-7x_3y_1+5x_3y_3\) on \(R^3\), to find the matrix representation B of \(f\) relative to the basis S = { (2,1,1),(1,2,1),(-1,1,1) }, form a matrix M whose rows are the given basis, and form a matrix A whose rows are the coefficients of \(f\). Obtain the matrix from "mat;M*A*tsp(M);M;[(2,1,1),(1,2,1),(-1,1,1)];A;[(3,2,-6),(-1,4,8),(-7,0,5)]", which gives B = MAM\(^T\).

Find the matrix representation C of \(f\) in the basis S' = { (1,0,1),(1,1,1),(1,1,0) } by "mat;G*A*tsp(G);G;[(1,0,1),(1,1,1),(1,1,0)];A;[(3,2,-6), (-1,4,8),(-7,0,5)]". Determine the change of basis matrix P from S to S' by "mat;rref(mrg(tsp(M),tsp(G)));M;[(2,1,1),(1,2,1),(-1,1,1)];G;[(1,0,1),(1,1,1),(1,1,0)]". Verify C = [(-5,-3,-2),(2,8,1),(4,10,8)] = P\(^T\)BP by "mat;tsp(P)*B*P;B;[(5,15,11),(13,29,31),(-1,12,32)];P;[(4/3,2/3,-1/3),(-1,0,1),(2/3,1/3,-2/3)]".

(3) Find a nonsingular matrix P such that D = P\(^T\)AP is diagonal for a symmetric matrix A = [(2,-3,5),(-3,4,1),(5,1,6)]. Form a matrix M = [A, I] and reduce M to echelon form "mat;elf(mrg(A,eye(3)));A;[(2,-3,5),(-3,4,1),(5,1,6)]". Observe that the matrix Q = [(1,0,0),(3,2,0),(-46,-34,-2)] is a lower triangular matrix at the right half of M, and P = Q\(^T\). Verify D by "mat;P*A*tsp(P);P;[(1,0,0),(3,2,0),(-46,-34,-2)];A;[(2,-3,5),(-3,4,1),(5,1,6)]". Note that p = 2, n = 1, rank(A) = 3, and the signature of A is 1.

(4) Determine if the quadratic form \(q(x,y)=x^2-3xy+5y^2\) is positive definite. Given that the coefficients are a = 1, b = -3, c = 5, a > 0 and b\(^2\) - 4ac = -11 < 0. Thus, \(q(x,y)> 0\) is positive definite. If \(q(x,y)=3x^2+2xy-4y^2,q\) is not positive definite because b\(^2\) - 4ac = 52 > 0.

Determine if \(q(x,y,z)=x^2-2xy+4xz+5y^2+6yz-6z^2\) is positive definite by "mat;elf(mrg(A,eye(3)));A;[(1,-1,2),(-1,5,3),(2,3,-6)]" and "mat;P*A*tsp(P);A;[(1,-1,2),(-1,5,3),(2,3,-6)];P;[(1,0,0),(1,1,0),(-13,-5,4)]", where the first result gets P, and the second finds D that has one negative diagonal entry. Thus, \(q\) is not positive definite. Also "mat;elf(tsp(elf(A)));A;[(1,-1,2),(-1,5,3),(2,3,-6)]" returns a diagonal matrix with one negative diagonal entry.

(5) Let \(q(x,y,z)=2x^2-2xy+6xz-4yz+7y^2-5z^2\) be a quadratic form. Substitute \(x=2r-3s+t,y=r-s+2t\),
\(z=5r+2s+3t\). Find \(q(r,s,t)\) in matrix equation by "mat;tsp(v)*tsp(P)*A*P*v;A;[(2,-1,3),(-1,7,-2),(3,-2,-5)];P;[(2,-3,1),(1,-1,2),(5,2,3)];v;[r,s,t]", where A is the matrix representation of \(q\) in the standard basis, and P consists of the coefficients corresponding to the substitution.

If the substitution \(x=r+s-38t,y=2s+2t,z=26t\), the matrix P for substitution is P = [(1,1,-38),(0,2,2),(0,0,26)], and "mat;tsp(v)*tsp(P)*A*P*v;A;[(2,-1,3),(-1,7,-2),(3,-2,-5)]; P;[(1,1,-38),(0,2,2),(0,0,26)];v;[r,s,t]" returns a quadratic form \(q(r,s,t)\) without cross product terms, because P\(^T\)AP is diagonal. Check the matrix Q by "mat;elf(mrg(A,eye(3)));A;[(2,-1,3),(-1,7,-2),(3,-2,-5)]" and P = Q\(^T\).

(6) Find a nonsingular matrix \(P\) such that \(D=P^*HP\) is diagonal, where \(H\) = [(2,6*I,-I),(-6*I,1,4+5*I),(I,4-5*I,5)] is Hermitian matrix. The matrix Q is at the right half of the result by "mat;elf(mrg(H,eye(3)));H;[(2,6*I,-I),(-6*I,1,4+5*I),(I,4-5*I,5)]". Verify the \(D=QHQ^*=P^*HP\) by "mat;q*A*tsp(q);A;[(2,6*I,-I),(-6*I,1,4+5*I),(I,4-5*I,5)];q;[(1,0,0),(6*I,2,0),(-60-50*I,-28+20*I,-68)]".

Find a nonsingular matrix such that D = P\(^*\)HP is a diagonal for H = [(2,-3*I),(3*I,1)] by "mat;elf(mrg(H,eye(2)));H;[(2,-3*I),(3*I,1)]" and "mat;tsp(P)*H*P;H;[(2,-3*I),(3*I,1)];P;[(1,3*I),(0,2)]".

(7) Find a nonsingular linear substitution so that \(q(x,y,z)=x^2+2xy-4xz+5y^2+6yz+7z^2\) is diagonal by "mat;elf(mrg(A,eye(3)));A;[(1,1,-2),(1,5,3),(-2,3,7)]". Observe that P = [(1,-1,13),(0,1,-5),(0,0,4)]. Thus, substitute \(x=r-s+13t,y=s-5t,z=4t\) to obtain \(q(r,s,t)=r^2+4s^2-52t^2\).

Q: mat;dgl(A);A;[(4,1,-1),(2,5,-2),(1,1,2)] A: P \(=\begin{bmatrix}-1&1&0\\1&0&2\\0&1&1\end{bmatrix}\) D \(=\begin{bmatrix}3&0&0\\0&3&0\\0&0&5\end{bmatrix}\)

Q: mat;dgl(A,1);A;[(4,1,-1),(2,5,-2),(1,1,2)] A: P \(=\begin{bmatrix}\frac{-\sqrt{2}}{2}&\frac{\sqrt{2}}{2}&\frac{\sqrt{6}}{6}\\\frac{\sqrt{2}}{2}&0&\frac{\sqrt{6}}{3}\\0&\frac{\sqrt{2}}{2}&\frac{\sqrt{6}}{6}\end{bmatrix}\) D \(=\begin{bmatrix}3&0&0\\0&3&0\\0&0&5\end{bmatrix}\)

Q: mat;inv(dgl(A,0,1))*A*dgl(A,0,1); A;[(4,1,-1),(2,5,-2),(1,1,2)] A: \(=\begin{bmatrix}3&0&0\\0&3&0\\0&0&5\end{bmatrix}\)

back to top

Matrix Decompositions

A matrix decomposition is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.

Common matrix decomposition methods are implemented, and they are (1) QR-decomposition ("qrd"), (2) LU-decomposition ("lud"), (3) singular value decomposition ("svd"), (4) Cholesky decomposition ("cho"), (5) rank decomposition ("rkd"), and (6) LDL-decomposition ("ldl").

1. QR Decomposition An m × n matrix \(A\) with linearly independent columns can be factored as \(A=QR\), where \(Q\) is an m × n matrix with orthonormal column vectors, and \(R\) is an n × n matrix invertible upper triangular matrix. The matrix \(Q\) can be obtained by applying the Gram-Schmidt process to the columns of \(A\). Suppose \(A=[{\bf u}_1,{\bf u}_2,\cdots,{\bf u}_n],Q=[{\bf q}_1,{\bf q}_2,\cdots,{\bf q}_n]\). Express \({\bf u}_i\) as a linear combination of the orthonormal set \(\{{\bf q}_1,{\bf q}_2,\cdots,{\bf q}_n\}\) and write the n equations in matrix form \(A=[{\bf u}_1,{\bf u}_2,\cdots,{\bf u}_n]=[{\bf q}_1,{\bf q}_2,\cdots,{\bf q}_n]R\), where \(R=[r_{ij}]=[⟨{\bf u}_i,{\bf q}_i⟩]\). Because \({\bf q}_i\) is orthogonal to \({\bf u}_1,{\bf u}_2,\cdots,{\bf u}_{i-1}\) for \(i>2\) and the columns of \(A\) are linearly independent, \(R\) is an invertible upper triangular matrix.

2. LU Decomposition Factor a square nonsingular matrix \(A\) into a lower and upper triangular matrices, \(A=LU\), where \(L\) is a lower triangular matrix with ones on the diagonal, and \(U\) is an upper triangular matrix with nonzero on the diagonal. The matrices \(L\) and \(U\) can be obtained by elementary row-addition operations.

3. Singular Value Decomposition An m × n matrix \(A\) can be expressed as \(A=UΣV\), where \(U,V\) have orthonormal columns, and \(Σ\) is an m × n nonnegative diagonal matrix whose entries are the singular values of \(A\) (the square roots of the eigenvalues of \(A^*A,σ_i=\sqrt{λ_i}\)). The columns of \(V=[{\bf v}_1,{\bf v}_2,\cdots,{\bf v}_k]\) are ordered according to the singular values \(σ_1≥σ_2≥\cdots,σ_k>0\), and the columns of \(U=[{\bf u}_1,{\bf u}_2,\cdots,{\bf u}_k]\) consists of the orthonormal basis for the column space of \(A\), where \({\bf u}_i=\frac{A{\bf v}_i}{||A{\bf v}_i||}=\frac{1}{σ_i}A{\bf v}_i\).

4. Cholesky Decomposition Factor a Hermitian and positive definite matrix \(A\) as \(A=LL^*\), where \(L\) is a nonsingular lower triangular matrix with real and positive diagonal entries. Cholesky decomposition is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for efficient numerical solutions.

5. Rank Decomposition Decomposition Factor an m × n matrix \(A\) with rank r as \(A=CF\), where \(C\) is an m × r matrix and \(F\) is an r × n matrix. Since rank\((A)\) = r, \(A\) has the r linearly independent columns \({\bf c}_1,{\bf c}_2,\cdots,{\bf c}_r\), and they form a basis of the column space of \(A\). Suppose \(A=[{\bf a}_1,{\bf a}_2,\cdots,{\bf a}_n]\). Then \({\bf a}_j=f_{1j}{\bf c}_1+f_{2j}{\bf c}_2+\cdots+f_{rj}{\bf c}_r\) for \(j=1,\cdots,n\). Then the columns of \(A\) that correspond to \({\bf c}_1,{\bf c}_2,\cdots,{\bf c}_r\) form the matrix \(C\) and the coefficients of \(f_{ij}\) form the matrix \(F\). The matrix \(F\) is the nonzero rows of the reduced row echelon form of \(A\). If \(A\) is nonsingular, \(C=A\) and \(F=I\). Note that rank decomposition is not unique.

6. LDL Decomposition Factor a Hermitian and positive definite matrix \(A\) as \(A=LDL^*\), where \(L\) is a lower triangular matrix with ones on its diagonal. LDL Decomposition is closely related to Cholesky decomposition, and it can be expressed as \(A=LD^{\frac{1}{2}}(LD^{\frac{1}{2}})^*=CC^*\).

Computing Matrix Decomposition The function "qrd(A)" finds the matrices Q and R such that A = QR, and the function "lud(A)" factors A as A = LU. In a similar fashion, the function "ldl(A)" returns matrices L and D for A = LDL^*; "cho(A)" returns matrix L for A = LL\(^*\); "svd(A)" returns matrices U, Σ and V for A = UΣV; "rkd(A)" returns matrices C and F for A = CF.

Examples mat;qrd(A);A;[(2,5,14),(6,9,-11)] || mat;lud(A);A;[(4,3,2),(1,6,3),(2,5,1)] || mat;rkd(A);A;[(9,5,4),(3,5,1)] || mat;ldl(A);A;[(4,12,-16),(12,37,-43),(-16,-43,98)] || mat;cho(A);A;[(4,12,-16),(12,37,-43),(-16,-43,98)] || mat;svd(A);A;[(1,1,0),(1,0,1)] ||.

Q: mat;qrd(A);A;[[1,1,1],[1,1,3],[2,3,4]] A: Q \(=\begin{bmatrix}\frac{\sqrt{6}}{6} & - \frac{\sqrt{3}}{3} & - \frac{\sqrt{2}}{2}\\\frac{\sqrt{6}}{6} & - \frac{\sqrt{3}}{3} & \frac{\sqrt{2}}{2}\\\frac{\sqrt{6}}{3} & \frac{\sqrt{3}}{3} & 0\end{bmatrix}\) R = \(\begin{bmatrix}\sqrt{6} & \frac{4 \sqrt{6}}{3} & 2 \sqrt{6}\\0 & \frac{\sqrt{3}}{3} & 0\\0 & 0 & \sqrt{2}\end{bmatrix}\)	Q: mat;lud(A);A;[[4,3,2],[1,6,3],[2,5,1]] A: L \(=\begin{bmatrix}1 & 0 & 0\\\frac{1}{4} & 1 & 0\\\frac{1}{2} & \frac{2}{3} & 1\end{bmatrix}\) U \(=\begin{bmatrix}4 & 3 & 2\\0 & \frac{21}{4} & \frac{5}{2}\\0 & 0 & - \frac{5}{3}\end{bmatrix}\)	Q: mat;ldl(A);A;((25,15,-5),(15,18,0),(-5,0,11)) A: L \(=\begin{bmatrix}1 & 0 & 0\\\frac{3}{5} & 1 & 0\\- \frac{1}{5} & \frac{1}{3} & 1\end{bmatrix}\) D \(=\begin{bmatrix}25 & 0 & 0\\0 & 9 & 0\\0 & 0 & 9\end{bmatrix} \)
Q: mat;cho(A);A;((25,15,-5),(15,18,0),(-5,0,11)) A: L \(=\begin{bmatrix}5 & 0 & 0\\3 & 3 & 0\\-1 & 1 & 3\end{bmatrix}\)	Q: mat;svd(A);A;[(1,2),(2,1)] A: U \(=\begin{bmatrix}\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2}\\- \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2}\end{bmatrix}\) Σ \(=\begin{bmatrix}1 & 0\\0 & 3\end{bmatrix}\) V \(=\begin{bmatrix}- \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2}\\\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2}\end{bmatrix}\)	Q: mat;rkd(A);A;[(1,1,1),(1,2,2)] A: C \(=\begin{bmatrix}1&1\\1&2\end{bmatrix}\) F \(=\begin{bmatrix}1&0&0\\0&1&1\end{bmatrix}\)

back to top