1. The Geometry of Linear Equations
MIT OpenCourseWare39:482019-09-24English
Chapters
The main idea of the first MIT linear algebra lecture is to read Ax=b as weighting the columns of A by the components of x, then asking whether those columns can be combined to produce the right-hand-side vector b.
Overview
1. The Geometry of Linear Equations
- Strang introduces the basic problem of solving systems of linear equations and uses a two-variable example to introduce the row picture, the column picture, and Ax=b.
- Plot the solution set of each equation separately and read off their common solution, (1,2), at the intersection of the two lines.
- Adding vectors tip to tail shows how the two columns combine to produce (0,3), and all their possible combinations fill the entire plane.
- The solution to the three equations is where three planes meet, while the column picture shows immediately that the right-hand side is exactly the third column, so the solution is (0,0,1).
- Strang first keeps the matrix fixed and changes the right-hand-side vector, then uses coplanar columns in three dimensions and a repeated column in nine dimensions to show why not every b can be reached.
- An example that gives (12,7) compares taking a weighted sum of the columns with taking dot products row by row, and the lecture ends with a preview of elimination in the next lecture.
Key points
- A system of linear equations is written as Ax=b, where A is the coefficient matrix, x is the vector of unknowns, and b is the right-hand-side vector.
- In the row picture, each equation has a solution set. In the two-variable example, the two lines intersect at (1,2). In the three-variable example, the three planes meet at one point.
- In the column picture, the unknowns are the weights on the column vectors. In the two-variable example, one copy of the first column plus two copies of the second gives b=(0,3).
- When A is fixed and b changes, the columns stay the same, and the goal is to find weights that produce the new b. If b is the sum of the first two columns, one solution is (1,1,0).
- Having a solution for every b is equivalent to saying that the linear combinations of A's columns fill the entire target space. If all three columns lie in the same plane, there is no solution for any b outside that plane.
- To multiply a matrix by a vector, you can either take a weighted sum of the columns or take the dot product of each row with the vector. Both methods give the same result.
Chapter notes
Course introduction and the matrix form of a system of equations
- Gilbert Strang begins teaching MIT 18.06 Linear Algebra. The textbook is Introduction to Linear Algebra, and the course website, web.mit.edu/18.06, provides exercises from previous years, MATLAB code, and the syllabus. The first lecture starts with the basic problem of linear algebra: solving a system of linear equations, beginning with the case of N equations in N unknowns.
- The same system can be viewed in three ways. The row picture looks at one equation at a time, which in the two-variable example means looking at how the lines intersect. The column picture looks at the coefficient matrix one column at a time, and Strang puts particular emphasis on this view. Matrix form uses the coefficient matrix A to represent the system algebraically.
- The two-variable example is 2x − y = 0 and −x + 2y = 3. Listing each equation's coefficients in a row, in the order of the unknowns, gives the 2×2 coefficient matrix A: the first row is (2, −1), and the second is (−1, 2).
- Put the unknowns x and y into the vector x and the numbers 0 and 3 from the right side of the equations into the vector b, and the system becomes Ax = b. Next, the lecture uses the row picture to look at and solve this example before returning to the more general problem.
The two-variable row picture: where two lines intersect
- Plot each equation's solution set separately in the xy-plane. For the first equation, 2x−y=0, setting y=0 gives the origin (0,0), and setting x=1 gives (1,2).
- Every solution to the first equation lies on the line through these two points. For example, (1/2,1) also satisfies 2x−y=0.
- The solution line for the second equation, −x+2y=3, does not pass through the origin, because the left-hand side is not 3 when x=y=0. Setting y=0 gives (−3,0), and setting x=−1 gives (−1,1). Connecting these two points gives the second line.
- The two lines intersect at (1,2), so this point solves both equations. It is already on the first line, and substituting it into the second equation gives −1+4=3.
The two-variable column picture: building the right-hand side from vectors
- For a system of two equations in two unknowns, the row picture treats the solution as the intersection of two lines. The column picture looks at the two equations together and seeks the right coefficients x and y so that the weighted sum of the two columns of A equals the right-hand-side vector b.
- In the plane, the first column is (2,−1), meaning 2 units right and 1 down. The second column is (−1,2), meaning 1 unit left and 2 up.
- Take x=1 and y=2. First draw one copy of the first column. Starting at the tip of its arrow, add two copies of the second column tip to tail, moving 1 unit left and 2 up each time.
- Adding component by component gives (2−2,−1+4)=(0,3), exactly the target right-hand-side vector b. Here, (0,3) gives the coordinates of the vector sum, not the values of the unknowns x and y.
- As x and y range over all possible values, the linear combinations of the two columns fill the entire plane, so they can be combined to produce any right-hand-side vector b.
Row and column pictures for a system in three unknowns
- The three-variable system is written in matrix form as A(x,y,z)ᵀ=b. The three rows of A are (2,−1,0), (−1,2,−1), and (0,−3,4), and the right-hand-side vector is b=(0,−1,4)ᵀ.
- In the row picture, the solution set of each equation is a plane in three-dimensional space. For example, the plane for the second equation, −x+2y−z=−1, does not pass through the origin. It can be determined by three points that satisfy the equation: (1,0,0), (0,0,1), and (0,−1/2,0).
- The first two planes intersect along a line, and the third plane meets that line at one point. This common intersection point is the solution to the system. The row picture explains what the solution means geometrically, but it is not easy to read off the coordinates of the intersection.
- In the column picture, x, y, and z are the coefficients of A's three column vectors, (2,−1,0)ᵀ, (−1,2,−3)ᵀ, and (0,−1,4)ᵀ, respectively. Solving the system means finding the coefficients that make the linear combination of these three vectors equal to b.
- In this example, b was deliberately chosen to be the third column, so setting the coefficients of the first two columns to 0 and that of the third column to 1 immediately gives the solution (0,0,1), which is also where the three planes meet. In general, you cannot read off the answer directly from the column picture. The next lecture will introduce elimination as a systematic way to solve the system.
Which right-hand-side vectors can be produced?
- Keep A fixed and add the first two columns to get a new right-hand-side vector b=(1,1,-3). The corresponding solution is (x,y,z)=(1,1,0). From the row perspective, the three new planes meet at the point given by this solution. From the column perspective, all three columns stay the same; the new b is obtained simply by adding the first and second columns.
- With A fixed, the first question is whether Ax=b has a solution for every b, not how to find the solution. If a solution exists, elimination can find it. Since Ax is the linear combination of A's columns weighted by the components of x, the question is equivalent to asking whether the linear combinations of those columns fill all of three-dimensional space.
- The matrix A in the example is nonsingular, meaning it is invertible, so every b can be produced. Conversely, if the three columns lie in the same plane, as when the third column is the sum of the first two, all their linear combinations stay in that plane.
- When the three columns lie in the same plane, there is a solution for b in the plane, but b outside it cannot be reached. In that case, A is singular and not invertible. The same question extends to nine equations in nine unknowns: can linear combinations of nine nine-dimensional column vectors produce every nine-dimensional right-hand-side vector b?
- Strang uses a 9×9 matrix randomly generated in MATLAB to show that nine columns can fill nine-dimensional space, making the matrix invertible. But if the ninth column is the same as the eighth, it adds no new direction, so some values of b cannot be produced.
- In the example where the ninth column repeats the eighth, combinations of the remaining columns might fill only an eight-dimensional plane in nine-dimensional space, not the whole space. For now, the lecture focuses on the case where every b can be produced by combining the columns.
Two ways to multiply a matrix by a vector
- A system of linear equations can be written in matrix form as Ax=b, where A is the coefficient matrix, x is the vector of unknowns, and b is the right-hand-side vector. To compare the two ways of multiplying, use a matrix A with first row (2,5) and second row (1,3), along with the vector x=(1,2).
- Working by columns, use the components of x as weights: take one copy of the first column (2,1) and add twice the second column (5,3) to get (2+10, 1+6)=(12,7).
- Working by rows, take the dot product of each row of A with x to get the corresponding component of the result. The first row gives 2×1+5×2=12, and the second gives 1×1+3×2=7.
- Both methods give the same result, but Strang recommends thinking of Ax as a linear combination of A's columns. In this 2×2 example, if the right-hand side is b=(12,7), then x=(1,2) is a solution to the system.
- The next lecture will show how to use elimination to systematically solve systems of linear equations of any size and determine when a system has no solution.
What it answers
- What do the row picture and the column picture look for?
- The row picture plots the solution set of each equation separately and looks for their common intersection. The column picture looks for weights that make the weighted sum of the coefficient matrix's columns equal to b.
- Why can you see the solution in the three-variable example without using elimination?
- The right-hand-side vector (0,-1,4) is exactly the third column of A, so assigning weights of 0 to the first two columns and 1 to the third gives the solution (0,0,1).
- When does a system in three unknowns fail to have a solution for every b?
- If all three column vectors lie in the same plane, their linear combinations stay in that plane too. So there is no solution for b outside the plane, and the matrix is not invertible.
- How do you multiply a matrix by a vector?
- You can weight the corresponding columns of the matrix by the components of the vector and add them up, or you can take the dot product of each row of the matrix with the vector to get the components of the result one at a time.
Quotes
It's asking us to find the right linear combination.
I quit on the row picture before I've successfully found the point where the three planes meet.
A times x is a combination of the columns.
Mentioned
- Gilbert Strang — The lecturer for the first MIT 18.06 Linear Algebra lecture.
- MIT 18.06 — The MIT linear algebra course this lecture is part of.
- Introduction to Linear Algebra — The course textbook Strang introduces at the start.
- 行图 — Looks at each equation's solution set one row at a time and represents the solution to the system as their common intersection.
- 列图 — Treats solving the system as finding a combination of the columns that produces the right-hand-side vector.
- 线性组合 — Multiply each vector by a scalar, then add the resulting vectors.
- 可逆矩阵 — Strang uses a 3×3 example to show that the columns of this kind of matrix can be combined to produce any right-hand-side vector.
- 点积 — When multiplying a matrix by a vector row by row, the dot product of each row with x gives one component of the result.
- 消元法 — A systematic method for solving systems that will be introduced in the next lecture and can also tell whether a system has no solution.