Column Space And Null Space Of A Matrix With Geometry

Here's a sample question from "Advanced Engineering Mathematics",

The above question gave rise to the rest of what I would talk about in this article. It was an unassuming question, because at first glance I considered it simple, because there was nothing deeply intuitive to think about except finding two matrices that would be my counterexamples (alluding the "Show the following" part). But as I attempted the question, the rest of what became of this article surfaced, and I decided to jot it down.

Vectors as units of Matrices

Depending on your familiarity with Linear Algebra, this might be simply intuitive. Every row and column of a matrix can be observed as a vector, either as a row vector or column vector. You can grow a matrix row-wise by combining existing rows through linear combination, and conversely, same holds true for column-wise growth. Also, adding arbitrary vectors either ways is a viable growth approach as well, but for the article sake, I want to establish relationships across the rows/columns, hence I will stick with linear combination of rows and columns.

Row-wise growth

For a single row vector, $R_1 = \begin{pmatrix} 1 & 2 \end{pmatrix}$. I can grow it into a $2 \times 2$ matrix by creating a second row, $R_2$, that is a linear combination of $R_1$ (for instance, multiplying it by 2):

Combining them row-wise yields:

Because the second row depends entirely on the first row, I grew the matrix vertically without adding new, independent directional information.

Column-wise growth

Now, I can apply the same logic horizontally using columns. Starting with a single column vector, $C_1 = \begin{pmatrix} 1 \\ 2 \end{pmatrix}$. I can grow the matrix by creating a second column, $C_2$, through a linear combination (for instance, multiplying it by 2):

Combining them column-wise yields:

By placing the dependent vector to the right of my original column, the matrix grows horizontally while maintaining a strict linear relationship between its columns.

What is the Rank of a Matrix?

Keep in mind how I grew the matrix to add more rows or columns. In a sense, I did not add any new information that isn't known already. You can think of rank as the "score" of unique, independent information contained within a matrix.

In the row-wise growth example, I started with one row and generated a second row by multiplying the first by 2. The second row ($R_2$) brings no new information to the table, because it is basically a multiple of a unit vector (which can be simplified back to $R_1$). Therefore, the rank of this matrix is 1.

The exact same logic applies to the column-wise growth example. I doubled one column to make the second, looking at the matrix horizontally, $C_2$ is just a shadow of $C_1$. There is only 1 independent column of data.

A beautiful fact of linear algebra is that the number of independent rows always equals the number of independent columns.

If a matrix has a size of $2 \times 2$ but a rank of $1$, it means the matrix grew in size, but it failed to grow in unique information.

What a Matrix does to a Vector

Earlier, I multiplied a vector by a scalar value to form new rows and columns of a matrix. What about when I multiply a $\text{matrix A}$ by a $\text{vector x}$, to get a new $\text{vector y} = Ax$? The matrix transforms the input vector into an output vector. Do we have an observable relationship? From here henceforth I will focus on the columns. Let's use $\text{matrix A}$ (where $R_2 = 2R_1$):

The $\text{vector } (3, 1)^T$ went in, and the $\text{vector } (5, 10)^T$ came out. If I was to add the output (call it $C_3$) to the matrix, growing it into:

Looking at the matrix-vector multiplication, $C_3 = 5C_1$ (without prior knowledge, one could imagine that I scaled $C_1$ by 5 to get $C_3$). Hence, matrix-vector multiplication is also a linear combination of the columns of a matrix.

Whichever linear combination I chose, be it scalar value or a matrix-vector multiplication, there's an observable linear relationship. The columns $(1,2)^T,(2,4)^T,(5,10)^T$ are all scalar multiples (linear combinations) of $(1,2)^T$. Geometrically speaking, this is a line through the origin in ℝ², in the direction of $(1,2)^T$.

Now, what if I try the below input:

The $\text{vector } (-2, 1)^T$ went in, and zero came out. The matrix completely annihilated that input, erasing all information. Same goes for $(-4, 2)^T$:

Also erased, and same goes for $(6, -3)^T$, you can try it yourself. All three of these are scalar multiples of $(-2, 1)^T$. Every vector along that direction gets annihilated by $A$.

The values might look arbitrarily chosen, but they are not. What values of $\mathbf{x}$ would you substitute into $A\mathbf{x} = 0$ to make the equality true? Solve $A\mathbf{x} = 0$:

Both equations say the same thing: $\mathbf{x_1} = -2\mathbf{x_2}$, meaning $\mathbf{x_2}$ is free. Let $\mathbf{x_2} = \mathbf{t}$, then:

Also, why does this matter one might ask? This means this sample matrix has a blind spot. There's an entire line of input vectors that $A$ cannot distinguish from the zero vector. If I handed you the output $A\mathbf{x} = 0$, you can't tell whether the original input was $(-2, 1)^T$ or $(-4, 2)^T$ or $(6, -3)^T$ or any other multiple of $(-2, 1)^T$. Information along that direction was permanently destroyed by the multiplication.

So there is a relationship between information preserved and information destroyed. Let me use the identity matrix (a $2 \times 2$ matrix with rank 2) to show you the contrast.

Both columns $(1, 0)^T$ and $(0, 1)^T$ are independent and their possible linear combinations $\alpha(1, 0)^T + \beta(0, 1)^T$ produces every vector in ℝ².

Solving $A\mathbf{x} = 0$:

This gives $\mathbf{x_1} = 0, \mathbf{x_2} = 0$. The only vector that $A$ sends to zero is the zero vector itself. That makes sense because the identity matrix doesn't destroy any information.

The pattern is clear. The rank-1 matrix had an entire line of inputs it annihilated. The rank-2 matrix annihilated nothing except the zero vector. The higher the rank, the less the matrix destroys.

Let me push this into three dimensions for a different perspective. Here is a $3\times2$ matrix with rank 2:

Row reduce $R_3 - R_1 - R_2$ gives $(0, 0)$, hence $rank(A) = 2$. The columns are $(1, 0, 1)^T$ and $(0, 1, 1)^T$ and are equally independent of each other. Both columns live in ℝ³ (3 components per column). Geometrically they form a plane through the origin in ℝ³, but not all of ℝ³ (just a flat 2-dimensional slice of it). For instance, the vector $(1, 1, 2)^T$ is reachable ($C_1 + C_2$), but the vector $(0, 0, 1)^T$ is not (you'd need $a = 0$, $b = 0$ from the first two components, but then the third component would be 0, not 1).

Then solving $A\mathbf{x} = 0$:

From the first two equations, $x_1 = x_2 = 0$. The blind spot is only the zero vector, meaning no information is lost. Again, the higher the rank, the less information is lost.

What a Matrix does to a Matrix

I showed that matrix-vector multiplication is a linear combination of $A$'s columns. Matrix-matrix multiplication is the same thing, just repeated. Each column of the second matrix acts as a separate input vector, and $A$ transforms each one independently.

I will reuse the rank-1 matrix and multiply it by itself:

The first column of $A^2$ is $A$ times the first column of $A$:

The second column of $A^2$ is $A$ times the second column of $A$:

So:

Both output columns, $(5, 10)^T$ and $(10, 20)^T$, are scalar multiples of $(1, 2)^T$. The same line, the same direction, and the outputs of $A^2$ land on the exact same line as the outputs of $A$.

What happened from the row perspective? Each row of $A^2$ is a linear combination of the rows of the second matrix. But for understanding what the matrix preserves and what it destroys, the column perspective is where the geometry lives. So I'll keep my focus there.

The observation here is that when I square a matrix, I am feeding the outputs of $A$ back into $A$ as inputs. The question becomes, what does $A$ do to vectors that it already produced? Does it preserve them, or does it destroy them? The answer depends on where those output vectors land relative to what the matrix annihilates.

Column Space and Null Space

By now, I have been pointing at two collections of vectors without formally naming them. Allow me to fix that.

The collection of all possible output vectors that a matrix can produce is called its column space. For our rank-1 matrix $A = \begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$, every output is a scalar multiple of $(1, 2)^T$. The column space is that entire line through the origin. It is the set of all reachable destinations. Any vector in ℝ² that does not sit on this line is unreachable by $A$. No input exists that would produce it.

The collection of all input vectors that $A$ annihilates (maps to zero) is called its null space. For the same matrix, every multiple of $(-2, 1)^T$ gets erased. The null space is that entire line through the origin. It is the set of all blind spots. Any input along this direction is invisible to the matrix.

Both of these collections are vector spaces (closed under linear combination), not just arbitrary sets. If $A$ erases $\mathbf{x}$ and $A$ erases $\mathbf{y}$, then $A$ erases any combination $\alpha\mathbf{x} + \beta\mathbf{y}$, because $A(\alpha\mathbf{x} + \beta\mathbf{y}) = \alpha A\mathbf{x} + \beta A\mathbf{y} = 0 + 0 = 0$.

The rank of a matrix is the dimension of its column space. It tells you how many independent directions the matrix can produce. The nullity of a matrix is the dimension of its null space. It tells you how many independent directions the matrix destroys. And the two always add up to the number of columns:

For our rank-1 matrix in ℝ², the column space is 1-dimensional (one line of outputs), the null space is 1-dimensional (one line of erased inputs), and $1 + 1 = 2$ columns.

For the identity matrix, the column space is 2-dimensional (all of ℝ²), the null space is 0-dimensional (only the zero vector), and $2 + 0 = 2$ columns.

Hopefully it is clearer now when I say "a matrix with more rank preserves more and destroys less, while a matrix with less rank preserves less and destroys more".

Now let me get back to the original question.

Let's solve the question

Take two $2 \times 2$ matrices, both with rank 1. Say $\text{matrix A and B}$, and for $\text{matrix A}$:

$rank(A) = 1$. One independent row, one zero row (which adds no new information). Let's compute $A^2$:

$rank(A^2) = 1$. Squaring did nothing as $A^2 = A$.

For $\text{matrix B}$:

$rank(B) = 1$. One independent row, one zero row. Compute $B^2$:

$rank(B^2) = 0$. Squaring removed all information inside the matrix, resulting into a zero matrix.

These two matrices give the counterexample:

$rank(A) = rank(B) = 1$, but $rank(A^2) = 1 \neq 0 = rank(B^2)$.

But why did squaring preserve $A$'s rank while destroying $B$'s entirely? This is where the column space and null space make the answer visible.

Why it happened

For matrix $A$, the column space is $\text{span}\{(1, 0)^T\}$ (that is the $x_1$-axis), and the null space is $\text{span}\{(0, 1)^T\}$ (which is the $x_2$-axis). These two lines are completely separate, and they share nothing except the zero vector.

When I square $A$, I feed $A$'s output back into $A$. The output lives on the $x_1$-axis (the column space), but the $x_1$-axis is not the null space. $A$ does not annihilate vectors on the $x_1$-axis. So the output survives the second application, hence information is preserved. $A^2 = A$.

For matrix $B$, the column space is $\text{span}\{(1, 0)^T\}$ (also the $x_1$-axis), but the null space is also $\text{span}\{(1, 0)^T\}$ (which is also the $x_1$-axis). They are the same line.

When I square $B$, I feed $B$'s output back into $B$. The output lives on the $x_1$-axis (the column space). But the $x_1$-axis is also the null space, hence $B$ annihilates every vector on that line. So the output of the first application gets completely erased by the second application. $B^2 = 0$.

Same rank, same column space dimension, same null space dimension, but the geometric relationship between the two spaces is entirely different. In $A$, the column space and null space point in different directions. In $B$, they point in the same direction. That one difference determines whether squaring preserves or destroys.

Observation

Two matrices can agree on every numerical summary. Same rank, same nullity, same dimensions for every space involved. And yet they behave completely differently under the same operation, because those numbers alone do not capture the geometric configuration of the spaces relative to each other.

Rank tells you "how much" a matrix preserves, but it does not tell you "where" the preserved information lands relative to what the matrix destroys. That "where" is the relationship between the column space and the null space. When the column space avoids the null space, information survives repeated application. When the column space falls inside the null space, information self-destructs on contact.

A simple question about counterexamples opened up into the geometric structure underneath matrix multiplication. The answer to "give a counterexample" was two lines of computation, but the answer to "why does this counterexample work" required me pausing and observing intuitively what a matrix preserves, what it destroys, and how those two collections of vectors sit relative to each other in space.