SOLID STATE PRESS
← Back to catalog
Singular Value Decomposition (SVD) cover
Coming soon
Coming soon to Amazon
This title is in our publishing queue.
Browse available titles
Mathematics

Singular Value Decomposition (SVD)

Singular Values, Rank-k Approximation, and the Geometry Behind PCA — A TLDR Primer

Singular value decomposition shows up in machine learning, statistics, image compression, and nearly every serious application of linear algebra — yet most courses either skip the intuition entirely or bury it under dense proofs. If you have an exam coming up, a homework set you can't crack, or you just sat through a lecture on SVD and walked out more confused than you walked in, this guide is for you.

**TLDR: Singular Value Decomposition** builds the full picture from the ground up, starting with the geometric meaning of A = UΣVᵀ — rotate, stretch, rotate — and working up through the eigendecomposition of AᵀA, fully worked 2×2 examples with sign-pitfall warnings, the Eckart-Young theorem for low-rank approximation, and the direct connection between SVD and principal component analysis (PCA). The final section surveys real-world uses: the Moore-Penrose pseudoinverse, least-squares solving, image compression, denoising, and recommender systems.

Written for high school students in advanced math courses and college freshmen in linear algebra or data science, the guide is short by design — no filler, no padding, no multi-chapter detour before the first useful idea. Every key term is defined in plain language the first time it appears, and every abstraction is grounded in a concrete worked example.

If you've been searching for a clear SVD linear algebra study guide that actually connects the formulas to the geometry, this is the one to grab. Scroll up and download it now.

What you'll learn
  • State what the SVD of a matrix is and what U, Sigma, and V represent geometrically
  • Compute the SVD of a small matrix by hand using eigenvalues of A^T A
  • Use the rank-k truncated SVD to approximate a matrix and quantify the error
  • Explain how SVD underlies principal component analysis and basic data compression
  • Identify when SVD exists, why it is unique up to sign, and how it differs from eigendecomposition
What's inside
  1. 1. What SVD Actually Says: Rotate, Stretch, Rotate
    Introduces SVD as a factorization A = U Sigma V^T and gives the geometric picture of any linear map as a rotation, an axis-aligned stretch, and another rotation.
  2. 2. Building SVD from A^T A: Where the Pieces Come From
    Derives the singular values and singular vectors from the eigendecomposition of A^T A and A A^T, and shows why singular values are non-negative.
  3. 3. Computing SVD by Hand on a 2x2
    Walks through full SVD computations on small matrices, including a non-square case, with checks and common sign pitfalls.
  4. 4. Low-Rank Approximation and the Eckart-Young Theorem
    Shows how truncating the SVD gives the best rank-k approximation under Frobenius and spectral norms, with image compression as the canonical example.
  5. 5. SVD and PCA: Same Math, Different Story
    Connects SVD of a centered data matrix to principal component analysis, explaining variance, loadings, and scores.
  6. 6. Where SVD Shows Up: Pseudoinverse, Noise, and Beyond
    Surveys the Moore-Penrose pseudoinverse, least-squares solutions, denoising, and recommender systems to show why SVD is everywhere.
Published by Solid State Press
Singular Value Decomposition (SVD) cover
TLDR STUDY GUIDES

Singular Value Decomposition (SVD)

Singular Values, Rank-k Approximation, and the Geometry Behind PCA — A TLDR Primer
Solid State Press

Singular Value Decomposition (SVD)

Singular Values, Rank-k Approximation, and the Geometry Behind PCA — A TLDR Primer

Copyright © 2026 Solid State Press.

Published by Solid State Press.

All rights reserved. No part of this book may be reproduced or transmitted in any form without prior written permission, except for brief quotations in critical reviews and educational use.

Part of the TLDR Study Guides series.

Contents

  1. 1 What SVD Actually Says: Rotate, Stretch, Rotate
  2. 2 Building SVD from A^T A: Where the Pieces Come From
  3. 3 Computing SVD by Hand on a 2x2
  4. 4 Low-Rank Approximation and the Eckart-Young Theorem
  5. 5 SVD and PCA: Same Math, Different Story
  6. 6 Where SVD Shows Up: Pseudoinverse, Noise, and Beyond
Chapter 1

What SVD Actually Says: Rotate, Stretch, Rotate

Every matrix is secretly three operations stacked on top of each other. That is what the Singular Value Decomposition (SVD) tells you.

Given any real $m \times n$ matrix $A$, the SVD writes it as a product of three matrices:

$A = U \Sigma V^T$

Here $U$ is an $m \times m$ orthogonal matrix (its columns are unit vectors that are mutually perpendicular), $\Sigma$ is an $m \times n$ diagonal matrix (non-zero entries only on the main diagonal), and $V$ is an $n \times n$ orthogonal matrix. The entries on the diagonal of $\Sigma$ are called singular values, written $\sigma_1 \geq \sigma_2 \geq \cdots \geq 0$, always arranged in decreasing order.

A common mistake is to confuse this with the eigendecomposition $A = P D P^{-1}$. The eigendecomposition requires $A$ to be square and diagonalizable, and uses the same matrix $P$ on both sides. SVD works for any matrix — rectangular included — and uses two different orthogonal matrices, $U$ and $V$.

The geometric picture

Think about what $A$ does to a vector $\mathbf{x}$ in $\mathbb{R}^n$. Rather than treating $A$ as one mysterious operation, the SVD says: split the work into three clean steps.

Step 1 — $V^T$ rotates (or reflects). Multiplying by $V^T$ is a rigid rotation of $\mathbb{R}^n$. It doesn't stretch anything; it just reorients the input vector into a new coordinate frame aligned with the right singular vectors — the columns of $V$.

Step 2 — $\Sigma$ stretches along axes. With the input now sitting in the right coordinate frame, $\Sigma$ scales each axis independently. The $i$-th axis is scaled by $\sigma_i$. If $\sigma_i = 0$, the whole direction collapses. This is the only step that changes lengths.

Step 3 — $U$ rotates again. The result is then rotated (or reflected) one more time by $U$, landing the output in $\mathbb{R}^m$. The columns of $U$ are the left singular vectors.

The phrase "rotate, stretch, rotate" is the entire geometric content of SVD.

What happens to the unit circle

About This Book

If you are sitting in a linear algebra course — whether that is a college freshman survey class, an applied math elective, or a machine learning prerequisite — and SVD just appeared on the syllabus, this is the book you need. It also works as a linear algebra exam prep resource if you are a college freshman heading into a final, or a high school student in an advanced course who wants a matrix decomposition visual intuition guide that does not bury you in abstraction first.

This SVD linear algebra study guide covers singular values, the geometry of matrix transformations, and singular value decomposition explained simply through a worked 2x2 example. It builds from $A^T A$ up through the Eckart-Young theorem on low rank approximation matrix factorization, then explains the PCA and SVD connection, and closes with the SVD pseudoinverse and least squares applications. Concise and tight, with no filler.

Read it straight through, work every example alongside the text, then test yourself on the problem set at the end.

Keep reading

You've read the first half of Chapter 1. The complete book covers 6 chapters in roughly fifteen pages — readable in one sitting.

Coming soon to Amazon