Linear Algebra
Build smarter skills with practical AI tutorials and powerful machine learning apps and Calculators.
Calculator 1 | Calculator 2 | Calculator 3 | Calculator 4 | Calculator 5 (PCA) | Calculator 6
Build smarter skills with practical AI tutorials and powerful machine learning apps and Calculators.
Explanation: A scalar is a single number (like 3 or -1.5) that has magnitude only, no direction. It is used to scale vectors and matrices, e.g. 3·v stretches v to three times its length.
Connection to AI/ML: Learning rates, regularization strengths, biases and the loss value are all scalars. Training boils down to minimizing one scalar loss.
Explanation: A vector is an ordered list of numbers that describes a point or an arrow with both magnitude and direction, written v = (v1, …, vn). It lives in n-dimensional space.
Connection to AI/ML: Every data sample is a feature vector. Word/image embeddings, weight vectors and gradients are all vectors.
Explanation: Vectors are added component-wise (tip-to-tail) and multiplied by scalars (stretching/flipping); norms measure their length. These two operations are the foundation of all of linear algebra.
Connection to AI/ML: Gradient descent is a vector update, w ← w − η∇L. Embedding analogies (king − man + woman ≈ queen) and L1/L2 regularization (norms) rely on these.
Explanation: The dot product multiplies matching components and sums them: a·b = |a||b|cosθ. It measures how aligned two vectors are; zero means perpendicular.
Connection to AI/ML: A neuron computes w·x + b. Transformer attention scores are query·key dot products, and cosine similarity powers search and recommendations.
Explanation: Defined in 3D, a × b is a vector perpendicular to both a and b, with length equal to the area of the parallelogram they span. Its direction follows the right-hand rule.
Connection to AI/ML: Less central to core ML, but key in 3D vision, robotics, graphics and point clouds: surface normals, camera/pose geometry and rotations.
Explanation: A vector space is a set of vectors closed under addition and scalar multiplication (R² and Rⁿ are examples). Any combination of its vectors stays inside the space.
Connection to AI/ML: Feature space, embedding space and parameter space are all vector spaces where learning happens; kernel methods work in function spaces.
Explanation: Vectors are linearly independent if none can be built as a combination of the others. Dependent vectors are redundant and add no new direction.
Connection to AI/ML: Redundant (collinear) features cause multicollinearity and unstable regression. Feature selection and dimensionality reduction remove such dependence.
Explanation: A basis is a minimal set of independent vectors that can build every vector in the space; the number of them is the dimension. Different bases describe the same space with different coordinates.
Connection to AI/ML: Features are coordinates in some basis. PCA, Fourier transforms and learned embeddings all change the basis to get better representations; embedding size is the chosen dimension.
Explanation: A matrix is a rectangular m×n grid of numbers that represents a linear transformation or a table of data. Its columns show where the basis vectors land.
Connection to AI/ML: A dataset is a matrix X (samples × features); every neural-network layer has a weight matrix W; images and covariance tables are matrices.
Explanation: Matrices of the same shape can be added, subtracted and multiplied by scalars element by element. The element-wise (Hadamard) product is also common.
Connection to AI/ML: Adding biases, applying activation functions, dropout masks and normalization are element-wise operations on whole batches of data.
Explanation: Each entry of AB is a row-of-A · column-of-B dot product, and the result composes the two transformations ((m×n)(n×p) → m×p). Order matters: AB ≠ BA in general.
Connection to AI/ML: It is the core computation of deep learning (Wx for each layer, QKᵀ in attention), and GPUs/TPUs are built to accelerate it.
Explanation: The transpose Aᵀ flips rows and columns, so (Aᵀ)ᵢⱼ = Aⱼᵢ, and (AB)ᵀ = BᵀAᵀ. It turns row vectors into column vectors.
Connection to AI/ML: Backpropagation sends gradients back through Wᵀ; the normal equations use XᵀX; covariance is XᵀX/n; attention uses Kᵀ.
Explanation: The inverse A⁻¹ undoes the effect of A, so A⁻¹A = I. It exists only for square, non-singular matrices.
Connection to AI/ML: Closed-form linear regression is w = (XᵀX)⁻¹Xᵀy; Gaussian processes and Kalman filters use inverses. In practice, solvers or the pseudo-inverse are used for numerical stability.
Explanation: The determinant is a single number giving the factor by which a transformation scales area/volume (negative = orientation flip). A determinant of 0 means the matrix collapses space and is not invertible.
Connection to AI/ML: Used to test invertibility, in Gaussian densities (det Σ), and in normalizing flows, which track the log-determinant of the Jacobian.
Explanation: Rank is the number of linearly independent rows or columns, i.e. the dimension of the output a matrix can produce. Lower rank means information is squashed into fewer dimensions.
Connection to AI/ML: Real data and weights are often approximately low-rank, which enables compression and LoRA fine-tuning. A rank-deficient XᵀX breaks ordinary regression.
Explanation: An orthogonal matrix Q has orthonormal columns (QᵀQ = I), so Q⁻¹ = Qᵀ. It rotates or reflects space while preserving lengths and angles.
Connection to AI/ML: Orthogonal weight initialization keeps gradients stable in deep/recurrent nets; QR decomposition, PCA and random projections rely on them.
Explanation: A symmetric matrix equals its transpose (A = Aᵀ). By the spectral theorem it has real eigenvalues and orthogonal eigenvectors.
Connection to AI/ML: Covariance matrices, Hessians, kernel (Gram) matrices and graph Laplacians are symmetric, which makes them easy to analyze and decompose.
Explanation: A symmetric matrix is positive definite if xᵀAx > 0 for every non-zero x, equivalently all eigenvalues are positive. Geometrically it makes a bowl-shaped quadratic with a single minimum.
Connection to AI/ML: Valid covariance matrices and kernels must be positive (semi)definite; a positive-definite Hessian guarantees a minimum and enables Newton methods and Cholesky solves.
Explanation: An eigenvalue λ satisfies Av = λv: it is the factor by which A stretches or shrinks space along a special direction. Large eigenvalues mark the strongest directions.
Connection to AI/ML: In PCA they measure the variance explained; the largest eigenvalue governs RNN stability; they drive PageRank, spectral clustering and Hessian curvature analysis.
Explanation: An eigenvector is a non-zero vector whose direction is unchanged by A; it is only scaled by its eigenvalue. They form the natural axes of a transformation.
Connection to AI/ML: Principal components are eigenvectors of the covariance matrix; PageRank is the dominant eigenvector of the web graph; spectral clustering uses Laplacian eigenvectors.
Explanation: Diagonalization writes A = PDP⁻¹: switch to the eigenvector basis, scale along each axis with diagonal D, then switch back. It makes powers easy: Aᵏ = PDᵏP⁻¹.
Connection to AI/ML: Used to analyze linear dynamical systems and RNN behavior, Markov chain steady states and PCA (diagonalizing the covariance matrix).
Explanation: SVD factors any matrix as A = UΣVᵀ: rotate, stretch along axes by the singular values, rotate again. It works for non-square matrices as well.
Connection to AI/ML: Powers dimensionality reduction, latent semantic analysis, low-rank compression, denoising, pseudo-inverses and classic recommender systems.
Explanation: PCA finds orthogonal directions of maximum variance (eigenvectors of the covariance matrix, or the SVD of centered data) and projects data onto the top few. It keeps the most information in the fewest dimensions.
Connection to AI/ML: Used for dimensionality reduction, visualization, noise removal and speeding up or de-correlating inputs to ML models.
Explanation: Matrix factorization splits a matrix into a product of simpler ones (LU, QR, SVD, NMF, or R ≈ UVᵀ). It exposes hidden structure and makes computation cheaper.
Connection to AI/ML: Recommender systems (users × items ≈ user factors × item factors), topic modeling (NMF), embeddings and LoRA all use it.
Important Note: All the Calculators listed in this site are for educational purpose only and we do not guarentee the accuracy of results. Please do consult with other sources as well.