Linear Algebra: The Language of Data
Linear algebra is the mathematical language that AI uses to represent and transform data. Every image, sentence, audio clip, and database row that enters an AI system is first converted into the structures that linear algebra provides: vectors, matrices, and tensors. This lesson introduces you to the core ideas of linear algebra and shows you exactly where each one appears in AI.
Vectors: Ordered Lists of Numbers
A vector is an ordered list of numbers. That is all it is. Each number in the list is called a component or element.
v = [3, 7, 2]
This vector has three components. The order matters: [3, 7, 2] is different from [7, 3, 2].
Vectors in AI
In AI, vectors represent data. Here are some examples:
| Data Type | Vector Representation |
|---|---|
| A house listing | [3 bedrooms, 1500 sqft, 10 years old] = [3, 1500, 10] |
| A pixel color | [red, green, blue] = [255, 128, 0] |
| A word (embedding) | [0.21, -0.45, 0.78, ...] (hundreds of numbers) |
| A user profile | [age, purchases, clicks, ...] = [28, 47, 312, ...] |
The key idea is that similar data produces similar vectors. Two houses with similar features will have similar vectors. Two words with similar meanings will have similar embedding vectors. AI exploits this property constantly.
Vector Operations
The two most important vector operations are:
Addition — combine two vectors element by element:
[1, 2, 3] + [4, 5, 6] = [5, 7, 9]
In AI, vector addition combines information. For example, residual connections in neural networks add the input of a layer to its output.
Dot product — multiply corresponding elements and sum them:
[1, 2, 3] · [4, 5, 6] = (1×4) + (2×5) + (3×6) = 4 + 10 + 18 = 32
The dot product measures how similar two vectors are. When two vectors point in the same direction, their dot product is large. When they point in opposite directions, it is negative. When they are perpendicular, it is zero.
In AI, the dot product is used in:
- Attention mechanisms in transformers (comparing query and key vectors)
- Similarity search (finding related documents or images)
- Neuron computations (each neuron computes a dot product of inputs and weights)
Matrices: Grids of Numbers
A matrix is a rectangular grid of numbers, organized in rows and columns:
M = | 1 2 3 |
| 4 5 6 |
This matrix has 2 rows and 3 columns, so it is called a "2 by 3" matrix (written 2×3).
Matrices in AI
Matrices are the core computational structure of neural networks. The weights of a neural network layer are stored as a matrix. When data flows through a layer, it is multiplied by the weight matrix:
Input vector: [x₁, x₂, x₃] (3 features)
Weight matrix: 3×4 matrix (3 inputs → 4 outputs)
Output vector: [y₁, y₂, y₃, y₄] (4 features)
This single operation, matrix-vector multiplication, is what a neural network layer does. The entire network is a series of these multiplications with non-linear functions in between.
Matrix Multiplication
When you multiply a matrix by a vector (or another matrix), you are performing a linear transformation. This can:
- Rotate data (change the orientation of vectors)
- Scale data (stretch or compress vectors)
- Project data (reduce the number of dimensions)
- Combine features (create new features from existing ones)
Every layer of a neural network performs one of these transformations. By stacking many layers, the network can learn extremely complex transformations that convert raw data into useful predictions.
Tensors: Multi-Dimensional Arrays
A tensor is a generalization of vectors and matrices to any number of dimensions:
| Structure | Dimensions | Example |
|---|---|---|
| Scalar | 0D | A single number: 7.5 |
| Vector | 1D | A list: [3, 7, 2] |
| Matrix | 2D | A grid: 3×4 |
| 3D Tensor | 3D | A cube of numbers: 3×4×5 |
| 4D Tensor | 4D | A batch of cubes: 32×3×224×224 |
Tensors in AI
AI frameworks like PyTorch and TensorFlow use tensors as their fundamental data structure. Here are some common tensor shapes in AI:
A batch of 32 color images, each 224×224 pixels:
Shape: [32, 3, 224, 224]
batch × channels × height × width
A batch of 16 sentences, each 128 tokens, each with a 768-dim embedding:
Shape: [16, 128, 768]
batch × sequence_length × embedding_dim
Understanding tensor shapes is essential for working with AI frameworks. When something goes wrong (and it often does), the error is frequently a shape mismatch that you can only debug if you understand what each dimension represents.
Key Concepts to Study
When you dive deeper into linear algebra for AI, focus on these topics:
- Vector spaces and dimensions — understanding what it means to work in high-dimensional spaces
- Matrix operations — multiplication, transpose, inverse
- Eigenvalues and eigenvectors — how matrices stretch and rotate space, used in PCA and stability analysis
- Norms and distances — measuring the size of vectors and the distance between them
Each of these concepts has direct, concrete applications in AI systems.
Where to Go Next
For a complete, in-depth treatment of linear algebra through the lens of AI, take the Linear Algebra for AI course. It covers vectors, matrices, dot products, matrix multiplication, eigenvalues, and tensors, with every concept connected to real AI applications like word embeddings, neural network layers, transformers, and GPUs.
Summary
Linear algebra provides the language of AI:
- Vectors represent data as ordered lists of numbers
- Matrices store the learned weights and perform transformations
- Tensors generalize these structures to any number of dimensions
- Operations like the dot product and matrix multiplication are the core computations that power every AI system
Without linear algebra, there is no AI. It is the first and most fundamental mathematical pillar.