Meta & resources

Probability, statistics and data analysis

Machine Learning: concepts & procedures

Machine Learning: fundamental algorithms

Machine Learning: model assessment

Artificial neural networks

Natural language processing

The computer science appendix

The mathematics appendix

Matrix algebra notes

Capital letters, like

$A$

, indicate matrices.Transpose of a matrix

Transposing a matrix

$A$

, of elements $[A]_{ij} = a_{ij}$

, is the operation which switches the row and column positions of each element:$[A^t]_{ij} = a_{ji}$

Properties

**Transpose of the transpose**:$(A^t)^t = A$**Transpose of the sum**:$(A + B)^t = A^t + B^t$**Transpose of the product**:$(AB)^t = B^t A^t$

The first one follows straightly from definition.

The second one is straightforward just because the elements of

$(A+B)^t$

are the sums of elements in$A^t$

and$B^t$

.The third one is easily proven using the fact that *, *so that we can say

$[AB]_{ij} = \sum_k a_{ik} b_{kj}$

$[(AB)^t]_{ij} = [AB]_{ji} = \sum_k a_{jk} b_{ki},$

and $[B^t A^t]_{ij} = \sum_k b^t_{ik} a^t_{kj} = \sum_k b_{ki} a_{jk}$

, so the two things are the same.Special types of matrices

:*IDEMPOTENT*$M^2 = M$

Matrix Convolution

Given two matrices A and B (typically kernel and image, as this is used in computer vision),

$A =
\begin{bmatrix}
a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\
a_{21} & a_{22} & a_{23} & \dots & a_{2n} \\
\dots & \dots & \dots & \dots & \dots \\
a_{n1} & a_{n2} & a_{n3} & \dots & a_{nn}
\end{bmatrix}, \ \
B =
\begin{bmatrix}
b_{11} & b_{12} & b_{13} & \dots & b_{1n} \\
b_{21} & b_{22} & b_{23} & \dots & b_{2n} \\
\dots & \dots & \dots & \dots & \dots \\
b_{n1} & b_{n2} & b_{n3} & \dots & b_{nn}
\end{bmatrix} \ ,$

their convolution is obtained via the multiplication of locationally similar entries and summing:

$\mathcal{C} = \sum_{i=0}^{i=} \sum_{j=1}^{j=} B_{ij} A_{n-in-j}$

This procedure is loosely related to mathematical convolution.

Frobenius norm of a matrix

Given matrix M, its Frobenious norm is the square root of the sum of the squares of its elements.

$||M|| = \sqrt{\sum_{i,j} M_{ij}^2}$

Last modified 9mo ago