Linear Algebra Basic

This is the review and cheatsheet of some basic linear algebra knowledge during my undergraduate study.

1. Basic

A scalar value that can be computed from elements of a square matrix.

where $a_{ij}$ is the element of A and $M_{ij}$ is minor.

Matrix Inverse
A square matrix M has an inverse that M is invertible if the determinant $|M| \neq 0$.

The inverse properties are

Orthogonal Matrices
A square matrix $A \in R^{nn}$ is orthogonal:

Matrix multiplication
\(A \in R^{mn}\) and \(B \in R^{np}\)

Matrix transpose

The trace of a square matrix $A \in R^{nn}$ is denoted as tr(A), which is the sum of diagonal elements in the matrix:

Properties are below:

The column rank of a matrix $A \in R^{mn}$ is the size of the largest subset of columns of A that constitute a linearly independent set. The same definition as for row rank. For any matrix, the column rank is equal to the row rank. Both are denoted as rank(A). Properties are below:

Eigenvalue and Eigenvectors
Given a square matrix A, \(\lambda\) is an eigenvalue and x is the corresponding eigenvector if

which equals to
The properties are
3.The rank of A is equal to the number of non-zero eigenvalues of A
4.If A is non-singular then $1/\lambda_{i}$ is an eigenvalue of $A^{-1}$ with associated eigenvector.
5.The eigenvalues of a diagonal matrix are just the diagonal entries.

The gradient
$f: R^{mn} \to R$ is a function that input a matrix A and returns a value. The gradient of f is the matrix of partial derivatives.

2.Other definitions and calculations

Laplace Matrix (simple graph)
Given a simple graph G with n vertices, its Laplace Matrix \(L_{nn}\) is defined as: L=D-A, where D is the degree matrix and A is the adjacency matrix. D is a diagonal matrix which includes the information about the degree of each vertex. A is the adjacency matrix which only includes 1 and 0 since G is a simple graph and the diagonal are all 0.
symmetrix normalized laplacian

Singular value decomposition
Assume $A \in R^{mn}$ and all elements in M belongs to real or plural values. There exists a decomposition that

where U and V are orthogonal that $U^{T}U=I_{mm}$ and $V^{T}V=I_{nn}$.

Eigen decomposition
$A \in R^{nn}$ is a square matrix with n linear independent eigenvectors $q_{i}$(i=1…n). A can be factorized as

where Q is the square n by n matrix with ith column is the eigenvector of A, and $\Lambda$ is the diagonal matrix with diagonal elements are the corresponding eigenvalues.