Kronecker Product
Matrix Representation of the Tensor Product
Intermediate
1. Definition
Definition: Kronecker product
For an $m \times n$ matrix $A = (a_{ij})$ and a $p \times q$ matrix $B$, the Kronecker product $A \otimes B$ is defined as the following $mp \times nq$ block matrix:
That is, each entry $a_{ij}$ of $A$ is replaced by the scalar multiple $a_{ij}B$ of the matrix $B$.
The Kronecker product is named after the German mathematician Leopold Kronecker (1823–1891). As a matrix representation of the tensor product it appears widely in numerical computation, signal processing, and quantum information.
Example: 2×2 and 3×3 matrices
$$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 6 & 7 \\ 8 & 9 & 10 \\ 11 & 12 & 13 \end{pmatrix}$$
Then $A \otimes B$ is a $6 \times 6$ block matrix:
$$A \otimes B = \begin{pmatrix} 1 \cdot B & 2 \cdot B \\ 3 \cdot B & 4 \cdot B \end{pmatrix} = \begin{pmatrix} 5 & 6 & 7 & 10 & 12 & 14 \\ 8 & 9 & 10 & 16 & 18 & 20 \\ 11 & 12 & 13 & 22 & 24 & 26 \\ 15 & 18 & 21 & 20 & 24 & 28 \\ 24 & 27 & 30 & 32 & 36 & 40 \\ 33 & 36 & 39 & 44 & 48 & 52 \end{pmatrix}$$
Figure 1: Block structure of the Kronecker product ($2 \times 2 \otimes 3 \times 3$)
2. Basic properties
Below, the matrix sizes are assumed to be such that all stated operations are defined.
2.1 Associativity
Property
$$(A \otimes B) \otimes C = A \otimes (B \otimes C)$$
Expanding the nested block-matrix structure on both sides yields the same matrix.
2.2 Distributivity
Property
$$A \otimes (B + C) = A \otimes B + A \otimes C$$
$$(A + B) \otimes C = A \otimes C + B \otimes C$$
2.3 Scalar multiplication
Property
For any scalar $c$:
$$(cA) \otimes B = A \otimes (cB) = c(A \otimes B)$$
2.4 Transpose
Property
$$(A \otimes B)^T = A^T \otimes B^T$$
For complex matrices the conjugate transpose satisfies the analogous identity $(A \otimes B)^* = A^* \otimes B^*$.
Proof
The $(i,j)$-block of $A \otimes B$ is $a_{ij}B$, so the $(j,i)$-block of $(A \otimes B)^T$ is $(a_{ij}B)^T = a_{ij}B^T$. On the other hand, the $(j,i)$-block of $A^T \otimes B^T$ is $(A^T)_{ji} B^T = a_{ij} B^T$. The two agree. $\square$
2.5 Inverse
Property
If $A$ and $B$ are both invertible:
$$(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$$
Proof
Using the mixed-product property (below):
$$(A \otimes B)(A^{-1} \otimes B^{-1}) = (AA^{-1}) \otimes (BB^{-1}) = I_m \otimes I_p = I_{mp}$$
Similarly $(A^{-1} \otimes B^{-1})(A \otimes B) = I_{mp}$. $\square$
2.6 Trace
Property
If $A$ is $m \times m$ and $B$ is $p \times p$:
$$\operatorname{tr}(A \otimes B) = \operatorname{tr}(A) \cdot \operatorname{tr}(B)$$
Proof
The diagonal blocks of $A \otimes B$ are $a_{11}B, a_{22}B, \ldots, a_{mm}B$, and the trace of each block is $a_{ii}\operatorname{tr}(B)$. Hence:
$$\operatorname{tr}(A \otimes B) = \displaystyle\sum_{i=1}^{m} a_{ii} \operatorname{tr}(B) = \operatorname{tr}(A) \cdot \operatorname{tr}(B) \quad \square$$
2.7 Determinant
Property
If $A$ is $m \times m$ and $B$ is $p \times p$:
$$\det(A \otimes B) = (\det A)^p \cdot (\det B)^m$$
This follows from the eigenvalue formula below. Letting $\lambda_1, \ldots, \lambda_m$ and $\mu_1, \ldots, \mu_p$ denote the eigenvalues of $A$ and $B$, the eigenvalues of $A \otimes B$ are $\lambda_i \mu_j$ for $i = 1, \ldots, m$ and $j = 1, \ldots, p$, so:
$$\det(A \otimes B) = \displaystyle\prod_{i=1}^{m}\displaystyle\prod_{j=1}^{p} \lambda_i \mu_j = \left(\displaystyle\prod_{i=1}^{m}\lambda_i\right)^p \left(\displaystyle\prod_{j=1}^{p}\mu_j\right)^m = (\det A)^p (\det B)^m$$
2.8 Non-commutativity
In general the Kronecker product is not commutative:
$$A \otimes B \neq B \otimes A$$
However, $A \otimes B$ and $B \otimes A$ are related by a similarity transformation via the perfect-shuffle permutation matrices $P, Q$, satisfying $P(A \otimes B)Q = B \otimes A$, so they share the same multiset of eigenvalues.
3. Mixed-product property
Theorem: Mixed-product property
Whenever the products are defined ($A$ is $m \times n$, $B$ is $p \times q$, $C$ is $n \times r$, $D$ is $q \times s$):
This is one of the most important identities for the Kronecker product and underlies many proofs and applications.
Proof
The $(i,k)$-block of $A \otimes B$ is $a_{ik}B$ (size $p \times q$), and the $(k,j)$-block of $C \otimes D$ is $c_{kj}D$ (size $q \times s$). Therefore the $(i,j)$-block of $(A \otimes B)(C \otimes D)$ is:
$$\displaystyle\sum_{k=1}^{n} (a_{ik}B)(c_{kj}D) = \displaystyle\sum_{k=1}^{n} a_{ik}c_{kj}(BD) = \left(\displaystyle\sum_{k=1}^{n} a_{ik}c_{kj}\right)(BD) = (AC)_{ij} \cdot (BD)$$
This is the $(i,j)$-block of $(AC) \otimes (BD)$. $\square$
Applications of the mixed-product property
The inverse identity $(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$ is an immediate corollary. The $k$-th power identity
$$(A \otimes B)^k = A^k \otimes B^k$$
follows by induction on $k$ using the mixed-product property.
4. Relation to the vec operator
4.1 Definition of the vec operator
Definition: vec operator
For an $m \times n$ matrix $X = (\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n)$, the vec operator stacks the columns of $X$ into an $mn \times 1$ vector:
$$\operatorname{vec}(X) = \begin{pmatrix} \mathbf{x}_1 \\ \mathbf{x}_2 \\ \vdots \\ \mathbf{x}_n \end{pmatrix}$$
4.2 Fundamental vec identity
Theorem
If $A$ is $m \times n$, $X$ is $n \times p$, and $B$ is $p \times q$:
Special cases:
- $B = I$: $\operatorname{vec}(AX) = (I \otimes A)\,\operatorname{vec}(X)$
- $A = I$: $\operatorname{vec}(XB) = (B^T \otimes I)\,\operatorname{vec}(X)$
Sketch of proof
Let $\mathbf{x}_j$ denote the $j$-th column of $X$. The $k$-th column of $AXB$ is:
$$(AXB)_k = A \displaystyle\sum_{j=1}^{p} x_j b_{jk} \cdot \mathbf{e}_j = \displaystyle\sum_{j=1}^{p} b_{jk} A \mathbf{x}_j$$
This is exactly the action of $(B^T \otimes A)$ on $\operatorname{vec}(X)$. $\square$
Figure 2: Vectorization by the vec operator
4.3 Commutation matrix
For an $m \times n$ matrix $X$, there is a commutation matrix $K_{m,n}$ that relates $\operatorname{vec}(X)$ and $\operatorname{vec}(X^T)$:
$$K_{m,n} \operatorname{vec}(X) = \operatorname{vec}(X^T)$$
The commutation matrix enables swapping the order of a Kronecker product:
$$K_{p,m}(A \otimes B)K_{n,q} = B \otimes A$$
where $A$ is $m \times n$ and $B$ is $p \times q$.
5. Eigenvalues
Theorem: Eigenvalues of the Kronecker product
Let $A$ be an $m \times m$ matrix with eigenvalues $\lambda_1, \ldots, \lambda_m$ and let $B$ be a $p \times p$ matrix with eigenvalues $\mu_1, \ldots, \mu_p$. The eigenvalues of $A \otimes B$ are:
with corresponding eigenvectors $\mathbf{u}_i \otimes \mathbf{v}_j$, where $A\mathbf{u}_i = \lambda_i \mathbf{u}_i$ and $B\mathbf{v}_j = \mu_j \mathbf{v}_j$.
Proof
By the mixed-product property, if $A\mathbf{u}_i = \lambda_i \mathbf{u}_i$ and $B\mathbf{v}_j = \mu_j \mathbf{v}_j$:
$$(A \otimes B)(\mathbf{u}_i \otimes \mathbf{v}_j) = (A\mathbf{u}_i) \otimes (B\mathbf{v}_j) = (\lambda_i \mathbf{u}_i) \otimes (\mu_j \mathbf{v}_j) = \lambda_i \mu_j \, (\mathbf{u}_i \otimes \mathbf{v}_j)$$
so $\mathbf{u}_i \otimes \mathbf{v}_j$ is an eigenvector of $A \otimes B$ with eigenvalue $\lambda_i \mu_j$. If $A$ and $B$ are both diagonalizable, the resulting $mp$ eigenvectors yield the full eigenstructure of $A \otimes B$. $\square$
Example: computing eigenvalues
$$A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}, \quad B = \begin{pmatrix} 1 & 0 \\ 0 & 4 \end{pmatrix}$$
The eigenvalues of $A$ are $\lambda_1 = 2, \lambda_2 = 3$ and those of $B$ are $\mu_1 = 1, \mu_2 = 4$.
Hence the eigenvalues of $A \otimes B$ are:
$$\lambda_1\mu_1 = 2, \quad \lambda_1\mu_2 = 8, \quad \lambda_2\mu_1 = 3, \quad \lambda_2\mu_2 = 12$$
Eigenvalues of the Kronecker sum
A related construction is the Kronecker sum:
$$A \oplus B = A \otimes I_p + I_m \otimes B$$
Its eigenvalues are $\lambda_i + \mu_j$, which plays an essential role in the theory of the matrix exponential identity $e^{(A \oplus B)t} = e^{At} \otimes e^{Bt}$.
6. Relation to the tensor product
The tensor product is a universal construction in abstract algebra that does not depend on a choice of basis. The Kronecker product is the matrix representation of the tensor product with respect to a specific basis.
6.1 Finite-dimensional case
Let $V$ ($\dim V = m$) and $W$ ($\dim W = p$) be finite-dimensional vector spaces, and let $T: V \to V$ and $S: W \to W$ be linear maps. If $A$ is the matrix representation of $T$ in the basis $\{e_1, \ldots, e_m\}$ and $B$ is the matrix representation of $S$ in the basis $\{f_1, \ldots, f_p\}$, then the matrix representation of $T \otimes S: V \otimes W \to V \otimes W$ in the basis $\{e_i \otimes f_j\}$ is the Kronecker product $A \otimes B$.
6.2 Effect of basis ordering
The ordering of the basis of $V \otimes W$ is not unique. Using the lexicographic ordering $e_1 \otimes f_1, e_1 \otimes f_2, \ldots, e_m \otimes f_p$ yields the Kronecker product $A \otimes B$; a different ordering corresponds to a row- and column-permutation of the matrix.
Note: Since the Kronecker product depends on a basis, identifying "Kronecker product = tensor product" is strictly speaking inaccurate. Precisely: the Kronecker product is the standard matrix representation of the tensor product.
7. Applications
7.1 Matrix equations
Sylvester equation
The Sylvester equation for an unknown matrix $X$ is:
$$AX + XB = C$$
Applying the vec operator:
$$\operatorname{vec}(AX + XB) = (I \otimes A + B^T \otimes I)\operatorname{vec}(X) = \operatorname{vec}(C)$$
This reduces to the $mn \times mn$ linear system $(I \otimes A + B^T \otimes I)\operatorname{vec}(X) = \operatorname{vec}(C)$. The coefficient matrix $I \otimes A + B^T \otimes I$ is exactly the Kronecker sum $A \oplus B^T$.
Lyapunov equation
The Lyapunov equation is the fundamental stability equation in control theory:
$$AX + XA^T = -Q$$
It is a special case of the Sylvester equation with $B = A^T$, $C = -Q$, and can be vectorized using the Kronecker product in the same way.
7.2 Signal processing
In signal processing the Kronecker product is used for multidimensional filtering. For instance, a separable 2-D filter is the Kronecker product $\mathbf{v} \otimes \mathbf{h}^T$ of a horizontal filter $\mathbf{h}$ and a vertical filter $\mathbf{v}$, reducing an $N^2$-multiplication 2-D convolution to $2N$ 1-D convolutions.
It also appears in modelling the channel matrices of MIMO (multiple-input multiple-output) communication systems: when the spatial and temporal correlations are separable, the channel covariance matrix has the Kronecker structure $R_t \otimes R_r$.
7.3 Quantum information
In quantum computing the state space of a multi-qubit system is built from the tensor product. An $n$-qubit system lives in a $2^n$-dimensional Hilbert space, and gate operations on individual qubits are expressed via Kronecker products.
Example: gate operation on two qubits
Applying a Pauli X gate on the first qubit and the identity on the second:
$$X \otimes I = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \otimes \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \end{pmatrix}$$
The Kronecker product is also indispensable for describing CNOT gates and entanglement.
8. Frequently asked questions
Q1: What is the difference between the Kronecker product and the tensor product?
The tensor product is an abstract algebraic construction that does not depend on a choice of basis. The Kronecker product is a matrix representation of the tensor product with respect to a specific basis, used for concrete computation. In short, the Kronecker product is the matrix (coordinate) representation of the tensor product.
Q2: Is the Kronecker product commutative?
In general the Kronecker product is not commutative: $A \otimes B \neq B \otimes A$. However, $A \otimes B$ and $B \otimes A$ are related by a similarity transformation via permutation matrices (the perfect-shuffle permutations), so they share the same multiset of eigenvalues.
Q3: How is the vec operator related to the Kronecker product?
The vec operator stacks the columns of a matrix into a single column vector and satisfies the fundamental identity $\operatorname{vec}(AXB) = (B^T \otimes A)\operatorname{vec}(X)$. This identity reduces matrix equations such as the Sylvester equation or the Lyapunov equation to ordinary linear systems.
9. References
- Horn, R. A., & Johnson, C. R. (2012). Matrix Analysis, 2nd ed., Cambridge University Press. Cambridge University Press — Chapter 4 develops the theory of the Kronecker product.
- Laub, A. J. (2005). Matrix Analysis for Scientists and Engineers, SIAM. SIAM
- Wikipedia: Kronecker product
- Wikipedia: Tensor product