Kronecker Product
Matrix Representation of the Tensor Product
Intermediate
1. Definition
Definition: Kronecker product
For an $m \times n$ matrix $A = (a_{ij})$ and a $p \times q$ matrix $B$, the Kronecker product $A \otimes B$ is defined as the following $mp \times nq$ block matrix:
That is, each entry $a_{ij}$ of $A$ is replaced by the scalar multiple $a_{ij}B$ of the matrix $B$.
The Kronecker product is named after the German mathematician Leopold Kronecker (1823–1891), although the operation was first described by J. G. Zehfuss in 1858 and was also called the “Zehfuss matrix” (Henderson, Pukelsheim & Searle, 1983). As a matrix representation of the tensor product it appears widely in numerical computation, signal processing, and quantum information.
Example: 2×2 and 3×3 matrices
$$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 6 & 7 \\ 8 & 9 & 10 \\ 11 & 12 & 13 \end{pmatrix}$$
Then $A \otimes B$ is a $6 \times 6$ block matrix:
$$A \otimes B = \begin{pmatrix} 1 \cdot B & 2 \cdot B \\ 3 \cdot B & 4 \cdot B \end{pmatrix} = \begin{pmatrix} 5 & 6 & 7 & 10 & 12 & 14 \\ 8 & 9 & 10 & 16 & 18 & 20 \\ 11 & 12 & 13 & 22 & 24 & 26 \\ 15 & 18 & 21 & 20 & 24 & 28 \\ 24 & 27 & 30 & 32 & 36 & 40 \\ 33 & 36 & 39 & 44 & 48 & 52 \end{pmatrix}$$
Figure 1: Block structure of the Kronecker product (when $A$ is $2 \times 2$)
2. Basic properties
Below, the matrix sizes are assumed to be such that all stated operations are defined.
2.1 Associativity
Property
$$(A \otimes B) \otimes C = A \otimes (B \otimes C)$$
Expanding the nested block-matrix structure on both sides yields the same matrix.
2.2 Distributivity
Property
$$A \otimes (B + C) = A \otimes B + A \otimes C$$
$$(A + B) \otimes C = A \otimes C + B \otimes C$$
2.3 Scalar multiplication
Property
For any scalar $c$:
$$(cA) \otimes B = A \otimes (cB) = c(A \otimes B)$$
2.4 Transpose
Property
$$(A \otimes B)^T = A^T \otimes B^T$$
For complex matrices the conjugate transpose satisfies the analogous identity $(A \otimes B)^* = A^* \otimes B^*$.
Proof
The $(i,j)$-block of $A \otimes B$ is $a_{ij}B$, so the $(j,i)$-block of $(A \otimes B)^T$ is $(a_{ij}B)^T = a_{ij}B^T$. On the other hand, the $(j,i)$-block of $A^T \otimes B^T$ is $(A^T)_{ji} B^T = a_{ij} B^T$. The two agree. $\square$
2.5 Inverse
Property
If $A$ and $B$ are both invertible:
$$(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$$
Proof
Using the mixed-product property (below):
$$(A \otimes B)(A^{-1} \otimes B^{-1}) = (AA^{-1}) \otimes (BB^{-1}) = I_m \otimes I_p = I_{mp}$$
Similarly $(A^{-1} \otimes B^{-1})(A \otimes B) = I_{mp}$. $\square$
2.6 Trace
Property
If $A$ is $m \times m$ and $B$ is $p \times p$:
$$\operatorname{tr}(A \otimes B) = \operatorname{tr}(A) \cdot \operatorname{tr}(B)$$
Proof
The diagonal blocks of $A \otimes B$ are $a_{11}B, a_{22}B, \ldots, a_{mm}B$, and the trace of each block is $a_{ii}\operatorname{tr}(B)$. Hence:
$$\operatorname{tr}(A \otimes B) = \displaystyle\sum_{i=1}^{m} a_{ii} \operatorname{tr}(B) = \operatorname{tr}(A) \cdot \operatorname{tr}(B) \quad \square$$
2.7 Determinant
Property
If $A$ is $m \times m$ and $B$ is $p \times p$:
$$\det(A \otimes B) = (\det A)^p \cdot (\det B)^m$$
This follows from the eigenvalue formula below (eigenvalues counted over the complex numbers, with algebraic multiplicity). Letting $\lambda_1, \ldots, \lambda_m$ and $\mu_1, \ldots, \mu_p$ denote the eigenvalues of $A$ and $B$, the eigenvalues of $A \otimes B$ are $\lambda_i \mu_j$ for $i = 1, \ldots, m$ and $j = 1, \ldots, p$, so:
$$\det(A \otimes B) = \displaystyle\prod_{i=1}^{m}\displaystyle\prod_{j=1}^{p} \lambda_i \mu_j = \left(\displaystyle\prod_{i=1}^{m}\lambda_i\right)^p \left(\displaystyle\prod_{j=1}^{p}\mu_j\right)^m = (\det A)^p (\det B)^m$$
2.8 Non-commutativity
In general the Kronecker product is not commutative:
$$A \otimes B \neq B \otimes A$$
However, if $A$ is $m \times n$ and $B$ is $p \times q$, the commutation matrices—perfect-shuffle permutation matrices—give $K_{p,m}(A \otimes B)K_{n,q} = B \otimes A$ (permutation equivalence; Section 4.3). When $A$ and $B$ are both square, $K_{n,q} = K_{p,m}^T = K_{p,m}^{-1}$, so this is a similarity transformation and $A \otimes B$ and $B \otimes A$ share the same multiset of eigenvalues.
3. Mixed-product property
Theorem: Mixed-product property
Whenever the products are defined ($A$ is $m \times n$, $B$ is $p \times q$, $C$ is $n \times r$, $D$ is $q \times s$):
This is one of the most important identities for the Kronecker product and underlies many proofs and applications.
Proof
The $(i,k)$-block of $A \otimes B$ is $a_{ik}B$ (size $p \times q$), and the $(k,j)$-block of $C \otimes D$ is $c_{kj}D$ (size $q \times s$). Therefore the $(i,j)$-block of $(A \otimes B)(C \otimes D)$ is:
$$\displaystyle\sum_{k=1}^{n} (a_{ik}B)(c_{kj}D) = \displaystyle\sum_{k=1}^{n} a_{ik}c_{kj}(BD) = \left(\displaystyle\sum_{k=1}^{n} a_{ik}c_{kj}\right)(BD) = (AC)_{ij} \cdot (BD)$$
This is the $(i,j)$-block of $(AC) \otimes (BD)$. $\square$
Applications of the mixed-product property
The inverse identity $(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$ is an immediate corollary. For square $A$, $B$ and a non-negative integer $k$, the $k$-th power identity
$$(A \otimes B)^k = A^k \otimes B^k$$
follows by induction on $k$ using the mixed-product property (for negative integers $k$ as well when $A$ and $B$ are invertible).
4. Relation to the vec operator
4.1 Definition of the vec operator
Definition: vec operator
For an $m \times n$ matrix $X = (\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n)$, the vec operator stacks the columns of $X$ into an $mn \times 1$ vector:
$$\operatorname{vec}(X) = \begin{pmatrix} \mathbf{x}_1 \\ \mathbf{x}_2 \\ \vdots \\ \mathbf{x}_n \end{pmatrix}$$
4.2 Fundamental vec identity
Theorem
If $A$ is $m \times n$, $X$ is $n \times p$, and $B$ is $p \times q$:
Special cases:
- $B = I$: $\operatorname{vec}(AX) = (I \otimes A)\,\operatorname{vec}(X)$
- $A = I$: $\operatorname{vec}(XB) = (B^T \otimes I)\,\operatorname{vec}(X)$
Sketch of proof
Let $\mathbf{x}_j$ denote the $j$-th column of $X$. The $k$-th column of $AXB$ is:
$$(AXB)_k = AX\mathbf{b}_k = A\left(\displaystyle\sum_{j=1}^{p} b_{jk}\mathbf{x}_j\right) = \displaystyle\sum_{j=1}^{p} b_{jk} A \mathbf{x}_j$$
where $\mathbf{b}_k$ is the $k$-th column of $B$. The $k$-th block of $(B^T \otimes A)\operatorname{vec}(X)$ is $\sum_{j} (B^T)_{kj} A\mathbf{x}_j = \sum_{j} b_{jk} A\mathbf{x}_j$, which is the same. $\square$
Figure 2: Vectorization by the vec operator
4.3 Commutation matrix
For an $m \times n$ matrix $X$, there is a commutation matrix $K_{m,n}$ that relates $\operatorname{vec}(X)$ and $\operatorname{vec}(X^T)$:
$$K_{m,n} \operatorname{vec}(X) = \operatorname{vec}(X^T)$$
The commutation matrix enables swapping the order of a Kronecker product:
$$K_{p,m}(A \otimes B)K_{n,q} = B \otimes A$$
where $A$ is $m \times n$ and $B$ is $p \times q$.
5. Eigenvalues
Theorem: Eigenvalues of the Kronecker product
Let $A$ be an $m \times m$ matrix with eigenvalues $\lambda_1, \ldots, \lambda_m$ and let $B$ be a $p \times p$ matrix with eigenvalues $\mu_1, \ldots, \mu_p$. The eigenvalues of $A \otimes B$ are:
(eigenvalues counted over the complex numbers, with algebraic multiplicity). Moreover, if $A\mathbf{u} = \lambda \mathbf{u}$ and $B\mathbf{v} = \mu \mathbf{v}$, then $\mathbf{u} \otimes \mathbf{v}$ is an eigenvector of $A \otimes B$ with eigenvalue $\lambda \mu$.
Proof
By the mixed-product property, if $A\mathbf{u}_i = \lambda_i \mathbf{u}_i$ and $B\mathbf{v}_j = \mu_j \mathbf{v}_j$:
$$(A \otimes B)(\mathbf{u}_i \otimes \mathbf{v}_j) = (A\mathbf{u}_i) \otimes (B\mathbf{v}_j) = (\lambda_i \mathbf{u}_i) \otimes (\mu_j \mathbf{v}_j) = \lambda_i \mu_j \, (\mathbf{u}_i \otimes \mathbf{v}_j)$$
so $\mathbf{u}_i \otimes \mathbf{v}_j$ is an eigenvector of $A \otimes B$ with eigenvalue $\lambda_i \mu_j$. If $A$ and $B$ are both diagonalizable, the resulting $mp$ eigenvectors yield the full eigenstructure of $A \otimes B$. In general, the Schur decompositions $A = UTU^*$ and $B = VSV^*$ ($U, V$ unitary, $T, S$ upper triangular) give $A \otimes B = (U \otimes V)(T \otimes S)(U \otimes V)^*$, where $T \otimes S$ is upper triangular with diagonal entries $\lambda_i \mu_j$, so the conclusion holds even without diagonalizability. $\square$
Example: computing eigenvalues
$$A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}, \quad B = \begin{pmatrix} 1 & 0 \\ 0 & 4 \end{pmatrix}$$
The eigenvalues of $A$ are $\lambda_1 = 2, \lambda_2 = 3$ and those of $B$ are $\mu_1 = 1, \mu_2 = 4$.
Hence the eigenvalues of $A \otimes B$ are:
$$\lambda_1\mu_1 = 2, \quad \lambda_1\mu_2 = 8, \quad \lambda_2\mu_1 = 3, \quad \lambda_2\mu_2 = 12$$
Eigenvalues of the Kronecker sum
A related construction is the Kronecker sum:
$$A \oplus B = A \otimes I_p + I_m \otimes B \quad (A \text{ is } m \times m,\ B \text{ is } p \times p)$$
Its eigenvalues are $\lambda_i + \mu_j$. Since $A \otimes I_p$ and $I_m \otimes B$ commute, the matrix exponential satisfies $e^{(A \oplus B)t} = e^{At} \otimes e^{Bt}$.
6. Relation to the tensor product
The tensor product is a universal construction in abstract algebra that does not depend on a choice of basis. The Kronecker product is the matrix representation of the tensor product with respect to a specific basis.
6.1 Finite-dimensional case
Let $V$ ($\dim V = m$) and $W$ ($\dim W = p$) be finite-dimensional vector spaces, and let $T: V \to V$ and $S: W \to W$ be linear maps. If $A$ is the matrix representation of $T$ in the basis $\{e_1, \ldots, e_m\}$ and $B$ is the matrix representation of $S$ in the basis $\{f_1, \ldots, f_p\}$, then the matrix representation of $T \otimes S: V \otimes W \to V \otimes W$ in the basis $\{e_i \otimes f_j\}$ is the Kronecker product $A \otimes B$. More generally, for linear maps $T: V \to V'$ and $S: W \to W'$ between different spaces, the matrix representation of $T \otimes S$ is a Kronecker product of rectangular matrices.
6.2 Effect of basis ordering
The ordering of the basis of $V \otimes W$ is not unique. Using the lexicographic ordering $e_1 \otimes f_1, e_1 \otimes f_2, \ldots, e_m \otimes f_p$ yields the Kronecker product $A \otimes B$; a different ordering applies the same permutation to the rows and the columns, i.e. a similarity transformation $P(A \otimes B)P^T$ ($B \otimes A$ is one example).
Note: Since the Kronecker product depends on a basis, identifying "Kronecker product = tensor product" is strictly speaking inaccurate. Precisely: the Kronecker product is the standard matrix representation of the tensor product.
7. Applications
7.1 Matrix equations
Sylvester equation
The Sylvester equation for an unknown $m \times n$ matrix $X$, with an $m \times m$ matrix $A$, an $n \times n$ matrix $B$, and an $m \times n$ matrix $C$, is:
$$AX + XB = C$$
Applying the vec operator:
$$\operatorname{vec}(AX + XB) = (I_n \otimes A + B^T \otimes I_m)\operatorname{vec}(X) = \operatorname{vec}(C)$$
This reduces to the $mn \times mn$ linear system $(I_n \otimes A + B^T \otimes I_m)\operatorname{vec}(X) = \operatorname{vec}(C)$. With the definition above, the coefficient matrix $I_n \otimes A + B^T \otimes I_m$ equals the Kronecker sum $B^T \oplus A$ (it is permutation-similar to $A \oplus B^T$ and has the same eigenvalues $\lambda_i + \mu_j$). Hence the solution is unique exactly when $A$ and $-B$ have no common eigenvalue (all $\lambda_i + \mu_j \neq 0$).
Lyapunov equation
The Lyapunov equation is the fundamental stability equation in control theory:
$$AX + XA^T = -Q$$
It is a special case of the Sylvester equation with $B = A^T$, $C = -Q$, and can be vectorized using the Kronecker product in the same way.
7.2 Signal processing
In signal processing the Kronecker product is used for multidimensional filtering. For instance, a separable (rank-one) 2-D filter is the Kronecker product $\mathbf{v} \otimes \mathbf{h}^T$ of a horizontal filter $\mathbf{h}$ and a vertical filter $\mathbf{v}$. For such an $N \times N$ filter, a 2-D convolution needing $N^2$ multiplications per output pixel splits into one vertical and one horizontal 1-D convolution ($2N$ multiplications per pixel in total). A general (non-separable) filter does not admit this decomposition.
It also appears in modelling the channel matrices of MIMO (multiple-input multiple-output) communication systems: assuming that the transmit-side and receive-side spatial correlations are separable (the Kronecker model), the channel covariance matrix takes the form $R_t \otimes R_r$ of the transmit correlation matrix $R_t$ and the receive correlation matrix $R_r$ (written $R_t^T \otimes R_r$ under some vec conventions).
7.3 Quantum information
In quantum computing the state space of a multi-qubit system is built from the tensor product. An $n$-qubit system lives in a $2^n$-dimensional Hilbert space, and a gate acting on a single qubit independently (a local gate) is expressed as its Kronecker product with the identity on the other qubits.
Example: gate operation on two qubits
Ordering the computational basis as $|00\rangle, |01\rangle, |10\rangle, |11\rangle$ with the first qubit as the left factor, applying a Pauli X gate on the first qubit and the identity on the second gives:
$$X \otimes I = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \otimes \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \end{pmatrix}$$
Entangling gates such as CNOT are generally not a single Kronecker product, but they can be written as sums of Kronecker products, e.g. $|0\rangle\langle 0| \otimes I + |1\rangle\langle 1| \otimes X$, so the Kronecker product remains indispensable for describing them.
8. Frequently asked questions
Q1: What is the difference between the Kronecker product and the tensor product?
The tensor product is an abstract algebraic construction that does not depend on a choice of basis. The Kronecker product is a matrix representation of the tensor product with respect to a specific basis, used for concrete computation. In short, the Kronecker product is the matrix (coordinate) representation of the tensor product.
Q2: Is the Kronecker product commutative?
In general the Kronecker product is not commutative: $A \otimes B \neq B \otimes A$. However, the two are related by permutation matrices (the perfect-shuffle permutations) as $K(A \otimes B)K' = B \otimes A$. When $A$ and $B$ are square, this is a similarity transformation, so they share the same multiset of eigenvalues.
Q3: How is the vec operator related to the Kronecker product?
The vec operator stacks the columns of a matrix into a single column vector and satisfies the fundamental identity $\operatorname{vec}(AXB) = (B^T \otimes A)\operatorname{vec}(X)$. This identity reduces matrix equations such as the Sylvester equation or the Lyapunov equation to ordinary linear systems. Whether the solution is unique depends on whether the resulting coefficient matrix is invertible (for the Sylvester equation, whether $A$ and $-B$ have no common eigenvalue).
9. References
- Horn, R. A., & Johnson, C. R. (2012). Matrix Analysis, 2nd ed., Cambridge University Press. Cambridge University Press — Chapter 4 develops the theory of the Kronecker product.
- Laub, A. J. (2005). Matrix Analysis for Scientists and Engineers, SIAM. SIAM
- Henderson, H. V., Pukelsheim, F., & Searle, S. R. (1983). On the history of the Kronecker product. Linear and Multilinear Algebra, 14(2), 113–120. doi:10.1080/03081088308817548
- Wikipedia: Kronecker product
- Wikipedia: Tensor product