Hadamard Product

Entrywise Matrix Product

Intermediate (second-year university level)

Published: 2026-03-20

Definition and basic properties

For matrices $A, B \in \mathbb{R}^{m \times n}$ of the same size, the Hadamard product (symbol $\circ$ or $\odot$) is defined entrywise:

$$(A \circ B)_{ij} = a_{ij} b_{ij}$$

The basic properties are as follows:

  • Commutativity: $A \circ B = B \circ A$
  • Associativity: $(A \circ B) \circ C = A \circ (B \circ C)$
  • Distributivity: $A \circ (B + C) = A \circ B + A \circ C$
  • Identity: the all-ones matrix $J$ ($A \circ J = A$)
  • Transpose: $(A \circ B)^T = A^T \circ B^T$
A 2 3 4 1 ∘ B 5 6 7 8 = A ∘ B 10 18 28 8 Multiply entries independently: (A∘B)_{ij} = a_{ij} · b_{ij}
Figure 1: The Hadamard product multiplies corresponding entries.

Schur product theorem

Theorem (Schur product theorem): If $A, B$ are $n \times n$ positive semidefinite matrices, then their Hadamard product $A \circ B$ is also positive semidefinite. Moreover, if both $A$ and $B$ are positive definite, then $A \circ B$ is also positive definite.

Sketch of proof

Let the eigendecomposition of $B$ be $B = \displaystyle\sum_{k=1}^n \lambda_k v_k v_k^T$ with $\lambda_k \geq 0$. For any vector $x$:

$$x^T(A \circ B)x = \displaystyle\sum_{i,j} a_{ij} b_{ij} x_i x_j = \displaystyle\sum_{k=1}^n \lambda_k \displaystyle\sum_{i,j} a_{ij} (v_k)_i (v_k)_j x_i x_j = \displaystyle\sum_{k=1}^n \lambda_k (x \circ v_k)^T A (x \circ v_k) \geq 0$$

Here $x \circ v_k$ denotes the entrywise product $(x_i (v_k)_i)_{i=1}^n$. The final inequality follows from $\lambda_k \geq 0$ for each $k$ and the positive semidefiniteness of $A$.

A ≥ 0 positive semidefinite B ≥ 0 positive semidefinite ∘ A ∘ B ≥ 0 positive semidefinite Schur product theorem: positive semidefiniteness is preserved under the Hadamard product (it is not preserved in general by the ordinary matrix product)
Figure 2: The Schur product theorem. The Hadamard product of positive semidefinite matrices is again positive semidefinite.

Oppenheim inequality

Theorem (Oppenheim): For $n \times n$ positive semidefinite matrices $A, B$:

$$\det(A \circ B) \geq \det(A) \displaystyle\prod_{i=1}^n b_{ii}$$

By symmetry, swapping the roles of $A$ and $B$ also yields:

$$\det(A \circ B) \geq \det(B) \displaystyle\prod_{i=1}^n a_{ii}$$

In particular, when $B$ is a correlation matrix with unit diagonal, $\det(A \circ B) \geq \det(A)$ holds. This inequality can be viewed as a generalization of Hadamard's determinant inequality $\det(A) \leq \displaystyle\prod_i a_{ii}$.

Relation to the ordinary matrix product

The Hadamard product and the ordinary matrix product are related as follows:

  • Diagonal-matrix case: If $A$ and $B$ are both diagonal, then $A \circ B = AB$ (a rare situation where the Hadamard product coincides with the ordinary matrix product)
  • Vector case: $a \circ b$ can be written using the diagonal matrix $\text{diag}(a)$ as $a \circ b = \text{diag}(a) b$
  • Trace identity: $\text{tr}(A^T B) = \mathbf{1}^T (A \circ B) \mathbf{1} = \displaystyle\sum_{ij} a_{ij} b_{ij}$ (the Frobenius inner product)
  • Kronecker product: $A \circ B$ appears as a submatrix (a diagonal block) of $A \otimes B$

Applications

  • Statistics: structured covariance matrices (spatial correlation $\circ$ temporal correlation)
  • Machine learning: gating mechanisms (entrywise products in LSTM/GRU), attention masks
  • Signal processing: window function application, spectral masking
  • Image processing: pixelwise filtering

Summary

  • The Hadamard product is the entrywise product and satisfies commutativity, associativity, and distributivity
  • By the Schur product theorem, the Hadamard product of positive semidefinite matrices preserves positive semidefiniteness
  • The Oppenheim inequality gives a lower bound on the determinant
  • It is widely used in settings where entrywise operations are essential, such as machine learning and signal processing

References