Learning NMF Through A Toy Example

Learning the mechanisms of the NMF update rule through a mathematical example.

Using NMF, we can understand and analyze the structure of complex matrices for a variety of different applications. Photo by charlesdeluvio on Unsplash.

Overview: What is NMF?

In this article, we’ll cover a common implementation of Non-Negative Matrix Factorization (NMF), a popular class of algorithms used across a variety of fields. Specifically, we’ll show how the multiplication rule [1] can be used for estimating the factor matrices in NMF.

Getting Started

Let’s say we want to apply NMF to the following matrix:

In other words, we want to find H and W such that:

With the further constraint imposed by NMF that all entries are non-negative in all three matrices. This X could stem from a variety of different data sources, and we can use and analyze W and H to understand the underlying structure of X better.

Since X is a 4x3 matrix, we can factorize this matrix into a 4x2
matrix multiplied with a 2x3 matrix. Written out:

We’ll first need to initialize W and H. We’ll start with them initialized as “all 1s”:

From here, we’re ready to see how the multiplication rule allows for iteratively refining W and H such that their product approximates X.