Non-negative matrix factorisation
Non-negative Matrix Factorisation, shortened as NMF, is a technique, often employed in recommender systems and in computer vision, to create an (approximate) factorisation of a big matrix into two sensibly smaller matrices in a way that elements are non-negative.
Let's say you have a large
, with NMF you are able to factorise it into the product of matrices
), with the property that
playing the role of the hidden features in the data. The typical use case is that of a big sparse matrix factorised into two smaller dense matrices, which are much easier to deal with for computations.
Often, an approximate factorisation is desired, so that in reality
is chosen precisely in such a way that
is a reasonable approximation of
. The real full decomposition is
is the residuals matrix, whose elements can be either positive or negative.
The usual way to proceed in finding
is to use gradient descent (see page) by looking at minimising the difference between
with the constraint of non-negativity.
Let's say we are working with text and doing some NLP on documents. We have a term-document matrix (see the page linked)
and we would like to extract 10 features in order to generate a features matrix
rows and 10 columns and a coefficient matrix
with 10 rows and
columns. The product of
would then have
columns, working as a reasonable approximation of the original
Other typical applications are those where you have the matrix of users' ratings on items (products, movies, ...) and you'd want to discover some
will give you the association of users to these hidden features and
the association of the items to these hidden features. Once obtained the factorisation, you can use it to predict the rating a given user
would give to an item