Facultad de Ingeniera Departamento de Sistemas e Industrial Machine Learning 2do Semestre de 2011

Assignment 1: Linear Algebra and Probability.

Anibal Montero L. 162253 Delivered to: Prof. Fabio A. Gonzalez. 1. Let D = {d1dn} be a set of documents and T = {t1tm} a set of terms (words). Let TD =

(TDij) i=1...m, j=1...n be a matrix such that TDij corresponds to the number of times the term ti appears in the document dj. Assume a process where a document dj is randomly choosen with uniform probability and then a term ti, present in dj, is randomly choosen with a probability proportional to the frequency of ti in dj.

(a) How do you transform the matrix TD to obtain a matrix P(T,D), such that P(T,D)ij= P(ti,

dj) (the joint probability of term ti and document dj)?. (b) How do you obtain the P(T |D) matrix?

According that was mentioned:

Then:

Where:

(c) How do you obtain the P(D |T) matrix?

According to the total probability theorem and the last results:

Besides that:

2. Let li be the length, number of characters, of term ti, and let L = (li...lm) be a column vector.

(a) Calculate E[l], the expected value of the random variable l corresponding to the length of a randomly choosen term. Let:

Then:

And thus:

(b) Calculate Var(l), the variance of l.

(c) Show that Var(al + b) = a2Var(l), where a and b are arbitrary constants.

. . .

3. Find an expression for P(T |D) that exclusively uses P(D |T), P(T, D) and, optionally, constant matrices/vectors. Thats simple, because according the last results:

4. Find an expression for the matrix Cov = ( Cov(ti,ti) ) i,i=1...m, the covariance of the random variables corresponding to terms. Given the definition of covariance between random vectors:

That matrix product leads the covariance between pairs of terms(rows) , it gives the generic value of a covariance matrix:

5. (optional) What is the meaning of the Eigenvector of Cov corresponding to the largest

eigenvalue?. The largest eigenvalue measures the variance of the first principal component, i.e., measures the variance of the axis that maximizes the projections of each element (row) of the matrix.