Similarity Models

Models studied (formulas & behavior):

1. cosine similarity: |v1.v2|/(||v1||*||v2||)

2. euclidean distance: |v1-v2| = ||v1||**2+||v2||**2-2|v1.v2|

3. hamming distance: how many bits are different, but this does not work in our case because here one parameter can give similar behavior in a range of values.

4. boolean model: better for comparing text not numbers
a) this model first takes a set of stemmed corpus of words
b) then for each word in query which belongs it checks in which docs it lies and make set of docs
c) then intersection of those set gives the result

5. extended boolean model: mixture of vector model and boolean model but better for comparing text not numbers

a) first find frequency of each word in vector in all documents then multiply it idf(x)/max(idf)

v(j)=[w1,j:w2,j....wn,j]
wx,j=f(x,j)*idf(x)/max(idf)

b) query is given as booleans (x and y) / (x or y)

sim(or,i,j)=sqrt(((w(x,i))^2+(w(y,j))^2/2)
sim(and,i,j)=1-sqrt(((1-w(x,i))^2+(1-w(y,j))^2)/2)


6. we can use weighted cosine similarity:
(sum w[i]*u[i]*v[i]) / sqrt[(sum w[i]*u[i]^2)*(sum w[i]*v[i]^2)]
but we don't know what weight to be given to individual parameter.

7. weighted euclidean distance: if elements are varying more than the threshold value then it has to be given higher weight so that sum of difference increases which is obvious.