Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- One thing that appears to be true is the ranking of the outputs are rather similar.
- While they are not identical, they are within a reasonable margin of error.
- This could be due to a number of causes, my implementation could not be calculating the cosine similarity the same way as Lucene.
- However, the fact that the outputrs are relatively similar is reassuring.
- Furthermore, Lucene could be weighing certain terms differently than I am.
- Lucene could know that "the" for example, can be dropped from the query or removed from the results entirely.
- In my implementation, this is not done.
- Another important thing to note is that in QueryID 4, there are plurals and singular versions of the same words:
- "bulb" - "bulbs" and "alternative" - "alternatives"
- The stem of these words could be obtained and the outputs would be much different.
- Furthermore, we are not considering the correlation between the terms in the query.
- We are not forcing "global warming potential" to all be in the same document.
- We are also not considering synonyms for words.
- An example would be "california" -> "The Golden State"
- Documents could mention one or the other, or both.
Advertisement
Add Comment
Please, Sign In to add comment