itsjube

Untitled

Nov 20th, 2016
131
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.45 KB | None | 0 0
  1. -- Query ID 1 --
  2. In both the Lucene result, and my search engine "Global warming" is the first result, for the query "global warming potential".
  3. More than likely, this is because the weight of each term is different for Lucene and my Search Engine.
  4. However, I would argue for this query my result is better. If you look at the second result for Lucene, it is: EnergyConsumption
  5. Mine is: 88888_Lights_Out.
  6. Lucene's result is a nearly empty page with links to various energy consumption pages in Wikipedia.
  7. Mine result is a campaign to raise awareness of global warming.
  8. Therefore, my result is more relevent to the query.
  9. If I were searching for "global warming potential" I would prefer to get a page about a campaign to raise its awareness, than a page with energy consumption links.
  10. I did not search for "energy consumption", I searched for "global warming potential".
  11. This trend continues for the 3rd result for Lucenece: Environmental_impact_of_the_energy_industry while mine is: Joseph_Romm, a leading scientist of global warming.
  12. This trend continues for the rest of the results.
  13. It would seem this is due to Lucene's customized scoring algorithm which is a combination of the Vector Space Model and the Boolean Model.
  14.  
  15. -- Query ID 2 --
  16.  
  17.  
  18. --Query ID 3 --
  19.  
  20.  
  21. --Query ID 4 --
  22.  
  23.  
  24.  
  25.  
  26.  
  27. ---- Generic ----
  28.  
  29. One thing that appears to be true is the ranking of the outputs are rather similar.
  30. While they are not identical, they are within a reasonable margin of error.
  31. This could be due to a number of causes, my implementation could not be calculating the cosine similarity the same way as Lucene.
  32. However, the fact that the outputs are relatively similar is reassuring.
  33. Furthermore, Lucene could be weighing certain terms differently than I am.
  34. Lucene could know that "the" for example, can be dropped from the query or removed from the results entirely.
  35. In my implementation, this is not done.
  36. Another important thing to note is that in QueryID 4, there are plurals and singular versions of the same words:
  37. "bulb" - "bulbs" and "alternative" - "alternatives"
  38. The stem of these words could be obtained and the outputs would be much different.
  39.  
  40.  
  41. Furthermore, we are not considering the correlation between the terms in the query.
  42. We are not forcing "global warming potential" to all be in the same document.
  43. We are also not considering synonyms for words.
  44. An example would be "california" -> "The Golden State"
  45.  
  46. Documents could mention one or the other, or both.
Advertisement
Add Comment
Please, Sign In to add comment