code_junkie

Detecting programming language from a snippet

Nov 13th, 2011
180
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.15 KB | None | 0 0
  1. print "Hello"
  2.  
  3. def foo
  4. puts "hi"
  5. end
  6.  
  7. class Classifier
  8. def initialize
  9. @data = {}
  10. @totals = Hash.new(1.0)
  11. end
  12.  
  13. def words(code)
  14. code.split(/[^a-z]/).reject{|w| w.empty?}
  15. end
  16.  
  17. def train(code,lang)
  18. @data[lang] ||= Hash.new(1)
  19. for word in words(code)
  20. @data[lang][word] += 1
  21. @totals[word] += 1
  22. end
  23. end
  24.  
  25. def prob(words,lang)
  26. # We really want to multiply here but I use logs
  27. # to avoid floating point underflow
  28. # (adding logs is equivalent to multiplication)
  29. words.inject(0.0){|c,w| c + Math.log(@data[lang][w]/@totals[w])}
  30. end
  31.  
  32. def classify(code)
  33. ws = words(code)
  34. @data.keys.sort_by{|lang| prob(ws,lang)}.last
  35. end
  36. end
  37.  
  38. # Example usage
  39.  
  40. c = Classifier.new
  41.  
  42. # Train from files
  43. c.train(open("code.rb").read, :ruby)
  44. c.train(open("code.py").read, :python)
  45. c.train(open("code.cs").read, :csharp)
  46.  
  47. # Test it on another file
  48. c.classify(open("code2.py").read) # => :python (hopefully)
  49.  
  50. int i = 5;
  51. int k = 0;
  52. for (int j = 100 ; j > i ; i++) {
  53. j = j + 1000 / i;
  54. k = k + i * j;
  55. }
  56.  
  57. "package, class, implements "=> JAVA
  58. "<?php " => PHP
  59. "include main fopen strcmp stdout "=>C
  60. "cout"=> C++
  61. etc...
Add Comment
Please, Sign In to add comment