Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- def getTokens(data:RDD[(String,String)]):RDD[(String,List[String])]={
- data.map(x => (Utils.parseLine(x._2)._1.toString(),
- EntityResolution.tokenize(Utils.parseLine(x._2)._1.toString(), stopWords)))
- }
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement