Advertisement
Guest User

Untitled

a guest
Nov 14th, 2018
83
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 0.47 KB | None | 0 0
  1. def processNlToks(nlToks):
  2.   return [tok.encode('ascii', 'replace').decode().strip() for tok in nlToks \
  3.            if tok != "-RCB-" and \
  4.            tok != "-LCB-" and \
  5.            tok != "-LSB-" and \
  6.            tok != "-RSB-" and \
  7.            tok != "-LRB-" and \
  8.            tok != "-RRB-" and \
  9.            tok != "@link" and \
  10.            tok != "@code" and \
  11.            tok != "@inheritDoc" and \
  12.            tok.encode('ascii', 'replace').decode().strip() != '']
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement