Advertisement
Guest User

Untitled

a guest
Sep 20th, 2019
120
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.55 KB | None | 0 0
  1. # loading the tokenizer and vocab processors
  2. processor = get_roberta_processor(tokenizer=fastai_tokenizer, vocab=fastai_roberta_vocab)
  3.  
  4. # creating our databunch
  5. data = ItemLists(".", RobertaTextList.from_df(train, ".", cols=feat_cols, processor=processor),
  6. RobertaTextList.from_df(val, ".", cols=feat_cols, processor=processor)
  7. ) \
  8. .label_from_df(cols=label_cols, label_cls=CategoryList) \
  9. .add_test(RobertaTextList.from_df(test, ".", cols=feat_cols, processor=processor)) \
  10. .databunch(bs=4,pad_first=False)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement