Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- if inputs_embeds is None:
- inputs_embeds = self.wte(input_ids)
- position_embeds = self.wpe(position_ids)
- if token_type_ids is not None:
- token_type_embeds = self.wte(token_type_ids)
- else:
- token_type_embeds = 0
- hidden_states = inputs_embeds + position_embeds + token_type_embeds
- hidden_states = self.drop(hidden_states)
- output_shape = input_shape + (hidden_states.size(-1),)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement