Advertisement
tinyevil

Untitled

Jun 6th, 2019
338
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.88 KB | None | 0 0
  1. 1. Lex keywords as identifiers. Compute simple hashsum while recognizing (acc = acc * N + x).
  2. 2. Make pass over tokens to resolve hashes
  3. consult ehash = EXACT_HASH[(hash >> (i * P)) & ((1 << P) - 1)] for i in 0..Q
  4. if ehash.key matches your hash, record ehash.value as the hash instead
  5. otherwise, record 0
  6.  
  7. N, P and Q make PHF
  8. N is choosen to minimize P and Q,
  9. P is the size of the EXACT_HASH table
  10. Q is the max collision size (usually 2)
  11.  
  12. 3. When new token begins:
  13. 3.a) add TOK[REC] to the retired token's value
  14. 3.b) set REC to REC_HASH[hash].
  15. 4. If CHECK[REC] == current character, advance to REC+1,
  16. otherwise, set REC=BADREC
  17.  
  18. Generation:
  19. TOK[X] = 0 if X does not recognize a keyword
  20. TOK[X] = TOKEN(Keyword) - TOKEN(Identifier) if X recognizes a keyword
  21.  
  22. REC_HASH[hash(X)] = X
  23.  
  24. CHECK[X + 0], CHECK[x + 1] etc equal to characters 0, 1, ... in X keyword
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement