SHARE
TWEET

Untitled

tinyevil Jun 6th, 2019 82 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. 1. Lex keywords as identifiers. Compute simple hashsum while recognizing (acc = acc * N + x).
  2. 2. Make pass over tokens to resolve hashes
  3.     consult ehash = EXACT_HASH[(hash >> (i * P)) & ((1 << P) - 1)] for i in 0..Q
  4.         if ehash.key matches your hash, record ehash.value as the hash instead
  5.         otherwise, record 0
  6.        
  7.     N, P and Q make PHF
  8.     N is choosen to minimize P and Q,
  9.     P is the size of the EXACT_HASH table
  10.     Q is the max collision size (usually 2)
  11.    
  12. 3. When new token begins:
  13.     3.a) add TOK[REC] to the retired token's value
  14.     3.b) set REC to REC_HASH[hash].
  15. 4. If CHECK[REC] == current character, advance to REC+1,
  16.     otherwise, set REC=BADREC
  17.  
  18. Generation:
  19.     TOK[X] = 0 if X does not recognize a keyword
  20.     TOK[X] = TOKEN(Keyword) - TOKEN(Identifier) if X recognizes a keyword
  21.    
  22.     REC_HASH[hash(X)] = X
  23.    
  24.     CHECK[X + 0], CHECK[x + 1] etc equal to characters 0, 1, ... in X keyword
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top