API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 15th, 2017
118
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 17.22 KB | None | 0 0
raw download clone embed print report
  1. 1489559394.850593 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  2. 1489559394.850667 [0 lua] "EXISTS" "icicle-generator-lock"
  3. 1489559394.850686 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  4. 1489559394.850702 [0 lua] "GET" "icicle-generator-logical-shard-id"
  5. 1489559394.851752 [0 10.42.91.195:54612] "SCRIPT" "load" "local lock_key = 'icicle-generator-lock'\nlocal sequence_key = 'icicle-generator-sequence'\nlocal logical_shard_id_key = 'icicle-generator-logical-shard-id'\n\nlocal max_sequence = tonumber(KEYS[1])\nlocal min_logical_shard_id = tonumber(KEYS[2])\nlocal max_logical_shard_id = tonumber(KEYS[3])\nlocal num_ids = tonumber(KEYS[4])\n\nif redis.call('EXISTS', lock_key) == 1 then\n redis.log(redis.LOG_INFO, 'Icicle: Cannot generate ID, waiting for lock to expire.')\n return redis.error_reply('Icicle: Cannot generate ID, waiting for lock to expire.')\nend\n\n--[[\nIncrement by a set number, this can\n--]]\nlocal end_sequence = redis.call('INCRBY', sequence_key, num_ids)\nlocal start_sequence = end_sequence - num_ids + 1\nlocal logical_shard_id = tonumber(redis.call('GET', logical_shard_id_key)) or -1\n\nif end_sequence >= max_sequence then\n --[[\n As the sequence is about to roll around, we can't generate another ID until we're sure we're not in the same\n millisecond since we last rolled. This is because we may have already generated an ID with the same time and\n sequence, and we cannot allow even the smallest possibility of duplicates. It's also because if we roll the sequence\n around, we will start generating IDs with smaller values than the ones previously in this millisecond - that would\n break our k-ordering guarantees!\n\n The only way we can handle this is to block for a millisecond, as we can't store the time due the purity constraints\n of Redis Lua scripts.\n\n In addition to a neat side-effect of handling leap seconds (where milliseconds will last a little bit longer to bring\n time back to where it should be) because Redis uses system time internally to expire keys, this prevents any duplicate\n IDs from being generated if the rate of generation is greater than the maximum sequence per millisecond.\n\n Note that it only blocks even it rolled around *not* in the same millisecond; this is because unless we do this, the\n IDs won't remain ordered.\n --]]\n redis.log(redis.LOG_INFO, 'Icicle: Rolling sequence back to the start, locking for 1ms.')\n redis.call('SET', sequence_key, '-1')\n redis.call('PSETEX', lock_key, 1, 'lock')\n end_sequence = max_sequence\nend\n\n--[[\nThe TIME command MUST be called after anything that mutates state, or the Redis server will error the script out.\nThis is to ensure the script is \"pure\" in the sense that randomness or time based input will not change the\noutcome of the writes.\n\nSee the \"Scripts as pure functions\" section at http://redis.io/commands/eval for more information.\n--]]\nlocal time = redis.call('TIME')\n\nreturn {\n start_sequence,\n end_sequence, -- Doesn't need conversion, the result of INCR or the variable set is always a number.\n logical_shard_id,\n tonumber(time[1]),\n tonumber(time[2])\n}\n"
  6. 1489559394.852512 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  7. 1489559394.852531 [0 lua] "EXISTS" "icicle-generator-lock"
  8. 1489559394.852539 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  9. 1489559394.852546 [0 lua] "GET" "icicle-generator-logical-shard-id"
  10. 1489559394.853024 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  11. 1489559394.853047 [0 lua] "EXISTS" "icicle-generator-lock"
  12. 1489559394.853057 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  13. 1489559394.853070 [0 lua] "GET" "icicle-generator-logical-shard-id"
  14. 1489559394.853809 [0 10.42.91.195:54612] "SCRIPT" "load" "local lock_key = 'icicle-generator-lock'\nlocal sequence_key = 'icicle-generator-sequence'\nlocal logical_shard_id_key = 'icicle-generator-logical-shard-id'\n\nlocal max_sequence = tonumber(KEYS[1])\nlocal min_logical_shard_id = tonumber(KEYS[2])\nlocal max_logical_shard_id = tonumber(KEYS[3])\nlocal num_ids = tonumber(KEYS[4])\n\nif redis.call('EXISTS', lock_key) == 1 then\n redis.log(redis.LOG_INFO, 'Icicle: Cannot generate ID, waiting for lock to expire.')\n return redis.error_reply('Icicle: Cannot generate ID, waiting for lock to expire.')\nend\n\n--[[\nIncrement by a set number, this can\n--]]\nlocal end_sequence = redis.call('INCRBY', sequence_key, num_ids)\nlocal start_sequence = end_sequence - num_ids + 1\nlocal logical_shard_id = tonumber(redis.call('GET', logical_shard_id_key)) or -1\n\nif end_sequence >= max_sequence then\n --[[\n As the sequence is about to roll around, we can't generate another ID until we're sure we're not in the same\n millisecond since we last rolled. This is because we may have already generated an ID with the same time and\n sequence, and we cannot allow even the smallest possibility of duplicates. It's also because if we roll the sequence\n around, we will start generating IDs with smaller values than the ones previously in this millisecond - that would\n break our k-ordering guarantees!\n\n The only way we can handle this is to block for a millisecond, as we can't store the time due the purity constraints\n of Redis Lua scripts.\n\n In addition to a neat side-effect of handling leap seconds (where milliseconds will last a little bit longer to bring\n time back to where it should be) because Redis uses system time internally to expire keys, this prevents any duplicate\n IDs from being generated if the rate of generation is greater than the maximum sequence per millisecond.\n\n Note that it only blocks even it rolled around *not* in the same millisecond; this is because unless we do this, the\n IDs won't remain ordered.\n --]]\n redis.log(redis.LOG_INFO, 'Icicle: Rolling sequence back to the start, locking for 1ms.')\n redis.call('SET', sequence_key, '-1')\n redis.call('PSETEX', lock_key, 1, 'lock')\n end_sequence = max_sequence\nend\n\n--[[\nThe TIME command MUST be called after anything that mutates state, or the Redis server will error the script out.\nThis is to ensure the script is \"pure\" in the sense that randomness or time based input will not change the\noutcome of the writes.\n\nSee the \"Scripts as pure functions\" section at http://redis.io/commands/eval for more information.\n--]]\nlocal time = redis.call('TIME')\n\nreturn {\n start_sequence,\n end_sequence, -- Doesn't need conversion, the result of INCR or the variable set is always a number.\n logical_shard_id,\n tonumber(time[1]),\n tonumber(time[2])\n}\n"
  15. 1489559394.854558 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  16. 1489559394.854581 [0 lua] "EXISTS" "icicle-generator-lock"
  17. 1489559394.854591 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  18. 1489559394.854601 [0 lua] "GET" "icicle-generator-logical-shard-id"
  19. 1489559394.856173 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  20. 1489559394.856196 [0 lua] "EXISTS" "icicle-generator-lock"
  21. 1489559394.856206 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  22. 1489559394.856215 [0 lua] "GET" "icicle-generator-logical-shard-id"
  23. 1489559394.856919 [0 10.42.91.195:54612] "SCRIPT" "load" "local lock_key = 'icicle-generator-lock'\nlocal sequence_key = 'icicle-generator-sequence'\nlocal logical_shard_id_key = 'icicle-generator-logical-shard-id'\n\nlocal max_sequence = tonumber(KEYS[1])\nlocal min_logical_shard_id = tonumber(KEYS[2])\nlocal max_logical_shard_id = tonumber(KEYS[3])\nlocal num_ids = tonumber(KEYS[4])\n\nif redis.call('EXISTS', lock_key) == 1 then\n redis.log(redis.LOG_INFO, 'Icicle: Cannot generate ID, waiting for lock to expire.')\n return redis.error_reply('Icicle: Cannot generate ID, waiting for lock to expire.')\nend\n\n--[[\nIncrement by a set number, this can\n--]]\nlocal end_sequence = redis.call('INCRBY', sequence_key, num_ids)\nlocal start_sequence = end_sequence - num_ids + 1\nlocal logical_shard_id = tonumber(redis.call('GET', logical_shard_id_key)) or -1\n\nif end_sequence >= max_sequence then\n --[[\n As the sequence is about to roll around, we can't generate another ID until we're sure we're not in the same\n millisecond since we last rolled. This is because we may have already generated an ID with the same time and\n sequence, and we cannot allow even the smallest possibility of duplicates. It's also because if we roll the sequence\n around, we will start generating IDs with smaller values than the ones previously in this millisecond - that would\n break our k-ordering guarantees!\n\n The only way we can handle this is to block for a millisecond, as we can't store the time due the purity constraints\n of Redis Lua scripts.\n\n In addition to a neat side-effect of handling leap seconds (where milliseconds will last a little bit longer to bring\n time back to where it should be) because Redis uses system time internally to expire keys, this prevents any duplicate\n IDs from being generated if the rate of generation is greater than the maximum sequence per millisecond.\n\n Note that it only blocks even it rolled around *not* in the same millisecond; this is because unless we do this, the\n IDs won't remain ordered.\n --]]\n redis.log(redis.LOG_INFO, 'Icicle: Rolling sequence back to the start, locking for 1ms.')\n redis.call('SET', sequence_key, '-1')\n redis.call('PSETEX', lock_key, 1, 'lock')\n end_sequence = max_sequence\nend\n\n--[[\nThe TIME command MUST be called after anything that mutates state, or the Redis server will error the script out.\nThis is to ensure the script is \"pure\" in the sense that randomness or time based input will not change the\noutcome of the writes.\n\nSee the \"Scripts as pure functions\" section at http://redis.io/commands/eval for more information.\n--]]\nlocal time = redis.call('TIME')\n\nreturn {\n start_sequence,\n end_sequence, -- Doesn't need conversion, the result of INCR or the variable set is always a number.\n logical_shard_id,\n tonumber(time[1]),\n tonumber(time[2])\n}\n"
  24. 1489559394.857593 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  25. 1489559394.857617 [0 lua] "EXISTS" "icicle-generator-lock"
  26. 1489559394.857627 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  27. 1489559394.857636 [0 lua] "GET" "icicle-generator-logical-shard-id"
  28. 1489559394.862116 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  29. 1489559394.862139 [0 lua] "EXISTS" "icicle-generator-lock"
  30. 1489559394.862149 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  31. 1489559394.862159 [0 lua] "GET" "icicle-generator-logical-shard-id"
  32. 1489559394.862870 [0 10.42.91.195:54612] "SCRIPT" "load" "local lock_key = 'icicle-generator-lock'\nlocal sequence_key = 'icicle-generator-sequence'\nlocal logical_shard_id_key = 'icicle-generator-logical-shard-id'\n\nlocal max_sequence = tonumber(KEYS[1])\nlocal min_logical_shard_id = tonumber(KEYS[2])\nlocal max_logical_shard_id = tonumber(KEYS[3])\nlocal num_ids = tonumber(KEYS[4])\n\nif redis.call('EXISTS', lock_key) == 1 then\n redis.log(redis.LOG_INFO, 'Icicle: Cannot generate ID, waiting for lock to expire.')\n return redis.error_reply('Icicle: Cannot generate ID, waiting for lock to expire.')\nend\n\n--[[\nIncrement by a set number, this can\n--]]\nlocal end_sequence = redis.call('INCRBY', sequence_key, num_ids)\nlocal start_sequence = end_sequence - num_ids + 1\nlocal logical_shard_id = tonumber(redis.call('GET', logical_shard_id_key)) or -1\n\nif end_sequence >= max_sequence then\n --[[\n As the sequence is about to roll around, we can't generate another ID until we're sure we're not in the same\n millisecond since we last rolled. This is because we may have already generated an ID with the same time and\n sequence, and we cannot allow even the smallest possibility of duplicates. It's also because if we roll the sequence\n around, we will start generating IDs with smaller values than the ones previously in this millisecond - that would\n break our k-ordering guarantees!\n\n The only way we can handle this is to block for a millisecond, as we can't store the time due the purity constraints\n of Redis Lua scripts.\n\n In addition to a neat side-effect of handling leap seconds (where milliseconds will last a little bit longer to bring\n time back to where it should be) because Redis uses system time internally to expire keys, this prevents any duplicate\n IDs from being generated if the rate of generation is greater than the maximum sequence per millisecond.\n\n Note that it only blocks even it rolled around *not* in the same millisecond; this is because unless we do this, the\n IDs won't remain ordered.\n --]]\n redis.log(redis.LOG_INFO, 'Icicle: Rolling sequence back to the start, locking for 1ms.')\n redis.call('SET', sequence_key, '-1')\n redis.call('PSETEX', lock_key, 1, 'lock')\n end_sequence = max_sequence\nend\n\n--[[\nThe TIME command MUST be called after anything that mutates state, or the Redis server will error the script out.\nThis is to ensure the script is \"pure\" in the sense that randomness or time based input will not change the\noutcome of the writes.\n\nSee the \"Scripts as pure functions\" section at http://redis.io/commands/eval for more information.\n--]]\nlocal time = redis.call('TIME')\n\nreturn {\n start_sequence,\n end_sequence, -- Doesn't need conversion, the result of INCR or the variable set is always a number.\n logical_shard_id,\n tonumber(time[1]),\n tonumber(time[2])\n}\n"
  33. 1489559394.863532 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  34. 1489559394.863555 [0 lua] "EXISTS" "icicle-generator-lock"
  35. 1489559394.863565 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  36. 1489559394.863575 [0 lua] "GET" "icicle-generator-logical-shard-id"
  37. 1489559394.873052 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  38. 1489559394.873075 [0 lua] "EXISTS" "icicle-generator-lock"
  39. 1489559394.873085 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  40. 1489559394.873095 [0 lua] "GET" "icicle-generator-logical-shard-id"
  41. 1489559394.873798 [0 10.42.91.195:54612] "SCRIPT" "load" "local lock_key = 'icicle-generator-lock'\nlocal sequence_key = 'icicle-generator-sequence'\nlocal logical_shard_id_key = 'icicle-generator-logical-shard-id'\n\nlocal max_sequence = tonumber(KEYS[1])\nlocal min_logical_shard_id = tonumber(KEYS[2])\nlocal max_logical_shard_id = tonumber(KEYS[3])\nlocal num_ids = tonumber(KEYS[4])\n\nif redis.call('EXISTS', lock_key) == 1 then\n redis.log(redis.LOG_INFO, 'Icicle: Cannot generate ID, waiting for lock to expire.')\n return redis.error_reply('Icicle: Cannot generate ID, waiting for lock to expire.')\nend\n\n--[[\nIncrement by a set number, this can\n--]]\nlocal end_sequence = redis.call('INCRBY', sequence_key, num_ids)\nlocal start_sequence = end_sequence - num_ids + 1\nlocal logical_shard_id = tonumber(redis.call('GET', logical_shard_id_key)) or -1\n\nif end_sequence >= max_sequence then\n --[[\n As the sequence is about to roll around, we can't generate another ID until we're sure we're not in the same\n millisecond since we last rolled. This is because we may have already generated an ID with the same time and\n sequence, and we cannot allow even the smallest possibility of duplicates. It's also because if we roll the sequence\n around, we will start generating IDs with smaller values than the ones previously in this millisecond - that would\n break our k-ordering guarantees!\n\n The only way we can handle this is to block for a millisecond, as we can't store the time due the purity constraints\n of Redis Lua scripts.\n\n In addition to a neat side-effect of handling leap seconds (where milliseconds will last a little bit longer to bring\n time back to where it should be) because Redis uses system time internally to expire keys, this prevents any duplicate\n IDs from being generated if the rate of generation is greater than the maximum sequence per millisecond.\n\n Note that it only blocks even it rolled around *not* in the same millisecond; this is because unless we do this, the\n IDs won't remain ordered.\n --]]\n redis.log(redis.LOG_INFO, 'Icicle: Rolling sequence back to the start, locking for 1ms.')\n redis.call('SET', sequence_key, '-1')\n redis.call('PSETEX', lock_key, 1, 'lock')\n end_sequence = max_sequence\nend\n\n--[[\nThe TIME command MUST be called after anything that mutates state, or the Redis server will error the script out.\nThis is to ensure the script is \"pure\" in the sense that randomness or time based input will not change the\noutcome of the writes.\n\nSee the \"Scripts as pure functions\" section at http://redis.io/commands/eval for more information.\n--]]\nlocal time = redis.call('TIME')\n\nreturn {\n start_sequence,\n end_sequence, -- Doesn't need conversion, the result of INCR or the variable set is always a number.\n logical_shard_id,\n tonumber(time[1]),\n tonumber(time[2])\n}\n"
  42. 1489559394.874520 [0 10.42.91.195:54612] "EVALSHA" "ea1abaf3fe8531d6620c759fcb98f30e6728341f" "4" "4095" "1" "1023" "1"
  43. 1489559394.874543 [0 lua] "EXISTS" "icicle-generator-lock"
  44. 1489559394.874554 [0 lua] "INCRBY" "icicle-generator-sequence" "1"
  45. 1489559394.874563 [0 lua] "GET" "icicle-generator-logical-shard-id"
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!