API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Jan 31st, 2015
181
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.66 KB | None | 0 0
raw download clone embed print report
  1. # Data actively being written - 100,000 counter updates
  2. bash-3.2# ./bin/nodetool flush
  3. bash-3.2# ./bin/sstableverify test metrics
  4. WARN 09:43:52 JNA link failure, one or more native method will be unavailable.
  5. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') (1987623 bytes)
  6. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') succeeded. All 20077 rows read successfully
  7. bash-3.2# ./bin/nodetool flush
  8. bash-3.2# ./bin/sstableverify test metrics
  9. WARN 09:44:02 JNA link failure, one or more native method will be unavailable.
  10. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-2-big-Data.db') (1315710 bytes)
  11. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-2-big-Data.db') succeeded. All 13290 rows read successfully
  12. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') (1987623 bytes)
  13. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') succeeded. All 20077 rows read successfully
  14. bash-3.2# ./bin/nodetool flush
  15. bash-3.2# ./bin/sstableverify test metrics
  16. WARN 09:44:16 JNA link failure, one or more native method will be unavailable.
  17. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-2-big-Data.db') (1315710 bytes)
  18. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-2-big-Data.db') succeeded. All 13290 rows read successfully
  19. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-3-big-Data.db') (1442628 bytes)
  20. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-3-big-Data.db') succeeded. All 14572 rows read successfully
  21. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') (1987623 bytes)
  22. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') succeeded. All 20077 rows read successfully
  23. bash-3.2# ./bin/nodetool verify
  24. bash-3.2# ./bin/nodetool verify test metrics
  25.  
  26. # At this point, all writing is done
  27. bash-3.2# ./bin/nodetool flush
  28. bash-3.2# ./bin/sstableverify test metrics
  29. WARN 09:44:58 JNA link failure, one or more native method will be unavailable.
  30. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') (1987623 bytes)
  31. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') succeeded. All 20077 rows read successfully
  32. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-4-big-Data.db') (1860210 bytes)
  33. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-4-big-Data.db') succeeded. All 18790 rows read successfully
  34. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-2-big-Data.db') (1315710 bytes)
  35. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-2-big-Data.db') succeeded. All 13290 rows read successfully
  36. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-3-big-Data.db') (1442628 bytes)
  37. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-3-big-Data.db') succeeded. All 14572 rows read successfully
  38. bash-3.2# ./bin/nodetool scrub
  39. bash-3.2# ./bin/sstableverify test metrics
  40. WARN 09:45:14 JNA link failure, one or more native method will be unavailable.
  41. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-5-big-Data.db') (1442628 bytes)
  42. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-5-big-Data.db') succeeded. All 14572 rows read successfully
  43. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-7-big-Data.db') (1860210 bytes)
  44. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-7-big-Data.db') succeeded. All 18790 rows read successfully
  45. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-8-big-Data.db') (1987623 bytes)
  46. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-8-big-Data.db') succeeded. All 20077 rows read successfully
  47. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-6-big-Data.db') (1315710 bytes)
  48. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-6-big-Data.db') succeeded. All 13290 rows read successfully
  49. bash-3.2# ./bin/nodetool compact
  50. bash-3.2# ./bin/sstableverify test metrics
  51. WARN 09:45:30 JNA link failure, one or more native method will be unavailable.
  52. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-9-big-Data.db') (6606171 bytes)
  53. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-9-big-Data.db') succeeded. All 66729 rows read successfully
  54.  
  55. # Now we intentionally corrupt the data, a single byte in the middle of the sstable
  56. bash-3.2# dd if=/dev/zero of=/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-9-big-Data.db bs=1 count=1 seek=657 conv=notrunc
  57. 1+0 records in
  58. 1+0 records out
  59. 1 bytes transferred in 0.000022 secs (45590 bytes/sec)
  60. bash-3.2# ./bin/sstableverify test metrics
  61. WARN 09:46:05 JNA link failure, one or more native method will be unavailable.
  62. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-9-big-Data.db') (6606171 bytes)
  63. Error verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-9-big-Data.db'): java.lang.Exception: Invalid SSTable, please force repair
  64. bash-3.2# ./bin/nodetool verify
  65. error: Invalid SSTable, please force repair
  66. -- StackTrace --
  67. java.lang.Exception: Invalid SSTable, please force repair
  68. at org.apache.cassandra.db.compaction.Verifier.verify(Verifier.java:190)
  69. at org.apache.cassandra.db.compaction.CompactionManager.verifyOne(CompactionManager.java:689)
  70. at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:98)
  71. at org.apache.cassandra.db.compaction.CompactionManager$4.execute(CompactionManager.java:348)
  72. at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:284)
  73. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  74. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  75. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  76. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  77. at java.lang.Thread.run(Thread.java:724)
  78.  
  79. # To recover, a full repair is required. Should discuss the 'right' way to trigger this (set sstable as unrepaired? immediately force repair? Unsure
  80. # For now, we kill the sstable and restart the node
  81. bash-3.2# ccm node1 stop
  82. bash-3.2# rm /Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-9-big-Data.db
  83. bash-3.2# ccm node1 start
  84. bash-3.2# ./bin/nodetool repair
  85. [2015-01-31 01:47:10,304] Starting repair command #1, repairing keyspace test with repair options (parallelism: parallel, primary range: false, incremental: true, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of ranges: 2)
  86. [2015-01-31 01:47:11,539] Repair session 2048cff0-a92e-11e4-bf65-8ff7b1eb6f10 for range (3074457345618258602,-9223372036854775808] finished
  87. [2015-01-31 01:47:11,805] Repair session 204a2f80-a92e-11e4-bf65-8ff7b1eb6f10 for range (-3074457345618258603,3074457345618258602] finished
  88. [2015-01-31 01:47:11,827] Repair command #1 finished in 1 second
  89. [2015-01-31 01:47:11,836] Nothing to repair for keyspace 'system_auth'
  90. [2015-01-31 01:47:11,844] Starting repair command #2, repairing keyspace system_traces with repair options (parallelism: parallel, primary range: false, incremental: true, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of ranges: 2)
  91. [2015-01-31 01:47:12,909] Repair session 21d31970-a92e-11e4-bf65-8ff7b1eb6f10 for range (3074457345618258602,-9223372036854775808] finished
  92. [2015-01-31 01:47:12,914] Repair session 21d31971-a92e-11e4-bf65-8ff7b1eb6f10 for range (-3074457345618258603,3074457345618258602] finished
  93. [2015-01-31 01:47:12,915] Repair command #2 finished in 1 second
  94. bash-3.2# ./bin/nodetool verify test metrics
  95. bash-3.2# ./bin/sstableverify test metrics
  96. WARN 09:47:39 JNA link failure, one or more native method will be unavailable.
  97. Verifying BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') (6606171 bytes)
  98. Verify of BigTableReader(path='/Users/jeff/.ccm/snapshot/node1/data/test/metrics-963efb90a92d11e48cd58ff7b1eb6f10/la-1-big-Data.db') succeeded. All 66729 rows read successfully
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!