Guest User

Untitled

a guest
Apr 10th, 2018
255
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.90 KB | None | 0 0
  1. ```sh
  2. mysql> select count(*) from users;
  3. +----------+
  4. | count(*) |
  5. +----------+
  6. | 25305798 |
  7. +----------+
  8. 1 row in set (10.24 sec)
  9.  
  10. # Inplace, rebuilds table, allows DML
  11. mysql -u root experiments --execute="alter table users add column ptfun varchar(50) default 'pt fun'"
  12. 106s (1min 46s)
  13.  
  14. # Starts with 1k chunk for first select query followed by 30k and then straight away to 51-56k
  15. pt-online-schema-change --alter "add column ptfun varchar(50) default 'pt fun'" D=experiments,t=users,u=root --execute --alter-foreign-keys-method rebuild_constraints
  16. 297s (4min 57s)
  17.  
  18. # made changes to the online schema code to work on host, instead of socket
  19. # With 9k chunks:
  20. osc_cli copy --ddl-file=alter.sql --database=experiments --socket=/tmp/mysql.socket --mysql-user=root --mysql-password= --outfile-dir=/Users/rishabhpugalia/www/dev/percona/outfile
  21. 689s (10min 29s)
  22.  
  23. # With 60k chunks:
  24. osc_cli copy --ddl-file=alter.sql --database=experiments --socket=/tmp/mysql.socket --mysql-user=root --mysql-password= --outfile-dir=/Users/rishabhpugalia/www/dev/percona/outfile --chunk-size=15000000
  25. 559s (9m 19s)
  26.  
  27. 2m vs 5m vs 10.5m || 9.2m
  28. ```
  29.  
  30. For FB, had to make these changes:
  31.  
  32. ```sh
  33. $ cat /etc/my.cnf
  34. [mysqld]
  35.  
  36. binlog-format = 'STATEMENT'
  37. secure-file-priv = ''
  38. ```
  39.  
  40. In `db.py`, made this change in `connection_config` variable:
  41.  
  42. ```sh
  43. 'host':'localhost',
  44. # 'unix_socket': socket,
  45. ```
  46.  
  47. ### Differences Noticed
  48.  
  49. DB selects data into OUTFILE and then LOADs data INFILE in the defined chunks. Finally it does checksumming based on queries like these:
  50.  
  51. ```sql
  52. 2018-04-03T04:09:23.520806Z 77 Query /* /usr/local/bin/osc_cli:db.pyc */ SELECT count(*) as cnt, bit_xor(crc32(@range_end_0:=`id`)), bit_xor(crc32(`name`)), bit_xor(crc32(`email`)), bit_xor(crc32(`active`)), bit_xor(crc32(`active_woi`)), bit_xor(crc32(`bambam5`)), bit_xor(crc32(`bambam6`)), bit_xor(crc32(`booboo5`)) FROM ( SELECT * FROM `__osc_new_users` FORCE INDEX (`PRIMARY`) WHERE ( `id` > @range_start_0 ) ORDER BY `id` LIMIT 64935 ) as tmp
  53. ```
  54.  
  55. PT does simple insert into select and doesn't execute any specific query for checksumming. Might be happening inside their code but nothing mentioned in the doc, command's output or general log.
  56.  
  57. ```
  58. 18-04-03T04:19:18.231881Z 81 Query EXPLAIN SELECT `id`, `name`, `email`, `active`, `active_woi`, `bambam5`, `bambam6`, `booboo5` FROM `experiments`.`users` FORCE INDEX(`PRIMARY`) WHERE ((`id` >= '29798504')) AND ((`id` <= '29854092')) LOCK IN SHARE MODE /*explain pt-online-schema-change 30864 copy nibble*/
  59. 2018-04-03T04:19:18.232317Z 81 Query INSERT LOW_PRIORITY IGNORE INTO `experiments`.`_users_new` (`id`, `name`, `email`, `active`, `active_woi`, `bambam5`, `bambam6`, `booboo5`) SELECT `id`, `name`, `email`, `active`, `active_woi`, `bambam5`, `bambam6`, `booboo5` FROM `experiments`.`users` FORCE INDEX(`PRIMARY`) WHERE ((`id` >= '29798504')) AND ((`id` <= '29854092')) LOCK IN SHARE MODE /*pt-online-schema-change 30864 copy nibble*/
  60. ```
Add Comment
Please, Sign In to add comment