‎[19:48] ‎<‎gryffus‎>‎  Anyone have idea why i cannot start mysql with same configuration that is working on other servers? I even get no output from mysql: https://pastebin.com/raw/dqQbdkwz  . What is errno= 32 ?
‎[19:49] <-- jdavila (~Davila@200.125.188.214) opustil server (Quit: Leaving).
‎[19:49] --> jdavila (~Davila@200.125.188.214) se připojil ke kanálu.
‎[19:50] ‎<‎litheum‎>‎ !perror 32
‎[19:50] ‎<‎ubiquity_bot‎>‎ OS error code 32: Broken pipe
‎[19:50] ‎<‎gryffus‎>‎ hmm, so it just dies without any message? How can i debug this?
‎[19:50] ‎<‎litheum‎>‎ check your syslog or journal or whatnot to see if there's anything else in there, that there must be some more info someplace
‎[19:50] <-- gugah (~gugah@181.169.151.140) opustil server (Quit: Leaving).
‎[19:52] ‎<‎gryffus‎>‎ litheum: there really is not... dmesg, journalctl, /var/log/mysqld.log, nothing, nowhere, only what i have posted
‎[19:53] ‎<‎litheum‎>‎ what OS are you using? and what package did you download and install?
‎[19:53] ‎<‎gryffus‎>‎ litheum: RHEL 7.3
‎[19:53] ‎<‎litheum‎>‎ i wonder if by default mysqld --initialize will try to use /var/lib/mysql, did you look there? have you tried mysqld --initialize --datadir=/opt/data/ ?
‎[19:54] ‎<‎gryffus‎>‎ and mysql-community-server-5.7.19-1.el7.x86_64
‎[19:54] ‎<‎salle‎>‎ gryffus: Check Selinux config
‎[19:54] <-- evilman_work (~evilman@87.244.6.228) opustil server (Read error: No route to host).
‎[19:54] ‎<‎litheum‎>‎ right, selinux could be a thing, i was hoping the message about that would show up in the journalctl output, but maybe it still goes to some separate audit log?
‎[19:55] ‎<‎gryffus‎>‎ salle: i had some errors with selinux even with permissive mode, so i disabled it completely...
‎[19:56] ‎<‎salle‎>‎ gryffus: Does mysqld --initalize work with the default datadir? litheum already asked that
‎[19:58] --> vcolombo (~vcolombo@208.184.112.78) se připojil ke kanálu.
‎[20:00] --> mmazing (~mmazing@unaffiliated/mmazing) se připojil ke kanálu.
‎[20:00] ‎<‎gryffus‎>‎ salle: no
‎[20:01] <-- jsmyth (~jsmyth@unaffiliated/jsmyth) opustil server (Quit: Leaving.).
‎[20:02] ‎<‎salle‎>‎ gryffus: No in what way? Any errors?
‎[20:02] <-- dbclk (~dbclk@27.34.21.132) opustil server (Ping timeout: 240 seconds).
‎[20:02] --> dbclk_ (~dbclk@27.34.21.132) se připojil ke kanálu.
‎[20:02] ‎<‎litheum‎>‎ heh
‎[20:02] ‎<‎gryffus‎>‎ salle: it behaves just the same, just exits silently
‎[20:04] <-- jdavila (~Davila@200.125.188.214) opustil server (Quit: Leaving).
‎[20:04] --> ToreadorVampire (~ToreadorV@cpc107563-bsfd9-2-0-cust806.5-3.cable.virginm.net) se připojil ke kanálu.
‎[20:04] <-- ToreadorVampire (~ToreadorV@cpc107563-bsfd9-2-0-cust806.5-3.cable.virginm.net) opustil server (Changing host).
‎[20:04] --> ToreadorVampire (~ToreadorV@unaffiliated/toreadorvampire) se připojil ke kanálu.
‎[20:05] ‎<‎litheum‎>‎ try it with --console and with --log-error pointing to some specific file and then try it as the mysql user instead of root and then i guess it's off to strace
‎[20:08] --> meandrain (~meandrain@89.39.123.241) se připojil ke kanálu.
‎[20:08] --> cidic (~cidic@96-88-240-57-static.hfc.comcastbusiness.net) se připojil ke kanálu.
‎[20:08] <-- denimsoft (~denimsoft@188.172.147.58) opustil server (Quit: Textual IRC Client: www.textualapp.com).
‎[20:09] <-- treegor (~gary@bzq-82-81-213-77.cablep.bezeqint.net) opustil server (Ping timeout: 248 seconds).
‎[20:09] ‎<‎gryffus‎>‎ litheum:  open("/opt/data/tmp/ibd0v9Zq", O_RDWR|O_CREAT|O_EXCL, 0600) = -1 ENOENT (No such file or directory)
‎[20:09] ‎<‎gryffus‎>‎ i skipped the rest
‎[20:09] ‎<‎gryffus‎>‎ and it seems that the it does not like tmp being in datadir.....
‎[20:09] ‎<‎gryffus‎>‎ but why it works on all other servers?
‎[20:09] ‎<‎litheum‎>‎ did the program exit after that ??
‎[20:09] ‎<‎gryffus‎>‎ yes
‎[20:10] ‎<‎litheum‎>‎ did you set tempdir to a directory that doesn't exist?
‎[20:10] ‎<‎litheum‎>‎ tmpdir? tempdir? whatever it is
‎[20:10] ‎<‎gryffus‎>‎ https://pastebin.com/gc573Jht end of strace
‎[20:10] --> treegor (~gary@bzq-82-81-213-77.cablep.bezeqint.net) se připojil ke kanálu.
‎[20:11] ‎<‎litheum‎>‎ look at that, it's trying to write to stderr, where is that message going??
‎[20:13] ‎<‎gryffus‎>‎ log-output=FILE, log-error=/var/log/mysqld.log
‎[20:14] ‎<‎gryffus‎>‎ but in my /var/log/mysqld.log there is nothing
‎[20:14] ‎<‎litheum‎>‎ nutty
‎[20:14] ‎<‎litheum‎>‎ but back to my other recent question: did you set tempdir to a directory that doesn't exist?
‎[20:14] <-- cidic (~cidic@96-88-240-57-static.hfc.comcastbusiness.net) opustil server (Quit: cidic).
‎[20:14] ‎<‎gryffus‎>‎ yes, that is possible
‎[20:14] ‎<‎gryffus‎>‎ let me check and try to create it
‎[20:15] --> gugah (~gugah@181.169.151.140) se připojil ke kanálu.
‎[20:15] <-- kneeki (~kneeki@cpe-76-173-67-35.hawaii.res.rr.com) opustil server (Remote host closed the connection).
‎[20:15] ‎<‎gryffus‎>‎ if i create it, i get [ERROR] --initialize specified but the data directory has files in it. Aborting.
‎[20:15] ‎<‎gryffus‎>‎ :(
‎[20:15] ‎<‎litheum‎>‎ great! that's something at least!
‎[20:15] ‎<‎gryffus‎>‎ but at least there is error message
‎[20:15] ‎<‎gryffus‎>‎ yeah
‎[20:16] ‎<‎litheum‎>‎ so set tmpdir somewhere else and get this mother rolling
‎[20:17] ‎<‎gryffus‎>‎ well, that is another problem... i need to set this same as on production, but on production we use datadir as /opt/data and tmpdir as /opt/data/tmp :(
‎[20:17] ‎<‎gryffus‎>‎ well but i got an idea
‎[20:17] ‎<‎gryffus‎>‎ maybe the /opt/data/tmp is not on our backups
‎[20:17] ‎<‎litheum‎>‎ ok, try mysql_install_db after creating the tmpdir
‎[20:18] ‎<‎litheum‎>‎ i don't think it cares whether the directory is empty or not
‎[20:19] ‎<‎gryffus‎>‎ well, i don't need to get clean db working
‎[20:19] ‎<‎gryffus‎>‎ just our backups
‎[20:19] ‎<‎salle‎>‎ gryffus: By the way tmpdir within datadir is quite bad idea
‎[20:19] ‎<‎gryffus‎>‎ so if running mysql does not care, i dont care about install db
‎[20:19] ‎<‎litheum‎>‎ gryffus: what?
‎[20:19] ‎<‎gryffus‎>‎ salle: yeah i have said that numerous times to our configuration team
‎[20:19] ‎<‎litheum‎>‎ gryffus: you don't actually need to initialize the datadir? what are you actually trying to do then?
‎[20:20] ‎<‎gryffus‎>‎ litheum: i was debugging why mysql stopped working after restoring backup
‎[20:20] ‎<‎salle‎>‎ Yep if the goal is to restore dump there there is no need to initialize datadir
‎[20:20] --> josefig (~josefig@unaffiliated/josefig) se připojil ke kanálu.
‎[20:21] ‎<‎litheum‎>‎ gryffus: so you had restored the backup, but mysqld wouldn't start, and you figured you'd debug that by trying mysql_install_db instead? i guess that makes sense. crazy crazy crazy that you don't get an error message to stderr or a log file when mysqld flips out at the missing tmpdir, that's awful
‎[20:23] ‎<‎gryffus‎>‎ litheum: i have restored the backup and mysqld wouldn't start, so i first tried to delete datadir and it did not started with same symptoms (not a single message in logs). So i tried init DB to get at least some debug info, but it was not working either... And you know rest of the story :)
‎[20:23] ‎<‎gryffus‎>‎ thanks :)
‎[20:23] ‎<‎gryffus‎>‎ strace actually helped me to find out
‎[20:24] ‎<‎litheum‎>‎ yah, strace is a ridiculously useful tool for lots of otherwise intractable problems
‎[20:24] ‎<‎gryffus‎>‎ litheum: yeah like networkmanager :D
‎[20:25] ‎<‎gryffus‎>‎ if you have ever tried to debug one of the plugins you know what i mean
‎[20:25] ‎<‎litheum‎>‎ i feel very fortunate to know not a single thing at all about networkmanager
‎[20:25] ‎<‎gryffus‎>‎ litheum: the feeling is right :D