* Now talking on #firefox
* Topic for #firefox is: Welcome to the Firefox community & support channel | Have a question/problem? Just ask. If no one's around say "help!" || Get Firefox 3.0.1: http://getfirefox.com || Firefox 2.0.0.16: http://www.mozilla.com/en-US/firefox/all-older.html
* Topic for #firefox set by Mossop at Tue Aug 26 02:33:11 2008
Lns Hi all
mzz WillPittenger: (try adding or removing a cookie and exception, then verify they make it to those files)
Lucy shab: besides google toolbar, you have other things listed in tools -> addons -> extensions?
Lns I need some support re: Bugzilla #453704 - https://bugzilla.mozilla.org/show_bug.cgi?id=453704 . Is anyone available to talk to me about this?
firebot New Firefox - General bug 455076 filed by lori@emotiv.com.
shab no only google toolbar
firebot Lns: Bug 453704 cri, --, ---, nobody@mozilla.org, UNCO, Extreme slowness, "Firefox is already running" error for >3 users launching Firefox in LTSP environm
firebot Bug https://bugzilla.mozilla.org/show_bug.cgi?id=455076 maj, --, ---, nobody@mozilla.org, UNCO, Inline tables separate
WillPittenger Export on exit was turned on.  I turned them off.  See what happens.
shab and i kind of want google toolbar back i used it alot, but when i enabled it again my old problem came back
shab could it be that my firefox has a bug
Owner_ can I run in Safe mode and not have to turn off my Filter for kids?
Lucy shab: more likely something in the settings is off
* Quits (Snorkel@72AD333D.CF53FA24.D038C61.IP) has joined #firefox
shab how do i fix it
* paulc (einsicht@moz-68505BE2.dsl.sntc01.pacbell.net) has joined #firefox
* Quits has quit (Connection reset by peer)
Lucy shab: well we can reset the prefs and see if that helps. If we do it by hand we can back them up in case it doesn't make a difference
mzz Lns: not familiar with that setup. I'm guessing that has the homedir on a network filesystem though?
Mardeg Owner_: that mode is for testing if the problem still happens, not for leaving it in that mode
shab should i get out of firefox and talk on another browser
Lns mzz: kind of. LTSP thin-clients request sessions from the server after PXE boot. Everything re: the session is done on the LTSP server (at least in my setups).
Mardeg Owner_: from that diagnosis you can progress to finding a fix
Lucy shab: well we can go most of the way, then you can come back in on another browser
Lucy shab: what operating system are you using?
shab windows vista office
WillPittenger mzz: Next, FF is telling me there is an extension which needs updated, but won't tell me which.  The list is blank.
Lucy shab: ok go to start and in the search box paste %appdata%\Mozilla\Firefox\Profiles
mzz Lns: sqlite (which is used among other things for history in firefox 3) requires working filesystem locks and does a rather noticable amount of io. Does disabling history affect the problem at all? I'm assuming history does *work*, just slowly?
Owner_ ok--thanks for the info--will try that for now and see what happens--have a great day!
* Owner_ has quit (Quit: ChatZilla 0.9.83 [Firefox 3.0.1/2008070208])
shab okay
mzz Lns: so the thin clients have no local storage at all, just ram, and everything is nfs mounted?
Lns mzz: I haven't tried disabling history, although i have disabled the anti-malware stuff to reduce network i/o, and some other things detailed here: http://lns.wikidot.com/firefox3ltspoptimizations
* damons has quit (Quit: damons)
Lns mzz: UnionFS
Lns But yes
Lucy shab: hit enter after you've pasted that, that should open an explorer window of the Profiles folder
mzz ah
shab i have to profile box open
mzz Lns: do you have the ram to try with the entire profile in a tmpfs?
Mardeg Lns: have you tried the ltsp support? http://www.ltsp.org/twiki/bin/view/Ltsp/Support and http://www.disklessworkstations.com/cgi-bin/web/350001.html
Lucy shab: ok you should have a folder called something like <random>.default
firebot Firefox: 'Linux mozilla-central leak test build' has changed state from Success to Burning.
firebot Firefox: 'Linux mozilla-central build' has changed state from Success to Burning.
Lns mzz: Not really. my TCs have 256MB max... which is generally just enough for FF not to crash via X11 Pixmap buffering.
firebot Firefox: 'Linux x86-64 mozilla-central build' has changed state from Success to Burning.
Lns Mardeg: yes, I'm in constant contact w/#ltsp and friends
firebot New Firefox - Phishing Protection bug 455078 filed by thomas@tgohome.com.
firebot Bug https://bugzilla.mozilla.org/show_bug.cgi?id=455078 enh, --, ---, nobody@mozilla.org, UNCO, Alert the user to fraudulent JavaScript popups/dialogs
mzz Lns: any significant cpu load on either the client or the server while it's being slow?
Lns mzz: yes, LTSP server CPU utilization pegs pretty badly.
mzz huh
Lns Not sure about client usage, however.
mzz Lns: what processes is that thing running then?
* WillPittenger is now known as WLP|Away
Lns mzz: "firefox" is the offending process mostly - even sitting after a successful launch will utilize 30-60% CPU
mzz oh, wait
shab lucy, i have found the profile
mzz Lns: firefox isn't actually running on the thin client, the thin client just runs an X server, with firefox running on the server with display on the client?
Lns mzz: that is correct.
Lucy shab: ok, in the profile there should be a file called prefs do you see it?
* joseph has quit (Quit: ChatZilla 0.9.83 [Firefox 3.0.1/2008070208])
mzz in that case, has anyone here recently ran firefox 3 with x remote?
shab yes
mzz (through an ssh link or whatever)
wsmwk Lucy in the sky :)
Mardeg mzz: would that involves automatic proxy configuration?
Lucy shab: ok, exit firefox first, then rename that file to anything else, oldprefs will do just fine, then give Firefox a start and see if that helps. You can come back in another browser then.
Lns mzz: generally LTSP clients will tunnel X11 through ssh by default (unless specified lts.conf variable LDM_DIRECTX = true, in which ssh tunneling is disabled).. just FYI.
mzz Mardeg: no, just two linux systems with firefox and ssh on both
mzz Lns: makes sense. Just wondering if this can be reproduced to some extent using just firefox tunneled through ssh instead of a full ltsp setup
* shab has quit (Quit: http://www.mibbit.com ajax IRC Client)
Lns mzz: gotcha.
* wayne (wayne@moz-17EBC7D3.nt.net) has joined #firefox
* shab (Mibbit@moz-51225C4C.socal.res.rr.com) has joined #firefox
Mardeg Lns: would https://help.ubuntu.com/community/UbuntuLTSP/KioskProjects work better for running firefox directly on the thin clients?
mzz Lns: so storage is in fact local, so it's not an nfs-related issue with that. Could be you're still hitting the overly agressive fsync-ing.
zzxc Mardeg: pong
shab lucy i renamed prefs and restarted firefo
Mardeg zzxc: you were helping shab before?
mzz Lns: you should be able to rule that out by putting the profile on a tmpfs on the server (I'm assuming it has the ram to try that for at least a single client)
Lns Mardeg: I'm not sure. I haven't tried that setup.
Lns mzz: I'm not really sure how to move the profile to a tmpfs.. they do have 8GB ram, though, yes.
mzz Lns: move ~/.mozilla somewhere else, mount a tmpfs on ~/.mozilla, copy the contents back into that, roughly
* Owner_ (chatzilla@moz-349BC5EE.dynamic.mts.net) has joined #firefox
Lucy shab: want to talk here or in #sumo ?
shab yes zzxc helped me alot yesterder
shab eitehr one
Lns mzz: actually that's gonna be real hard to try, as the bug appears when > 10 users are launching FF at the same time.
Owner_ it's me again--its sytill doing it even in safe mode
* Ricmacas has quit (Quit: http://www.mibbit.com ajax IRC Client)
mzz grr
shab whichever is best for you
Lucy shab: #sumo is probably quieter, look over there and respond to what I said?
Mardeg Owner_: okay, the next thing to test is whether it happens in an additonal newly created clean profile
Mardeg firebot: profile manager
firebot Mardeg: The Profile Manager is a way to easily create, or switch profiles.  For Firefox: http://support.mozilla.com/kb/Managing+profiles  For other Mozilla applications:  http://kb.mozillazine.org/Profile_Manager - You must exit Chatzilla before use.
Lns mzz: is there any way I can trace what, exactly, is happening while FF is lagging? Basically the experience I've heard is this: In a lab full of 35 students, all logging on and launching Firefox at the same time, 3-5 will launch successfully (first) - then the rest of the class will see Firefox attempting to launch (bottom gnome panel gives a tab) - and ~25-30 will see the panel tab dissapear - others will get "Firefox is already running" error. A
Lns LL others will get "Firefox is already running" error after they try to launch again.
Mardeg Owner_: use the instructions in the first link to create and switch to a new profile, without deleting your current profile
mzz Lns: wait, "already running"? Are you sharing profiles somehow?
Lns mzz: no, that's the weird thing.
* mzz digs up the code
Lns Other bug comments seem to wonder about profile locking in FF3 vs. FF2 (which didn't ever experience this issue)
Lns mzz: FYI, https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/269188 has useful info as well.
Owner_ when I followed the instructions it went straight to the start page--is that what its supposed to do?
Owner_ didn't give me the option of creating a new profile
dr|z3d Sounds like you may already have an instance of Fx in memory, Owner_
dr|z3d Owner_: Can you check the task manager and quit any instance of firefox.exe in the processes tab.
Lns mzz: I'm 95% sure this isn't it because only 2 of my 7 labs have this setup, but users DO have their own FF profiles in homedirs..but they are named the same (via /etc/skel copy for old FF2 global bookmark linking reasons). Would simply having separate, yet NAMED the same profiles have an effect?
firebot New Firefox - Bookmarks & History bug 455079 filed by ivensmg@ig.com.br.
firebot Bug https://bugzilla.mozilla.org/show_bug.cgi?id=455079 nor, --, ---, nobody@mozilla.org, UNCO, Bookmarks
* Owner_ has quit (Connection reset by peer)
* mcr (mcr@moz-D7D52E9.sandelman.ca) has joined #firefox
mzz Lns: so from the point of view of the firefox process the profile is on a local fs, right? What kind of fs?
firebot Firefox: 'Linux mozilla-central leak test build' has changed state from Burning to Success.
Lns mzz: yes. ext3 FS on all servers.
mzz Lns: I agree it seems unlikely the profiles having the same name has anything to do with it, mainly because if it did I'd expect the second one to fail already, not the fifth or so.
Lns mzz: yeah.
* Owner_ (chatzilla@moz-349BC5EE.dynamic.mts.net) has joined #firefox
Owner_ ok that worked to create a new profile, but I thought it would save my bookmarks and add-ons and it didn't
firebot Firefox: 'Linux mozilla-central build' has changed state from Burning to Success.
* timrosenblatt has quit (Ping timeout)
Mardeg Owner_: that's the point of creating a new profile, to see if something in your existing profile was causing the problem
* shab has quit (Quit: http://www.mibbit.com ajax IRC Client)
Mardeg Owner_: so does the problem still happen in that new profile?
firebot Firefox: 'Linux x86-64 mozilla-central build' has changed state from Burning to Success.
Owner_ I guess that's just a wait and see kind of thing--it only did it just whenever, but it only started the past week or so and I don't recall changing anything
Owner_ is there a free virus/spyware tool anyone would reccommend
Owner_ ;maybe there's hidden junk in here or something---you can probably tell I'm not very somputer savvy
stevee for spyware, the best two are SuperAntiSpyware and MalwareBytes Anti-Malware imo
stevee for virus killers, pick any two and two nerds will fight to the death over which one is best ;-)
Mardeg firebot: malware
firebot Mardeg: For help removing malware, install http://www.safer-networking.org/en/mirrors/index.html and ask in irc://irc.freenode.net/malware-removal
Mardeg firebot: rootkit
firebot Mardeg: Rootkits are detected using http://download.sysinternals.com/Files/RootkitRevealer.zip and http://research.pandasoftware.com/blogs/images/AntiRootkit.rar and prevented using http://www.diamondcs.com.au/download.php?file=pg
stevee i like kaspersky, but i hear good things about NOD32
mzz Lns: fwiw, assuming the code in http://mxr.mozilla.org/firefox/source/profile/dirserviceprovider/src/nsProfileLock.cpp#524 is the right code (and I think it is) it first tries to take an fcntl lock on $profiledir/.parentlock, after that succeeds it tries to create $profiledir/lock as a symlink with the host + pid stored as its destination
Mardeg Owner_: try the links from firebot
stevee spybot search&destroy is so 2005
mzz Lns: that lock symlink is there for legacy reasons (so an ancient Firefox still understands the profile is locked)
stevee just like ad-aware... they've had their time with removing tracking cookies OMG SCARY!!!
Owner_ My PC got"hijacked" a year or so ago and every day it "crashes" and says "has recovered form a serious error"...somehow its still working though--they didn't know how to fix it at the computer fixing place
Mardeg Owner_: sounds like getting the program "hijackthis" and going to bleepingcomputer forums would be the go
mzz Lns: so I'm curious if in the failing case it gets as far as creating that lockfile (I'm guessing it doesn't)
Lns mzz: gimme one sec - was afk for a min.
Noah stevee: hehe nerds fighting to the death... yeaaah
stevee :)
* BASEman has quit (Quit: )
mzz Lns: otoh I can't really think of an obvious reason for the fcntl to fail unless the server's under a silly amount of load
Noah stevee: oh yeah, wanna test that myspace oddness?
stevee sure
* BASEman (patrick@moz-8E531C1D.coditel.net) has joined #firefox
dr|z3d Owner_: If you have an XP disc you might try booting off it and repairing Windows.
dr|z3d Owner_: You might also want to try a full scan with http://spywareterminator.com (with ClamAV enabled).
mzz Lns: I think I'd strace it to verify something goes wrong with those locking attempts (I can't think of anything else that'd trigger a profile is locked error message, but I could be wrong)
Lns mzz: I think you're on the right track there as it does definitely have something to do w/locking
Lns mzz: how do you strace it?
firebot New Firefox - General bug 455082 filed by keagans@gmail.com.
Lns mzz: I know that disabling the anti-*ware download from google.com did speed things up a bit (though not fully).
firebot Bug https://bugzilla.mozilla.org/show_bug.cgi?id=455082 nor, --, ---, nobody@mozilla.org, UNCO, firefox 3.0.1 will not install any addons
mzz Lns: strace -f firefox 2>/path/to/logfile (which will be insanely huge but better too much data than too little at this point imho)
Lns mzz: as you can imagine, 35 seperate ff instances all trying to DL anti-malware files > 10MB can put a load on the server. ;)
mzz Lns: yeah, I don't know a sane way to share that db though (I'd kind of like that for multiple profiles in use on the same system, which is basically the same thing)
Lns mzz: Yes, that's my big beef w/sqlite in FF3. It breaks a lot of things for multi-user setups such as LTSP unfortunately (like global bookmarks)
mzz Lns: don't know if you followed that during the betas, but one problem affecting some linux systems before the 3 final release was any history action triggering an sqlite commit, which triggered an fsync, which on (at least some) ext3 filesystems flushes *all* pending io, not just to one file
mzz Lns: which caused things to get spectacularly slow if you were doing lots of io on the same partition firefox is on
mzz Lns: now 3 does fsync less than the betas, but it still fsyncs, and I'm pretty sure that ext3 issue is also still there
dr|z3d Owner_: Ys
Lns mzz: ok... Basically most of my servers were upgraded after the final Ubuntu release was out (3.0 I believe, then 3.0.1 upgrade later)
mzz Lns: so I'm wondering if having a few dozen firefox instances on the same ext3 partition is simply causing so much io flushing that things become unusable
dr|z3d Owner_: Sorry, yes, do give spywareterminator a go.. let us know how you get in with that.
* Elen has quit (Connection reset by peer)
dr|z3d Owner_: And if you don't have a boot disk, you can always try reinstalling the latest service pack.
Lns mzz: well the performance of the server would definitely reflect that.. the whole gnome session becomes unstable
Lns when launching FF for all usres
Lns users*
Mardeg background to mzz's comments - http://shawnwilsher.com/archives/169
Lns wow ty Mardeg
Owner_ I just installed the latest yesterday I think
mzz Lns: oh hey, a hidden pref
Lns mzz: and FYI this *mostly* happens when new profiles are created. I have heard varying later success with created profiles (2-4 unsuccessful launches of Firefox might complete the profile creation and then let them in)..but I don't *think* this is across the board.
mzz Lns: if you don't have any important permanent profiles I'm curious if setting toolkit.storage.synchronous to 0 (just to confirm this is the problem, that pref is all kinds of unsafe) would help
mzz Lns: although I think to really confirm that you'd have to do it for the whole lot of them, not for one test profile while the rest is still using the default value, if you know what I mean
mzz Lns: (afaik that cuts out sqlite's fsyncs entirely)
Lns mzz: I can definitely try this for an entire lab - I can wipe their current profile and then set that pref in /etc/firefox/pref/firefox.js
* Owner_ has quit (Quit: ChatZilla 0.9.83 [Firefox 3.0.1/2008070208])
mzz Lns: that's what I was hoping
* Steffen (chatzilla@moz-737049F0.adsl.alicedsl.de) has joined #firefox
mzz Lns: see https://bugzilla.mozilla.org/show_bug.cgi?id=421482 also
firebot mzz: Bug 421482 maj, --, ---, sdwilsh@forerunnerdesigns.com, RESO FIXED, Firefox 3 uses fsync excessively
Lns mzz: ok, yeah. Definitely.
* dholbert_sheriff is now known as dholbert_sheriff|afk
Lns mzz: Unfortunately I can't get to it today :( I'm sure you're in this chan regularly .. ?
Lns I can try on Monday in the morning (PST)
mzz Lns: I'm in here regularly, and my client is connected pretty much 24/7, so you can leave messages that way
firebot Firefox3.0: 'Linux fx-linux-1.9-slave09 dep unit test' has changed state from Test Failed to Success.
* Domdabomb (Dombev30@moz-6C29BEFC.cable.ubr06.jarr.blueyonder.co.uk) has joined #firefox
Lns mzz: awesome. Thanks so much for your help so far. I feel good about getting this fixed. I'll post our chat to the bug reports if that's ok, too.
mzz Lns: basically I can't think of any obvious way to get a bogus "profile is locked" unless something weird happens at the fs level, and more calls to fsync is known to be an issue on ext3
mzz Lns: (sqlite on nfs is also known to be a problem, but afaict nfs should not be involved here)
mzz Lns: if you have something running on the actual thin clients that accesses the profile through nfs that'd almost certainly make matters worse though, so you might want to confirm that's not the case too
* Dombev30 has quit (Ping timeout)
mzz also, I'm sort of curious if it's possible to run out of fcntl locks
Lns mzz: yeah - AFAIK the *only* thing that runs in the client chroot (thinclient session) is X server.
Lns mzz: which had caused issues in FF2 regarding X11 pixmap caching locking up the thinclient session entirely
mzz Lns: that's what I thought (after recovering from my assumption that the thin clients ran firefox)
mzz Lns: yeah, I can imagine you still hitting that problem in 3 (my Xorg process tends to grow quite a bit as I browse, although I probably open more tabs than the average user)
Lns mzz: some people *do* run FF locally on TC via "localapps" in LTSP - to provide more responsiveness for plugins like flash/shockwave
Lns mzz: you know, i've never had an issue personally, i use LTSP and for the past month Ive never had firefox crash or lock up my TC session. I know the code was greatly cleaned up from FF2 due to many LTSP user complaints.
mzz Lns: if you do that (with the profile on nfs) you can run into problems caused by fcntl locks over nfs not working all that well, or so the sqlite docs say
Lns I was pretty well involved with that whole process (besides the coding)
Lns mzz: seems like sqlite + multi-user setups just don't seem like a good fit
firebot Just appeared in Planet Mozilla - http://planet.mozilla.org/ :
firebot    Jeremy Orem: My Life is a Little Bit Easier Now
mzz Lns: it can't cause corruption because mozilla isn't actually using those locks on the sqlite files to ensure no concurrent access, but I'm pretty sure sqlite still takes the locks. Which could cause problems, especially if the nfsd is having problems of its own.
mzz Lns: actually I think the problem here is that sqlite ensures no concurrent access all by itself, while mozilla is already doing that through the profile-level lock.
mzz Lns: as long as you don't poke the sqlite data through something other than firefox I'm pretty sure disabling the locking sqlite does would be safe.
mzz Lns: (you still need the fsync calls to protect against data loss on crash, but the locking is rather redundant afaik)
Lns mzz: yeah, nothing else is accessing sqlite at all
Mardeg sqlite manager - http://shawnwilsher.com/archives/169
Mardeg oops, I meant - https://addons.mozilla.org/en-US/firefox/addon/5817
Mardeg good thing I didn't have anything embarrasing in my clipboard :P
firebot Firefox: 'Linux mozilla-central qm-centos5-03 dep unit test' has changed state from Success to Test Failed.
Lns mzz: What can I do to help Mozilla devs see that multi-user setups such as LTSP are on the rise, and to keep that in mind?
mzz Lns: I'm not sure (I'm also no actual moz dev :)
Lns mzz: ah. =) It sure would be nice to see some integration with other sql servers for multi-user setups
mzz Lns: filing bugs on issues you encounter is obviously a good idea, but you're doing that already
* damons (gnubeard@moz-F673A528.mountainview.mozilla.com) has joined #firefox
mzz Lns: well, for most of the data sharing it across instances doesn't actually make any sense
firebot Firefox3.0: 'Linux fxdbug-linux-tbox Depend' has changed state from Success to Test Failed.
mzz Lns: sqlite is pretty decent as long as you're either only accessing it through one client at the time or it's almost read-only
mzz Lns: it's only concurrent writes that don't really work all that well
mzz Lns: (the obvious exception here is the urlclassifier, I don't know if it'd be possible to share that data)
Lns mzz: hold on a min..sorry
mzz (mainly because I'm not sure how often the db gets written to)
* Odr` has quit (Quit: Quit)
* dholbert_sheriff|afk is now known as dholbert_sheriff
* Runiat has quit (Quit: ChatZilla 0.9.83 [Firefox 3.0.1/2008070208])
* logan has quit (Quit: Leaving)
Mardeg yeah, the ltsp setup seems to be encountering a weakness that was meant to be a strength of the "lightness" of sqlite
* dholbert_sheriff has quit (Quit: Ex-Chat)
mzz mmm, the fsync issue is a bit orthogonal to all this
* dholbert_sheriff (dholbert@moz-334F2010.mountainview.mozilla.com) has joined #firefox
mzz the problem with that one is that (in some cases) linux ends up flushing the entire journal
firebot Firefox: 'WINNT 5.2 mozilla-central qm-win2k3-03 dep unit test' has changed state from Success to Test Failed.
mzz while all sqlite wants to actually do is *order* operations
mzz so the syscall it uses makes sure the ordering happens, but does a ton more work than sqlite wants it to do
* asztal has quit (Quit: @)
* asztal (asztal@moz-35612F6.range86-169.btcentralplus.com) has joined #firefox
mzz at least that's what I gathered from the sqlite docs: it's important that the data actually makes it to disk there, but the absolutely critical thing is that it makes it to disk in the right order (or you'd end up with a corrupt db if things crash at the wrong time)
* Carlos (chatzilla@moz-41511CA7.hsd1.co.comcast.net) has joined #firefox
firebot Firefox3.0: 'Linux fxdbug-linux-tbox Depend' has changed state from Test Failed to Success.
* damons has quit (Quit: damons)
* Kazie has quit (Quit: #Kazie)
* Tomcat is now known as Tomcat|afk
Mardeg Lns: might be worth inviting the places team to do an ltsp setup for testing, or invite them to your school if they are close by :) Builds of what they've worked on fixing so far are linked from http://shawnwilsher.com/archives/172
Noah mzz: so is sqlite still a step up from the RDF format?
Noah mzz: how do IE and Opera balance their data saving?
Lns Mardeg: mzz: hold on still..sorry, talking w/someone
mzz Noah: that depends on what you're doing. You can't really sensibly partially read or write to rdf (well, to xml in general really, I'm assuming you're talking about that form of rdf serialisation)
* Tomcat_xp has quit (Connection reset by peer)
* MaK77 (chatzilla@moz-DB680DD5.retail.telecomitalia.it) has joined #firefox
Mardeg Shawn Wilsher is begging for feedback on those builds :)
mzz Noah: I don't know what underlying storage format IE and Opera use.
mzz Mardeg: I'm not really hurt by the fsync issue myself, so I don't think I'd make a good tester
Mardeg that was aimed at Lns :)
Noah mzz: I see / I meant how we formally used RDF as a storage type basically for all for firefox 2 lifecycle
Noah it seemed RDF was more prone to been corrupted
Noah *being
Mardeg Noah: except it wasn't just RDF, it was also MORK for the history
mzz Noah: yep, but when using rdf I'm pretty sure the whole lot got read into ram on first use and stayed there until it was written out again on shutdown. There's a reason the default url history size got bumped along with the move to the new storage format :)
Noah Mardeg: yup, I'll never forget good ol' mork
* Carlos has quit (Quit: ChatZilla 0.9.83 [Firefox 3.0.1/2008070208])
mzz oh wait, history wasn't rdf
Lucy sqlite doesn't replace the file until the write is successful. If rdf got interrupted mid write, the file was done
mzz sqlite is transactionally safe and you can do things with the data without reading all of it
Mardeg it's a shame we have to keep mork in firefox for importing legacy upgrades :/
Noah mzz: ah, I see / I just thought it was bumped up for other reasons
mzz (iirc you can do things with mork without reading all of it too, but I *think* only on a single primary key, not using arbitrary indices. And I don't know about incremental updates at all)
Noah Lucy: done as in fried?
mzz Noah: sqlite scales pretty decently
Lucy Noah: yeah, it'd be the same number of chars but they'd all be nulls
* trmanco has quit (Quit: I just hit the close button :))
mzz Noah: the main thing it really doesn't do well at all is concurrent writers (processes or threads), but mozilla can get by without those.
Mardeg I'm guessing that was from the corrupted rdf files people emailed to you Lucy?
mzz Noah: pretty much any plain text based format (including anything xml-based) is hard or impossible to partially write, often to partially read as well. I don't know enough about mork to comment on that.
Noah mzz: yeah, see in IE's case I think they do some kind of db thing simliar to ms excel type stuff
* mzz nods
Noah but I've never investigated
mzz Noah: ms certainly has at least one in-house db engine they can use for this kind of thing
mzz think "sql server lite" :)
Noah yeah that term actually popped in to my minus the lite :D
Noah *my head
* mzz is an sqlite fanboy
mzz it does what it aims to do pretty well, and it's occasionally convenient that you can still poke at the data through sqlite manager or commandline tools
Noah mzz: yeah, I heard back in the years I investigated that mork was a horribly documented code and many people were angry at it - and that guy who wrote it was AWOL
* wayne has quit (Quit: Leaving)
mzz Noah: from what I've heard mork was never intended as a general-purpose db, it was written for storing mail stuff after using an existing db was shot down by management
* IT-Pro has quit (Quit: IT-Pro)
Noah mzz: I always like to compare how one product does what it does to another, maybe you see what is more solid in different situations - like for instance, Google Chrome's history saving [does it copy us/mimic our sqlite db?]
Mardeg so it got put to more use than intended? Like when Microsoft bought QDOS and turned it into MSDOS?
Noah mzz: ah, I see - also I just remembered MS access - that's another db!
Noah although isn't access slated towards saving of huge corporate data?
mzz Mardeg: I think I read that, give me a minute to hunt down the page
arooni sometimes firefox doesnt let me save files to /tmp... wahts up with that?
dr|z3d Access is good for mailmerge. That's about it.
mzz Mardeg: http://developer.mozilla.org/En/Mork_Why
mzz arooni: works for me
mzz arooni: that is: I'm normally downloading to /tmp these days, and I haven't hit a case where it wouldn't let me yet
mzz arooni: how's it failing?
mzz arooni: also, what kind of fs is /tmp, how's it mounted, what are the permissions?
arooni drwxrwxrwt  17 root   root  53248 2008-09-12 15:58 tmp
firebot New Firefox - Tabbed Browser bug 455090 filed by james@dsgamemaker.com.
firebot Bug https://bugzilla.mozilla.org/show_bug.cgi?id=455090 nor, --, ---, nobody@mozilla.org, UNCO, List all tabs button does not compress image tabs resulting in oversized menu
* Quan (chatzilla@moz-5BE136C0.bb.sky.com) has joined #firefox
mzz Noah: oh, and http://en.wikipedia.org/wiki/SQLite mentions google chrome uses sqlite (but not what for)
* HardcoreRockabilly (chatzilla@moz-79141C3A.dsl.skt2ca.sbcglobal.net) has joined #firefox
firebot Firefox: 'Linux mozilla-central qm-centos5-moz2-01 dep unit test' has changed state from Success to Test Failed.
* Mardeg wonders if the "separate process per tab" would make chrome work better in LTSP
mzz Mardeg: I'd expect the memory hit to be a potential problem
mzz Mardeg: other than that I'd expect the amount of io to be roughly the same, so I wouldn't expect it to matter that much
* AWatkins has quit (Ping timeout)
* LionsMane has quit (Quit: Leaving.)
mzz Mardeg: although I'm not actually sure how this'd be implemented on X, if whatever it does involves more roundtrips to the server that'd be a noticable hit in this case
* Lns tries to absorb all of this
* Lns ...fails ;)
mzz Lns: no real need, most of it's offtopic
* Adam has quit (Quit: ChatZilla 0.9.83 [Firefox 3.0.1/2008070208])
* Adam (chatzilla@moz-D37DD2EC.hsd1.pa.comcast.net) has joined #firefox
Lns mzz: oh cool. =) SO.. I'll try the pref change like we discussed, was there anything else I should do that I didn't catch from the buffer?
Lns Besides talking w/the places people and possibly getting them more involved?
mzz Lns: strace the failing startup (preferably on a clean profile, since whatever goes wrong the first time might leave a stale lock)
Lns mzz: oh, right.
mzz Lns: it would be nice if we could come up with a way to confirm this is triggered through places and/or sqlite
Lns mzz: and stracing will def. do that right?
Noah mzz: ltsp is what exactly?
Lns Noah: www.ltsp.org
mzz Lns: if that pref change makes the problem go away that pretty much confirms sqlite's fsync calls trigger the problem, and making places commit less often should help with that (and be safer than that pref change)
Lns mzz: ok.
mzz Lns: I'm still curious about that strace though, since I don't really understand why that lock attempt would fail even under fairly significant load
Lns mzz: did you see the one comment in the bugzilla re: locking?
mzz Noah: the two main things from a firefox perspective here as far as I can tell is that this involves running a few dozen firefox instances on one system (and filesystem) and that the display is remote (X through ssh)
mzz Lns: might've missed it, sec
Lns  Comment #4 From  Jordan Erickson   2008-09-11 11:47:03 PDT
mzz Lns: yep, but you're looking for "lock", while one of the lock files is .parentlock
Noah Lns: comparatively what would be the Windows equilavalent of what LTSP does?
Lns Noah: Windows Terminal Server
Noah ah alright
Lns mzz: what's the diff between the two locks? was it the backward compat. thing?
mzz Lns: sec, let me see what firefox 2 creates locking-wise
* NeX_ (Alive@moz-31B303C2.rev.libertysurf.net) has joined #firefox
Lns ok..i'll brb
Noah Lns: terminal services allows a user to access applications and data on a remote computer over a network - so this would ideal over installing applications on every computer for example?
* BlueStreak has quit (Quit: )
mzz Lns: creates the same two lockfiles here (both .parentlock and lock). I don't know in which version .parentlock was introduced.
mzz Lns: afaik older versions just look for the existence of anything called "lock"
* NeX has quit (Ping timeout)
Noah mzz: the version of firefox that introduced .parentlock?
mzz Lns: advantage of .parentlock is that I'd expect the fcntl lock on that to reliably go away when the process does (not the actual file, the lock it has on that file)
* mzz tries to confirm that
mzz Lns: if I kill -9 firefox 2 it leaves behind its lock symlink (as well as .parentlock), but it restarts without complaining
Lns Noah: yes (re: term. svcs). Centralized administration, software - just think of LTSP like a big beefy computer with tons of keyboards, mice and monitors hanging off of it (like the old mainframe days w/serial dumb terminals)
Noah Lns: :D
* Nathaniel (nathaniel@acm.bw.edu) has joined #firefox
mzz Noah: thing to keep in mind is that (at least in the setup Lns is using) stuff actually *runs* on the server too
mzz Noah: if I understand correctly it's also possible through ltsp or an addon to it to keep the app stored on the server but run it on the clients (through an nfs mount)
Lns mzz: right. Everything besides the X "server" (which again, is the traditional X "server" sense) runs on the LTSP server(s) itself.
mzz Noah: but in his case only the X server runs that way, the rest of the session runs remotely
Lns mzz: you'd be talking about "localapps", which is still experemental
mzz nod
* mw22____ (chatzilla@moz-58B890CA.ls1.ov.home.nl) has joined #firefox
Lns mzz: Noah: but mzz was correct before in comparing it simply to running Firefox from a different computer through X11 (possibly tunneled through ssh)
* Adam has quit (Connection reset by peer)
* mw22____ is now known as mw22
Mardeg so the X server runs on the client terminals, and the client browser runs on the "mainframe" server? got it :)
mzz Lns: since you're tunneling through ssh I'd compare it to tunneling through ssh too (and that's not all that hard to set up either)
Lns Mardeg: right.
mzz Mardeg: yeah, the "server" in X feels a bit backwards
Lns mzz: right. And LTSP, in Ubuntu anyway, by default, tunnels through SSH for compression and security.
* Zarggg (z@moz-CF4F6E16.c3-0.eas-ubr6.atw-eas.pa.cable.rcn.com) has joined #firefox
mzz Lns: yeah, getting your X traffic sniffed sucks, so that makes perfect sense
Lns yep
* BASEman has quit (Quit: )
mzz (and I'd expect the overhead for that to be no serious issue)
Lns mzz: it's not, normally - it just depends on what kind of thin-client hardware you're using. Some older machines, and notably newer AMD Geode family of chipsets are very low-power thus don't have a lot of umph..so turning off SSH helps performance.
Lns And in LAN environments security really isn't an issue anyway (normally)
Mardeg Lns: so you're gonna try those builds linked earlier?
mzz Lns: hmm, makes me randomly wonder if playing games with the underlying compression and encryption algorithms ssh is using is worth it
firebot Just appeared in Planet Mozilla - http://planet.mozilla.org/ :
Lns Mardeg: I didn't catch the builds - newer FF builds not in ubuntu repos yet?
firebot    David Rolnitzky
mzz Lns: mentioned on that blog by Shawn Wilsher (sp?) Mardeg linked to
Mardeg Lns: test builds with sqlite patches
Lns oh ok
firebot The dictionary service is not accessible right now, sorry.
mzz Lns: contain patches making places (history/bookmarks) call sqlite's commit much less often, which might help.
Lns I can try, I should probably try the pref thing first as to not completely yank FF out of the server yet =)
firebot Firefox: 'Linux mozilla-central qm-centos5-03 dep unit test' has changed state from Test Failed to Success.
Mardeg mock up a stage server?
* mzz has no clue how well that'd work
* martincleaver has quit (Quit: martincleaver)
Mardeg https://build.mozilla.org/tryserver-builds/2008-08-28_11:12-sdwilsh@shawnwilsher.com-try-df9b4f955b9/sdwilsh@shawnwilsher.com-try-df9b4f955b9-firefox-try-linux.tar.bz2 is the linux build
* Adam (chatzilla@moz-D37DD2EC.hsd1.pa.comcast.net) has joined #firefox
* Adam (chatzilla@moz-D37DD2EC.hsd1.pa.comcast.net) has left #firefox
Mardeg that'd probably have the other goodness of things like jit enabled javascript, <video> support, text-shadow css, etc.
* SJrX (none@64CFB83B.AD939BCE.7A95930C.IP) has joined #firefox
SJrX Hmmmmm it seems I can't change my bookmarks
Lns mzz: ok - I'll try the pref, because that sounds like it might help. I'll definitely keep you posted, maybe you can subscribe to the bugs I reffed earlier too - you guys seem like gems in possibly figuring this thing out for us
SJrX Any changes I make disappear later
SJrX When I restart firefox
mzz heh
mzz so far all I have is general ideas on what area to look in
mzz don't get too excited yet
* jmoore3rd (Compaq_Own@moz-9B77C465.asm.bellsouth.net) has joined #firefox