[11:18 AM] <GeKo> mikeperry: Do you already have a branch with the rebased ff24 stuff somewhere?
[11:56 AM] <armadev> microchip_: ok, i'm here a little bit more now. why not contribute your file to suse itself?
[12:00 PM] <mikeperry> GeKo: https://gitweb.torproject.org/user/mikeperry/tor-browser.git
[12:01 PM] <GeKo> thanks
[12:01 PM] <mikeperry> GeKo: I believe that branch builds. Pearl Crescent reported issues with a build that they made using Xcode using that, but they suspect compiler issues
[12:02 PM] <mikeperry> a few patches also havent been rebased yet: https://gitweb.torproject.org/user/mikeperry/tor-browser.git/tree/refs/heads/torbrowser-firefox24.0:/BADPATCHES
[12:15 PM] <arlolra> armadev: https://check2.torproject.org/cgi-bin/TorBulkExitList.py?ip=123.123.123.123&port=443
[12:18 PM] <StrangeCharm> armadev: You around?
[12:18 PM] <armadev> spiffy. can we run that through sort -n
[12:18 PM] <armadev> strangecharm: i am!
[12:19 PM] <armadev> arlolra: the sort -n suggestion was for you
[12:20 PM] <arlolra> we can
[12:21 PM] <armadev> (the previous bulkexitlist does, and somebody might be relying on that. plus it's easier to tell that it doesn't have duplicates)
[12:21 PM] <armadev> also, great!
[12:21 PM] <armadev> did you see that weasel set up the cron to give you tor's cached dir info? i guess you probably did since this url uses it?
[12:22 PM] <arlolra> yup, using it
[12:22 PM] <armadev> great. and also that you can modify your apache vhost file
[12:23 PM] <arlolra> are you happy with what it says here?
[12:23 PM] <arlolra> https://check2.torproject.org/cgi-bin/TorBulkExitList.py
[12:23 PM] <arlolra> should I update the part about svn to my github repo?
[12:24 PM] <arlolra> anything else you can think of re: that page or https://check2.torproject.org/?
[12:24 PM] <arlolra> otherwise, I think we're good to give it a shot
[12:24 PM] <armadev> https://check2.torproject.org/cgi-bin/TorBulkExitList.py?ip=128.31.0.34%3A443 is what i tried to input into the box
[12:24 PM] <armadev> it didn't like my input :)
[12:25 PM] <arlolra> doesn't on check either .. I think you need &port=
[12:25 PM] <arlolra> should it support that syntax?
[12:25 PM] <armadev> well, how do you use the box to put the port of your service?
[12:25 PM] <armadev> (i agree that it's not obvious on the original either)
[12:26 PM] <arlolra> can you do that on the original?
[12:26 PM] <arlolra> I don't think so ... but I agree that there should be an easier way
[12:28 PM] <armadev> "we do keep a cache of answers for all queries made" is this still true on your new script?
[12:30 PM] <arlolra> no. we played with some caching mechanisms but it's plenty fast as is
[12:30 PM] <GeKo> mikeperry: Okay, that should be fine. I'd like to build a TBB3.0 with esr24 as I might have a (simple) fix for #9783 which is not working with esr17.
[12:31 PM] <armadev> sounds like we should remove that sentence then :)
[12:31 PM] <armadev> but yes, sounds great, i think we're all set for a switchover. what did you have in mind?
[12:36 PM] <arlolra> armadev: sorry, cut out. sounds good
[12:58 PM] <mikeperry> GeKo: So we have a build that matches for a commit btw. I suppose we should try to release it and then do another release shortly after with your Windows fix and a torbutton hack to watch for this XPCOM issue
[12:58 PM] <GeKo> mikeperry: Sounds good to me.
[12:59 PM] <GeKo> If you create a tag I'll start building it tomorrow.
[1:00 PM] <GeKo> (or just tell me the commit you want to use)
[1:00 PM] <mikeperry> oh, let me tag a new torbutton release for the startpage change
[1:00 PM] <GeKo> sure
[1:04 PM] <GeKo> mikeperry: Btw #9790 is really, really scary. I never thought that such cross-extenion influencing would be possible...
[1:05 PM] <GeKo> so, having something that screams at the user when such central components are not working would be neat
[1:07 PM] <Lunar^> mikeperry: you should close #8839 :)
[1:11 PM] <mikeperry> GeKo: Oh wow, that's the same bug as #9763
[1:11 PM] <GeKo> mikeperry: No.
[1:11 PM] <GeKo> cache isolation is working fine even if the component is not loading
[1:12 PM] <mikeperry> not for me
[1:12 PM] <GeKo> that's interesting
[1:12 PM] <mikeperry> it's broken by the startup pref being in the wrong state
[1:12 PM] <GeKo> could you try if my workaround is fixing the issue for you as well?
[1:13 PM] <GeKo> if so convince the HTTPSE-people to catch their exception and you are done :D
[1:13 PM] <mikeperry> we created that exception by disabling ctypes
[1:14 PM] <microchip_> armadev: i just inspected the tor package on the suse build service. They have indeed added a systemd file
[1:14 PM] <GeKo> mikeperry: yes, exactly.
[1:14 PM] <mikeperry> so I am thinking this means for thi 17.0.9 build we just re-enable it
[1:14 PM] <mikeperry> since not having cache isolation is superbad
[1:14 PM] <GeKo> yes
[1:14 PM] <mikeperry> and we can fix the ctypes issues later
[1:15 PM] <microchip_> armadev: the reason i wrote my systemd file is because i package tor in my home repo as suse is often late in providing up to date tor packages, so i wasn't aware they have added a systemd file
[1:26 PM] <arlolra_> hmm, how come this relay has a running flag and yet status says running is false? https://atlas.torproject.org/#details/0FB356FBE623B3F6C8A2CC3CA42E0E7681F72FE7
[1:26 PM] <arlolra_> armadev: sorted https://check2.torproject.org/cgi-bin/TorBulkExitList.py?ip=123.123.123.123
[1:31 PM] <armadev> arlolra: exciting! my guess is that it's not running now, but the last time we saw it in the consensus, it had the Running flag.
[1:31 PM] <armadev> that is pretty clearly a bug in atlas.
[1:31 PM] <armadev> (it can't get into the consensus without having the Running flag.)
[1:32 PM] <wfn> i think that's the case, just wanted to say sth like that - the flags included are from the last available consensus (2013-09-23 15:00:00 in this case), which is not the newest one, hence router is not reported as running
[1:32 PM] <armadev> ah. perhaps that's because Properties refers to the last consensus stanza we have for it,
[1:32 PM] <wfn> i wonder what atlas should do - it makes sense to report latest available flags; problem is, the name "Running" has a certain meaning to it that can confuse people, i guess
[1:33 PM] <armadev> and Status refers to ... but i bet the Uptime is from...uh.
[1:33 PM] <armadev> right. i think part of the problem is that atlas doesn't tell you *when* each piece of information is from.
[1:33 PM] <wfn> yeah, that'd be useful, hm
[1:33 PM] <armadev> i bet Uptime would do better if it were "Uptime (as of 17 hours ago"
[1:34 PM] <wfn> not familiar with atlas' code, but yeah that'd help, hm
[1:34 PM] <armadev> and similarly, "Flags (as of x ago)"
[1:34 PM] <armadev> wfn, can i ask you to file a ticket for this feature / bugfix?
[1:35 PM] <wfn> armadev: okay, will do :)
[1:35 PM] <armadev> thanks!
[1:35 PM] <wfn> armadev: likewise with rndm's tool: http://globe.rndm.de/#/relay/0FB356FBE623B3F6C8A2CC3CA42E0E7681F72FE7
[1:35 PM] <wfn> 'Running' flag reported, but not the consensus from which it was extracted
[1:35 PM] <wfn> okay, will file in couple of h or sooner
[1:36 PM] <armadev> looks like it, yes
[1:37 PM] <arlolra_> interesting. I ask because the current check has that relay in the bulk list but since it's not in the consensus, check2 isn't reporting it
[1:37 PM] <armadev> ah yes
[1:37 PM] <armadev> i forgot that feature
[1:37 PM] * armadev plays some more circus music as he multitasks in too many directions
[1:37 PM] <armadev> so, here's the issue. if your relay falls out of the consensus briefly, but clients have the consensus from two hours ago, and they use the relay and exit from it,
[1:38 PM] <armadev> and check.tp.o says "nope, not an exit!"
[1:38 PM] <wfn> ooh
[1:38 PM] <armadev> then people will be surprised.
[1:39 PM] <armadev> the general shape of the solution so far has been "if we would have listed it in any of the last n consensus periods" where n is typically 16 or something
[1:39 PM] <wfn> ideally, the "fresh_until" field in consensuses should be used to decide whether exit addresses in that particular consensus are valid, i suppose?
[1:39 PM] <armadev> a related issue is: if you're a relay and you stop being a relay, odds are not bad that you'll show back up pretty soon. so if we stop listing you, when you come back, you'll be a false negative. better to be a false positive for a short time period instead.
[1:40 PM] <wfn> i.e., won't tor clients refuse to use an outdated consensus, where outdated means "more fresh than valid_until / fresh_until" (not sure of field name)
[1:40 PM] <armadev> wfn: tor clients will continue to use a consensus they have for 24 hours, even if it's out of date, if they can't get a better one
[1:40 PM] <wfn> oh. hm.
[1:40 PM] <armadev> this is both a bug and a feature
[1:40 PM] <armadev> the 'feature' side has won out so far, since it makes the network not break if a few directory authorities fall over briefly
[1:41 PM] <armadev> (when phrased like that, it seems like a pretty important feature)
[1:41 PM] <wfn> yeah, it'd make sense to me (total newcomer) at least for sure
[1:41 PM] <wfn> can the bulk exit list checker than include all exit addrs in the last 24h maybe?
[1:42 PM] <arlolra_> it can if that's the right solution .. which it seems to be
[1:43 PM] <armadev> unless we can produce a better solution
[1:43 PM] <armadev> also, the main check.tp.o should probably use the same list
[1:45 PM] <armadev> isn't this exciting? all these changing requirements? :)
[1:45 PM] <armadev> someday somebody should write up a description of how the check algorithm behaves, and should behave
[1:45 PM] <armadev> wfn: if you could open a ticket for that too, assuming you want to help answer it, that'd be grand
[1:46 PM] <arlolra_> Rym wrote some docs
[1:46 PM] <arlolra_> https://github.com/arlolra/check/blob/master/docs/ABOUT.md
[1:46 PM] <smallproblem> I think I have a repoducable quirky bug.  The latest vidalia relay bundle for windows on windows xp, if you're running vidalia/tor and open notepad, a shortcut to torrc is created
[1:46 PM] <wfn> armadev: of course, very glad to help, will do!
[1:47 PM] <armadev> smallproblem: open notepad for anything? or for torrc?
[1:47 PM] <smallproblem> start--> run--> notepad
[1:48 PM] <smallproblem> that's it, not opening a file or anything
[1:49 PM] <armadev> smallproblem: exciting. can you help track it down? i fear it is unlikely to be solved otherwise.
[1:50 PM] <armadev> smallproblem: step one is to open a ticket for it on trac
[1:52 PM] <armadev> arlolra: the broader question here is: what are likely false positives and false negatives, and which ones are worse when they occur?
[1:52 PM] <armadev> right now we have check.tp.o telling tbb users "you're not using tor!" and it's a lie
[1:52 PM] <armadev> that's a pretty bad false negative
[1:52 PM] <Lever> armadev : no one answer me
[1:53 PM] <armadev> lever: still. here is bad.
[1:53 PM] <smallproblem> ok, send me that info again, i was trying to reproduce the issue on win7, \
[1:53 PM] <armadev> > smallproblem: exciting. can you help track it down? i fear it is unlikely to
[1:53 PM] <armadev> +be solved otherwise.
[1:53 PM] <armadev> > smallproblem: step one is to open a ticket for it on trac
[1:53 PM] <smallproblem> open ticket how?
[1:53 PM] <wfn> armadev: i'm not sure which component the second ticket (exit list strategy, check{1,2}.tp.o) should be filed under. i suppose it doesn't have to do anything directly with tordnsel, etc.? or does it?
[1:53 PM] <armadev> smallproblem: bugs.torproject.org, make an account, new ticket
[1:53 PM] <wfn> arlolra: i guess you're not using the exit lists provided? https://metrics.torproject.org/data.html#exitlist
[1:54 PM] <armadev> wfn: i'd go with component Check
[1:54 PM] <armadev> (if there is one)
[1:54 PM] <wfn> agh! there's "Tor Check" - missed that one
[1:54 PM] <armadev> wfn: i believe arlolra's code takes those exitlists as input. but see also the ticket where they're missing a lot of entries.
[1:54 PM] <arlolra> wfn: we're using the output of TorDNSEL, which produces those
[1:54 PM] <wfn> ok, ok [absorbing new info]
[1:54 PM] <wfn> thanks
[1:55 PM] <smallproblem> hold on
[1:55 PM] <arlolra> armadev: ok, so err on the side of telling tbb users they are using tor
[1:56 PM] <armadev> i think so. i guess the false positive side is when an exit relay operator loads this page in her safari.
[1:56 PM] <arlolra> in #9204 B) ... For users who want to consider all relays in the past n hours, this script should be able to handle reading multiple consensus files (ideally just by concatenating them before they go in).
[1:56 PM] <armadev> another false positive side is if your neighbor ran an exit relay, and then the dynamic IP game happened and you inherited her IP
[1:56 PM] <wfn> or, a tor relay is running in a large local network / e.g. uni network.. :(
[1:57 PM] <armadev> wfn: ah yes, another one is if you're behind a nat and share it with an exit
[1:57 PM] <armadev> arlolra: oh good, i did write about this problem in #9204 then. :)
[1:57 PM] <armadev> s/write about/hint at/
[1:59 PM] <wfn> so, uh, did anyone think about the totally insane and probably non-scalable idea of providing a 'paranoid tor check' service which would attempt to establish a circuit to/through a given exit adderss, and then immediately report results in http response?
[1:59 PM] <wfn> though, uh, that still doesn't solve the 'large local network' problem, so eh
[1:59 PM] <armadev> that wouldn't be a complete solution, right, since exits can be multihomed?
[1:59 PM] <armadev> and since that
[1:59 PM] <wfn> indeed, indeed..
[2:00 PM] <wfn> bugger.
[2:03 PM] <armadev> wfn: sounds like a good opportunity for somebody to write up all the pros and cons, so the next conversation goes easier.
[2:03 PM] <armadev> the right answer for tbb is for tbb to check itself. that's what tbb 3.0 does. it doesn't use an external website (which, heck, could lie to it).
[2:03 PM] <smallproblem> @armadev here's the ticket!  see: https://trac.torproject.org/projects/tor/ticket/9808
[2:04 PM] <wfn> oh, right! forgot about that, hey that's great.
[2:04 PM] <armadev> for example, imagine the following attack. alice is not using tor, and has an adversarial network. she asks for check.torproject.org. her attacking network notices that, and redirects her request through tor, so that a) she gets the right ssl cert for check.tp.o, and b) check says she's using tor.
[2:04 PM] <armadev> but the adversarial network doesn't redirect anything else. or redirects everything but a few things.
[2:06 PM] <wfn> and this is trivially possible even when she's making an https request from the beginning?
[2:06 PM] <smallproblem> alright guys, thanks so much.  I'm going back to working on other stuff.
[2:06 PM] <armadev> wfn: yes, i think so
[2:06 PM] <armadev> smallproblem: thanks. i think you may actually have the bug backwards -- i wonder if the relay bundle tries to make the shortcut always, and it doesn't show up sometimes.
[2:06 PM] <wfn> yes, that sounds bad. i mean, elaborate model of evil, but that's not good for sure.
[2:07 PM] <armadev> arlolra: in general, the question for torbulkexitlist, especially if external services are going to cache their answer for a while, is "what list shall i give them that accurately predicts the right answer for x hours into the future?"
[2:07 PM] <smallproblem> @armadev why would relay bunde want to make a shortcut to torrc?
[2:07 PM] <smallproblem> that makes no sense
[2:07 PM] <armadev> to help you edit it?
[2:08 PM] <armadev> i dunno. these bundles are nearly unmaintained.
[2:09 PM] <smallproblem> But you can do that inside Vidalia anyway.  creating an all users shortcut on the start menu to my account's specific torrc user is silly.
[2:09 PM] <wfn> to add to the circus, re: atlas flag confusion, thing is, the onionoo backend is basically reporting that same info:
[2:09 PM] <wfn> https://onionoo.torproject.org/details?lookup=0FB356FBE623B3F6C8A2CC3CA42E0E7681F72FE7
[2:09 PM] <wfn> *but* it is implied that those flags are coming from the last consensus available (hopefully i'm not mixing things up, but i think they are.)
[2:09 PM] <wfn> on its part, atlas could simply add a warning icon or whatnot to the flags if it sees that running=false as reported by onionoo. if it's more complicated than that, then it might require karsten's intervention. i'll write this in the ticket, end of wall of text
[2:09 PM] <mttpgn> Sherief: I added something about the Cyberoam firewall to Tor Weekly News. Can you see if there's anything you'd like to add?
[2:09 PM] <armadev> the feature to edit the torrc from inside vidalia is a) quite new, and b) mostly broken
[2:10 PM] <smallproblem> oh well.
[2:10 PM] <Sherief> mttpgn: Will check it right now, I should've done it my self so thanks a bunch :)
[2:10 PM] <armadev> wfn: that "last_seen" line looks important
[2:10 PM] <armadev> (in onionoo)
[2:11 PM] <wfn> yeah, it's the same that you saw when you linked to metrics relay info page about turtles - last consensus this was present in.
[2:11 PM] <wfn> i've reimplemented these parts in my backend as well:
[2:11 PM] <wfn> http://ts.mkj.lt:5555/details?lookup=0FB356FBE623B3F6C8A2CC3CA42E0E7681F72FE7
[2:11 PM] <wfn> http://ts.mkj.lt:5555/statuses?lookup=0FB356FBE623B3F6C8A2CC3CA42E0E7681F72FE7&condensed=true
[2:12 PM] <wfn> the same that you saw - semantically the same meaning
[2:12 PM] <helix> smallproblem: thanks for reporting 9808. wtf.
[2:12 PM] <smallproblem> @helix, you're welcome.
[2:13 PM] <smallproblem> @helix I thought I got hacked
[2:13 PM] <helix> I haven't seen this on win7 but I'll see if I can find an XP to check it out with
[2:14 PM] <helix> is winxp your normal OS?
[2:14 PM] <smallproblem> somewhat
[2:15 PM] <Sherief> mttpgn: I have nothing to add, Thanks again.
[2:16 PM] <smallproblem> I use linux and win7, it only happens in winxp
[2:16 PM] <arlolra> armadev: alright, well, I'll start by implementing the current solution. doesn't seem like a bad predictor
[2:17 PM] <arlolra> and seems straightforward with what we have
[2:17 PM] <mttpgn> Sherief: great. I tried to find something on the support mailing list about it, but I didn't see it. Did you summarize this issue on any of the mailing lists