* Now talking on #galaxyproject
 * Topic for #galaxyproject is: The Galaxy Project, high powered genomics for everyone - http://galaxyproject.org | Use now! http://usegalaxy.org
 * Topic for #galaxyproject set by natefoo at Mon Mar  7 23:00:14 2011
 * #galaxyproject :http://galaxyproject.org
 * jenjackson (~jenjackso@adsl-71-134-238-186.dsl.pltn13.pacbell.net) has joined #galaxyproject
 * ceberhard (~carleberh@107.194.88.162) has joined #galaxyproject
 <jasperk> When visualising a gtf file in the trackster, does someone knows why auto visualisation works but pack visualisation does not work locally but it works on the galaxy online instance...
 <jasperk> I see now that I get an error in the terminal...
 <dannon> Did you figure out your issue based on the error message?
 <jasperk> I think it has to do with my "custom" genome as the error is
 <dannon> Ahh, k.
 <jasperk> http://pastebin.com/mVN1dGS1
 <mrscribe> Title: galaxy.webapps.galaxy.api.datasets ERROR 2014-01-03 15:37:35,907 Error in datase - Pastebin.com (at pastebin.com)
 <jasperk> I am looking for a way to load the entire bacterial database into galaxy, but it seems like a lot of manual work...
 <dannon> So you're following https://wiki.galaxyproject.org/Learn/CustomGenomes ?
 <mrscribe> Title: Learn/CustomGenomes - Galaxy Wiki (at wiki.galaxyproject.org)
 <jasperk> Yeah, I have a fasta formatted genome, I have a GTF file and I set the dbkey such that it can be mapped against the fasta genome. And when I visualise it in trickster I do see a line, but as it is a GTF file I prefer to have the pack visualization
 <dannon> What revision are you running locally?
 <jasperk> hmm, when I upload the custom genome to usegalaxy.org and the gtf, it works... I think I updated to the latest version let me check again
 <jasperk> hg update stable
 <jasperk> 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
 <dannon> 'hg summary' ?
 <jasperk> hg summary
 <jasperk> parent: 11247:a9a0ac9c1afa tip
 <jasperk>  Fix for determining a guid in the install_manager.
 <jasperk> branch: stable
 <jasperk> commit: 1 modified, 7 unknown
 <jasperk> update: (current)
 * martenson (~martenson@martenson.bx.psu.edu) has joined #galaxyproject
 <dannon> Is this a galaxy instance you're able to update?  There should be more changes in the pipeline.
 <jasperk> yes I have admin rights
 * martenson has quit (Client Quit)
 <dannon> Ok, then that's what I'd try first; it may be that this is an older bug that's been fixed.
 * martenson (~martenson@2610:8:7800:14:c80:91f:bf8a:7a93) has joined #galaxyproject
 <jasperk> but the update state that there is none?
 <jasperk> hg update stable
 <dannon> update only updates local state, you need to 'pull' to get new changesets from the upstream repository.
 <dannon> So, 'hg pull -u <repository>'
 <jasperk> hg pull -u
 <jasperk> pulling from https://bitbucket.org/galaxy/galaxy-dist/
 <jasperk> searching for changes
 <jasperk> no changes found
 <mrscribe> Title: galaxy / galaxy-dist Bitbucket (at bitbucket.org)
 <dannon> This may or may not fix your issue, but it'll get you as up to date as galaxy main -- pull from galaxy-central
 <dannon> You'll stay on the stable branch, so no worries there
 <jasperk> added 691 changesets with 1664 changes to 521 files (-1 heads)
 <jasperk> 2 files updated, 0 files merged, 0 files removed, 0 files unresolved
 <jasperk> lets restart galaxy
 <dannon> So only two updates.  Yeah, restart and see if it fixes it, but we may have to dig deeper.
 <jasperk> hg pull -u https://bitbucket.org/galaxy/galaxy-central/ right?
 <mrscribe> Title: galaxy / galaxy-central Bitbucket (at bitbucket.org)
 <dannon> Yep
 <dannon> All that meant is that you got all of the changesets in -central that aren't yet in -dist, but 'update' only applied those changes from the stable branch
 <dannon> So, only two files, apparently.
 <jasperk> there is a problem with galaxy.tools ERROR 2014-01-03 16:04:40,330 Error reading tool from path: genetrack/genetrack_indexer.xml
 <jasperk> but galaxy starts
 <dannon> Tool was removed, I think.
 <jasperk> no still the same errors
 <jasperk> galaxy.webapps.galaxy.api.datasets ERROR 2014-01-03 16:05:44,038 Error in dataset API at listing contents: 'data': 'data'
 <jasperk> Traceback (most recent call last):
 <jasperk>   File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 46, in show
 <jasperk>     rval = self._data( trans, dataset, **kwd )
 <jasperk>   File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 200, in _data
 <jasperk>     ref_seq = data_dict[ 'data' ]
 <jasperk> KeyError: 'data'
 <ceberhard> jasperk: this may have to do with the 'chrom' selection in that part of the code.
 <ceberhard> What value are you using for the chromosome selection in trackster?
 <dannon> ceberhard: You were my next move, heh.
 <ceberhard> (since prokaryotes don't necc. have that value)
 <ceberhard> dannon: jgoecks might be the best for this
 <ceberhard> So hopefully he's not packing :)
 <jasperk> I have cur as a header in the gtf file
 <jasperk> chr gb2gtf gene 74 1171 . - . gene_id "TT_C0001"; transcript_id "TT_C0001"
 <jasperk> chr gb2gtf CDS 74 1171 . - . gene_id "TT_C0001"; transcript_id "TT_C0001"; protein_id "AAS80349.1"; db_xref "GI:46195931"
 <dannon> I assumed he'd be packing/something, yeah :)
 <jasperk> and this is the fasta header of the genome >chrom
 <jasperk> hum cur i mean
 <jasperk> stupid apple auto correct... >chr is the fasta header
 <ceberhard> Ok: two disclaimers:
 <ceberhard> This might be a better question for galaxy-dev depending on how soon you need to get this done. Jeremy might be better able to answer there.
 <ceberhard> 2) I'm not 100% familiar with this code in trackster. But I'm happy to try and work this out now if you need to.
 <ceberhard> We could start by seeing if the chrom selection is making it to that part of the code.
 <jasperk> I can always post on the mailing list if that is preferred, I do see that 2 chr are detected (chromosome and plasmid) in the current custom builds
 <jasperk> I think the trackster detects it nicely as a line is drawn in the autoplot in trickster
 <jasperk> http://picpaste.com/pics/Screen_Shot_2014-01-03_at_16.19.43-P3aeOn2v.1388762361.png
 <ceberhard> Ok. I'd encourage you to ask over email, then, since I believe you'll get a better, easier soln. But - if you're willing to dig into the code a bit - you might add a print to line 200 of lib/galaxy/webapps/galaxy/api/datasets.py: 'print data_dict' (without the single quotes).
 <ceberhard> Yeah - looks like it's getting some data. No idea why no seq_data in the dictionary, tho.
 * martenson has quit (Remote host closed the connection)
 <jasperk> {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'}
 <dannon> Oh.
 <ceberhard> So you need a 2bit version of your reference I'm guessing.
 <ceberhard> Nope.
 <dannon> iirc, it'll automatically happen if you install those tools.
 <ceberhard> You need the tool.
 <dannon> I had to do this before.
 <ceberhard> Huh.
 <dannon> Yep.
 * martenson (~martenson@martenson.bx.psu.edu) has joined #galaxyproject
 <dannon> You can snag the stable executables here for various platforms:  http://hgdownload.cse.ucsc.edu/admin/exe/
 <mrscribe> Title: Index of /admin/exe (at hgdownload.cse.ucsc.edu)
 <dannon> Just download faToTwoBit for your platform and drop it on your path somewhere.
 <jasperk> downloading restarting put it in /usr/bin i guess?
 <dannon> No need to restart
 <dannon> But yeah, you could put it there.
 <jasperk> download the other as well? {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'{'kind': 'error', {'kind': 'error', {'message': '/bin/sh: 1: fa'TmesosTwoBiatge': not found\n'}
 <jasperk> hmm TmesosTwoBiatge is not located in the exe dir.
 <dannon> Did that get mangled?
 <jasperk> just  a sec forgot chmod +x
 <jasperk> in the galaxy account I can perform in the terminal /usr/bin/faToTwoBit
 <jasperk> faToTwoBit - Convert DNA from fasta to 2bit format
 <jasperk> {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'}
 <jasperk> where does galaxy look for faToTwoBit?
 <dannon> Local single node install?  Are you using a virtual environment or anything?
 <dannon> It'd check on the galaxy user's path, whatever that's set to.
 <jasperk> yeah local node, but I think the terminal needed a refresh
 <jasperk> {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'{'kind': }
 <jasperk> 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'}
 <jasperk> $ faToTwoBit
 <jasperk> faToTwoBit - Convert DNA from fasta to 2bit format
 <jasperk> hmm it should be accessible for galaxy
 <dannon> And you're not using a virtualenv or anything else that'd overwrite the path?
 <jasperk> it is running on a ubuntu 64 bit server directly, no virtual machine and such
 <jasperk> and under echo $ are no galaxy paths or whatsoever
 <jasperk> hmm
 <jasperk> might it have to do with /bin/sh?
 <dannon> That should just be echoing the error you're getting in the shell; that is, that faToTwoBit isn't found
 <dannon> in run.sh, can you ensure that the faToTwoBit location is in the path?
 <jasperk> yeah but if i run /bin/sh faToTwoBit in the terminal /bin/sh: 0: Can't open faToTwoBit
 <jasperk> as I am in bash
 <dannon> It doesn't get executed quite like that; red herring.
 * martenson has quit (Remote host closed the connection)
 <dannon> Something I've done before just to verify the path is to edit a tool to echo $PATH back and return
 * martenson (~martenson@martenson.bx.psu.edu) has joined #galaxyproject
 <jasperk> /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/bin
 <jasperk> and the file is located /usr/bin/faToTwoBit
 <jasperk> thus in theory it should find it
 <jasperk> but no luck
 <jasperk> I think the error comes from File: extract_genomic_dna.py
 <dannon> Well, I'm not sure what's going on with the path there, but the *super* hacky thing to do to short cut all this is to replace faToTwoBit with the full path.
 <jasperk> I could only find faToTwoBit in extract_genomic_dna.py are there other locations?
 <ceberhard> Converters
 <dannon> I think it's in the datatype converter, yeah.
 <ceberhard> galaxy/datatypes/converters/fasta_to_2bit.xml
 <dannon> That's the one.  I'm assuming trackster is trying to convert the reference for you, and that's what it'd use.
 <ceberhard> Although: '<!-- <description>__NOT_USED_CURRENTLY_FOR_CONVERTERS__</description> -->'
 <ceberhard> No idea what that means or if it stands since it's commented out.
 <dannon> Could be a lie.
 <ceberhard> Right.
 <dannon> Love debugging stuff I've never touched, heh :)
 <ceberhard> :P
 <ceberhard> You could try editing that with your full path.
 <ceberhard> To see if it works?
 <dannon> A note, you *will* need to restart galaxy after editing this one; the tool will need to be reloaded.
 <ceberhard> Sorry this has gotten complicated, jasperk, and thanks for being patient.
 <jasperk> yeah it surprises me too I mean a bash path should be simple
 <dannon> Should be.
 <ceberhard> Right.
 <jasperk> I already called that I would be home later
 <jasperk> yeah I learned it the hard way that xmls need restart
 <jasperk> just to be sure like this right? <command>/usr/bin/faToTwoBit $input $output</command>
 <jasperk> {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'}
 <jasperk> {'kind'{'kind': 'error'galaxy.webapps.galaxy.api.datasets ERROR 2014-01-03 17:15:44,479 Error in dataset API at listing contents: 'data': 'data'
 <jasperk> Traceback (most recent call last):
 <jasperk> dammit :P
 <dannon> Ok, so it may not be trying to use that one.
 <dannon> /lib/galaxy/tools/genome_index/index_genome.py ?
 <dannon> Seems like something trackster would potentially do.
 <jasperk> {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n'}
 <jasperk> {'kind': {'kind': 'error', 'message': '/bin/sh: 1: faToTwoBit: not found\n''error'}
 <jasperk> same
 <ceberhard> Yeah - the only other way I'd try at this point (besides galaxy-dev) is create the 2bit file outside galaxy and edit the loc file. https://wiki.galaxyproject.org/Visualization%20Setup?highlight=%28faToTwoBit%29
 <mrscribe> Title: Visualization Setup - Galaxy Wiki (at wiki.galaxyproject.org)
 <ceberhard> (Or head home and have a beer)
 <ceberhard> I'm not entirely sure that would work without creating a new visualization, tho.
 <jasperk> yeah its already dark outside. ILL setup a galaxy-dev post and grab a beer or two
 <dannon> Sounds like a plan to me
 <jasperk> as I never did the 2bit set up it might take a moment to do so.
 <ceberhard> Right.
 <jasperk> thanks a lot anyway! It sure helped me to get a feeling on whats in the engine of galaxy
 <dannon> Sure, sorry we couldn't actually get it working.
 <ceberhard> Yeah - hope there's a simple soln.
 <jasperk> well in theory it should be. I mean it can't find a file that is in /usr/bin i mean thats just completely odd
 <jasperk> and I think it is almost certainly a ubuntu os issue
 <ceberhard> Yes - very.
 <dannon> The other 'fix' I mentioned was to explicitly set your path in run.sh
 <dannon> Acutally, that might not help, because the path seemed to include /usr/bin and it just wasn't accessible
 <dannon> What were the permissions and ownership of faToTwoBit?
 <jasperk> i eventually chmodded it to 777
 <jasperk> to leave out any permission issues
 <dannon> Hmm, ok.