Tackling big data: Storage and transfer

In their recently published paper titled "The Widening Gulf between Genomics Data Generation and Consumption: A Practical Guide to Big Data Transfer Technology," Feltus, Wang and six other co-authors at Clemson, the University of Utah and the National Center for Biotechnology Information discussed the careful planning and engineering required to move and manage Big Data at the speeds needed for high-throughput science. If properly executed, sophisticated data transfer networks, such as Internet2's Advanced Layer2 Service, as well as the inclusion of advanced applications and software, can improve transfer efficiency by orders of magnitude.

"Universities and other research organizations can spend a lot of money building supercomputers and really fast networks," Feltus said. "But with research computing systems, there's a gulf between the 'technology people' and the 'research people.' We're trying to bring these two groups of experts together and learn to speak a common dialect. The goal of our paper is to expose some of this information technology to the research scientists so that they can better see the big picture."

http://www.sciencedaily.com/releases/2015/10/151029134644.htm