↓ Skip to main content

A quantitative assessment of the Hadoop framework for analyzing massively parallel DNA sequencing data

Overview of attention for article published in Giga Science, June 2015
Altmetric Badge

About this Attention Score

  • In the top 25% of all research outputs scored by Altmetric
  • High Attention Score compared to outputs of the same age (91st percentile)

Mentioned by

twitter
28 tweeters
peer_reviews
1 peer review site
facebook
1 Facebook page
googleplus
1 Google+ user

Citations

dimensions_citation
12 Dimensions

Readers on

mendeley
44 Mendeley
citeulike
4 CiteULike
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
A quantitative assessment of the Hadoop framework for analyzing massively parallel DNA sequencing data
Published in
Giga Science, June 2015
DOI 10.1186/s13742-015-0058-5
Pubmed ID
Authors

Alexey Siretskiy, Tore Sundqvist, Mikhail Voznesenskiy, Ola Spjuth

Abstract

New high-throughput technologies, such as massively parallel sequencing, have transformed the life sciences into a data-intensive field. The most common e-infrastructure for analyzing this data consists of batch systems that are based on high-performance computing resources; however, the bioinformatics software that is built on this platform does not scale well in the general case. Recently, the Hadoop platform has emerged as an interesting option to address the challenges of increasingly large datasets with distributed storage, distributed processing, built-in data locality, fault tolerance, and an appealing programming methodology. In this work we introduce metrics and report on a quantitative comparison between Hadoop and a single node of conventional high-performance computing resources for the tasks of short read mapping and variant calling. We calculate efficiency as a function of data size and observe that the Hadoop platform is more efficient for biologically relevant data sizes in terms of computing hours for both split and un-split data files. We also quantify the advantages of the data locality provided by Hadoop for NGS problems, and show that a classical architecture with network-attached storage will not scale when computing resources increase in numbers. Measurements were performed using ten datasets of different sizes, up to 100 gigabases, using the pipeline implemented in Crossbow. To make a fair comparison, we implemented an improved preprocessor for Hadoop with better performance for splittable data files. For improved usability, we implemented a graphical user interface for Crossbow in a private cloud environment using the CloudGene platform. All of the code and data in this study are freely available as open source in public repositories. From our experiments we can conclude that the improved Hadoop pipeline scales better than the same pipeline on high-performance computing resources, we also conclude that Hadoop is an economically viable option for the common data sizes that are currently used in massively parallel sequencing. Given that datasets are expected to increase over time, Hadoop is a framework that we envision will have an increasingly important role in future biological data analysis.

Twitter Demographics

The data shown below were collected from the profiles of 28 tweeters who shared this research output. Click here to find out more about how the information was compiled.

Mendeley readers

The data shown below were compiled from readership statistics for 44 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Sweden 2 5%
United States 2 5%
Norway 1 2%
United Kingdom 1 2%
Germany 1 2%
Unknown 37 84%

Demographic breakdown

Readers by professional status Count As %
Researcher 14 32%
Student > Ph. D. Student 7 16%
Student > Bachelor 6 14%
Student > Master 4 9%
Student > Postgraduate 3 7%
Other 8 18%
Unknown 2 5%
Readers by discipline Count As %
Agricultural and Biological Sciences 14 32%
Computer Science 11 25%
Biochemistry, Genetics and Molecular Biology 5 11%
Engineering 5 11%
Pharmacology, Toxicology and Pharmaceutical Science 2 5%
Other 5 11%
Unknown 2 5%

Attention Score in Context

This research output has an Altmetric Attention Score of 19. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 23 September 2015.
All research outputs
#1,161,456
of 16,597,904 outputs
Outputs from Giga Science
#284
of 825 outputs
Outputs of similar age
#20,278
of 238,515 outputs
Outputs of similar age from Giga Science
#1
of 1 outputs
Altmetric has tracked 16,597,904 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 92nd percentile: it's in the top 10% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 825 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 23.1. This one has gotten more attention than average, scoring higher than 65% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 238,515 tracked outputs that were published within six weeks on either side of this one in any source. This one has done particularly well, scoring higher than 91% of its contemporaries.
We're also able to compare this research output to 1 others from the same source and published within six weeks on either side of this one. This one has scored higher than all of them