critical assessment of metagenome...
TRANSCRIPT
Critical Assessment of
Metagenome Interpretation
Alice C. McHardy
Computational Biology of Infection Research
Helmholtz Centre for Infection Research
and the CAMI Initiative
Computational Metagenomics
AGCGATTTGACACGGAGAGTATAGAA
AGCGATTTGACACGATGATAG
GAGAGATATTTTAGACCCACGATGATCC
AGCGATTTGTAGTATATCGAGGGG
Sequencing
[…]sdATGACGATTCCGAAACACGAGCCGCGCGAGGTCTTCGATCGGGCGATCGAGCATACGCGGGCGCTTTCCC CGGCCACTTTCGATCAGTGGTTCGGGGGAGTTCAGTTCGATGATCTGACCGACGGCGTCCTGACGCTGCG CGAGCAGCAGCGCGCCATAGCGCGCCAGCACGTCGCGGGCCCGCGCCATGCGCCTCGCCGCGTCGTCCTC GGATCCCGTGAAGATGAGCACGAGCGCCGTGACCTCGCGCCGCTCGC
CCTCGACGGGGGTGCGCTCGTTGGCCCCGTTCGGCTCCTCTTGGAGCACGGCGCTCGCCTGGATCTCGGC
GCTCTTCTCCTCGTGGAACGGCTCCAGGAAATCGGCGAGGTCGTTCGCCCCGAACCGCTCGGCCGTCGCG TAGAAGAAGCCGAGGAGCGCCTCGTGCAGCCGCCCGGCGTCGGGGAACCGATCCTCCGGGTTCTTCGCGA GCATCGTGCTCACGACGTCGGCGAGCGACCGGGGCACGTCGGGGCGCACGAGCGCGAGCGGCGGATACTC
GCTCGCCTGCAGCCGCCGCATGATCTCGAACGCCGTCGGCGCCGCGAACGGGTTCGACCCGGCGAGCATC GATGGTCGTGTTCCGCGGGATGAGCTTGAAGCTCACGCCGCCGCCGGTCTCGACGCCGAGCGACAGCGGC
GTCACGTCGAGGAGCAGCACCTCGTCCACCTGCCCGGAGAGCGCGGCGCCCTGCAGCGCAGCGCCGACCG CGACGACCTCGTCGGGGTTGACGCCCTTGTTCGGCTCCCGGCCGAAGAACTCGCGCACCGCCGCCTGCAC CGCGGGCATCCGGGTCATGCCGCCGACGAGGACCACCGTGTTGACGGCGGAGACGGGGAGCTTGGCGTCC CCGAGCGTCGCGCGACAGACGTCGATGGTCCGCCGGATGAGGCCCTCGCAGAGCATCTCGAGCTCGTTGC GCCGCATCGTCCGCTCGAGGTGGAGCGGCCCTCCGCCCGGGCCGACGGCGATGAAGGGGATGTTGATCTC
GGTCTCGAGCGACGACGAGAGCTCGTGCTTCGCCTTCTCGGCCGCCTCCTTGAGCCGCTGCAGCGCCATG CGATCCCGGCGCAGATCGATGCGGCTCTTCCCCTCGAACTCGTCGGCGAGCAGGTCGATGATCCGCTGGT CGAAGTCCTCGCCGCCGAGGTGCGTGTCGCCGCCGGTCGCCTTGACGCTGAAGACCCCGCTCGCGATCTCGAGGATCGAGATATCGAACGTGCCTCCGCCGAGATCGTAGACCGCGATCGTCTCGGCCTTCACCTTGTCG […]
From Garcia Martin
et al. Nature
Biotech. 2006
Assembly
Genome &
Taxonomic
‘binning’[…]sdATGACGATTCCGAAACACGAGCCGCGCGAGGTCTTCGATCGGGCGATCGAGCATACGCGGGCGCTTTCCC CGGCCACTTTCGATCAGTGGTTCGGGGGAGTTCAGTTCGATGATCTGACCGACGGCGTCCTGACGCTGCG CGAGCAGCAGCGCGCCATAGCGCGCCAGCACGTCGCGGGCCCGCGCCATGCGCCTCGCCGCGTCGTCCTC GGATCCCGTGAAGATGAGCACGAGCGCCGTGACCTCGCGCCGCTCGC
CCTCGACGGGGGTGCGCTCGTTGGCCCCGTTCGGCTCCTCTTGGAGCACGGCGCTCGCCTGGATCTCGGC
GCTCTTCTCCTCGTGGAACGGCTCCAGGAAATCGGCGAGGTCGTTCGCCCCGAACCGCTCGGCCGTCGCG TAGAAGAAGCCGAGGAGCGCCTCGTGCAGCCGCCCGGCGTCGGGGAACCGATCCTCCGGGTTCTTCGCGA GCATCGTGCTCACGACGTCGGCGAGCGACCGGGGCACGTCGGGGCGCACGAGCGCGAGCGGCGGATACTC
GCTCGCCTGCAGCCGCCGCATGATCTCGAACGCCGTCGGCGCCGCGAACGGGTTCGACCCGGCGAGCATC GATGGTCGTGTTCCGCGGGATGAGCTTGAAGCTCACGCCGCCGCCGGTCTCGACGCCGAGCGACAGCGGC
GTCACGTCGAGGAGCAGCACCTCGTCCACCTGCCCGGAGAGCGCGGCGCCCTGCAGCGCAGCGCCGACCG CGACGACCTCGTCGGGGTTGACGCCCTTGTTCGGCTCCCGGCCGAAGAACTCGCGCACCGCCGCCTGCAC CGCGGGCATCCGGGTCATGCCGCCGACGAGGACCACCGTGTTGACGGCGGAGACGGGGAGCTTGGCGTCC CCGAGCGTCGCGCGACAGACGTCGATGGTCCGCCGGATGAGGCCCTCGCAGAGCATCTCGAGCTCGTTGC GCCGCATCGTCCGCTCGAGGTGGAGCGGCCCTCCGCCCGGGCCGACGGCGATGAAGGGGATGTTGATCTC
GGTCTCGAGCGACGACGAGAGCTCGTGCTTCGCCTTCTCGGCCGCCTCCTTGAGCCGCTGCAGCGCCATG CGATCCCGGCGCAGATCGATGCGGCTCTTCCCCTCGAACTCGTCGGCGAGCAGGTCGATGATCCGCTGGT CGAAGTCCTCGCCGCCGAGGTGCGTGTCGCCGCCGGTCGCCTTGACGCTGAAGACCCCGCTCGCGATCTC GAGGATCGAGATATCGAACGTGCCTCCGCCGAGATCGTAGACCGCGATCGTCTCGGCCTTCACCTTGTCG […]
Annotation
Bacteria 0.7
Archaea 0.3
Proteobacteria 0.2
Firmicutes 0.05
…
A big challenge in computational metagenomics
Lacking consensus about benchmarking data sets, evaluation procedures and metrics complicates proper
performance assessment
Towards a comprehensive and objective evaluation ofcomputational metagenomics software
Page 4 |
Principles
Design decisions made by the community (data sets, evaluation measures and principles)
Extensive, high-quality benchmark datasets fromunpublished data
Evaluation measures: informative to developers and theapplied community
Reproducibility: data generation, programs, evaluation
Benchmark assembly, (tax.) binning and taxonomicprofiling software
Timeline 1st CAMI Challenge
Sczyrba et al., Nature Methods 2017
Reproducibility and Standardization
Belmann et al., Gigascience 2015
Output formats for binning
and profiling
Standard interfaces
Docker-based Bioboxes
for programs and metrics
Semi-automatic
benchmarking in future
CAMI Challenge Datasets
Common experimental setups and community types
Strain-level variation
Different ev. distances to public genomes
Non-bacterial sequences (archaea, plasmids, viruses)
Seite 9 |
CAMI_low
1 sample
15 Gb
2 x150 bp
Insert size:
270 bp
CAMI_medium
(differential
abundance)
2 samples
40 Gb
2 x150 bp
Insert sizes:
270 bp & 5kbp
CAMI_high
(time series)
5 samples
75 Gb
2 x150 bp
Insert size:
270 bp
Simulated from ~700 novel microbial genomes, 600 novel
viruses, plasmids and other circular elements
https://github.com/CAMI-challenge/MetagenomeSimulationPipeline
0 200 400 600 800 1000 1200
high
medium
low
CAMI Challenge Datasets
Genomes Strains Strains (evolved) plasmids/viruses
80 groups
ANI ≥ 95%
High
Challenge Participants
https://data.cami-challenge.org
Early 2015: >40 registered
participants
Submissions to the CAMI Challenge
With consent to publish: 215 submissions;16 teams; 25 programs; 36
bioboxes
Seite 14 |
Results of the Assembly Challenge
Seite 15 |
Unique strainsCommon strainsAll
Good performance in genome assembly for unique strains
Subspecies diversity is a challenge!
Genome Recovery at different Coverages
Page 16 |
• Assemblers using multiple kmers performed better
• Minia (and Meraga) are good in plasmid assembly
Meraga
Velour (k=63)
Megahit
Ray (k=51)
Minia
Gold
standard
Results for Genome Binners
• Good performance in genome reconstruction for unique strains
• Subspecies diversity is a challenge!
Seite 17 |
Results for Taxonomic Binners
• Good performance until family rank and substantial decrease below
• Small predicted bins are oftentimes false positives
Seite 18 |
MC - All data
MC - 1% of dataset (smallest
predicted bins) removed
Results for Taxonomic Profilers
• Performance decreases substantially below family rank
• Virus and plasmid data affect abundance estimates
Seite 19 |
Presence/absence of taxa
Recall (Completeness)
Precision (Purity)
Abundance estimates
L1-Norm & weighted Unifrac
Main Conclusions
• Good assembly and genome binning of „single strain“ species
• Strain-level diversity is a challenge
• Taxonomic profiling and taxon binning good until familylevel
• Reproducibility is very important, large variability ofprogram performances with parameter settings
What is next?
Benchmark programs on CAMI challenge datasets with CAMI
benchmarking platform (www.data.cami-challenge.org)
2nd CAMI challenge
• Illumina/PacBio/ONP data sets
• specific environments
• strain madness
• workflows
Get in touch: [email protected];contact@cami-
challenge.org
Seite 22 |
Page 23 |
Participants of CAMI Workshop at INI in Cambridge May 2016
Contributors 1st CAMI Challenge
P. Hofmann, P. Belmann, D. Koslicki, T. Woyke, N.
Shapiro, S. Janssen, M. Barton, P. D. Blood, S.
Majda, J. Dröge, I. Gregor, J. Fiedler, E. Dahms, R.
Garrido-Oter, A. Bremges, A. Fritz, M. Z. DeMaere,
C. Quince, T. Sparholt Jørgensen, L. Hestbjerg
Hansen, S. J. Sørensen, Y. Bai, D. Turaev, M.
Beckstette, M. Balvociute, F. Meyer, N. Nagarajan,
B. Chia, B. Denis, J. L. Froula, Z. Wang, R. Egan, D.
Don Kang, J. Cook, R. Chikhi, C. Deltel, C.
Lemaitre, P. Peterlongo, G. Rizk, D. Lavenier, Y.-W.
Wu, S. Singer, C. Jain, M. Strous, H. Klingenberg,
P. Meinicke, T. Lingner, H.-H. Lin, Y.-C. Liao, G.
Gueiros Z. Silva, D. A. Cuevas, R. A. Edwards, S.
Saha, V. C. Piro, B. Y. Renard, M. Pop, H.-P. Klenk,
M. Goeker, M. Balvociute, N. Kyrpides, J. Vorholt, P.
Schulze-Lefert, E. M. Rubin, A. E. Darling, T. Rattei,
A. Sczyrba
46 institutions
Why CAMI?
Tool development for shotgun metagenome data sets is a very active area: Assembly, (tax.) binning, taxonomic profiling
Method papers present evaluations using many different metrics, simulated data sets (snapshots) and are difficult to compare
It is unclear to everyone which tools are most suitable for a particular task and for particular data sets
Comparative benchmarking requires extensive resources and there are pitfalls
Towards a comprehensive, independent and unbiasedevaluation of computational metagenome analysis methods
First CAMI challenge
Benchmark assembly, (tax.) binning and taxonomic profiling software
Extensive, high-quality benchmark data sets from unpublished data
Publication with participants and data contributors
Aims
Overview of tools and use cases
Standards
Facilitate future benchmarking
Indicate promising directions for tool development
Suggestions for experimental design
Contest opened in early 2015
IKey principles
All design decisions (data sets, evaluation measures and principles)
should involve the community
Data sets should be as realistic as possible
Evaluation measures should be informative to developers and
understandable also by applied community
Reproducibility (data generation, tools, evaluation)
Participants should not see any of the data before
Google+ community
Community Involvement
ISME Roundtable, Hackathons &
workshops
Announcements in blogs & tweets
www.cami-challenge.org with
newsletter
Page 31 |
Timeline first CAMI Challenge
Reproducibility and Standardization
Barton et al., Gigascience 2015
Standard formats
for binning and
profiling
Standard interfaces
for tool execution
Bioboxes (docker
containers) for tools
and metrics
Currently 25 tools in
bioboxes – semi-
automatic
benchmarking in
future challenges
Challenge Data sets – Design principles
As realistic as possible, challenging
Common experimental setups and community types
Unpublished data
Strain-level variation
Different taxonomic distances to sequenced genomes (deep
branchers included)
State-of-the-art sequencing technologies
Non-bacterial sequences
Seite 34 |
WP1
CAMI Datasets
CAMI_low
1 sample
15 GBp
2 x150 bp
Insert size:
270 bp
CAMI_medium
(differential
abundance)
2 samples
40 GBp
2 x150 bp
Insert sizes:
270 bp & 5kbp
CAMI_high
(time series)
5 samples
75 GBp
2 x150 bp
Insert size:
270 bp
Datasets simulated from ~700 unpublished microbial genomes and
additional sequence material
1st Challenge Timeline
March 27, 2015
• Challenge opens
• Read data sets posted
• Assembly contest starts
May 20, 2015
• Assembly challenge closes
• Assembly gold standard posted
• Binning & profiling contest starts
July 18, 2015
• Challenge closes
• Evaluation starts
8 weeks
8 weeks
CAMI Evaluation Metrics
Basic Statistics:
• Number of contigs
• Number of large contigs (i.e. > 1000 bp)
• Largest contig length
• Total assembly length
• N50:
The length for which the collection of all contigs of that
length or longer covers at least half an assembly (50%)
Assembly Evaluation Metrics
Reference-based statistics:
• Reference length
• Reference GC %
• Number of chromosomes
• Number of genes/operons
• NGx, LGx
Assembly Evaluation Metrics
Alignment Statistics
Alignment Statistics
• Genome fraction %
Alignment Statistics
• Genome fraction %
• Duplication ratio
Alignment Statistics
• Genome fraction %
• Duplication ratio
• Number of gaps
Alignment Statistics
• Genome fraction %
• Duplication ratio
• Number of gaps
• Largest alignment length
Alignment Statistics
• Genome fraction %
• Duplication ratio
• Number of gaps
• Largest alignment length
• Number of unaligned contigs (full & partial)
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●
●●●
●●●●●●
●●
●●
●●●
●
●
●
●●
●
●
●
0
200
400
600
1030
719
1036
623
1036
800
1053
028
1048
936
1036
002
1036
563
1035
999
1035
996
1052
944
1036
584
1035
900
1035
993
1048
582
1036
554
1035
975
1048
855
1036
701
1048
771
1036
011
1053
022
1035
969
1035
978
1036
764
1036
569
1036
578
1035
963
1035
954
1036
686
1036
695
1030
890
1035
903
1030
824
1035
948
1053
019
1035
945
1035
906
1035
984
1035
921
1035
915
1036
587
1048
960
1030
827
1036
542
1036
572
1036
755
1036
617
1036
680
1036
650
1030
794
1030
872
1048
885
1049
053
1036
632
1036
659
1053
013
1036
770
1030
722
1048
918
1052
953
1048
780
1053
007
1030
881
1048
774
1030
803
1036
773
1030
776
1048
981
1036
548
1036
761
1048
903
1048
924
1048
873
1052
965
1035
924
1036
596
1036
698
1036
794
1036
719
1053
010
1053
016
1048
777
1036
671
1048
957
1036
683
1030
821
1048
999
1030
731
1036
665
1036
713
1049
026
1052
995
1036
743
1030
836
1048
933
1030
788
1036
674
1036
656
1048
984
1036
008
1030
845
1030
860
1030
857
1036
752
1030
770
1030
734
1048
954
1036
593
1036
722
1036
005
1030
725
1048
867
1035
990
1048
657
1030
956
1030
878
1052
992
1030
791
1010
920
1053
040
1036
734
1052
959
1053
025
1035
897
1049
032
1030
755
1036
599
1048
894
1035
987
1048
870
1036
560
1048
861
1052
986
1035
960
1035
936
1030
800
1030
947
1030
806
1048
864
1049
023
1035
930
1048
906
1036
662
1049
071
1053
058
1036
620
1048
888
1053
055
1036
749
1036
629
1048
783
1030
929
1030
818
1048
942
1048
768
1049
083
1030
728
1048
765
1048
876
1030
752
1048
945
1049
041
1030
830
1030
905
1049
086
1030
917
1030
761
1053
031
1030
743
1036
638
1036
539
1030
797
1035
909
1035
951
1030
764
1030
896
1030
809
1036
605
1030
839
1030
710
1048
909
1052
989
1036
608
1052
980
1052
947
1030
815
1030
746
1030
812
1048
882
1030
785
1049
080
1030
707
1052
983
1030
704
1052
950
1048
912
1053
052
1036
611
1048
879
1035
933
1049
014
1053
046
1049
011
1030
833
1030
968
1048
576
1052
956
1048
915
1052
968
1036
785
1036
575
1030
875
1049
008
1035
939
1053
004
1049
005
1030
713
1053
049
1035
912
1030
935
1030
869
1035
981
1049
002
1030
749
1048
948
1036
704
1049
038
1036
668
1030
893
1048
963
1036
581
1036
803
1049
029
1053
043
1048
762
1036
644
1036
746
1052
941
1049
068
1036
635
1036
677
1049
077
1030
965
1049
017
1036
707
1048
891
1030
758
1052
998
1048
930
1036
590
1053
034
1048
900
1052
962
1052
977
1049
089
1030
740
1036
641
1036
566
1036
557
1048
951
1049
062
1052
974
1036
767
1036
731
1036
689
1030
782
1036
692
1030
767
1048
939
1053
037
1048
786
1036
602
1049
065
1048
978
1036
710
1036
779
1030
911
1049
035
1048
987
1049
059
1030
737
1048
927
1036
776
1036
614
1036
782
1049
047
1048
972
1049
211
1048
966
1030
773
1048
969
1036
797
1036
647
1048
993
1036
758
1048
990
1049
056
1036
737
1030
716
1049
074
1049
044
1048
975
1036
716
1036
728
ID
tcov
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
● ●●●
●
●●● ●
●
●
●
●
●
●●
●
●●●
●●
●
●●
●
●
●
●
●●●●●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●●●
●
●
●
●●●●●
●
●●●●●
●
●● ●●
●
●●●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●●●
●●●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●●
●
●●
●
●
●
●
●●●●●●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●● ●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●●●●●
●
●
●
●●
●●
●●●●●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●●●
●
●●
●●
●
●●
●
● ●●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●
●● ●
●
●●
●
●●
●
●●
●
● ●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●●
●
●
●
●
●
●●
●●●●● ●
●
●●
●
●●
●●●
●
●●●
●
●●●
●
●
●
●●●●
●
●
●●
●
●
●●
●● ●●●
●
●
●●●
●
●●●
●
●
●●●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●●●
●
●●
●
●●
●
●●●●
●
●
●
●
● ●●●
●●●●●
●
●
●
● ●●●
●●●
●
●
●●
●
●●
●
●●●●
●
●
●●●
●
●●
●
●●
●
●
●
●●
●
●●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●●
● ●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●
●
●
●●● ●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●●●
●
●
●●●●
●
●● ●●●
●
●
●●●
●
●●
●●
●
●
●
●
●● ●●
●●●● ●●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●● ●●
●
●●●●
●
●●
●
●
●
●
●
●
● ●●●
●
●
●●●●
●●
●
●
●
●●●
●
●●●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●●
●
●
●
● ●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●●●
●●●
●
●●
● ●●
●
●●
●●●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●●●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
0
2500
5000
7500
10000
1030
719
1036
623
1036
800
1053
028
1048
936
1036
002
1036
563
1035
999
1035
996
1052
944
1036
584
1035
900
1035
993
1048
582
1036
554
1035
975
1048
855
1036
701
1048
771
1036
011
1053
022
1035
969
1035
978
1036
764
1036
569
1036
578
1035
963
1035
954
1036
686
1036
695
1030
890
1035
903
1030
824
1035
948
1053
019
1035
945
1035
906
1035
984
1035
921
1035
915
1036
587
1048
960
1030
827
1036
542
1036
572
1036
755
1036
617
1036
680
1036
650
1030
794
1030
872
1048
885
1049
053
1036
632
1036
659
1053
013
1036
770
1030
722
1048
918
1052
953
1048
780
1053
007
1030
881
1048
774
1030
803
1036
773
1030
776
1048
981
1036
548
1036
761
1048
903
1048
924
1048
873
1052
965
1035
924
1036
596
1036
698
1036
794
1036
719
1053
010
1053
016
1048
777
1036
671
1048
957
1036
683
1030
821
1048
999
1030
731
1036
665
1036
713
1049
026
1052
995
1036
743
1030
836
1048
933
1030
788
1036
674
1036
656
1048
984
1036
008
1030
845
1030
860
1030
857
1036
752
1030
770
1030
734
1048
954
1036
593
1036
722
1036
005
1030
725
1048
867
1035
990
1048
657
1030
956
1030
878
1052
992
1030
791
1010
920
1053
040
1036
734
1052
959
1053
025
1035
897
1049
032
1030
755
1036
599
1048
894
1035
987
1048
870
1036
560
1048
861
1052
986
1035
960
1035
936
1030
800
1030
947
1030
806
1048
864
1049
023
1035
930
1048
906
1036
662
1049
071
1053
058
1036
620
1048
888
1053
055
1036
749
1036
629
1048
783
1030
929
1030
818
1048
942
1048
768
1049
083
1030
728
1048
765
1048
876
1030
752
1048
945
1049
041
1030
830
1030
905
1049
086
1030
917
1030
761
1053
031
1030
743
1036
638
1036
539
1030
797
1035
909
1035
951
1030
764
1030
896
1030
809
1036
605
1030
839
1030
710
1048
909
1052
989
1036
608
1052
980
1052
947
1030
815
1030
746
1030
812
1048
882
1030
785
1049
080
1030
707
1052
983
1030
704
1052
950
1048
912
1053
052
1036
611
1048
879
1035
933
1049
014
1053
046
1049
011
1030
833
1030
968
1048
576
1052
956
1048
915
1052
968
1036
785
1036
575
1030
875
1049
008
1035
939
1053
004
1049
005
1030
713
1053
049
1035
912
1030
935
1030
869
1035
981
1049
002
1030
749
1048
948
1036
704
1049
038
1036
668
1030
893
1048
963
1036
581
1036
803
1049
029
1053
043
1048
762
1036
644
1036
746
1052
941
1049
068
1036
635
1036
677
1049
077
1030
965
1049
017
1036
707
1048
891
1030
758
1052
998
1048
930
1036
590
1053
034
1048
900
1052
962
1052
977
1049
089
1030
740
1036
641
1036
566
1036
557
1048
951
1049
062
1052
974
1036
767
1036
731
1036
689
1030
782
1036
692
1030
767
1048
939
1053
037
1048
786
1036
602
1049
065
1048
978
1036
710
1036
779
1030
911
1049
035
1048
987
1049
059
1030
737
1048
927
1036
776
1036
614
1036
782
1049
047
1048
972
1049
211
1048
966
1030
773
1048
969
1036
797
1036
647
1048
993
1036
758
1048
990
1049
056
1036
737
1030
716
1049
074
1049
044
1048
975
1036
716
1036
728
gID
X..
co
ntig
s
factor(Assembly)
●
●
●
●
●
●
Gold
Megahit
RayK41
RayK61
RayK81
Soap
●●●●
●
●●●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●●
●
● ●●● ●●●●
●
●●●●
●
●●●
●
●●●
●
●
●
●●
●
●
●●●●●
●
●●
●
●
●
●●●
●
●
●
●
●●●
●
●
●●●●
●
●●●●
● ●
●●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●●
●
●●●●
●
●●●●●●
●●●●●
● ●●
●
●●●
●●
●
●
●
●●●●
●
●●●●
●
●● ●●●●
●
●●●●
●
●●
●
●●●●●
●
●●●●●
●
●●
●
●
●
●●
●
●
●
●●●●
●
●●●●
●
●● ●● ●●
●
●
●
●●●
●
●
●
●●●●●
●
●●●●●
●
●●●
●
●●●●
●
●●●
●
●
●●
●
●
●
●●●
●●
●
●
●
●●●●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●●●
●●
●
●
●
●●●●
●
●
●●
●
●
●
●●●●
●
●●●●
●
●
●●
●
●
●
●●●●●
●
●●●●
●
●● ●● ●● ●●
●
●
●
●
●●
●
●
●
●●●● ●●●
●●●●
●
●●●
●
●
●
●●●●
●
●
●●
●
●
●
●●●●●● ●●●●●
●
●● ●●●●
●
●●●●●● ●●●
●
●●
●
●
●
●● ●●●
●
●●●
●
●●●●●● ●●●
●
●●●
●
●● ●●●●●
●
●● ●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●●
●●●●●●
●● ●●
●
●●
●
●
●
●●
●
●●
●
●
●●●
●● ●●●●●
●
●●
●
●
●
●●●
●
●●
●
●●●●
●
●
●
●
●●
●
●●
●
●
●
●●●●●
●
●●●●●
●●●●●●●
●● ●●●●●
●
●●●● ●●●●●
●●
●
●●
●
●
●
●●●
●
●
●
●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●●●● ●●●
●
●●● ●●●●
●
●●●●●
●
●
●●
●
●
●
●●
●
●●
●●
●●
●
●
●
●●● ●●●
●
●●●
●●
●
●●
●
●●
●
●●●
●●● ●●●
●
●●●●●
●
●●
●
●
●
●●●●●
●
●
●
●
●●
●
●●●
●
●
●●
●●
●
●●
●
●●●
●
●●●
●●
●
●●
●
●●
●
●●●
●
●●●●●●
●●●●
●
●●●●●
●
●●●●
●
●●●●
●
●●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●● ●●
● ●●●●●●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●● ●●●
●
●
●●
●
●
●
●●●●
●
●●●●
●
●● ●●
●
●●
●
●●●
●●●●
●●●
●
●●
●
●● ●●●●
●
●●●●
●
●●●●
●
●●●●●
●
●●●
●
●●●●
●
●●●
●
●
●
●●●●
●
●●● ●●●●
●
●
●●
●
●
●
●●●●●
●
●
●●●●
●
●●●
●
●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●● ●●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●●●●
●
●●●●●
●
●
●●
●
●
●
●●
●
●●
●
●●●●
●
●●●●●
●●
●
●●
●
●
●
●●●●●
●
●●
●●●
●
●
●●
●●
●
●●
●
●●
●●●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●●
●
●
●
●●●●●
●
●●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●●●●
●
●●●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●●●
●
●●●●●
●
●●
●●●
●
●●●●●
●
●●●●●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●●
●●
●●●
●
●●
●●●
●
●
●
●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●
●●●●
●
●●●●
●
●●●●●
●
●●●●●
●
●●●●●
●●●●
●
●●●●●
●●●●
●
●●●●●●●●●● ●●●●
●
●●●●● ●●●●●
●
●●●●●●
●●●●●●
●●●
●
●
●●
●
●
●
●●●●
●
●
●●
●●
●
●●●●●
●
●●●●●
●
●●●●●
●
0
500000
1000000
1500000
2000000
1030
719
1036
623
1036
800
1053
028
1048
936
1036
002
1036
563
1035
999
1035
996
1052
944
1036
584
1035
900
1035
993
1048
582
1036
554
1035
975
1048
855
1036
701
1048
771
1036
011
1053
022
1035
969
1035
978
1036
764
1036
569
1036
578
1035
963
1035
954
1036
686
1036
695
1030
890
1035
903
1030
824
1035
948
1053
019
1035
945
1035
906
1035
984
1035
921
1035
915
1036
587
1048
960
1030
827
1036
542
1036
572
1036
755
1036
617
1036
680
1036
650
1030
794
1030
872
1048
885
1049
053
1036
632
1036
659
1053
013
1036
770
1030
722
1048
918
1052
953
1048
780
1053
007
1030
881
1048
774
1030
803
1036
773
1030
776
1048
981
1036
548
1036
761
1048
903
1048
924
1048
873
1052
965
1035
924
1036
596
1036
698
1036
794
1036
719
1053
010
1053
016
1048
777
1036
671
1048
957
1036
683
1030
821
1048
999
1030
731
1036
665
1036
713
1049
026
1052
995
1036
743
1030
836
1048
933
1030
788
1036
674
1036
656
1048
984
1036
008
1030
845
1030
860
1030
857
1036
752
1030
770
1030
734
1048
954
1036
593
1036
722
1036
005
1030
725
1048
867
1035
990
1048
657
1030
956
1030
878
1052
992
1030
791
1010
920
1053
040
1036
734
1052
959
1053
025
1035
897
1049
032
1030
755
1036
599
1048
894
1035
987
1048
870
1036
560
1048
861
1052
986
1035
960
1035
936
1030
800
1030
947
1030
806
1048
864
1049
023
1035
930
1048
906
1036
662
1049
071
1053
058
1036
620
1048
888
1053
055
1036
749
1036
629
1048
783
1030
929
1030
818
1048
942
1048
768
1049
083
1030
728
1048
765
1048
876
1030
752
1048
945
1049
041
1030
830
1030
905
1049
086
1030
917
1030
761
1053
031
1030
743
1036
638
1036
539
1030
797
1035
909
1035
951
1030
764
1030
896
1030
809
1036
605
1030
839
1030
710
1048
909
1052
989
1036
608
1052
980
1052
947
1030
815
1030
746
1030
812
1048
882
1030
785
1049
080
1030
707
1052
983
1030
704
1052
950
1048
912
1053
052
1036
611
1048
879
1035
933
1049
014
1053
046
1049
011
1030
833
1030
968
1048
576
1052
956
1048
915
1052
968
1036
785
1036
575
1030
875
1049
008
1035
939
1053
004
1049
005
1030
713
1053
049
1035
912
1030
935
1030
869
1035
981
1049
002
1030
749
1048
948
1036
704
1049
038
1036
668
1030
893
1048
963
1036
581
1036
803
1049
029
1053
043
1048
762
1036
644
1036
746
1052
941
1049
068
1036
635
1036
677
1049
077
1030
965
1049
017
1036
707
1048
891
1030
758
1052
998
1048
930
1036
590
1053
034
1048
900
1052
962
1052
977
1049
089
1030
740
1036
641
1036
566
1036
557
1048
951
1049
062
1052
974
1036
767
1036
731
1036
689
1030
782
1036
692
1030
767
1048
939
1053
037
1048
786
1036
602
1049
065
1048
978
1036
710
1036
779
1030
911
1049
035
1048
987
1049
059
1030
737
1048
927
1036
776
1036
614
1036
782
1049
047
1048
972
1049
211
1048
966
1030
773
1048
969
1036
797
1036
647
1048
993
1036
758
1048
990
1049
056
1036
737
1030
716
1049
074
1049
044
1048
975
1036
716
1036
728
gID
N5
0
factor(Assembly)
●
●
●
●
●
●
Gold
Megahit
RayK41
RayK61
RayK81
Soap
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●●● ●
●
● ●●
●
●●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●●●● ●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●● ●●
●
●●
●
●
●
●
●●
●●●
●
●●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●●
●
●●●●
●
●●●
●
●●●●●
●
●●
●
●
●
●●
●●
●●●
●
●●●
●
●
●
●●●●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●●●●
●
●●
●
●●
●
●●
●
●
●●●
●
●
●
●
●●●●●●● ●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●●●●●
●
●
●●●
●
●●
●●
●
●●
●
●
●
●
● ●●
●
●●●●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●
●
●●●
●●
●●●
●
●
●
●
●
●●●●● ●●
●
●●
●
●
●●●●●
●●●●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●
●
●●●●●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●●●●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●●
●
● ●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●●● ●
●
●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●● ●●● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●
●
●●●●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●● ●●●
●
●●●●●●●●
●●●
●
●● ●
●
● ●●●●●●
●
●●
●
●●●
●
●●●
●
●
●
●●
●
●
●●●●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●●
●
●●
●●
●
●●●●
●
●● ●●●●●●●●●●●●
●●● ●●
●●●●●●●●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●●0
20
40
60
1030
719
1036
623
1036
800
1053
028
1048
936
1036
002
1036
563
1035
999
1035
996
1052
944
1036
584
1035
900
1035
993
1048
582
1036
554
1035
975
1048
855
1036
701
1048
771
1036
011
1053
022
1035
969
1035
978
1036
764
1036
569
1036
578
1035
963
1035
954
1036
686
1036
695
1030
890
1035
903
1030
824
1035
948
1053
019
1035
945
1035
906
1035
984
1035
921
1035
915
1036
587
1048
960
1030
827
1036
542
1036
572
1036
755
1036
617
1036
680
1036
650
1030
794
1030
872
1048
885
1049
053
1036
632
1036
659
1053
013
1036
770
1030
722
1048
918
1052
953
1048
780
1053
007
1030
881
1048
774
1030
803
1036
773
1030
776
1048
981
1036
548
1036
761
1048
903
1048
924
1048
873
1052
965
1035
924
1036
596
1036
698
1036
794
1036
719
1053
010
1053
016
1048
777
1036
671
1048
957
1036
683
1030
821
1048
999
1030
731
1036
665
1036
713
1049
026
1052
995
1036
743
1030
836
1048
933
1030
788
1036
674
1036
656
1048
984
1036
008
1030
845
1030
860
1030
857
1036
752
1030
770
1030
734
1048
954
1036
593
1036
722
1036
005
1030
725
1048
867
1035
990
1048
657
1030
956
1030
878
1052
992
1030
791
1010
920
1053
040
1036
734
1052
959
1053
025
1035
897
1049
032
1030
755
1036
599
1048
894
1035
987
1048
870
1036
560
1048
861
1052
986
1035
960
1035
936
1030
800
1030
947
1030
806
1048
864
1049
023
1035
930
1048
906
1036
662
1049
071
1053
058
1036
620
1048
888
1053
055
1036
749
1036
629
1048
783
1030
929
1030
818
1048
942
1048
768
1049
083
1030
728
1048
765
1048
876
1030
752
1048
945
1049
041
1030
830
1030
905
1049
086
1030
917
1030
761
1053
031
1030
743
1036
638
1036
539
1030
797
1035
909
1035
951
1030
764
1030
896
1030
809
1036
605
1030
839
1030
710
1048
909
1052
989
1036
608
1052
980
1052
947
1030
815
1030
746
1030
812
1048
882
1030
785
1049
080
1030
707
1052
983
1030
704
1052
950
1048
912
1053
052
1036
611
1048
879
1035
933
1049
014
1053
046
1049
011
1030
833
1030
968
1048
576
1052
956
1048
915
1052
968
1036
785
1036
575
1030
875
1049
008
1035
939
1053
004
1049
005
1030
713
1053
049
1035
912
1030
935
1030
869
1035
981
1049
002
1030
749
1048
948
1036
704
1049
038
1036
668
1030
893
1048
963
1036
581
1036
803
1049
029
1053
043
1048
762
1036
644
1036
746
1052
941
1049
068
1036
635
1036
677
1049
077
1030
965
1049
017
1036
707
1048
891
1030
758
1052
998
1048
930
1036
590
1053
034
1048
900
1052
962
1052
977
1049
089
1030
740
1036
641
1036
566
1036
557
1048
951
1049
062
1052
974
1036
767
1036
731
1036
689
1030
782
1036
692
1030
767
1048
939
1053
037
1048
786
1036
602
1049
065
1048
978
1036
710
1036
779
1030
911
1049
035
1048
987
1049
059
1030
737
1048
927
1036
776
1036
614
1036
782
1049
047
1048
972
1049
211
1048
966
1030
773
1048
969
1036
797
1036
647
1048
993
1036
758
1048
990
1049
056
1036
737
1030
716
1049
074
1049
044
1048
975
1036
716
1036
728
gID
X..
mis
asse
mblie
s
factor(Assembly)
●
●
●
●
●
●
Gold
Megahit
RayK41
RayK61
RayK81
Soap
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●●
●●●●●
●●●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●● ●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
● ●●
●
●●●
●●
●●●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●●●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●●
●
●●●
●
●
●
●
●●
●●●●●●
●●●●●●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●●●●●●
●●
●
●●●
●
●●
●●
●
●●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●
●
●●●●●
●
●
●
●
●●●●
●●●●
●
●●
●
●
●
●
●
●●●●●
●●●
●●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●●●
●
●
●
●●
●●●
●
●●●●●
●●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●●●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●●●●
●
●
●●
●
●
●
●●●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
0e+00
5e+06
1e+07
1030
719
1036
623
1036
800
1053
028
1048
936
1036
002
1036
563
1035
999
1035
996
1052
944
1036
584
1035
900
1035
993
1048
582
1036
554
1035
975
1048
855
1036
701
1048
771
1036
011
1053
022
1035
969
1035
978
1036
764
1036
569
1036
578
1035
963
1035
954
1036
686
1036
695
1030
890
1035
903
1030
824
1035
948
1053
019
1035
945
1035
906
1035
984
1035
921
1035
915
1036
587
1048
960
1030
827
1036
542
1036
572
1036
755
1036
617
1036
680
1036
650
1030
794
1030
872
1048
885
1049
053
1036
632
1036
659
1053
013
1036
770
1030
722
1048
918
1052
953
1048
780
1053
007
1030
881
1048
774
1030
803
1036
773
1030
776
1048
981
1036
548
1036
761
1048
903
1048
924
1048
873
1052
965
1035
924
1036
596
1036
698
1036
794
1036
719
1053
010
1053
016
1048
777
1036
671
1048
957
1036
683
1030
821
1048
999
1030
731
1036
665
1036
713
1049
026
1052
995
1036
743
1030
836
1048
933
1030
788
1036
674
1036
656
1048
984
1036
008
1030
845
1030
860
1030
857
1036
752
1030
770
1030
734
1048
954
1036
593
1036
722
1036
005
1030
725
1048
867
1035
990
1048
657
1030
956
1030
878
1052
992
1030
791
1010
920
1053
040
1036
734
1052
959
1053
025
1035
897
1049
032
1030
755
1036
599
1048
894
1035
987
1048
870
1036
560
1048
861
1052
986
1035
960
1035
936
1030
800
1030
947
1030
806
1048
864
1049
023
1035
930
1048
906
1036
662
1049
071
1053
058
1036
620
1048
888
1053
055
1036
749
1036
629
1048
783
1030
929
1030
818
1048
942
1048
768
1049
083
1030
728
1048
765
1048
876
1030
752
1048
945
1049
041
1030
830
1030
905
1049
086
1030
917
1030
761
1053
031
1030
743
1036
638
1036
539
1030
797
1035
909
1035
951
1030
764
1030
896
1030
809
1036
605
1030
839
1030
710
1048
909
1052
989
1036
608
1052
980
1052
947
1030
815
1030
746
1030
812
1048
882
1030
785
1049
080
1030
707
1052
983
1030
704
1052
950
1048
912
1053
052
1036
611
1048
879
1035
933
1049
014
1053
046
1049
011
1030
833
1030
968
1048
576
1052
956
1048
915
1052
968
1036
785
1036
575
1030
875
1049
008
1035
939
1053
004
1049
005
1030
713
1053
049
1035
912
1030
935
1030
869
1035
981
1049
002
1030
749
1048
948
1036
704
1049
038
1036
668
1030
893
1048
963
1036
581
1036
803
1049
029
1053
043
1048
762
1036
644
1036
746
1052
941
1049
068
1036
635
1036
677
1049
077
1030
965
1049
017
1036
707
1048
891
1030
758
1052
998
1048
930
1036
590
1053
034
1048
900
1052
962
1052
977
1049
089
1030
740
1036
641
1036
566
1036
557
1048
951
1049
062
1052
974
1036
767
1036
731
1036
689
1030
782
1036
692
1030
767
1048
939
1053
037
1048
786
1036
602
1049
065
1048
978
1036
710
1036
779
1030
911
1049
035
1048
987
1049
059
1030
737
1048
927
1036
776
1036
614
1036
782
1049
047
1048
972
1049
211
1048
966
1030
773
1048
969
1036
797
1036
647
1048
993
1036
758
1048
990
1049
056
1036
737
1030
716
1049
074
1049
044
1048
975
1036
716
1036
728
gID
To
tal.le
ng
th
factor(Assembly)
●
●
●
●
●
●
Gold
Megahit
RayK41
RayK61
RayK81
Soap
Overlap graph vs de Bruijn graph for assembly.
Schatz M C et al. Genome Res. 2010;20:1165-1173
Copyright © 2010 by Cold Spring Harbor Laboratory Press
R1: GACCTACA
R2: ACCTACAA
R3: CCTACAAG
R4: CTACAAGT
A: TACAAGTT
B: ACAAGTTA
C: CAAGTTAG
X: TACAAGTC
Y: ACAAGTCC
Z: CAAGTCCG
k=3
ovlp > 5
Effect of kmer size: 51-mer
https://github.com/rrwick/Bandage/wiki/Effect-of-kmer-size
Salmonella genome
100 bp Illumina reads
Bandage
Effect of kmer size: 61-mer
https://github.com/rrwick/Bandage/wiki/Effect-of-kmer-size
Salmonella genome
100 bp Illumina reads
Bandage
Effect of kmer size: 71-mer
https://github.com/rrwick/Bandage/wiki/Effect-of-kmer-size
Salmonella genome
100 bp Illumina reads
Bandage
Effect of kmer size: 81-mer
https://github.com/rrwick/Bandage/wiki/Effect-of-kmer-size
Salmonella genome
100 bp Illumina reads
Bandage
Effect of kmer size: 91-mer
https://github.com/rrwick/Bandage/wiki/Effect-of-kmer-size
Salmonella genome
100 bp Illumina reads
Bandage
Alexander Sczyrba
Peter Belmann
Andreas Bremges
Liren Huang
Alice McHardy
Johannes Dröge, Ivan Gregor
Peter Hofmann, Jessika Fiedler
Stephan Majda, Eik Dahms,
Stefan Jannsen Tanja Woyke
Michael Barton
Nikos Kyrpides
Eddy Rubin
Paul Schulze-Lefert, Yang Bai
Ruben Garrido-Oter
Hans-Peter Klenk
Phil Blood
Mihai Pop, Chris Hill
Aaron Darling
Matthew DeMaere
Thomas Rattei
Dmitri Turaev
Julia Vorholt
Michael Barton
Tue Sparholt
Jørgensen, Lars
Hestbjerg Hansen
Søren J Sørensen
David Koslicki
Isaak Newton Institute for Mathematical Sciences
