g-nome surfer: a tabletop interface for collaborative...

G-nome Surfer: a Tabletop Interface for CollaborativeExploration of Genomic Data

Orit Shaer1, Guy Kol2, Megan Strait1, Chloe Fan3, Catherine Grevet4, and Sarah Elfenbein1

1Wellesley College, 106 Central St, Wellesley, MA, USA, 024822Babson College, 231 Forest Street, Babson Park, MA, USA, 02457

3HCII, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, USA, 152134GVU Center, Georgia Institute of Technology, Atlanta, GA, USA, 30332

ABSTRACTMolecular and computational biologists develop new insightsby gathering heterogeneous data from genomic databasesand leveraging bioinformatics tools. Through a qualitativestudy with 17 participants, we found that molecular and com-putational biologists experience difficulties interpreting, com-paring, annotating, sharing, and relating this vast amount ofbiological information. We further observed that such in-teractions are critical for forming new scientific hypothe-ses. These observations motivated the creation of G-nomeSurfer, a tabletop interface for collaborative exploration ofgenomic data that implements multi-touch and tangible in-teraction techniques. G-nome Surfer was developed in closecollaboration with domain scientists and is aimed at lower-ing the threshold for using bioinformatics tools. A first-usestudy with 16 participants found that G-nome Surfer enablesusers to gain biological insights that are based on multipleforms of evidence with minimal overhead.

Author KeywordsReality-Based Interaction, Tabletop interaction, Bioinformat-ics, Genome Browser

ACM Classification KeywordsH.5.2 Information Interfaces and Presentation: User Inter-faces

General TermsDesign, Human Factors

INTRODUCTIONOver the past two decades, Human-Computer Interaction(HCI) research has generated a broad range of interactionstyles that move beyond the desktop into new physical andsocial contexts. Key areas of innovation in this respect are

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.CHI 2010, April 10 – 15, 2010, Atlanta, Georgia, USACopyright 2010 ACM 978-1-60558-929-9/10/04...$10.00.

tangible, tabletop, and embodied user interfaces. These in-teraction styles share an important commonality: leverag-ing users’ existing knowledge and skills of interaction withthe real non-digital world such as naıve physics, spatial, so-cial and motor skills [11]. Drawing upon users’ pre-existingknowledge and skills of interaction with the real non-digitalworld, these interaction styles are often unified under theumbrella of Reality-based Interfaces (RBIs) [11]. By basinginteraction on pre-existing real world knowledge and skills,RBIs offer a more natural, intuitive, and accessible form ofinteraction that reduces the mental effort required to learnand operate a computational system and supports high-levelcognition [11].

While these advances in Human-Computer Interaction havebeen applied to a broad range of application domains in-cluding problem-solving, education, and entertainment, lit-tle HCI research has been devoted to investigating RBI in thecontext of professional scientific research. However, it is im-portant to study RBI in this context where reducing the men-tal workload associated with accessing information and sup-porting collaborative high-level reasoning could potentiallylead to new scientific discoveries. Those RBIs that examinedthe possibilities of supporting scientific discovery in fieldssuch as molecular biology [8], chemistry [4], and geophysics[6], highlight the potential contribution of RBIs to support-ing scientific discovery, but focus on the representation andmanipulation of information that has an inherent physical orspatial structure such as proteins, molecules, and maps. Weare interested in a broader use case, investigating whetherreality-based interaction techniques can enhance scientificdiscovery in areas where vast amount of abstract informationis accessed and manipulated. Examples include molecularenergy levels, aggregated physiological data, and genomicinformation.

Advances in genomic technologies have led to an explosivegrowth in the quantity and quality of biological informationavailable to the scientific community. The ability to simulta-neously collect detailed information about the structure andactivity of multiple genes has fundamentally changed theway molecular biology research is conducted. Rather thanfocusing on small scale, lab-based experiments, researchersoften conduct large scale experiments in which informationfrom multiple genes is simultaneously measured, recorded,

CHI 2010: Bang a Table April 10–15, 2010, Atlanta, GA, USA

1427

and stored in a database. The need to analyze such large andcomplex data sets has driven a change in the tools used in bi-ological research: next to having a pipette and a pen, a webbrowser is currently the most widespread tool available forbiologists as it provides access to powerful computationaland statistical tools [21]. Web technologies have been mas-sively adopted by molecular biology software developers toallow easier access for biologists who are not computer ex-perts. However, existing web-based genomic tools show se-vere limitations in terms of persistence, usability, and sup-port of high-level reasoning [3, 15, 23].

Through a study of molecular and computational biologistswe observed that to develop insights, biologists gather a wealthof heterogeneous data from genomic databases and lever-age a diverse set of bioinformatics tools. However, theyexperience difficulties interpreting, comparing, annotating,sharing, and relating biological information. We further ob-served that these manipulations are critical for developingdeep insights and forming hypotheses. These observationsmotivated the creation of G-nome Surfer, a tabletop interfacethat provides collaborative and fluid interaction with hetero-geneous genomic data. G-nome Surfer supports searching,comparing, annotating, and relating large amounts of ge-nomic information.

The rest of the paper is organized as follows: the next sec-tion summarizes our study of molecular and computationalbiologists. Following that, we present the two other contri-butions of this paper. The first is G-nome Surfer, a tabletopinterface for collaborative exploration of genomic informa-tion that was designed based on findings from the observa-tional study. The second contribution is a first-use study ofthis system and the lessons we learned. The study demon-strates that G-nome Surfer enables users to gain biologicalinsights that are based on the multiple forms of evidence itprovides, with minimal overhead.

SEMI-STRUCTURED INTERVIEWS WITH BIOLOGISTSTo understand the current work practices of genomic researchers,we conducted a series of interviews with 17 molecular andcomputational biologists (eight female, nine male) from lead-ing genomic research institutions, industry, and an under-graduate research institution. The title and research areaof each participant are listed in table 1. Most interviewstook place at the researcher’s primary work place (exceptthree that took place in our HCI laboratory). The inter-views were semi-structured and lasted 45-60 minutes. Dur-ing the interviews, we asked participants to educate us abouttheir research goals, their work practices, and the computa-tional tools they use. We asked each participant to walk usthrough a particular instance of research work. We collecteddata by audio-taping the interviews, taking pictures of part-cipants’ workspace, collecting relevant work samples, andsaving screen captures of participants’ computers as theywere demonstrating how they perform various tasks. Thesecond author of this paper, a bioinformatician with experi-ence building commercial genomic analysis tools, aided ourneed-finding efforts and directed us towards issues most crit-ical for genomic researchers. Two additional authors have

background in biology. We analyzed this data by identifyingcommon high-level tasks and themes. We then distilled de-sign implications for genomic exploration tools. Following,we describe our findings.

ID Title Research AreaP1 Faculty Developmental GeneticsP2 Faculty Evolutionary BiologyP3 Faculty Animal PhysiologyP4 MSc Student MetagenomicsP5 Faculty Cell Biology and GeneticsP6 Industry Researcher Cancer Drug TherapyP7 Postdoc Viral InfectionsP8 Faculty NeuroscienceP9 Faculty Proteins StructureP10 Industry Researcher Next Generation SequencingP11 Research Assistant Evolutionary BiologyP12 Postdoc Genetic NetworksP13 Student Researcher Animal PhysiologyP14 Student Researcher Protein StructureP15 Faculty Bacterial SystemsP16 Faculty Proteins StructureP17 Postdoc Computational Genetics

Table 1. The background of study participants. All participants wereinterviewed individually except P3 and P13, and P9 and P14 who wereinterviewed in pairs.

Information Tasks and Bioinformatics ToolsWe found that although our subjects are interested in find-ing answers to a diverse set of biological problems, they usesimilar techniques for accessing and analyzing genomic in-formation. Specifically, we identified five basic informationtasks that are commonly performed by our subjects:

• literature searching: presents what is already known abouta gene, a condition, or a biological function.

• locating a gene on a genome: verifies the structure of agene and its relationship with neighboring genes.

• retrieving a genomic sequence: provides the base-pair oramino-acid sequence of a gene.

• searching for similarity between sequences: highlightsthe similarity between the researched sequence and othersequences.

• annotating genomic information: adds finding and con-clusions so that the researcher or other team members canfurther explore and query the information.

DatabasesWhile some of our subjects maintain their own database forstoring and managing molecular data and experiment results(mainly for confidentiality reasons), there exists a large num-ber of public online databases for depositing and import-ing genomic data. Public databases are used by all of oursubjects on a frequent basis. Such databases range fromthose that contain genomic information of a specific organ-ism to large databases that contain DNA sequence data formany different organisms. Also useful in designing exper-iments and forming hypotheses are literature databases (i.e.PubMed) and ontology databases (e.g. GeneOntology). TheNational Center for Biotechnology Information (NCBI) web


1428

site is typically a starting point for researchers looking forresources. All of these databases provide access through aweb browser but implement different methods and tools forstoring and retrieving information.

Genome BrowserA genome browser [5] is an online tool that visualizes thespatial relationships between different pieces of genomic data.In genomics, because spatial relationships often indicate func-tional relationships, a genome browser aims to help usersform hypotheses about the function of different genomic el-ements. A genome browser displays several collections ofdata (i.e. tracks) that are aligned in respect to the genomicsequence. Often, the most important tracks are those thatindicate genes, but tracks could also contain other informa-tion. Users can pan left and right on a genomic sequence aswell as zoom in and out. For example, a user may requestto view TP53, a human gene that is known to be related tocancer. A genome browser then represents the gene as a rect-angle along a chromosome. The coordinates of the rectanglerepresent the location of the gene on the chromosome. Theuser may also pan left and right through the chromosome toview other genes in the region. Upon zooming, a gene isrepresented as a series of rectangles and arrows. Rectanglesrepresent exons (areas of the gene that code for proteins),arrows represent introns (areas of a gene that are not trans-lated to proteins), the direction of the arrows represents thedirection in which the sequence of a gene is read (becausethe DNA is double-stranded, some genes are located on theleading strand and are read from left to right while others arelocated on the lagging strand and are read from right to left).Figure 1, shows a screen capture of the widely used UCSCGenome Browser. The center of the screen displays the genetrack which shows the structure of the TP53 gene in termsof exons and introns. The top part of the screen shows a pic-ture of the chromosome and highlights the area on the chro-mosome that is currently displayed. The bottom part of thescreen displays the expression track, which uses color cod-ing to convey the expression level of the TP53 gene in differ-ent tissues. Most current web-based genome browsers (e.g.

Figure 1. The UCSC Genome Browser. In the center, the gene trackshows the structure of the TP53 gene. At the top, a figure of the chromo-some highlights the viewed area. At the bottom, the expression tracksuse color coding to convey the expression level of the TP53 in differenttissues.

UCSC [22], Ensemble [7]) are implemented using HTML,allowing users to navigate genomic sequences using form-based controls. Thus, when users navigate through a regionon the chromosome, they proceed through a series of static

pages. These discontinuous page transitions often impairusers’ sense of location and context, leaving them wonder-ing how the displayed data points are related [21]. Emergingweb-based genome browsers (e.g. [1, 21]) attempt to ad-dress this problem by using Ajax technology to implementcontinuous zooming and panning. Also, a genome browseroften serves as a gateway for a myriad of other web-basedsources of information. Our subjects often access the litera-ture database PubMed through a genome browser as well assearch for ontology information. However, current genomebrowsers do not provide means for organizing and relatingmultiple forms of evidence. Thus, our subjects often developad-hoc techniques for relating the information they collect:three of our subjects maintain their own databases in whichthey link related publications to sequence information, twoother subjects create visual networks of related genes wherethey annotate the connection between two genes, while oth-ers print out text files and publications and annotate themmanually.

BLASTThe Basic Local Alignment Search Tool (BLAST) finds re-gions of local similarity between genomic sequences. TheBLAST tool is often accessed through a web browser. Itcompares a nucleotide (DNA or RNA) or protein (aminoacid) sequence to sequences in different databases and cal-culates the statistical significance of the matches. All thematches that are above a certain threshold are displayed. Re-searchers use BLAST for a wide variety of tasks. For exam-ple, a researcher that plans to test a new cancer drug thattargets the TP53 gene may use BLAST to compare the se-quence of the human TP53 with the mouse and rat genomesin order to determine which of the animals is a better candi-date for drug testing (one-to-many search). Or, a researcherthat designs drugs that target particular areas on a gene mayuse BLAST to verify that this area has minimal similarity toother genes on the human genome so that the drug will onlyaffect that particular area (one-to-one search). Finally, oneof our subjects uses BLAST to find the extent to which thebacteria population in the human gut is genetically similar tothe bacteria population on the human skin. To do so he usesthe BLAST algorithm to compare all the genomic sequencesextracted from a sample of the gut with all the genomic se-quences extracted from a sample of the skin (many-to-manysearch). Existing web based BLAST tools present the resultsof one-to-one and one-to-many searches visually so that thematching sequences are aligned with the reference sequencesorted according to a similarity score (see Figure 2). Fol-lowing the visual presentation are the details of the match-ing sequences (the matching score, location on the genome,and length). It is important to note that a BLAST searchcan return hundreds (or in some cases thousands) of results.Some of our subjects find the visual display of BLAST re-sults confusing, while others ignore the visual display andonly review the results in a text format.

Even though other bioinformatics tools are available for re-searchers, the above tools are fundamental to accessing andanalyzing genomic sequences.


1429

Figure 2. A visual display of similarity search results (by the NCBIBLAST). The human TP53 gene is represented by a red rectangle; be-low this rectangle two short sections of the rat genome that are similarto the human TP53 gene are represented by small red rectangles thatare aligned with the the human TP53 gene.

Data ExplosionIn the last decade, the cost per reaction of DNA sequenc-ing has fallen with a Moore‘s Law precision [14]. In 2005,a person could sequence his whole genome for the cost of350k, four years later several companies offer to sequencehuman genome for 60k, and soon the cost mark of 5k or even1k will be reached. Although subsistence DNA sequencingis currently conducted at a single investigator, departmental,or university facility setting, high-throughput DNA sequenc-ing is currently only performed in a handful of sites. How-ever, the recent introduction of next-generation sequencingtechnology, capable of producing millions of DNA sequencereads in a single run, is rapidly changing the landscape ofgenomics. In the near future, such sequencing instrumentswill become available for more researchers, allowing a sin-gle lab to create in one year the same amount of data thatwas held in all the NIH sequence databases just 3 years ago.In the context of HCI, these advances present the challengeof providing researchers with means for sharing, searching,comparing, connecting, and organizing this vast amount ofdata.

One of our subjects (P4) is involved in the Human Micro-biome Project that seeks to sequence the human genome to-gether with the genome of the entire human micro-bacterialpopulation. Every experiment that he conducts results inhundreds of millions of short genomic sequences that to-gether describe the several thousands of bacteria species thatpopulate the human body. The challenge he faces is howto organize these complex data sets into a form that will al-low him to learn about the similarities and differences in thegenomes of those micro organisms. In several cases, we ob-served that the ability to visualize large data sets is extremelyhelpful for biologists and can help them to understand andgeneralize complex phenomena they were not capable of un-derstanding before the visualization existed.

Integrated WorkflowTo gain insight into complex biological systems, our sub-jects often link together several data sets, each being han-dled with a special bioinformatics tool. We observed thatsubjects with background in biology often create a linearworkflow by manually fetching data from one spot, refor-matting the data, applying the next bioinformatic tool, pars-ing the results, reformatting the results, and so on. Often notcomfortable with programming, they only rarely automatea workflow (typically by asking for help from a bioinfor-

matician) and instead repeat required steps as necessary. Asgenomic sets grow larger, this method of operation becomesmore and more time consuming. Thus, there is a clear needfor providing means for easily linking both data and tools tocreate a workflow that can be repeated across experiments.In the words of one of our subjects:

We searched for NOS3 in the Zebra fish genome, andwhen we got the DNA sequence we picked the largestexon. Taking the sequence for that exon, we pasted itinto NCBI BLAST. Since it returned the NOS3 gene,we proceeded to the IDT site to input our sequenceand get suggestions for possible primers for this exon.(P13)

Subjects with background in computational biology typicallywrite scripts to execute a linear or parallel workflow. Oursubjects work with shell scripts, Perl, Python, Matlab, andC. Some scripts are designed for single-use, other scripts aredesigned to be used in multiple experiments and sometimesby other users.

Multiple Forms of EvidenceBiological data is often noisy due to the complexity of livingsystems and the imperfection of measurement technologies.As one of our subjects describes:

Sequencing DNA is not black and white, and thereforeit is not so simple that the sequence we receive is 100percent correct. Depending on the quality of the se-quence, we need to adjust the acceptable level of agree-ments. (P2)

To address this uncertainty, researchers often combine mul-tiple forms of evidence, as well as examine evidence frommultiple resources. For example, because genes are detectedthrough experimental evidence, no data indicates ‘the genes’unambiguously [5]. Thus researchers often combine evi-dence from multiple gene data sources and view it in paral-lel. Our subjects often combine multiple forms of evidencenot only to overcome uncertainty but also to discover con-nections and casual relationships. For example, some of oursubjects often compare the genomes of multiple organismsto infer evolutionary relationships. To do so, they displaythe genome from each organism in a single genome browsertrack (i.e. row). This sometimes results in displaying morethan a dozen tracks in parallel. Once inferring relationshipsbetween multiple genomes, users are often interested in ag-gregating the information and presenting their findings in asingle track (i.e. row). Some subjects often examine infor-mation in different levels of granularity - moving back andforth between viewing a large chromosome area containingmultiple genes and the base-pair level showing only a smallarea of a single gene:

Since we don’t know the exact location sometimes, weneed to zoom in to the base-pair level, and zoom out tothe level of viewing 10 or so genes to see their context.(P15)


1430

We found that existing tools often overwhelm users with theamount of data presented on the screen, making it difficultfor the user to organize the information in a way that high-lights the connections between multiple forms of evidence.This observation is also supported in the literature [3, 15].We thereby identify a clear need to support the display ofmultiple forms of evidence while allowing users to comparepieces of evidence, highlight the similarities and differences,and aggregate comparison results.

From Novice to ExpertOur subjects differ in their level of expertise both in termsof domain knowledge and of computer experience. Studentresearchers typically have 2-4 years of experience in theirfield of study, while researchers and faculty typically havemore than 10 years of experience in their field. Although allof our subjects are comfortable using bioinformatics applica-tions that have a GUI, subjects with background in molecularbiology typically use only limited functionality of bioinfor-matics tools and are often cautious of trying new features.Computational biologists on the other hand, are trained incomputer science and are expert users. When needed, theydevelop new computational tools to solve a particular bi-ological problem. We observed that bioinformatics toolsin general and genome browsers in particular have a highthreshold: they are powerful, but in order for one to reallytake advantage of their power, they require both a broad do-main knowledge and an extensive training. To support awider range of researchers and empower users to examinenew forms of evidence, there is a need for tools with lowerthreshold that encourage exploration of advanced features.

CollaborationIn leading research institutions and in industry, biologicalresearch is often conducted in multidisciplinary groups thatare made of biologists, computational biologists and bioin-formaticians. We learned from our subjects that in suchteams biologists often come up with a biological question,and computational biologists translate the question into aprocess in which data is collected and analyzed. Communi-cation in such teams is often the key to success. Our subjectsindicated that collaborative work is typically based on emailsand research meetings during which one of the researcherspresents slides or distributes hard copy of results. In addi-tion, researchers store their results in a shared database sothat other group members can access the information. Insmaller labs that consist of faculty and student researchers,several researchers often work together on the same com-puter exploring, analyzing, and discussing biological data.We found that despite the importance of collaboration in bio-logical research, bioinformatics tools do not provide supportfor collaborative exploration.

THE G-NOME SURFER SYSTEMOur user study helped us define a set of user-driven designgoals for a visual computing tool for genomic exploration.These goals include: 1) facilitating collaborative, immediateand fluid interaction with large amounts of heterogeneousgenomic information, 2) lowering the threshold for usingadvanced bioinformatics tools, 3) reducing mental workload

associated with accessing and manipulating genomic infor-mation, and 4) improving current information workflow pro-cesses in genomic research. Based on existing research oftabletop user interfaces indicating that tabletop interfacessupport collaboration through visibility of actions and egali-tarian input [10], facilitate active reading [16], as well as af-ford distributed cognition (that could potentially lower men-tal workload) [17], we decided to utilize tabletop interactionin our system design. We thereby created G-nome Surfer,a tabletop interface for genomic research with the intentionof meeting our user-driven goals. G-nome Surfer supportsthe five information tasks identified in the study: searchingliterature, locating a gene on a genome, retrieving genomicsequences, searching for similarity between sequences, andannotating genomic information.

Seven of our study participants participated in the iterativedevelopment process of G-nome Surfer by providing feed-back to a series of prototypes in increasing fidelity. Theirfeedback informed the current design of G-nome Surfer thatis implemented on top of the Microsoft Surface platform.Following, we describe G-nome Surfer‘s primary interactiontechniques and implementation.

Navigating Genomic MapsG-nome Surfer supports the navigation of genomic maps inmultiple zooming levels. To access a genomic map, a userfirst selects a clade, an organism, and an assembly, and thenspecifies a chromosome, a range, or a name of a particulargene. The gene is displayed in the center of a chromosometrack. The default zooming level captures approximately fiveadjacent genes. An indicator that is shaped as a box belowthe chromosome track shows what portion of the chromo-some is displayed. Each gene is represented as an arrowupon one of the two DNA strands. The direction of the ar-row represents the direction in which a gene is read. Thecoordinates of a gene represent its location upon the chromo-some in terms of base-pairs (see Figure 3). Users can thenpan right and left through the chromosome simply by usingflick gestures. Continuous visual feedback maintains users‘sense of location. When the user taps on a gene, its struc-ture in terms of exons and introns is displayed in a structuretrack below the chromosome. A polygon connects the geneand its structure track to support the user‘s sense of location.Exons are represented as rectangles upon the structure track.Introns are represented by the empty space between exons(see Figure 3). To display the DNA, RNA, or amino acidsequence of a gene, the user taps on the structure track. Ifthe user taps on an exon, she can choose either to displaythe sequence of the entire gene or only the sequence of thatparticular exon. The sequence is then displayed in a separatewindow that is connected to its source gene. Users can dis-play multiple sequences of the same gene (e.g. a user maydisplay the RNA and amino acid sequences of a gene), eachsequence opens in a new window that is connected to itssource exon. Users can move, orient, resize, and arrange thewindows as well as annotate genomic sequences. To com-pare two sequences, a user can overlay two windows andalign the sequences. Figure 3 shows the aligned RNA andamino acid sequences of TP53.


1431

Figure 3. A genomic map of TP53. A chromosome track (top) showsTP53 chromosomal environment. A structure track (center) displaysits structure in terms of exons and introns. Aligned RNA and aminoacid sequences of TP53 are displayed at the bottom.

Heterogeneous Information Upon RequestG-nome Surfer enables researchers to access and relate het-erogeneous information. When holding a finger upon a gene,the gene star-shaped context menu appears, allowing the userto choose between ontology, publications, and gene expres-sion information. Ontology displays a summary of the pub-licly known information about the gene from the Entrez Genedatabase. Publications displays the titles of publications re-lated to this gene from the PubMed database. When tappingon a publication, the abstract (with a link to the full paper)opens in a separate window. Gene expression presents ex-pression levels in different tissues. While the gene expres-sion information is taken from the UCSC Genome Browser[22], we created a new visualization that uses the acceptedcolor coding scheme (red for high expression, green for lowexpression) but displays the information in hierarchical struc-ture upon an organism‘s body. Figure 4 shows the gene con-text menu and the different pieces of information that relateto the human gene TP53. Each piece of information is dis-played in a separate window so that users can move, orient,resize, and spatially arrange the information upon the sur-face.

To allow users to save all the information related to a par-ticular gene, we decided to use a tangible test tube container(see figure 5) that is associated with a particular gene. Whenplaced upon the surface, the tube attracts all the pieces ofevidence related to that particular gene, then when removed,that information disappears. When placed upon the surfaceagain the information reappears on the surface. We selectedto represent storage with a physical tube because it is a fa-miliar object - tubes are often used as a portable storage ob-ject in the lab. In the future, we plan to allow users to placemultiple tubes on the surface to explore genes interaction,as well as to move information between surfaces. We be-lieve that the immediacy and persistence of tangible interac-

tion will enhance these tasks. A recent paper by Kirk et al.[13] informed our design by highlighting design considera-tions and tradeoffs of choosing between physical and digitalrepresentations. Currently, we pre-assigned a limited num-ber of tubes to represent a pre-defined set of human genes.However, we are developing a mechanism for dynamicallycoupling tube objects to genes.

Figure 4. The gene context menu and heterogeneous information re-lated to TP53 (structure, publications list, expression, ontology, andvarious papers).

Figure 5. A tangible test tube container stores all the heterogeneousinformation related to a particular gene associated with the tube.

Similarity SearchAfter a particular sequence is displayed on the surface, a usercan perform a BLAST search to find regions of local similar-ity between that sequence and the genomes of other organ-isms. To do so, the user places a tangible BLAST tool uponthe sequence (see Figure 6). A semi-transparent layer thencovers the surface (i.e. BLAST layer) and a BLAST contextmenu is presented. We chose to represent the BLAST toolusing a playful tangible object to make this mode change im-mediate, visible, and easily reversed. After the user selectsfrom the menu one or more organisms, the BLAST searchis invoked and search results are displayed. Because such


1432

search can yield numerous results, and considering the limi-tations of current representations of BLAST results, we cre-ated a new visualization for BLAST results. Figure 6 showsthe BLAST results of TP53 against the mouse, rat, and mon-key genomes. Each Blast result (i.e. an alignment to a ge-nomic region with a similarity score that crosses a certainthreshold) is represented as a rectangle in a shade of green.The color of the rectangle represents the similarity score ofthat result - the brighter the color the higher the similarityscore. Each rectangle contains further details about that par-ticular BLAST result including the target organism, the ex-act similarity score, the length of that genomic region, andits location upon the genome. The results are organized ina flower-like structure around a target organism so that tar-get organisms with more results are displayed closer to thebottom of the surface.

In order to align a BLAST result with the source gene, a usertaps a result. The result is then automatically aligned withthe source gene. The alignment is presented at the bottom ofthe BLAST later. A user can align multiple results with thesource gene. Finally, upon locating a set of BLAST resultsof interest, a user can save those results to the main G-nomeSurfer layer. When the user removes the tangible BLASTtool, the BLAST layer disappears and the selected resultsappear on the main layer.

Figure 6. Interacting with BLAST results of the human TP53 geneagainst the mouse, rat, and monkey genomes. A physical BLASTtool (shaped as a spacecraft) presents a star-shaped search menu whenplaced upon the surface. Search results are organized in a flower-likestructure around a target organism, with bright results represent highsimilarity score.

Reality-Based InteractionThe design of G-nome Surfer draws on users’ existing knowl-edge and skills to provide a tabletop reality-based interface[11]. Specifically, G-nome Surfer uses naıve physics metaphorssuch as inertia, transparency, and mass in the layout of chro-mosomes and genes and in the representation of BLAST re-sults. The interface also leverage users’ spatial skills, al-lowing them to spatially organize information upon the sur-face to express relationships between multiple forms of ev-

idence. Like tabletop interfaces in general [10], G-nomeSurfer draws upon users’ social skills and existing socialprotocols to afford collaborative interaction: the system pro-vides multiple points of entry (through multiple forms of ev-idence that can be simultaneously manipulated), and makesmodes visible to all users through the use of visual and phys-ical physical objects (e.g. the BLAST tool).

ImplementationG-nome Surfer is written in C# using the Microsoft Sur-face SDK. It uses web services to draw genomic informa-tion from various databases including UCSC, Pub Med, andEntrez Gene. The BLAST search is implemented using theWashington University BLAST (WU-BLAST) web service.In order to improve the performance of G-nome Surfer, weare currently working on implementing a local database andBLAST search. Tangible objects are tagged with the Mi-crosoft Surface fiducial tags.

ExtensibilityG-nome Surfer was designed with extensibility in mind. Forexample, additional forms of evidence (e.g. protein struc-ture and gene family information) could be added throughweb services, the star-shaped context menus could be ex-tended using hierarchical organization to support additionalfunctionality, and the layer metaphor that is used for display-ing BLAST results could be reused to support additional ser-vices. We are currently extending the system to support thedisplay of multiple genomes in parallel (e.g. human next toother organisms), and the display of isoforms (i.e. multipleRNA sequence sections from the same gene). In addition,we are currently developing a gene interaction layer that willsupport the investigation of gene interaction.

SYSTEM EVALUATIONTo evaluate the usability of our design, we conducted a first-use study of G-nome Surfer. In particular, we were inter-ested in validating that: 1) G-nome Surfer enables users togain biological insights based on the multiple forms of evi-dence it provides, with minimal overhead, 2) G-nome Surferpresents BLAST results in a way that facilitates rapid identi-fication of relevant results, and 3) G-nome Surfer supportssmooth transition between the chromosome level and thebase-pair/protein levels.

To define an insight, we draw on Saraiya et al. [19] that viewinsight as “an individual observation about the data by theparticipant, a unit of discovery” (p. 444). They group bioin-formatics insights into four categories: overview (overalldistribution), patterns (identification or comparison acrossdata attributes), groups (identification of comparison of groupsof entities), and details (focused information about a specificentity). In our study, we asked subjects to find answers tobiological questions that require surface biological insightsfrom all four categories. Surface insights are based on in-formation findings and typically answer “what” questions.We define overhead as training time combined with the timeusers spend on tasks that are not directly related to biologicalresearch such as reformatting data, importing and exportingfiles, and sorting data.


1433

Sessions were held at the HCI lab. In each 40-minutes ses-sion we asked two participants to work together to completea task. The 16 participants (all female) included undergradu-ate biology students with background in genomics and someresearch experience. Some participants had limited previ-ous experience with bioinformatics tools. We chose to testthe system with students because they represent an impor-tant user population (i.e. student researchers) and were mostavailable for first-use testing. Following a five minutes intro-duction to the Microsoft Surface and to the basic function-ality of G-nome Surfer, we asked the participants to worktogether to complete a task on their own: provide evidencethat the human gene TP53 is a candidate for the developmentof cancer gene therapy treatment and select a model organ-ism for testing the treatment. We modeled this task to mimica real scenario that we observed in our background study. Toaccomplish this task, subjects needed to complete four sub-tasks that include: gene location, evidence search, sequenceretrieval, and similarity search. We asked our subjects tofollow steps and record answers to biological questions thatreflect surface insights as they proceed toward accomplish-ing the task. Following the task, we asked the participants tocomplete a questionnaire with their opinion on the task andG-nome Surfer. We also conducted an informal debriefingwith participants and examined their answers to the biolog-ical questions. We observed the participants and took notesduring the session as well as videotaped each of the sessions.

ResultsFollowing a brief training, all participants were able to com-plete the task (and found correct answers to the biologi-cal questions) within a reasonable time. Table 2, showsmean times for completion, and mean confidence score (ona 7-points scale where 7 is most confident) for the overalltask and for each of the four subtasks. In general, partici-pants reported that the overall task was moderately difficult(mean score of 5.3, where 7 is most difficult), that they werefairly confident in their overall findings (mean score of 5.8,where 7 is most confident), and that the task had a relativelylow mental workload (mean score of 2.5, where 7 is highestworkload).

Description Mean SD ConfidenceLocating a gene and examining 6.2 2.3 6.4its struture and environmentGathering heterogeneous information 5 0.7 5.9(expression, publications, and ontology)Retrieving and exmaining genomic 3.3 1.2 6.1sequencesConducting and interperting 3 0.5 6.2similarity searchOverall task- investigation of TP53 17.5 4 5.8

Table 2. Time for completion (in minutes) of the study sub tasks.

Identifying BLAST ResultsAll users successfully conducted a similarity search (BLAST).The times for completion of the BLAST task (see Table 2)indicate that users were able to rapidly identify relevant BLASTresults with high confidence level. In the debrief interviewparticipants mentioned that the “Blaster as an external tool

was great” and that “it is easy to compare different organ-isms”. However, some participants reported that due to thecolor contrast we used in the BLAST visualization, it washard to read the details of BLAST results with low scores.Because some biologists are interested in further investigat-ing results with low similarity scores, we plan to address thisissue promptly.

Transition Between Zooming LevelsThe subtask of locating a gene and examining its structureand environment required users to transition from the chro-mosome level to the base-pair level and back several times.All users successfully completed this task with high confi-dence level (see Table 2). This includes correctly answer-ing biological questions about the location, structure, and thechromosomal environment of the TP53 gene. In the debriefinterview, participants mentioned that transitions are “easyto follow” and that “it is easy to overlay amino acid / RNAsequences”.

CollaborationAs expected, participants collaborated during the task. Weasked each participant to what extent she and her partnerwere working together, and responses had a mean score of5.9 where 7 is ‘we highly collaborated when exploring theinformation’. We further observed that in all teams, bothparticipants were actively interacting with the information.Some teams collaborated in a turn-taking manner, while oth-ers worked in parallel. We also observed that all teams werefrequently gesturing and actively discussing the task, results,and interaction techniques aloud as advised by the study in-structions.

SummaryOverall, participants were excited about working with G-nome Surfer, with a mean enjoyment score of 6.3 where 7is most enjoyable. They described the experience as “fun”,“hands on”, “easy to use, smooth, good response time” and“visually stimulating”. We allot some of this excitementto the novelty of the Microsoft Surface. More specifically,participants liked the integration of information, flexibilityof moving and resizing screens, and searching for specificgenes and linking them to pertinent articles. During the de-brief interview, one participant described G-nome Surfer as“designed for multiple users but simple enough that one usercan also accomplish the task”. Another participant men-tioned that G-nome Surfer “helped visualize gene locationand synthesized a lot of information in one place”.

Unfortunately, at the time of the study our mechanism fordeleting information from the surface was not stable, so mul-tiple participants reported that their surface became too clut-tered with information that they wished to remove. Partic-ipants also suggested to add more text-based information(e.g. adding coordinates on the structure track, exon length)that will increase confidence level in results, as well as ad-ditional functionality for searching within publications (e.g.for keywords), sorting and annotating publications. We planto address these issues as we expand G-nome Surfer.


1434

DiscussionThis study was intended as a first-use evaluation of our de-sign. Its results show that G-nome Surfer supports the fiveinformation tasks identified by our user study and sets a rel-atively low threshold for using bioinformatics tools. Thisstudy also demonstrates that G-nome Surfer provides a col-laborative, immediate and fluid interaction with heteroge-neous genomic information. However, an additional inves-tigation is required to evaluate whether G-nome Surfer re-duces the mental workload associated with accessing andmanipulating genomic information in comparison to currentGUI tools. Also, to distinguish the strengths and limitationsof G-nome Surfer in comparison to a collaborative GUI, acomparative study is required: specifically, we are interestedin separating the limitations of current tools from limitationsthat are inherent to conventional GUIs by identifying whichof the G-nome Surfer’s advantages are products of its visualdesign and integrated workflow, and which are products ofa horizontal tabletop interface. To truly assess whether G-nome Surfer improves current workflow processes, a longi-tudinal study within a research lab is required. Finally, it isimportant to note that all of our subjects in this study werefemale. This eliminated a confounding variable of gender,but does not represent most research settings of mixed gen-der. In the future, we will look at male and mixed gendergroups.

RELATED WORKEvaluation of Bioinformatics ToolsSeveral studies indicate that current web-based bioinformat-ics tools show severe limitations in supporting users to findanswers to complex biological questions [23, 3, 15, 12].To better understand the requirement for supporting deepcausal insights, Mirel performed a longitudinal field studyof biomedical researchers [15]. Her findings indicate thatusing the tested tool, scientists could successfully pose andanswer “what” questions that are based on information find-ings, but could not easily develop explanatory insights thatrequire answering “how” and “why” questions. Her study re-sults further suggest that design choices of such tools shouldsupport contextualizing relationships, highlighting biologi-cally meaningful concepts, and flexible interactivity. Thesefindings informed our design.

Reality-Based Interfaces for ScientistsA number of systems illustrate the vast possibilities of sup-porting scientific discovery through reality-based interaction.Brooks et al. [4] developed the first haptic display for sci-entific visualization. This display improved the perceptionand understanding of force fields and was used by chemiststo investigate docking positions for drugs. Gillet et al. [8]presented a tangible user interface for molecular biology thatused augmented reality technology to augment 3D molecularmodels. While users collaboratively manipulate the physicalobject, the system superimposes graphical information uponthe physical model. The system was only preliminarily eval-uated with professional researchers. Schkolne et al. [20]developed an immersive interface for the design of DNAmolecules that uses tangible objects to create and edit dig-ital representations of DNA molecules. A user study with

scientists finds this system to be more satisfying for usersthan a corresponding 2D system. While these systems fo-cus on the representation and manipulation of objects thathave an inherent physical structure. We are interested in abroader use case, where abstract information is representedand manipulated. Labscape [2], is a smart environment forthe cell biology laboratory that helps biologists to producemore complete records of their work. The system allowsbiologists to easily record, relate, and share heterogeneousinformation about their lab work. ButterflyNet [25], is a mo-bile capture and access system for field biologists that inte-grates paper notes with digital photographs captured duringfield research. While addressing different needs and require-ments, both systems share our challenge of organizing andrelating heterogeneous information.

To date, a few systems have been developed to facilitate col-laboration among scientists across large displays and multi-touch tables. WeSpace [24], integrates a large data wall witha multi-user multi-touch table and personal laptops to pro-vide group members equal access to touch manipulation aswell as live rendering and interactive visualization. Team-Tag [18] is a tabletop interface that allows biodiversity re-searchers to collaboratively search, label, and browse digitalphotos. Finally, Involv [9] is a tabletop application that usesthe Voronoi treemap algorithm to create an interactive visu-alization for the Encyclopedia of Life. Involv is the closestto our work as it shares the challenge of creating effectivetabletop interaction for exploring massive data spaces.

CONCLUSIONS AND FUTURE WORKThis paper makes three contributions. First, we described astudy of molecular and computational biologists that iden-tifes design requirements for supporting scientists searchingthrough and organizing vast amounts of heterogeneous in-formation in order to gain biological insights. Second, wedescribed G-nome Surfer, a tabletop user interface for col-laborative exploration of genomic data that employs multi-touch and tangible interaction techniques. G-nome Surferenables users to gain biological insights based on the mul-tiple forms of evidence it provides with minimal overhead.It also facilitates rapid identification of BLAST results, andsupports smooth transition between the chromosome leveland the base-pairs level. As a result, it lowers the thresholdfor using bioinformatics tools and encourages exploration ofmultiple forms of evidence. Finally, we presented resultsfrom a first-use usability study of G-nome Surfer.

We plan to expand G-nome Surfer so it will be suitable foruse in scientific and educational (college level) settings. Thisincludes improving G-nome Surfer’s performance throughlocal database and search, as well as supporting the displayof multiple genomes in parallel (e.g. human next to otherorganisms), and the display of isoforms (i.e. multiple formsof the same gene). We intend to further evaluate G-nomeSurfer’s strengths and limitations in comparison to currentstate-of-the-art GUI based tools, and to collaborative GUIwith multiple mice. We also plan to conduct a longitudinalstudy in both research and educational settings.


1435

While the domain of bioinformatics provides the frame forthis work, this research contributes to tabletop interaction ingeneral by demonstrating its application for supporting do-main experts interacting with large amount of abstract, het-erogeneous and complex information.

ACKNOWLEDGEMENTSWe would like to thank the biologists that participated in ourstudy. We also thank the Brachman Hoffman foundation forsupporting this work.

REFERENCES1. Argo Genome Browser,

http://www.broadinstitute.org/annotation/argo/.

2. Arnstein, L., Hung, C.-Y., Franza, R., Zhou, Q.H.,Borriello, G,Consolvo, S., Su, J., Labscape: A SmartEnvironment for the Cell Biology Laboratory. In IEEEPervasive Computing Magazine, 1(3). pp.13-2, 2002.

3. Bolchini, D., Finkelstein, A., Perrone, V., Nagl, S.Better bioinformatics through usability analysis,Bioinformatics, 25(3), 406-12, February 2009.

4. Brooks, F. P., Ouh-Young, M., Batter, J. J., and JeromeKilpatrick, P. Project GROPEHaptic displays forscientific visualization. In Proceedings of the 17thAnnual Conference on Computer Graphics andinteractive Techniques, Dallas, TX, USA, 1990.

5. Cline, M.S. and Kent, J.W., Understanding genomebrowsing, Nature Biotechnology, 27(2):153155.

6. Coutere, N., Riviere, G., Reuter, P., GeoTUI: ATangible User Interface for Geoscience, In Proceedingsof the Second International Conference on Tangibleand Embedded Interaction, ACM Press, 2008.

7. Ensembl, http://www.ensembl.org/index.html.

8. Gillet, A., Sanner, M., Stoffler, D., Goodsell, D., andOlson, A. 2004. Augmented Reality with TangibleAuto-Fabricated Models for Molecular BiologyApplications. In Proceedings of the Conference onVisualization ’04. IEEE Visualization, October 2004.

9. Horn, M.S., Tobiasz, M., Shen, C., VisualizingBiodiversity with Voronoi Treemaps. In Proc. SixthInternational Symposium on Voronoi Diagrams inScience and Engineering, Copenhagen, Denmark. June23-26, 2009.

10. Hornecker, E., Marshall, P., Dalton, N.S., Rogers, Y.,Collaboration and Interference: Awareness with Miceor Touch Input. Proc. of ACM CSCW Conference, 2008.

11. Jacob, R.J.K. , Girouard,A., Hirshfield, L.M., Horn,M.S., Shaer, O., Solovey, E.T., and Zigelbaum, J.,Reality-Based Interaction: A Framework forPost-WIMP Interfaces, In Proc. ACM CHI 2008Human Factors in Computing Systems Conference, pp.201-210, ACM Press, 2008.

12. Javahery, H. and Seffah, A., Refining the UsabilityEngineering Toolbox: Lessons Learned from a UserStudy on a Visualization Tool, HCI and Usability forMedicine and Health Care,185–198, 2007.

13. Kirk, D., A. Sellen, S. Taylor, N. Villar, and S. Izadi.2009. Putting the physical into the digital: Issues indesigning hybrid interactive surfaces. In HCI 2009, pp.35-44. British Computer Society.

14. Mardis, E., The impact of next-generation sequencingtechnology on genetics. Trends in Genetics, 24(3),133-141, March 2008.

15. Mirel, B. Supporting cognition in systems biologyanalysis: findings on users’ processes and designimplications, J Biomed Discov Collab. 27(2),153155,February 2009.

16. Morris, M.R., Brush, A.J.B., and Meyers, B., ReadingRevisited: Evaluating the Usability of Digital DisplaySurfaces for Active Reading Tasks. Proceedings ofIEEE Tabletop, 79-86, 2007.

17. Patten, J., and Ishii, H., A Comparison of SpatialOrganization Strategies in Graphical and Tangible UserInterfaces, in Proceedings of Designing AugmentedReality Environments , pp. 41-50, 2000.

18. Ryall, K., Forlines, C., Shen, C., Morris, M.R., Everitt,K., Experiences with and Observations of Direct-TouchTabletops. In Proc. IEEE Tabletop’06, 2006.

19. Saraiya, P. and North, C. and Duca, K., Aninsight-based methodology for evaluatingbioinformatics visualizations, Visualization andComputer Graphics, 11(4), 443-456, 2005.

20. Schkolne, S., Ishii, H., Schroder, P., Immersive designof DMA molecules with a tangible interface,Visualization, IEEE, pp. 227- 234, 2004.

21. Skinner, M.E., Uzilov, A. V., Stein, L. D., Mungall,C.J., and Holmes, I.H., JBrowse: A next-generationgenome browser, Genome Research, 19(9), 1630-1638,September 2009.

22. UCSC Genome Browser, http://genome.ucsc.edu/.

23. Veretnik, S. and Fink, J. and Bourne, P., Computationalbiology resources lack persistence and usability, PLoScomputational biology, 7(4), July 2008.

24. Wigdor, D., Jiang, H., Forlines, C., Borkin, M., Shen,C. 2009. The WeSpace: The Design, Development andDeployment of a Walk-Up and Share Multi-SurfaceVisual Collaboration System. Proceedings of CHI2009, Boston, MA.

25. Yeh, R., Liao, C., Klemmer, S., Guimbretiere, F., Lee,B., Kakaradov, B., Stamberger, J. and Paepcke, A.,ButterflyNet: a mobile capture and access system forfield biology research, in Proceedings of the SIGCHIconference on Human Factors in computingsystems,571–580, 2006.


1436

g-nome surfer: a tabletop interface for collaborative...

Documents