a subjunctive exploratory search interface to support media studies researchers

A SUBJUNCTIVE EXPLORATORY SEARCH INTERFACE TO SUPPORT MEDIA STUDIES RESEARCHERS

M. Bron1, J. van Gorp2, F. Nack1, M. de Rijke1, A. Vishneuski1, S. de Leeuw2 (15-08-2012)

(1) University of Amsterdam (2) Utrecht Universityvrijdag 10 augustus 2012

definition: media studies concerns the study of production, content and/or reception of various types of media, e.g., newspapers, films, and television programs

INTRODUCTION TO

MEDIA STUDIES RESEARCH

vrijdag 10 augustus 2012

definition: media studies concerns the study of production, content and/or reception of various types of media, e.g., newspapers, films, and television programs

INTRODUCTION TO


example: how does the representation of women smokers change in magazines and fiction shows between 1900 and 1950?


INTRODUCTION TO



how can we support media studies researchers in gathering material in electronic archives?

methodology: interviews; 30 mins; topic list

pick a recent research project?

where did you get the idea?

how did you search for data?

did you encounter any problems?

did your research questions change?

can you describe the various stages during the project?

can you write them down on index cards and order them chronologically?

ANALYSIS OF THE

RESEARCH CYCLE


ANALYSIS OF THE

RESEARCH CYCLE

exploredomain

analysisdata collection

literature study writing report


ANALYSIS OF THE

RESEARCH CYCLE

exploredomain

data collection

exploration

literaturestudy

developing the research question

determining available material

literature

broad gathering of material


ANALYSIS OF THE

RESEARCH CYCLE

exploredomain

data collection

exploration

analysis

data collection

contextualization

literaturestudy

literaturestudy

analyzing material

refining the research question

finding context

finding additional material


ANALYSIS OF THE

RESEARCH CYCLE

exploredomain

data collection

exploration

analysis

data collection

contextualization

writing report

presentation

literaturestudy

literaturestudy


ANALYSIS OF THE

RESEARCH CYCLE

exploredomain

data collection

exploration

analysis

data collection

contextualization

writing report

presentation

literaturestudy

literaturestudy

analysis of data, e.g., watching and transcribing films costs a lot of time, money, and effort.


DEVELOPMENT OF AN

INTERFACE

our example research question:

how does the representation of women smokers change in magazines and fiction shows between 1900 and 1950?

requirements of the material:

time and genre constraints, e.g., fiction shows

a specific group, e.g., women smokers

is it possible to contrast findings in the data, e.g., a change


DEVELOPMENT OF AN

INTERFACE

our example research question:

how does the representation of women smokers change in magazines and fiction shows between 1900 and 1950?

requirements of the material:

time and genre constraints, e.g., fiction shows

a specific group, e.g., women smokers

is it possible to contrast findings in the data, e.g., a change

Major problem: whether the material satisfies all these criteria is only known after everything has been analyzed!


DEVELOPMENT OF AN

INTERFACE

key question:

does the option to compare statistics of multiple results sets support media studies researchers in finding material that satisfies the contrast requirement in their research questions?


EXPERIMENTAL SETUP

EXPERIMENTAL DESIGN

Total of 36 media studies researchers and MSc. students (experience MD = 3; IQR = 0-6.5); between subject design; remote user study; data is 1.2 M tv programs with metadata from audio visual archive.

pre search task survey

a complex search task (30 mins):

Collect material relevant to develop and answer a research question on the topic of how minorities are described in the Dutch media over the last 40 years. After collection, formulate a research question.

post search task: write down the research question developed during searching

exit survey


EXPERIMENTAL SETUP

EVALUATION

subjunctive interface interaction statistics:

more query formulations

more documents bookmarked

wider variety of documents bookmarked (average pairwise similarity)

research questions formulated with the subjunctive interface:

a more refined scope

usability of the interface

usefulness for research; interesting

difficult to use; intuitive


RESULTS

INTERACTION STATISTICS

showing median and (inter quartile ranges)

for the query formulations

for the number of documents bookmarked

for wider variety of documents bookmarked indicated by the average pairwise cosine similarity

interaction baseline subjunctive

queries 3 (2-6) 5 (3-8)*

bookmarks 9 (6-22) 9 (3-13)

* significant at the alpha = 0.05 level using Wilcoxon rank sum test


bookmark variety .62 (.56-.69) .52 (.43-.63)*


ANALYSIS

INTERACTION PATTERNS

Query -> Inspect -> Query -> Inspect

is more often seen with the subjunctive interface

Query -> Inspect -> Filter -> Inspect

is seen more often with the standard exploratory search interface

Bookmark -> Paginate -> Bookmark -> Paginate

is a frequent pattern when using the baseline interface;

pagination does not occur in the top interaction patterns with the subjunctive interface


ANALYSIS

INTERACTION PATTERNS

Query -> Inspect -> Query -> Inspect

is more often seen with the subjunctive interface

Query -> Inspect -> Filter -> Inspect

is seen more often with the standard exploratory search interface

Bookmark -> Paginate -> Bookmark -> Paginate

is a frequent pattern when using the baseline interface;

pagination does not occur in the top interaction patterns with the subjunctive interface

comparison visualizations invite more queries

bookmarking form two result lists i.s.o. single long list promotes more diverse documents


RESULTS

RESEARCH QUESTIONS

Do the research questions formulated with the subjunctive interface have a more refined scope determined by a contrast or comparison?

three judges; experts in the field of media and migration assessed the research questions

showing median and (inter quartile ranges)/ 5 point Likert scale 1 is very bad; 5 is very good

judge 1judge 1 judge 2judge 2 judge 3judge 3baseline subjunctive baseline subjunctive baseline subjunctive

3 (3-4) 4 (3-4) 2 (2-4) 2 (2-3) 4 (3-4) 4 (2-4)

It is hard too judge whether the scope of research questions is more or less refined

what if we look at the research questions?


ANALYSIS

RESEARCH QUESTIONS

analysis of two research questions...

Investigation of the evolution of the term integration in news and human interest programs broadcasted between 1992 and 2012

In 1987 we observe a diminishing in the use of the term migrant worker and a rise in the use of the term immigrant. Is it possible to identify a cause in the broadcasting schedule of that time?

interface genre group topics period technicalbaseline 11 14 8 10 2

subjunctive 10 17 12 9 3


ANALYSIS

RESEARCH QUESTIONS

analysis of two research questions...

Investigation of the evolution of the term integration in news and human interest programs broadcasted between 1992 and 2012

In 1987 we observe a diminishing in the use of the term migrant worker and a rise in the use of the term immigrant. Is it possible to identify a cause in the broadcasting schedule of that time?

interface genre group topics period technicalbaseline 11 14 8 10 2

subjunctive 10 17 12 9 3

In research questions formulated with the subjunctive interface the contrast is more concrete.


RESULTS

USABILITY

usability of the interfaces

median and (inter quartile ranges)/ 5 point Likert scale 1 is absolutely not; 5 is very much


intuitive 4 (4-4) 4 (3-4)

interesting 4 (4-5) 4 (4-5)

useful 4 (4-5) 5 (4-5)

difficult to use 2 (2-2) 2 (2-4)

both interfaces are found to be (very) useful for media studies research

surprisingly: the subjunctive interface is not considered to be difficult to use


CONCLUSIONS

a subjunctive exploratory interface supports media researchers in

exploration: it invites more query formulations and more diverse documents to be gathered

formulating research questions in which a contrast is considered based on patterns in the material observed during exploration

moreover

the subjunctive interface, while more complex, is not considered to be difficult to use and is considered very useful for media studies research

the media studies research process is a dynamic and iterative process, in which it is difficult to make research questions concrete until material has been collected and analyzed


CONCLUSIONS

a subjunctive exploratory interface supports media researchers in

exploration: it invites more query formulations and more diverse documents to be gathered

formulating research questions in which a contrast is considered based on patterns in the material observed during exploration

moreover

the subjunctive interface, while more complex, is not considered to be difficult to use and is considered very useful for media studies research

Thank you; are there any questions?

the media studies research process is a dynamic and iterative process, in which it is difficult to make research questions concrete until material has been collected and analyzed


a subjunctive exploratory search interface to support media studies researchers

Technology

media studies researchvrijdag

dutch media

research questions

recent research project

material satises

study of production

nding material

material relevant