MLA 7th 1

Team Research Proposal

Team POLITIC

Political Opinions in Literature: Identifying Themes in International Compositions

Robert Cai, Matthew Carr, Adam Elrafei, Alexander Goniprow,

Adrian Hamins-Puertolas, Manpreet Khural, Andrew Li, Alexandra Winter,

Soumya Yanamandra, Dan Yang, and Kay Zhang

University of Maryland Gemstone Program

Mentor: Dr. Peter Mallios

Librarian: Timothy Hackman

and

The Maryland Institute for Technology in the Humanities

We pledge on our honor that we have not given or received any unauthorized assistance on this

assignment.

Team POLITIC

Introduction

The United States was involved in numerous international conflicts throughout the 20th

century. A prevalent theory suggests deeper public understanding of foreign cultures might have

allowed the United States to avoid several of these conflicts, including the Iran Hostage Crisis

and the Vietnam War (Li). Since the United States is a democracy, citizen perception of foreign

countries has a direct relationship with foreign policies enacted. A thorough understanding of

how the American public gathers its perceptions of foreign cultures is crucial to fully

comprehend American foreign policy and international relations. Foreign literature is one

important medium that exposes the United States to the political and cultural ideologies of other

countries (Griswold 1077). The American public reads novels by foreign authors to gain an

intimate perspective of foreign societies—views unavailable through domestic media. Readers

can also connect to other cultures because novels create emotional ties by appealing to universal

human themes (Aubry 27). At the same time, international and domestic political concerns guide

the United States’ public interest in foreign literature. For instance, it is not a coincidence that the

peaceful writings of Gandhi became important in the United States during the Civil Rights

Movement (Mallios 10-19).

        However, different foreign authors often provide opposing viewpoints of their societies.

The most popular works form a selective base of foreign literature that potentially accommodates

elites’ self-serving political biases. Using experimental methods, Gilens asserts that the United

States’ ignorance and misinformation “leads many [citizens] to hold political views different

from those they would hold otherwise” (379). Therefore, understanding public intent and attitude

requires knowing why certain novels and authors seem representative of a cultural canon. To

become a better-informed political citizen of the United States, one must think critically about

2

Team POLITIC

the uses of foreign literature.

        Our study will investigate how publicly available United States media received foreign

novels and authors and how these portrayals work toward social and political ends of

government support and criticism (Mallios 10-19). Specifically, we will conduct a low-constraint

case study of Russian literature to address the following question: Did the reception of Russian

novels and authors in the United States and United States foreign policy toward Russia reflect

each other from 1900-1923? We hypothesize that the reception of Russian literature in the

United States significantly correlates with United States policies toward Russia, due to inherent

ties between literary evaluation and political understanding. Scholars, politicians, and other

government officials will likely take interest in our study.

        We will use the portrayals of selected Russian novels and authors in nationally available

print media to define the reception of Russian literature in the United States during this time

period. We recognize scholars could investigate how alternative forms of media, such as pictures

or political cartoons, influence public understanding. However, we chose print media because it

is the easiest to quantitatively analyze. We will define United States foreign policy toward

Russia through quantifiable measures such as foreign aid, military investment, and trade deals

from 1900-1923. This will take the form of overarching topics that describe the types of policies

enacted, such as interventionism and humanitarianism. Our analysis will include keyword

searches relative to both literary reception and foreign policy. We will track how these themes

have evolved over time using techniques of topic modeling.1

        Our study does not seek to determine a relationship between political climates and

messages found in novels, opinions held by authors, or motivations behind translators. Instead,

we will determine the extent to which there is a relationship between media reception of Russian

1 See Appendix H for an example of topic modeling output.

3

Team POLITIC

literature in the United States and the political climate. Our research is distinguished from

previous studies in two ways: it analyzes reception in United States media and not the intent of

authors or translators, and we will accomplish our analysis through quantitative, not just

qualitative, methods.

Throughout the rest of our proposal, we will summarize our literature review, outline our

methodology, explain the limitations of our research, list confounding variables, and conclude

with descriptions of our anticipated results, our budget, our timeline, and the statistical tools we

will use throughout the project.

Literature Review

Introduction of Russian Literature in the Western World

Eugene-Melchoir de Vogue's Le Roman Russe (The Russian Novel) in 1886 represented

the increasing interest in Russian literature in Western Europe and America. Many writers,

including Isabel Hapgood and Constance Garnett, published English translations of Russian

novels, short stories, and poems to critical acclaim in subsequent decades (Moser 431). In other

words, the late nineteenth and early twentieth centuries marked the availability of Russian

literature to US public and intellectuals.

Many studies have sought to understand literary themes found in major Russian works.

For example, Emerson analyzes Leo Tolstoy’s views on war through a close reading of his many

texts (1855). However, only a few studies address Russian literary reception in the United States

during the early twentieth century. One of these rarities is Goldfarb’s account of how a

prominent literary critic, William Dean Howells, supported Tolstoy’s works in the United States

during the twentieth century (318). However, this study is limited in that it only contemplates

4

Team POLITIC

Russian literary reception through Howells’ and his critics’ views. We intend to expand on such

studies by using comprehensive statistical tools to analyze a wider base of reception material.

Canon Formation and Politics

Political motivations shape a nation’s literary canon, which in turn projects that nation’s

identity. The idea of a national literature emerged in the late eighteenth century as a way of

proving cultural independence on an international level (Corse, Nationalism and Literature 7-

14). Original research studies suggest canonical or high-culture literature does not reveal how

citizens perceive themselves, but rather how elites want to envision their nation (ibid 74). These

previous studies turn to college syllabi and literary prizes to define the most frequently appearing

works as canonical or high-culture (Brown, 1; Corse, Nations and Novels 1279-82). Unlike

bestsellers or popular culture novels, canonical texts differ greatly between countries, as they are

symbolic in value and not simply “economic commodities.” Theories of canon formation state

novels have to experience a conjunction of large sales and certain types of recognition to reach

canonical status (Ohmann 206). This recognition refers to the critical reception of works found in

publications that “carried special weight in forming cultural judgments,” such as the New York

Times Book Review and the New Republic (204). However, scholars have never specified the

ways in which elites have translated cross-cultural differences into literature.

Topic Modeling

Researchers use topic modeling to analyze large corpora of data. Topic modeling affirms

“documents are mixtures of topics, where a topic is a probability distribution over words”

(Steyvers 2). Furthermore, Latent Dirichlet Allocation (LDA), a more specific type of topic

modeling, asserts each document from a larger corpus consists of a plurality of topics (Chaney

and Blei 2). In past studies, researchers have used topic modeling in general and LDA

5

Team POLITIC

specifically to analyze large corpora of data. For example, a 100-topic LDA model generated

word probabilities under each topic for all articles in the journal Science between 1880 and 2002

(ibid 4).

More complex versions of topic modeling, however, can gather more information from

our Russian author database. For example, Topics over Time (TOT) models can account for the

chronology of documents in a corpus (ibid 9). Since our documents are dynamic in that they

change over time, LDA would confound the topics’ changes and lose any perceivable patterns.

Xuerui Wang and Andrew McCallum explain the topic analysis of US Presidential State-of-the-

Union addresses, where LDA “confounds Mexican-American War (1846-1848) with some

aspects of World War I (1914-1918)” since it is “unaware of the 70-year separation between the

two events” (1). Modeling topics over time serves to address this issue.

In Wang and McCallum’s study, they incorporated timestamps to help track “changes in

the occurrence of the topics themselves” as a function of time (2). They tested their model on

three data sets: “more than two centuries of U.S. Presidential State-of-the-Union addresses,” “17-

year history of the NIPS [Neural Information Processing Systems] conference,” and “nine

months of email archive” (ibid). The results of their study show the TOT model is able to predict

the timestamps of documents and generates topics that are “more distinct from each other than

LDA topics” (ibid 5). In our research, we will also use a TOT model on the databases we

anticipate constructing to account for time.

Furthermore, modified versions of LDA can relate metadata to topics. Metadata is

information about the documents we collect such as “author, title, geographic location, [and]

links” (Blei 10). Therefore, we can also correlate influences such as the gender and ethnicity of

the authors of the reception material to word probabilities found in topics in our corpus.

6

Team POLITIC

Sentiment Analysis

Sentiment analysis is also useful for sorting through large corpora of data. While topic

modeling focuses on the subject of the data in question, sentiment analysis focuses on the

opinion expressed about the subject matter of the data (Lee and Pang 1). Multiple methods can

determine the sentiment of a piece of data. Lee and Pang compared three different algorithms

used for sentiment analysis: the Naive Bayes, maximum entropy classification, and support

vector machines (ibid 3). The Naive Bayes algorithm is a simplistic algorithm. It may not hold to

high accuracy rates with complicated sets of data, but it “tends to perform surprisingly well” and

is even the ideal algorithm for use with “problem classes with highly dependent features” (ibid).

Maximum entropy classification and support vector machines are both much more sophisticated

methods. Maximum entropy classification algorithms “make no assumptions about the

relationships between features”, which will make it better than Naive Bayes with data that has

little or no dependence on similar features (ibid 4). Support vector machines differ from both of

the previous methods in that they do not focus on probability, which brings them much closer to

traditional methods used for normal topic modeling adapted to work with sentiment analysis

(ibid 4).

For our project, sentiment analysis methods will allow us to quickly categorize articles by

gauging how American periodicals perceive and discuss Russian authors and novels during the

time period of interest. In addition, incorporating a sentiment categorization into our database

will allow future researchers to quickly add to and examine our data.

Foreign Policy Analysis

Political scientists have devised several models and theories to explain how foreign

policy develops (Boyer 185). One such theory is the rational actor model, which states stimuli

7

Team POLITIC

and immediate responses lead to the creation of foreign policy (Boyer 189). However, the

political aspect of our study does not seek to determine how political leaders create foreign

policy, but rather attempts to measure and quantify it. Many previous studies have determined

United States foreign policy towards various nations by analyzing its components. For example,

Rick Travis analyzes foreign policy towards Africa by focusing on foreign aid to the continent

(798). Haslam focuses on direct foreign investment and the corresponding treaties to determine

United States foreign policy toward other nations (1182). For our study, we will gather data on

“exports, imports, investments, arms sales, and categories of foreign aid (bilateral, aggregate, and

per capita)” between the United States and the Russian Empire (and later the Soviet Union) to

define United States foreign policy (Watson 253).

Methodology

Our first tasks were to determine a time range and country to investigate, as outlined in

the literature review. We selected an upper time bound of 1923, since all preceding publications

are in the public domain and we can publicly release all collected data. We chose 1900 as our

lower time bound to guarantee a significant number of periodicals will be available.2 Time

allowing, we may be able to expand the time period of interest, guaranteeing more articles for

analysis. We decided to investigate Russian literature for several reasons. First, Russia was a

focal point of the United States during the twentieth century. World War I, the Bolshevik

Revolution, and the threat of communism led to increased public and governmental interest in

Russia during our selected time period. Second, only a relatively small number of significant

Russian authors had works available in English at the time. A narrow range of Russian literary

figures suggests American periodicals interested in examining Russian literature had to invoke

2 We anticipate finding a significant number of periodicals referencing Russian literary figures during the selected time period, as shown in Appendix E. By the beginning of our time period of interest, many national periodicals had already been well established (Baldasty).

8

Team POLITIC

certain Russian literary figures and works frequently, leading to larger sample sizes for the

selected authors. Subsequently, we will be able to construct a more exhaustive corpus3 of

Russian literature than of the more readily available literature from other countries, such as

Britain or France.

        To decide which literary figures to study, we compiled a list Russian literary figures

whose works had English translations during our time period of interest. Using that list, we

cataloged the number of search results found in the Readers’ Guide Retrospective4 for each

literary figure of interest.5 From this preliminary summary of the availability of periodicals in the

United States specifically discussing Russian literary figures, we chose to investigate Dostoevsky

and Tolstoy to maintain the feasibility of our study. We bring some bias in our selection of

literary figures, as we have chosen two of the most renowned Russian literary figures in the

United States. Therefore, our data regarding the reception of selected Russian literary figures in

the United States will not be representative of the entirety of Russian literary figures. We could

add one or two minor Russian authors to our research to increase the external validity of our

project if time permits.

        We resolved to capture a large, representative sample of the body of articles that

explicitly mention our selected Russian literary figures in periodicals popular in the United

States between 1900 and 1923. We will construct a database containing these articles using the

Readers’ Guide Retrospective index. The Retrospective’s emphasis on more popular periodicals

fits well with our intent to gain an understanding of how the general American public perceived

significant Russian literary figures in the early twentieth century. We will use a subject search of

3 See our Glossary of Terms in Appendix H4 The Readers’ Guide Retrospective is a comprehensive index of 608 popular periodicals published in the United States spanning from 1890 to 1982. 224 periodicals in the Readers’ Guide Retrospective – almost 37% of the database – are available prior to 1923. See Appendix G.5 An abridged version of this list can be found in Appendix E.

9

Team POLITIC

selected literary authors to explore the Readers’ Guide Retrospective and find articles

appropriate for the constructed database.

Scanning

Since most articles in the Readers’ Guide are not digitized, we have to digitize the

physical or microfilm versions of articles that fall within search parameters. We are currently

scanning articles by using publicly available resources at the University of Maryland McKeldin

Library. Therefore, our initial database construction will contain only articles available within

the University of Maryland archive system. Should time permit, it may be feasible to explore

other academic archives for articles from the Readers’ Guide Retrospective.

We have standardized scanning techniques to reduce preventable variations in image

quality and size.6 Systematic errors, including the presence of dust particles, stains, and other

debris on the scanning glass, also contribute to poor image quality and complicate analysis of the

database. We will therefore wipe down the scanning glass with glass cleaner solution and a

microfiber cloth before and after each scan to reduce this source of error.

Preservation of the scanned material is essential to data accuracy and reliability. During

microfilm scanning, an auto-adjust function adjusts the brightness and scanned size of each page

to produce an optimally clear image. Furthermore, we must adjust the resolution of the scanner

up from the default 300 dots per inch (DPI) to the maximum setting of 600 DPI. Similar settings

are also present on the non-print source scanners. Once saved, the file is left unmodified with the

exception of cropping. We will not manipulate images after scanning to retain the original image

data, quality, and integrity.

6 Examples of standardization in scanning articles include: uniform Scanner type, Scanner settings, and format in which material is saved. Images will be saved in the Tag Image File Format , a standard “for distributing high quality scanned images or finished photographic files” (“TIFF Files”).

10

Team POLITIC

We will convert these files to readable documents through Optical Character Recognition

software. We are using ABBYY FineReader 11 to save the files as plain text documents, DjVu

files, and FineReader documents. Topic modeling and sentiment analysis software can analyze

plain text files; the DjVu format compresses documents and maintains the layout of text on each

page; and we save FineReader files to document the transition from scanned image to readable

text. At this stage, we remove pictures from the pages.

Foreign Policy Analysis

The second portion of the project focuses on United States foreign policy toward Russia.

Our goal is to quantify the United States’ changing attitude and foreign policy towards Russian

over the established time period for the study of the authors. As mentioned earlier on, one

method of defining this relationship is to examine statistical data that relates to foreign policy

including foreign aid to Russia, trade relations, and America’s military presence in Russia. We

will also examine Presidential speeches delivered during the time period of interest; we will

simply run searches for references to Russia and transfer Presidential speeches that produce hits

into a database for future analysis.

With sufficient time, we will also collect and analyze newspaper editorials in a similar

manner. A theory discovered in preliminary research indicates that editorials of major

newspapers of the late nineteenth and early twentieth centuries, specifically The New York

Times, reflected political motivations of the United States government (“Deductions” 42;

Lippmann and Merz 3). If pursued, a newspaper editorial database provides our project with a

wider scope because it provides an additional level of comparison with other foreign policy data.

11

Team POLITIC

Annotation

As we assemble a corpus of articles regarding literary authors of interest, one priority is

to ensure we effectively organize the constructed database. We can more easily analyze an

organized corpus, making it essential for generating metadata7. Beyond ease of analysis,

metadata will give us the ability to categorize and analyze articles that deal with a specific topic

or exhibit similar traits, an approach that will yield more significant and interesting results than a

simple keyword search. The assembled corpus’s metadata will include, at a minimum, historical

and archival data concerning each article. We will also attempt to capture metadata regarding the

characteristics of each article, such as whether articles include explicit references to radical

politics, by annotating8 each article.

Annotation questions may reflect biases and stereotypes that we bring individually to the

project and it is difficult to ensure our uniformity in annotation. We determined what kind of

metadata to capture and refined annotation questions by annotating a sample of articles from the

assembled database.9 The goal of refining annotation questions is to confirm we will arrive at

similar answers if annotating independently.

In conjunction with the Maryland Institute for Technology in the Humanities (MITH), we

will attempt to automate the process by which we construct metadata, reducing time spent on this

portion of our methodology. It is feasible to automate metadata collection through computer

scripts, including collection of spelling variations in literary author names across the constructed

corpus,10 or, more abstractly, performing sentiment analysis on articles in the corpus. The end

7 According to the National Information Standards Organization, metadata is “structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource…metadata is often called data about data” (National Information Standards Organization).8 Annotation is a way to produce variables that will allow us to understand the political significance of Russian Literature in the United States and catalog the constructed corpus.9 Reference to revisions of Annotation Questions in Appendix D.10 The names of Russian authors often have a number of accepted spellings and are subject to frequent mistranslation (Pasterczyk). We will catalog alternative spellings of selected literary figures. The use of Boolean operators to

12

Team POLITIC

goal of our research project is to form conclusions about the relationship between the reception

of Russian literature in the United States and United States foreign policies. To reach these

conclusions, we will need to analyze both an annotated database of articles that pertain to

literature and an annotated database of articles that pertain to foreign policies.

In the data analysis section of the methodology, we expect to discover trends in the

databases that provide answers to certain questions. For the Russian literature database, the

questions will focus on the discourse throughout the United States surrounding the predominant

Russian authors.11

To conduct this style of data analysis, we will use a collection of data mining strategies.

Data mining refers to the process of collecting unknown properties of a database. Two basic

strategies are keyword frequencies12 and semantic parsing.13

The most important data mining analysis we plan to conduct is probabilistic topic

modeling, “a suite of algorithms that aim to discover and annotate large archives of documents

with thematic information” (Blei 2). A topic is a collection of words that all have a high

probability of being associated to one another. The basic probabilistic topic modeling is Latent

Dirichlet Allocation (LDA), as described in our literature review. The end result is that all the

articles in the database will have labels with proportions of various topics, which can then be

search for common name variations in a keyword search of the Readers’ Guide Retrospective will increase the number of articles found that relate to Russian literary authors of interest. An example of common name variations can be found in Appendix F.11 See Appendix C for current annotation guidelines.12 Keyword frequencies, achieved by using the publicly available Text Analysis Portal for Research (TAPoR) tool, will allow for organization of data on a more general level (Berson). An example of the information that TAPor can provide are the frequencies of author references and how often author names are found near each other.13 We will achieve semantic parsing by using software programs Shalmaneser and FrameNet, developed by the International Computer Science Institute at the University of California, Berkley. These programs will allow us to analyze databases using ‘frames,’ which, according to FrameNet, are semantic representations of situations. These tools highlight the types of sentences used in specific articles. For example, If an article contains many sentences framed under the semantic categories of ‘Judgment’ and ‘Assessment,’ we can safely conclude that article contains a number of opinionated statements. See Appendix H for more information.

13

Team POLITIC

categorized based on topic frequency. By comparison, we will implement a supervised version of

LDA (sLDA) in the automation of metadata creation.14 Finally, the last form of topic modeling

that we will use is the Topics Over Time model (TOT), described in the literature review, which

will introduce a time variable into our analysis (Wang and McCallum 5).

At the conclusion of this step in the process, we will have fully annotated and labeled the

databases by all the various data mining strategies. From this data, we can determine certain

trends in the topics in the articles. It is these trends that will allow us to make certain inferences

about the relationship between the reception of Russian literature in the United States and United

States foreign policy.

Conclusion

Our research aims to provide new insight into how the United States receives

foreign authors and novels and how this reception relates to US foreign policy. Our anticipated

results are vital to a recent development in the humanities known as the globalization of

American literary studies, given that “the mechanisms by which [differences between countries]

are translated into literature have never been fully specified” (Corse, Nations and Novels 1279).

Foreign novels are an inherent part of United States culture and if one were to ignore the

presence of foreign literature in United States politics, then one would be ignoring a major factor

that shaped both the citizens and government of the United States. “A sound public opinion

cannot exist without access to the news” and “evidence is needed” to reveal inherent biases in

publicly available portrayals of political events (Lippmann and Merz 1). Experts in fields of

literary studies claim scholars reach “little agreement about what constitutes literary value in this

field” and there exists “unnecessary confusion as to clear standards and goals” in evaluating 14 In sLDA, “each document is paired with a response. The goal is to infer latent topics predictive of the response” (Blei and McAuliffe 1). Instead of letting the software construct its own distribution over topics, we will provide a fitted model, specifically the annotation form previously mentioned in the methodology (ibid). Then the software can predict a response for the previously designated topics, such as sentiment, nationality, racism, politics, etc.

14

Team POLITIC

these types of literature (Brown 1-8). We are also pioneering relatively new software and

technology in the realm of literary analysis.

By May 9th, 2012, we plan to have compiled a sample database of several hundred

articles scanned and processed through the OCR software in preparation for a technical seminar

with MITH. Our annotation team hopes to annotate 150 of these articles. The goal of this

seminar is to experiment with some of the available database analysis software to determine how

effectively the computer programs can learn to annotate articles independently and whether any

trends in the metadata begin to surface. We anticipate finding a distinct correlation between the

reception of foreign literature and public attitudes toward foreign policy. We will compile our

completed findings into an additive online database, to which other scholars can contribute

similar research. Over time, our foundation will pave the way to understanding overall patterns

in foreign literature reception.

15

Team POLITIC

Works Cited

Aubry, Timothy. "Afghanistan Meets the Amazon: Reading the Kite Runner in America."

PMLA: Publications of the Modern Language Association of America 124.1 (2009): 25-

43. EBSCO. Web. 10 Sept. 2011.

Baldasty, Gerald J. E.W. Scripps and the Business of Newspapers. Urbana-Champaign: U of

Illinois P, 1999. Print.

Berson, Alex, Stephen Smith, and Kurt Thearling. Building Data Mining Applications for CRM.

New York: McGraw Hill, (1999): n. pag. Print.

Blei, David M., and Jon D. McAuliffe. “Supervised Topic Models.” Princeton U and U of

California, Berkeley, 2010. Web. 17 Mar. 2012.

Blei, David M. “Introduction to Probabilistic Topic Models.” Communications of the ACM.

Princeton U, n.d. Web. 17 Mar. 2012.

Boyer, Mark A. "Issue Definition and Two-Level Negotiations: An Application to the American

Foreign Policy Process." Diplomacy & Statecraft 11.2 (2000): 185-212. America: History

and Life with Full Text. Web. 27 Nov. 2011.

Brown, Joan L., and Crista Johnson. "Required Reading: The Canon in Spanish and Spanish

American Literature." Hispania 81.1 (1998): 1-19. JSTOR. Web. 12 Sept. 2011.

Chaney, Allison J.B., and David M. Blei. “Visualizing Topic Models.” International AAAI

Conference on Social Media and Weblogs. Princeton U Dept. of Computer Science,

2012. Web. 15 Mar. 2012.

Corse, Sarah M. Nationalism and Literature: The Politics of Culture in Canada and the United

States. Cambridge: Cambridge University Press, 1997. Print.

16

Team POLITIC

---. "Nations and Novels: Cultural Politics and Literary Use." Social Forces 73.4 (1995): 1279-

308. JSTOR. Web. 8 Sept. 2011.

“Deductions.” New Republic 4 Aug. 1920: 42-3. EBSCOhost. Web. 20 Mar. 2012.

Emerson, Caryl. "Leo Tolstoy On Peace And War." PMLA: Publications Of The Modern

Language Association Of America 124.5 (2009): 1855-58. Academic Search Premier.

Web. 15 Mar. 2012.

Gilens, Martin. “Political Ignorance and Collective Policy Preferences.” American Political

Science Review. 95.2 (2001): 379-96. Web. 29 Nov. 2011.

Goldfarb, Charles. “William Dean Howells: An American Reaction to Tolstoy.” Comparative

Literature Studies 8.4 (1971): 317-37. JSTOR. Web. 12 Mar. 2012.

Griswold, Wendy. "The Fabrication of Meaning: Literary Interpretation in the United States,

Great Britain, and the West Indies." American Journal of Sociology 92.5 (1987): 1077-

115. JSTOR. Web. 13 Sept. 2011.

Haslam, Paul Alexander. "The Evolution of the Foreign Direct Investment Regime in the

Americas." Third World Quarterly 31.7 (2010): 1181-203. Academic Search Premier.

Web. 27 Nov. 2011.

Lee, Lillian, and Bo Pang. “Sentiment of Two Women: Sentiment Analysis and Social Media.”

1900 University Avenue, Cornell University, New York. 22 Mar. 2011. Lecture.

Li, V. “Misgivings of a Tongue-Tied Nation.” Editorial Research Reports 2 (1990): n. pag. Web.

CQ Researcher. 13 Sept. 2011.

Lippmann, Walter, and Charles Merz. “A Test of the News: Introduction.” New Republic 4 Aug.

1920: 1-4. EBSCOhost. Web. 17 Mar. 2012.

17

Team POLITIC

Mallios, Peter Lancelot. Our Conrad: Constituting American Modernity. Stanford: Stanford UP,

2010. Google Books. Web. 15 Sept. 2011.

Moser, Charles A. "The Achievement Of Constance Garnett." American Scholar 57.3 (1988):

431. Academic Search Premier. Web. 20 Mar. 2012.

National Information Standards Organization. Understanding Metadata. Bethesda: NISO P,

2004. Web. 17 Mar. 2012.

Ohmann, Richard. "The Shaping Of A Canon: U.S. Fiction, 1960-1975." Critical Inquiry 10.1

(1983): 199-223. MLA International Bibliography. Web. 13 Nov. 2011.

Pasterczyk, Catherine E. “Russian Transliteration Variations for Searchers.” Education

Resources Information Center 8.1 (1985): n. pag. Web. 20 Mar. 2012.

Steyvers, Mark. "Probabilistic Topic Models." Handbook of Latent Semantic Analysis. Mahwah,

NJ: Lawrence Erlbaum Associates, 2007.

“TIFF Files.” John Salim Photographic Glossary of Terms. 2012. Web. 20 Mar. 2012.

Travis, Rick. "Problems, Politics, and Policy Streams: A Reconsideration US Foreign Aid

Behavior toward Africa." International Studies Quarterly 54.3 (2010): 797-821.

Academic Search Premier. Web. 27 Nov. 2011.

Wang, Xuerui, and Andrew McCallum. “Topics over Time: A Non-Markov Continuous-Time

Model of Topical Trends.” U of Massachusetts Dept. of Computer Science, 2006. Web.

15 Mar. 2012.

Watson, Robert P., and Sean McCluskie. "Human Rights Considerations and U.S. Foreign

Policy: The Latin American Experience." Social Science Journal 34.2 (1997): 249-57.

Academic Search Premier. Web. 27 Nov. 2011.

18

Team POLITIC

Appendices

Appendix A: Team Budget

Cost Per Item Cost

Immediate Expenses:

MLA Guide Book (already purchased from MLA.org)

$22.00

Large External Hard Drive (1+ Terabyte)

$300.00

Subtotal: $322.00

Foreseeable Expenses:

Hiring Technical Consultant for Enhancement of Existing Tools

$1,500.00

Travel Expenses (Conferences) $3,000.00

Subtotal: $4,500.00

TOTAL: $4,822.00

19

Team POLITIC

Appendix B: Team TimelineSpring 2012

o Complete team website

o Continue literature review

o Begin scanning periodicals into constructed Russian literature database

o Begin annotating Russian literature database and select metadata to capture

o Begin coordination with MITH and start to familiarize team with methods of

constructing and analyzing databases

Attempt to automate metadata collection

Summer 2012

o Continue scanning and annotation of Russian literature database

Fall 2012

o Prepare for and present at Junior Colloquium

o Determine methods by which to quantify American foreign policies

Begin construction of Foreign attitude / policy database

Spring 2013

o Present at Undergraduate Research Day

o Being drafting team thesis

Summer 2013

o Continue to draft team thesis

Fall 2013

o Obtain feedback for our thesis paper from Dr. Mallios

o Gather data regarding American foreign policy toward Russia

o Draw conclusions regarding the relationship between American foreign policies

and reception of Russian literature

Winter 2013-14

o Prepare presentation for Thesis Conference

o Revise and edit team thesis

Spring 2014

o Present at Senior Thesis Conference

20

Team POLITIC

Appendix C: Current Annotation Guidelines

1. Author (or authors) of principal concern in article. What literary author or authors, if any, is this article primarily about?

• Spelling:

--Be sure to spell any names given in answer to this question as accurately as possible, exactly reproducing how the name is spelled in this article. (Spellings will differ between articles: we want to capture the differences.

--Include the fullest version of the author’s name included in the article: i.e., include an author’s first and/or middle names and/or initials if these names are included at any point in the article.

• Individuals: Only literary authors named by personal name (i.e., not anonymous figures or those referenced only by job title) and who are persons (i.e., not publications) count as “authors” for purposes of this question.

• “Literary author” means an author of fiction, poetry, plays, or related forms of creative writing. This applies whether the author is being invoked in his or her capacity as a literary writer or not. Academic professors, literary critics, and journalistic and other commentators on literature do not fall into this category, unless they have significant literary accomplishments of their own.

• An author is of “principal” or “primary” concern in an article when an author is a major, continual, or focal concern that runs and receives explicit mention throughout an article as part of its general field of concerns, not just in discrete or severable paragraphs of it.

• Some more rules of thumb on identifying whether an author is a “primary” or “principal” concern in an article:

• if a literary author’s name is included in the article’s title, it is likely that s/he should be included in the answer to this question

• if there is a large disproportion between the number of times different authors are mentioned or referred to, this is a good indicator that those mentioned less should likely not be included in the answer to this question

21

Team POLITIC

• if the excising of relatively few paragraphs from this article would result in the elimination of reference to an author, that author should generally not be included in the answer to this question

• as a general matter, construe answers to this question narrowly: only an author (or authors) comprising the main and consistent focus of an article should be included—although articles whose explicit focus is evenly to compare two (or more) authors throughout may be described as having multiple “principal” authors

2. Sentiment Analysis 1: the Opinion of the Article Writer. Which of the following ratings comes closest to the article writer’s expressed opinion of the literary author(s) this article principally concerns? [Note: this question concerns the opinion ultimately taken by the article writer him/herself on the literary authors question. This is so even though the article writer may quote or reference opposing opinions along the way.] This question should be answered separately for each author named in question1.

2 – A Positive Opinion: a generally or ultimately positive opinion as an overall matter.0 – A Negative Opinion: a generally or ultimately negative opinion as an overall matter.U – A Mixed or Unclear Opinion, or No Opinion Offered: it is not possible to say whether the writer’s overall opinion of an author is either positive or negative because the writer’s opinions are mixed, unclear, or not offered at all.

3. Sentiment Analysis 2: Uncertainty of Article Writer’s Opinion. If the answer to Question 2 is “U,” answer the following question; if not skip it. Which of the following ratings comes closest to describing why the article writer’s opinion of a principal literary author is unclear? This question should be answered separately for each author named in question 1.

1 – A Mixed or Unclear Opinion: the article writer either expresses mixed opinions about the literary author, or does not make clear how the opinions, judgments, or values s/he holds clearly relates to the literary authorX – Straight Factual Account: this is not an article in which the article writer’s personality, opinions, judgments, are in evidence; the article writer assumes the position of the “straight,” factual, objective newspaper reporter; the article writer’s stance is neutral with respect to his/her own opinions and values, not evaluative.

4. Sentiment Analysis 3: Principal Author as Subject of Debate. (Y/N) Does this article contain any explicit reference to the literary author(s) it principally concerns as a subject of debate, either because interpretations of that literary author’s meaning are explicitly disputed, or because opposing positive and negative opinions of an author are explicitly referenced?

22

Team POLITIC

5. Books mentioned? (Y/N). Does this article explicitly mention by title any specific books, poems, or texts written by any literary author it is principally about? Note: this question should be answered separately for each author named in question 1.

6. National identification. (Y/N) Does this article specifically identify the nationality of any literary author it is principally about? Note: this question should be answered separately for each author named in question 1.

7. Style or literary artistry as issue. (Y/N) With respect to any literary author this article is principally about, is the author explicitly described in terms of “art” or as an “artist” or in terms of his or her “artistic” vision, or is at least one paragraph of the article devoted to the style (not the content) of his or her writing? (A “yes” answer to any part of this question means a YES answer to the question as a whole.) Note: this question should be answered separately for each author named in question1.

8. Foreign Place Names. (Y/N) Are there any non-U.S. place names mentioned in this article?

9. Gender of Article Writer. Use the following scale to identify the apparent gender of the writer of this article (i.e., not the gender of the literary figure(s) in question, but the gender of the article writer who is writing about the literary figure(s)):

M – MaleF – FemaleU – Unclear (i.e., because name is ambiguous or initials are used; the article is unsigned; or for another reason)

10. Gender as Issue. (Y/N) Is gender ever explicitly discussed as an issue in this article?

• Note: The fact that a character or author discussed in the article is a man or woman is not sufficient to constitute a Yes answer to this question; there needs to be some explicit attention drawn to gender as a matter of significance—(if only in a single phrase)--or reflection on or significance attributed to the categories of “man” or “woman,” “masculine” or “feminine,” or other gender ideas.

11. Race as Issue. (Y/N) Is race ever explicitly raised as an issue in this article?

• Note: this question should be answered “Yes” only if: (i) the article explicitly uses the term “race” (or some direct variant on it: “racial,” “racism,” etc.); (ii) there is explicit discussion about general ideas of race; or (iii) one of the following radicalized categories is explicitly invoked: black or African; white or Aryan or Caucasian; Slavic; Jewish or Hebrew.

23

Team POLITIC

12. Socioeconomic class as issue. (Y/N) Does socioeconomic class receive explicit discussion in this article?

• Note: Any explicit mention of social class (for example, “aristocratic,” “peasant,” “the poor,” “Count,” “prince”) will qualify as a YES answer to this question. (Czar, however, as a state figure, does not alone qualify.)

13. Religion as Issue. (Y/N) Does religion receive explicit discussion in this article?

14. Radical Politics as issue. (Y/N) Do any radical political movements including anarchism, nihilism, bolshevism, socialism, or communism receive explicit mention in this article?

15. America/West invoked as a point of similarity with Russia. (Y/N) Does this article make any specific and explicit claims that Russia shares any quality in common with the U.S., “the West,” or any of the countries, cultures, and/or literatures of Western Europe?

16. America/West invoked as point of contrast with Russia. (Y/N) Does this article draw any specific and explicit contrasts between Russia or anything Russian and any qualities or aspects of the U.S., “the West,” or any of the countries, cultures, and/or literatures Western Europe?

24

Team POLITIC

Appendix D: Sample Annotation Question Evolution

Current Sample Annotation Question

4. Sentiment Analysis: Principal Author as Subject of Debate. (Y/N) Does this article contain any explicit reference to the literary author(s) it principally concerns as a subject of debate, either because interpretations of that literary author’s meaning are explicitly disputed, or because opposing positive and negative opinions of an author are explicitly referenced?

Original Sample Annotation Question

4. Sentiment Analysis: All Opinions Expressed in the Article. [This question concerns all opinions expressed in the article concerning the literary writers in question—whether they express the article’s own point of view or other perspectives quoted and referenced in the article.] Which of the following ratings comes closest to the entire field of opinions quoted or mentioned in this article concerning each of the literary authors the article principally concerns? Note: this question should be answered separately for each author named in question 1.

2 – A Positive Opinion: a generally or ultimately positive opinion as an overall matter1 – A Mixed or Unclear Opinion: such that it is not possible to say whether the article’s

overall opinion of an author is positive or negative0 – A Negative Opinion: a generally or ultimately negative opinion as an overall matterX – Neutral: This article is not evaluative: it does not express opinions about the

author(s) in question, but is rather strictly and neutrally factual

25

Team POLITIC

Appendix E: Search Results Using the Readers’ Guide Retrospective

Author / Subject Total # Search Results Tolstoy, L.N. 432Chekhov, A.P. 266“russian literature” 193Dostoevsky, F.M. 128Gorky, M. 123Turgenev, I.S. 96Breshko-Breshovskaya, E.K. 53

Appendix F: Alternative spellings of “Dostoevsky”

“dostoevsky” OR “dostoyevsky” OR “dostoevskii” OR “dostoyevskii” OR “dostojevsky” OR

“dostojevskii” OR “dostoeffsky” OR “dostoyeffsky” OR “dostoeffskii” OR “dostoyeffskii” OR

“dostoieffsky” OR “dostoievsky” OR “dostoieffskii” OR “dostoievskii” OR “dosteovsky” OR

“dostoyefsky” OR “dostoievski” OR “dosteoffsky” OR “dosteovskii” OR “dostoefsky” OR

“dostoefskii” OR “dostojefsky” OR “dostojefskii” OR “dostojefski” OR “dostoevski” OR

“dosteovski” OR “dostoyevski” OR “dostojevski” OR “dostojeffski” OR “dostoyeffski” OR

“dostoeffski” OR “dostoieffski” OR “dostoievski” OR “dostojefski” OR “dostoyefski” OR

“dostoefski” OR “dostoiefski”

Alternative spellings research conducted by Nick Slaughter of the Foreign Literatures in America project.

26

Team POLITIC

Appendix G: Sample Chart of Periodicals within Readers’ Guide Retrospective: 1890-1982

Source Type

ISSN / ISBN

Publication Name Publisher Indexing Start

IndexingStop

Magazine 0163-2027 50 Plus Reader's Digest Association, Inc. 1/1/83 11/1/88

Magazine 1548-2014 AARP the Magazine. AARP 5/1/03

Magazine 1041-102X Ad Astra National Space Society 1/1/89

Magazine 0955-2308 Adults Learning National Institute of Adult Continuing Education

1/1/95

Magazine 0001-8996 Advocate Regent Media 1/16/01

Magazine 0002-0966 Aging Superintendent of Documents 11/1/82 1/1/96

Academic Journal

1205-7398 Alternatives Journal University of Waterloo 1/1/05

Magazine Amazing Wellness Active Interest Media, Inc. 1/1/10

Magazine 1545-8741 Amber Waves: The Economics of Food, Farming, Natural Resources, & Rural America

U.S. Dept. of Agriculture Economic Research Service

2/2/04

Magazine 0002-7049 America America Press 1/1/83

Magazine 0002-7375 American Artist Interweave Press, LLC 1/1/83

Magazine 1540-966X American Conservative American Conservative 1/16/06

Magazine 1079-3690 American Cowboy Active Interest Media, Inc. 2/1/11

Magazine 0194-8008 American Craft American Craft Council 2/1/83

Academic Journal

0002-8304 American Education US Department of Education 12/1/82 1/3/85

Magazine 0361-4751 American Film BPI Communications 1/1/88 1/1/92

Magazine 0002-8541 American Forests American Forests 9/1/92

Academic Journal

1549-4934 American Geographical Society's Focus on Geography

Wiley-Blackwell 10/15/05

Magazine 1523-3359 American Health RD Publications Inc. 1/1/99 10/1/99

Magazine 0730-7004 American Health (0730-7004)

RD Publications Inc. 1/1/88 1/1/97

Magazine 1092-1656 American Health for RD Publications Inc. 12/1/96 1/1/99

27

Team POLITIC

Women

Magazine 0002-8738 American Heritage AHMC Inc. 2/1/83

Magazine 1076-8866 American History Weider History Group 6/1/94

Magazine 0002-8770 American History Illustrated

Weider History Group 1/1/83 3/1/94

Academic Journal

0095-182X American Indian Quarterly

University of Nebraska Press 1/1/90

Academic Journal

1067-8654 American Journalism Review

University of Maryland 3/1/93

Academic Journal

0003-0937 American Scholar Phi Beta Kappa Society 1/15/83

28

Team POLITIC

Appendix H: Glossary of Terms

Classification: Supervised (requires human input) method of analyzing text in which the user

first defines labels of how they want a collection of words, sentences, etc, to be classified.

Next, the user creates a training corpus of words, sentences, etc that is already classified

according the specified labels to train the software. The user can then input the collection

of words, sentences, etc. they want to “classify” by the labels.

Corpus: a large body of texts, often the entirety of works by an author, articles by a newspaper,

or writings about a certain subject

Keyword frequencies: How often a word appears in literature

Latent Dirichlet Allocation: Abbreviated LDA, attributes each word in a written document to a

select number of topics determined to compose the document

Optical Character Recognition Software: Abbreviated OCR, translates PDFs and scans of either

handwritten or typed texts into electronic machine readable text

Semantic Parsing/Analysis: Also known as opinion mining, using text analysis to determine

subjective information in written works

Shalmaneser: A supervised tool (requires human input) for semantic and syntactic parsing, which

automatically assigns text to semantic and syntactic classes. Generates output such as the

following figure:

29

Team POLITIC

Where the original sentence is: “Creeping in its shadow I reached a point whence I could

look straight through the uncurtained window.”

The green text is the generated analysis of the semantics of the sentence and the gray text

is the generated analysis of the syntax of the sentence.

Text Analysis Portal for Research: Abbreviated TAPoR, a collaborative project that permits

researchers to use text analysis tools for the Humanities

Topic modeling: The use of a type of statistical model that generates abstract “topics” in a

database of documents

30