The Stroop Test on the iPad

Neurocognitive Testing Made Easier

Vanderbilt University

April 26, 2011 Sarah E. Waring and Emily S. Whitaker, Department of Biomedical Engineering

Advisor: James L. Blair, DO: Assistant Professor, Clinical Anesthesiology, Department of Anesthesiology, Vanderbilt University Medical Center

Abstract Recently, concerns have risen over the safety of some anesthetic agents in pediatric and geriatric patients due to recent evidence that indicates neurotoxic effects are leading to postoperative cognitive dysfunction (POCD) of the central nervous system (CNS). This is most commonly seen in elderly patients, who may end up with their ability to perform everyday tasks impaired. Although the risk of anesthesia to almost every other organ is assessed pre-operatively, the brain is neglected. This is due, at least partially, to our lack of understanding of exactly how anesthesia affects the brain. Currently, neurocognitive tests (NCT) are performed post-surgery to check for POCD; this assessment is subjective since there is not usually a pre-operative measurement to compare to, only the patient’s own idea of their pre-surgery function. In order to better understand the functional and structural changes caused by anesthesia, Dr. Blair has proposed a study that will compare pre- and post-operative results for both MR imaging and NCT. The study’s aim is to determine if changes in CNS structure are seen by MR imaging and how NCT results might correlate to these structural changes. Our project consisted of creating an iPad program that can be used in this study to easily administer the Stroop Test, (a NCT) in a user friendly, touchscreen format.

IntroductionCognitive dysfunction is defined as the impairment of the mental processes required for everyday living – perception, memory, and information processing (Hanning; Price, Garvan and Monk). When this type of cognitive decline occurs following a surgery (Newman, Stygall and Hirani; de Lau, van Meurs and Uitterlinden; Gao, Taha and Gauvin; Price, Garvan and Monk), we call it postoperative cognitive dysfunction (POCD). The etiology of POCD is unknown. Manifestations of POCD can include memory lapses, lack of problem solving abilities, difficulty retaining information, and the sudden inability to perform tasks that were previously easy. Elderly patients are at the highest risk of developing POCD (Moller, Cluitmans and Rasmussen; Perouansky and Hemmings). For 2010, it was estimated that over one million elderly patients would develop POCD that would have long-term, debilitating effects on daily living that result in a drastic (almost doubled) increase in their risk of death within one year post-op (Monk, Weldon and Garvan; Moller, Cluitmans and Rasmussen; Monk, Saini and Weldon; Lindholm, Traff and Granath).

Neurocognitive tests (NCT) can be administered to patients to assess their current cognitive function. The NCT used for this study focus on checking memory and executive function (Moller, Cluitmans and Rasmussen). Currently, various neurocognitive tests (NCT) for POCD are administered one week and three months post-surgery, but not before surgery. To evaluate the amount of cognitive decline

that actually occurs, preoperative baseline assessments are needed (Moller, Cluitmans and Rasmussen; Rasmussen, Larsen and Houx).

Since the etiology of POCD is unknown, our advisor has proposed a study that will look at functional and anatomic neuroimaging (fMRI and MRI), as well as neurocognitive tests to evaluate changes in neurocognitive function associated with POCD. All tests will be performed pre- and post-operatively, and the results will be analyzed to determine if there is a correlation between NCT and MR imaging changes. It is hypothesized that the neurocognitive tests will reflect physiological and functional changes in the CNS). Hence, the development of a pre-operative battery of neurocognitive tests to screen for patients at a high risk of POCD would be of great use. We want to develop a computerized program that can be easily used to administer these neurocognitive tests to elderly patients via a touch screen format.

Our focus in developing our iPad program was on the Stroop Word and Color Test, which is a measure of selective attention and cognitive flexibility that was originally developed by Stroop. Traditionally, this test has been administered on paper, by a trained proctor, but recently this test has been adapted for electronic use (Stroop; Strauss and Spreen; Golden). The original test consisted of four parts, but ours consists of just two parts. In our Part 1, the subject is shown color words printed in colored ink that matches the word and must choose the color of the ink. In Part 2, the subject still chooses the color of the ink, but the ink color no longer matches the word (e.g. the word “blue” printed in green ink should be answered “green”). Of major interest is the subject’s behavior when presented with color words printed in nonmatching colored inks. Stroop reported that normal people can read color words printed in colored ink as fast as fast as when the words are printed in black ink. However, the time to complete the task increases significantly when the subject is asked to name the color of the ink rather than the word. This decrease in color-naming speed is called the “color-word interference effect.” (Stroop; Strauss and Spreen; Golden).

Methodology The ultimate goal of this project was to create a working program to be used on an iPad for administration. Programming for this device allowed for an easily-administrable test due to its unique qualities; the iPad is both portable and lightweight, has a long battery life and exhibits instant start-up, and utilizes a touch screen, which is easy for patients to navigate. Because the program is specific to this device, to formulate the test we used Apple’s iOS application software – Xcode and SDK. The program was written in Objective-C language; therefore, our first step was to learn how to code in that specific language. Initial program development arose from the idea of a simple “quiz” game, which we modified for our purposes. To form our specific test, the Stroop test, we used current paper- and computer-

based tests as resources. Questions were chosen from a written Stroop test example and answer choices were selected using a random number generator. All questions are presented on a simple, white background to avoid skewing the user’s perception of the words displayed. Each question prompt was coded to contain a color word written in a specific ink color, with four answer choices exhibited on the touch screen. Other features that were added to the program include the ability to enter a patient ID number and store and export scores. Our scoring method was based on both response time and accuracy of answers; we implemented this in the program by setting a timer for each question and scoring one point per correct answer (zero points for missed question). Final processing of the program required enrollment as an Apple Developer, allowing us to run the application on an iPad for testing.

ResultsOur Xcode programming and thorough troubleshooting resulted in a working Stroop test program that runs on the iPad. The program has many features, each of which corresponds to important aspects of the Stroop test and data management. Each screen plays an important role in the test as a whole. The first is a start-up screen where the physician can enter a patient ID number that will be stored with the score. It also includes patient instructions and a “Start” button. This screen is shown in Figure 1.

Figure 1. Screenshot of the Stroop test start screen. The physician can enter a patient ID number to be stored along with the scores.

The subsequent screens display 32 quiz questions that progress from easy (the word matches the ink color) to more challenging (the word and ink color do not match). To answer the question, the user must tap the button corresponding to the correct color of the ink. Each question worth one point and is timed for five seconds; if the patient cannot answer in time, he or she misses the question and receives zero points. The score is stored for later use. Both types of question can be seen in Figure 2.

The last screen is a results page which displays the patient’s score (this feature can be disabled for actual neurocognitive testing, but remains visible for program demonstration) and a button which allows the physician to export the score along with the patient ID number via email. Thus, the patient’s identity is not compromised and no privacy laws are violated. This view is shown in Figure 3.

Figure 2. Screenshots of 2 example Stroop test questions. The user must identify the color of the ink.

Figure 3. Screenshot of the Stroop test start screen. The physician can enter a patient ID number to be stored along with the scores.

Our final program is thus successful in fulfilling our stated requirements for a functioning Stroop test. The number and order of questions can be altered as well as the screen view; however, we felt that it was better to keep the program looking clean and simple to avoid unnecessary distractions to the patient (which could skew results). The program and test were found to be simple and easy to use when tested on university students; thus, we hope that Dr. Blair will eventually be able to use the program on patients in his study without problems.

The cost of developing this program was quite low, and the benefits are great. Two students worked around 100 hours coding and debugging, and costs included Apple Developer program enrollment (for application testing and future distribution) at $99 for the year and the purchase of an iPad at $500. There are little to no safety risks associated with program development and, since it is being created for use as a research tool, does not require any sort of federal approval. Future benefits include an increase in knowledge of anesthesia’s effect on the brain and an eventual decrease in number of cases of post-operative cognitive dysfunction. Once tested by Dr. Blair in his study and approved by the App Store, the program can be shared with other hospitals for pre-operative screening. This could ultimately result in reduced lawsuits and legal fees.

ConclusionsThe final iPad program will be used in conjunction with physiological imaging (performed by Dr. Blair) to detect changes in cognitive function and brain structures, and will provide easier methods of test administration to facilitate his study. The creation of a successful program will allow for the combination of multiple tests and easy data transfer for analysis, thus cutting costs. We hope our program will help continue research into POCD and help researchers gain more knowledge of how surgery and anesthesia affect the brain. Ultimately, we met our design specifications and produced a working Stroop test program for the iPad.

RecommendationsIn the future, we hope this study will ultimately lead to a development of better anesthetic practices to decrease cases of POCD in the elderly. Finally, the program could eventually appear in Apple’s App Store (after getting approval) for distribution so that other hospitals can use it for screening. To continue to improve the application for Dr. Blair’s study, more neurocognitive tests could be programmed and inserted into the existing program to collect multiple scores (and cognitive baseline readings) per patient. Future versions of the test could also include a more detailed score report, displaying the time it took to answer each question as well as

the exact questions missed. We hope that Dr. Blair will be able to test and use this application in his study to help better determine baseline cognitive function.

Ideation ProcessInnovation Situation Questionnaire

1. Brief description of the situationNeurocognitive tests are used to diagnose post-operative cognitive dysfunction. Currently, tests

are administered either via the traditional pencil-and-paper format, or an electronic format on a computer. Development of a single application for the Apple iPad that can administer several tests quickly has been proposed. The problems to be solved are that the current methods of administering these tests are time-consuming and there is no touchscreen-formatted battery of tests available.

2. Detailed description of the situation

2.1. Supersystem - System - Subsystems

2.1.1. System nameApplication for the Apple iPad

2.1.2. System structureThe application will consist of C++ coding.

2.1.3. Supersystems and environmentThe Apple iPad is used to administer tests. Application must be formatted for touchscreen.

2.1.4. Systems with similar problemsElectronic versions of some of these tests are available, so coding and scoring issues have been worked out in some cases. Randomization of answer choices has also been resolved in some electronic versions, but the specific coding is unavailable to us.

2.2. Input - Process - Output

2.2.1. Functioning of the systemThe primary function is to administer neurocognitive tests to assess the cognitive function of a patient.

2.2.2. System inputsThe inputs to the application are the answers submitted by the user to the questions.

2.2.3. System outputsThe output, at the end of all the tests, consists of the patient's scores for each test. These scores will indicate their cognitive function.

2.3. Cause - Problem - Effect

2.3.1. Problem to be resolvedShorten length of time taken to administer a battery of neurocognitive tests.

2.3.2. Mechanism causing the problemThe nature of the tests themselves.

2.3.3. Undesirable consequences if the problem is not resolvedNone

2.3.4. Other problems to be solvedFormatting the tests for a touchscreen presentation

2.4. Past - Present - Future

2.4.1. History of the problemPaper-and-pencil versions of some neurocognitive tests have been supplemented by electronic versions. Some games are also available that utilize sections of some tests.

2.4.2. Pre-process timeTime before the tests are administered.

2.4.3. Post-process timeTime after the tests are administered and scored

3. Resources, constraints and limitations

3.1. Available resources Apple iPad Knowledge of coding Neurocognitive tests in paper and in electronic forms Scoring information

3.2. Allowable changes to the system Only minimal changes are to the neurocognitive tests themselves Small changes to presentation of test material Small/minimal changes to the scoring methods

3.3. Constraints and limitations Validity of neurocognitive tests must be preserved Test content cannot be altered Touchscreen format required Take about 10 minutes to administer tests

3.4. Criteria for selecting solution concepts Program runs as desired, administering neurocognitive pre-operative tests to patients Scoring data is statistically valid, i.e. matches data we would obtain using paper-and-pencil versions of the same tests.

Problem Formulation and Brainstorming

Neurocognitiveassessment of

patient

iPad applicationadministers NCTs

Coding ofapplication iscompleted

Application is onthe iPad

Patient cannotread English

Patient is colorblind

iPad battery dies Current tests aretime-consuming

Diagram 1

12/7/2010 10:14:41 PM.

8. Find a way to eliminate, reduce, or prevent Current tests are time-consuming.

Develop Concepts1. Categorize preliminary ideasCoding of application Determine what language to use Research/learn how to code for necessary functions Accept touchscreen input Multiple sets of test questions for repeated use

Electronic test presentation Use currently available electronic versions: pay licensing fee Write our own coding for each test Try to mimic presentation of test material in existing electronic version Randomized answer choices Method of input of answer choice on the touchscreen Present directions Start with sample questions

Validation of results Compare to existing normative data

Build our own set of data Develop scoring based on response time and accuracy Compare to results from paper-and-pencil version

Evaluate Results1. Meet criteria for evaluating Concepts Created working iPad program version of the Stroop Word and Color Test Developed scoring method based on response time and accuracy Tested on fellow students Touchscreen administration of test

2. Reveal and prevent potential failures Compatibility with different versions of iPad software

3. Apply Patterns/Lines of Evolution Continue to work on adding other neurocognitive tests to expand battery in this program Compare to results from paper-and-pencil version and existing normative data

4. Plan the implementation Use in Dr. Blair’s proposed study. Possibly sell in Apple’s App Store to expand market.

BibliographyBaranov, D, et al. "Consensus statement: First International Workshop on

Anesthetics and Alzheimer's disease." Anesth Analg (2009): 1627-1630.

Bokeriia, L A, et al. "Asymmetric cerebral embolic load and postoperative cognitive dysfunction in cardiac surgery." Cerebrovasc Dis (2007): 50-56.

de Lau, L M, et al. "Genetic variation in homocysteine metabolism, cognition, and white matter lesions." Neurobiol Aging (2008).

Gabel, R A. "Counting deaths due to medical errors." JAMA (2002): 2404, 2405.

Gao, L, et al. "Postoperative cognitive dysfunction after cardiac surgery." Chest (2005): 3664-3670.

Golden, C J. "Identification of brain disorders by the Stroop Color and Word Test." J Clin Psychol (1976): 654-658.

Hanning, C D. "Postoperative Cognitive Dysfunction." Br J Anaesth. (2005): 82-87.

Johnson, T, et al. "Postoperative cognitive dysfunction in middle-aged patients." Anesthesiology (2002): 1351-1357.

Lindholm, M L, et al. "Mortality within 2 years after surgery in relation to low intraoperative bispectral index values and preexisting malignant disease." Anesth Analg (2009): 508-512.

Mahanna, E P, et al. "Defining neuropsychological dysfunction after coronary artery bypass grafting." Ann Thorac Surg (1996): 1342-1347.

Moller, J T, et al. "Long-term postoperative cognitive dysfunction in the elderly ISPOCD1 study." Lancet (1998): 857-861.

Monk, T G, et al. "Anesthetic management and one-year mortality after noncardiac surgery." Anesth Analg. (2005): 4-10.

—. "Predictors of cognitive dysfunction after major noncardiac surgery." Anesthesiology 108.1 (2008): 18-30.

Newman, S, et al. "Postoperative cognitive dysfunction after noncardiac surgery: a systematic review." Anesthesiology (2007): 572-590.

Pantoni, L, A Poggesi and D. Inzitari. "The relation between white-matter lesions and cognition." Curr Opin Neurol. (2007): 390-397.

Perouansky, M and H C Jr Hemmings. "Neurotoxicity of general anesthetics: cause for concern?" Anesthesiology (2009): 1365-1371.

Price, C C, C W Garvan and T G Monk. "Type and severity of cognitive decline in older adults after noncardiac surgery." Anesthesiology (2008): 8-17.

Rasmussen, L S. "Postoperative cognitive dysfunction: incidence and prevention." Best Pract Res Clin Anaesthesiol (2006): 315-330.

Rasmussen, L S, et al. "The assessment of postoperative cognitive function." Acta Anaesthesiol Scand (2001): 275-289.

Silverstein, J H, et al. "Postoperative cognitive dysfunction in patients with preoperative cognitive impairment: which domains are most vulnerable?" Anesthesiology (2007): 431-435.

Steinmetz, J, et al. "Long-term consequences of postoperative cognitive dysfunction." Anesthesiology (2009): 548-555.

Strauss, E S E and Otfried Spreen. A Compendium of Neuropsychological Tests - Test Administration - Commentary. New York: Oxford University Press, 2006.

Stroop, J R. "Studies of the interference in serial verbal reaction." Journal of Experimental Psychology (1935): 643-662.

Warner, M A and T G Monk. "The impact of lack of standardized definitions on the specialty." Anesthesiology (2007): 198-199.