1

Newsandnarrativesinfinancialsystems:Exploitingbigdataforsystemicriskassessment1

RickardNyman,DavidGregory,SujitKapadia,PaulOrmerod*,DavidTuckett&RobertSmith*

September2016

Abstract:A number of recent contributions have tried to add to the understanding andforecastingofthemacroeconomybyanalysingnewsandnarratives.Thispaperappliesalgorithmicanalysistolargeamountsoffinancialmarkettext‐baseddatatoassesshownarrativesandemotionsplayaroleindrivingdevelopmentsinthefinancial system. We find that changes in the emotional content in marketnarratives are highly correlated across data sources. They show clearly theformation (and subsequent collapse) of very high levels of sentiment – highexcitementrelativetoanxiety–leadinguptotheglobalfinancialcrisis.Andwefindthattheshiftshavepredictivepowerforothercommonlyusedmeasuresofsentimentandvolatility.Wealsoshowthatanewmethodologythatattemptstocapture the emergence of narrative topic consensus gives an intuitiverepresentationoftheincreasinghomogeneityofbeliefsaroundanewparadigmprior to the crisis. With increasing consensus around narratives high inexcitement and lacking anxiety likely to be an important warning sign ofimpending financial system distress, the quantitativemetricswe developmaycomplementotherindicatorsandanalysisinhelpingtogaugesystemicrisk.

1Theviewsexpressedinthispaperaresolelythoseoftheauthor(s)andshouldnotbetakentorepresentthoseoftheBankofEngland,theMonetaryPolicyCommitteeortheFinancialPolicyCommittee.UniversityCollegeLondon,CentrefortheStudyofDecision‐MakingUncertaintyBankofEngland

2

1. Introduction  

Theyearsprecedingtheglobalfinancialcrisiswerecharacterisedby

widespreadexuberanceinthefinancialsector.Ashasoftenoccurredthroughout

history(ReinhartandRogoff,2009),consensusemergedoveranewparadigm,

underwhichthegreaterefficiencyofmarketsanddistributionofriskaroundthe

systemwasthoughttojustifythestrongpositivesentiment.Whenthecrash

cameduring2007and2008,sentimentreversedrapidlywithfearandanxiety

pervadingthefinancialsystem.

Thispaperappliesalgorithmicanalysistolargeamountsofunstructuredtext‐

baseddatatoidentifyquantitativemetricsthattrytocaptureshiftsinsentiment

alongwiththeextentofconsensusinthemarket.Wefindthatthesemetrics

capturekeydevelopmentsinthefinancialsystemrelativelywellpriortoand

duringtheglobalfinancialcrisis,aswellashavingpredictivepowerforother

commonlyusedmeasuresofsentimentandvolatility.Assuch,thesemetrics

couldpotentiallybeusedforgaugingsystemicriskinfinancialsystemsand

helpingtosignaltheprospectoffuturedistressasacomplementtomore

traditionalindicatorsandanalysis(see,forexample,Drehmannetal,2011,Bank

ofEngland,2014orGieseetal,2014).

Withrapidadvancesinwaystostoreandanalyselargeamountsof

unstructureddata,thereisincreasingawarenessthatthesedatamayprovidea

richsourceofusefulinformationforassessingeconomictrends.Forexample,a

growingliteratureexploitsindividualuser‐generatedsearchenginedata,suchas

GoogleTrends,totrytopredictthecurrentvalueof(‘nowcast’)economic

variablessuchasGDP(seeforexampleChoiandVarian,2012).However,some

recentstudiessuggestsearchenginedatashouldbetreatedwithcare,either

3

becauseofalackoftransparencyabouthowthedatahasbeencreated(Lazer,et

al.2014)oruncertaintyaboutthemotivationforsearching–independentlyor

becauseofsocialinfluence(Ormerodetal.,2014).

Bycontrast,themeasuresofsentimentweusearepre‐definedwordlists

representingtwospecificemotionalgroups.Thewordshavebeendeveloped

throughthelensofasocial‐psychologicaltheoryof“convictionnarratives”(CNT)

(TuckettandNikolic,2016),whichisonewayofempiricallyformulatingtheidea

of“animalspirits”asdescribedbyKeynes(1936).CNTisanewtheorywhich

emphasisestheroleofnarrativesandparticulargroupsofaction‐enablingor

disablingemotionsindrivingdecision‐makingunderuncertainty(Chongand

Tuckett,2014;TuckettandNikolic,2016).Ithasbeensuccessfullyusedinother

applications,forexampleasameasureofchangingmacroeconomicconfidence

(Tuckettetal.2015)andispartofagrowingbodyofworkinsideandoutside

economicsdevelopingbroadermodelsofhumancognitionanddecision‐making

behaviour(forexample,AkerlofandShiller,2009;Bruner,1990;Damasio,1999;

LaneandMaxfield,2005;MarandOatley,2008;Beckert,2011andTuckett,

2011)2.

2Narrativecanbeconsideredafundamentalformofmentalorganization(Bruner,1991)thatallowsexperiencetobeorderedinto“chunks”(Miller,1956)withimplicitrelevancetoplans(Pribriametal,1960)andcausalmodels(RottmanandHastie,2013;SlomanandLagnado,2015)andsoexplanationsandpredictionsaboutoutcome.Convictionnarrativesenableactorstodrawonthebeliefs,causalmodelsandrulesofthumbsituatedintheirsocialcontexttoidentifyopportunitiesworthactingon,tosimulatethefutureoutcomeoftheactionsbymeansofwhichtheyplantoachievethoseopportunitiesandtofeelsufficientlyconvincedabouttheanticipatedoutcomestoact(TuckettandNikolic,2016).Theyarefoundedonbiologicallyandsociallyevolvedcopingcapacitiesthatallowindividualstopreparetoexecuteparticularactionseventhoughtheycannotaccuratelyknowwhattheoutcomeswillbe.Convictionnarrativesalsoprovideaneasymeansforactorstocommunicateandgainsupportfromothersfortheirselectedactionsaswellastojustifythemselves.Ideasabouttheroleofsimulationandembodiedcognitionthatarecentraltothesupportiveroleofnarrativesindecision‐makingbuildonexistingworkinaffectiveandcognitiveneuroscience(e.g.Baumeister and Masicampo, 2010; Barsalou, 2008; DamasioandCarvalho,2013;SuddendorfandCorballis,2007.)

4

CNTsuggeststhatwithinthecontextofdeep,orKnightian(Knight,1921),

uncertainty(i.e.,uncertaintycharacterisedbyacontextinwhichthespaceof

potentialoutcomesofsomeeventcannotbearticulated)agentsdo(andhaveto)

constructnarrativessupportingtheirexpectationsandthatthesecreateafeeling

ofaccuracythatreadiesthemtoact.Suchconvictionnarrativescombine

cognitionandemotiontointerpretdata,envisionthefutureandsupportaction.

Convictionnarrativescontainafewfundamentalcomponents,notablyafocuson

thespecificemotionalelementsofnarrativesthatevokeattractionorapproach

toanobjectofinvestment(broadlyconceived),versusemotionsthatevoke

repulsionoravoidanceofthatobject.Thisemphasisonapproachandavoidance

inconvictionnarrativetheoryfocusestheideaofsentimentonitsimplications

foractioninuncertaindecision‐making,thusfocusingtheoften‐vaguetopicof

positive/negativesentiment.Inmoreordinarylanguagewefocusonexcitement

aboutthepotentialgainsfromanactionrelativetoanxietyaboutthepotential

losses.Ifexcitementcomestodominaterelativetoanxiety,investmentwillbe

undertaken(TuckettandNikolic,2016).Thus,inthesimplestcase,thekey

variablesofinterestaretheaggregaterelativedifferencebetweenexcitement

andanxietyandshiftsinthisdifferenceovertime.

Atanygivenmoment,therewillbeseveralnarrativesandassociated

emotionscirculatingamongallfinancialagents.Someofthesenarratives,or

piecesofthem,arelikelytobecontainedwithinrelevanttext‐baseddata

sources.Andiftherelativeshiftsintheemotionalcontentcorrelateacross

differentsources,itisplausiblethatatleastsomefinancialagentshadadopteda

subsetofthenarrativesandheldthemastrue,thoughitisimportanttonotethat

onecannotconclude,anditisinsomecaseshighlyunlikely(dependingonthe

5

typeofdata),thatthecontentcreatorsthemselveshadadoptedastruethe

narrativesportrayedintheirdocuments–forexample,onecaneasilyimaginea

bigdifferencebetweenfinancialnewsdocumentsandsocialmediadata,inthe

extenttowhichcontentcreatorsfeelwhattheywrite.

Withthisinmind,weanalysethreeunstructuredtext‐baseddatasourcesof

potentialinterest:internalBankofEnglanddailycommentaryonmarketnews

andevents;brokerresearchreports;andReuters’newsarticlesintheUnited

Kingdom.Motivatedbytheconvictionnarrativemethodology,wecapturean

emotionalsummarystatistic(RelativeSentimentShiftorRSS)basedonthese

sources,andexplorechangesinthestatisticovertimetoassesshow

convincinglyandrobustlyitmeasuresshiftsinconfidence.Thismeasureaimsat

capturingtheextenttowhichthecreatorsofthedocumentsportrayemotions

withinthenarrativesand,inparticular,shiftsinthebalancebetweenthe

proportionsofexcitementversusanxietywords.Aswithothertext‐basedand

bigdataapproacheswhichtrytooperationalisetheconceptofsentiment,for

exampleusingdifferentwordlists,akeystrengthofthismethodliesinitstop

downapproach,capturingaggregateshiftslargelyundetectabletothehuman

eye.

Therelativesentimentmetricsthatweextractappear,withthebenefitof

hindsight,togiveearlywarningsignsofsignificantfinancialeventsinrecent

years.Inparticular,overallsentimentwasatveryhighandstablelevelsinthe

mid‐2000s,arguablyindicativeofexuberanceinthefinancialsystemandtherisk

offuturedistress.Frommid‐2007,asurgeinanxietydroverapidfallsin

sentimentthatcontinueduntilsoonafterthecollapseofLehmanBrothers.And

therewerefurtherfallsinsentimentpriortothestartoftheEuroareasovereign

6

crisisin2011‐2.Inarelatedexercise,wealsoillustratehowourmethodscanbe

focussedonparticulartopicsorentities,suchas‘property’,thuspotentially

helpingtoshedlightonspecificsectorsoftheeconomy.

Togaugetherobustnessofouraggregatesentimentmetrics,wecompare

themwithbothwithstandardaggregatedmeasuresofconsumerconfidenceand

marketvolatilityandwithsomerelevantbutmoreatheoreticmeasuresof

uncertaintyfromtheliteratureexploitingtext‐basedinformation.Strikingly,we

findthatoursentimentmetricsoftenactasaleadingindicatorofsuchother

measuresandcanpotentiallyhelpustounderstandthem.

Financialbehaviourcanalsooftenbehomogenous.Therefore,inthesecond,

moreexploratory,partofthepaper,weaskwhetherwecanmeasurestructural

changesinthedistributionofnarratives.Specifically,wedevelopamethodology

tomeasure‘narrativeconsensus/disagreement’inthedistributionofnarratives

astheydevelopovertime.Thiscouldbearelevantmeasureoftheextentto

whichsomenarrativeshavebeensubjecttosocial‐psychologicalprocessesand

adoptedastrue(groupfeel(Tuckett,2011)).Forexample,priortotheglobal

financialcrisis,consensusappearedtodevelopacrossinvestorsbothabouta

newparadigminthefinancialsystemandinthebeliefthatitwaspossibleto

achievehigherreturnsthanpreviously–indeed,claimingtodosoarguably

becamenecessaryforfinancialinstitutionstoattractnewinvestment(Aikman

et.al.,2011).Butsuchconsensusinanenvironmentofhighsentimentcouldbe

suggestiveofover‐confidenceorirrationalexuberanceandthetheorypredicts

thatsuchsituationsarelikelytobeunsustainable.Theabilitytomeasurethe

emergenceofconsensusordisagreementwithintextdocumentscouldtherefore

proveusefulinidentifyingfinancialsystemrisks.

7

Usingournewlydevelopedmeasureofnarrativedispersal,wefindthat

consensusintheReutersnewsarticlesgrewsignificantlyoveraperiodspanning

severalyearspriortotheglobalfinancialcrisis.Whenviewedtogetherwiththe

sentimentseries,thiscouldbeindicativeofagrowing,predominantlyexcited

consensusaboutanewparadigminthefinancialsystem.Inotherwords,our

top‐downtextanalysismethodologysuggestsevidencethatconsensual,

convictionnarrativesemergedpriortothecrisisinwhichanxietyanddoubt

substantiallydiminished,indicativeofpossibleimpendingdistress.

Otherstudiesthatattempttoquantifysentimenthaveusedtext‐baseddata

sourcessuchascorporatereportsandnewsmediaanalysedwithmuchmore

generalwordliststocaptureemotion.Theyhaveattempted,forexample,to

predictvariousaspectsofassetprices(e.g.,LoughranandMacDonald,2011;

Tetlock2007;Tetlocketal.2008;Tetlock2011;Soo2013)ortocapture

economicpolicyuncertainty(Bakeret.al.,2013).Researchhasalsousedtext

datatoexploreopinionformationincentralbanks(Hansenetal,2014)orhow

thetoneandlanguageofstatementsbycentralbanksmayinfluencevariables

suchasinflationforecastsandinflationexpectations(seeforexampleBlinderet

al.,2008,Sturm&DeHaan,2011,Hubert2012).Morebroadly,thereisalsoa

widerliteratureonhowsentiment,ascapturedviasurveys,marketproxiesor

events,mayaffectfinancialmarketsandrelatedopiniondynamics(e.g.,Baker

andStein2004;BakerandWurgler2006,2007;Bakeretal.,2012;Brownand

Cliff,2005;Edmansetal.2007;Lux,2008;GreenwoodandNagel2009).

Ouremphasisdepartsfromtheaboveliteratureinseveralways.First,by

focusingonarestricteddictionaryofwordswedevelopourmeasuresof

sentimentfromthepointofviewofasocial‐psychologicaltheoryofactionunder

8

uncertainty(TuckettandNikolic,2016).Inthiswayweapplyatheoreticalfilter

whichshouldmoreaccuratelydetectfeatureswehypothesisetobeimportant

andavoidsomeofthedifficultiesassociatedwithdatamining.Whenprocessing

‘bigdata’thereisariskofobtainingseeminglysignificantcorrelationsthatdo

notgeneralise.Data‐miningtechniquesmayalsogeneralisepoorlytonewdata

sourcesandtobehighlycontextspecific.Secondourprimaryfocusisspecifically

ongaugingthesystemicrisk,ratherthanonmovementsinparticularasset

pricesorbroadermacroeconomicdevelopments.ConvictionNarrativeTheory

postulatesthatsystemicriskcanbegeneratedwhenmarketagentsgetcaptured

bygroupnarrativesthatopenupagapintheusualbalancebetweenexcitement

andanxietyinnarratives,suggestingthatsomekindof“thistimeisdifferent”

process(ReinhartandRogoff,2009)isgoingon(Tuckett,2011).Ifso,asevere

correctioncanbeexpectedtofollow.Third,muchcurrentresearchthatapplies

someformoftext‐basedsentimentanalysistostudytheeconomyorfinancial

marketstendstoexploitsocialmediagenerateddata.Bycontrast,wefocuson

datasourcesmorespecificallyconnectedtothefinancialsystem,includingone

sourcewrittenwithinacentralbank.

Section2explainsthedatawehaveanalysedandfocusesonourmeasureof

emotion,orsentiment,andexplainsthemethodology.Section3setsoutresults,

includingGrangercausalitytestsbetweentheemotionindicesandvarious

financial/economicindicators.Section4focusesonthemeasureof‘narrative

consensus’,explainingthemethodologyandresults.Section5discusseshow

thesemeasuresmightcomplementmoretraditionalindicatorsandanalysisused

insystemicriskassessment,andSection6concludes.Acertainamountof

technicalmaterialismadeavailableinanAppendix.Fulldetailsofallthe

9

statisticaltestsinvolvedinouranalysis,alongwithfurthertechnicalmaterial,is

inaSupplementaryMaterialdocument,availableonrequestfromtheauthors.

2. Data and Methodology 

Wemakeuseofavarietyofdatasourceswithamacroeconomicandfinancial

sectorfocus.

2.1 Bank of England internal market commentary 

TheMarketsDirectorateoftheBankofEnglandproducesarangeofinternal

reportsonfinancialmarketsandthefinancialsystem,someofwhichprovide

‘high‐frequency’commentaryoneventsandsomeofwhichprovidedeeper,or

morethematic,analysis.Forthisstudy,weanalysedsomedocumentsofthe

formerkind,morespecificallydailyreportsonthecurrentstateofmarkets,

giventhatforthekindofanalysisweemployhere,theidealtypeofdatashould

remainas‘raw’aspossibleinordernotto‘distort’themarketemotionsreflected

within.Thesedocumentsmainlycoverfinancialnewsandhowmarketsappear

torespondtosuchnews.Wethereforeexpectthesedocumentstocorrelatewell

withfinancialsentimentintheUKandpotentiallycontainusefulinformationon

systemicrisk.

Weanalyseonaverage26documentspermonthfromJanuary2000untilJuly

2010.Thedocumentsaretypicallyrelativelyshort,around2‐3pagesofemail

text.Fortherestofthepaper,werefertothesedocumentsas‘Market

CommentaryDaily(MCDAILY)’.

2.2 Broker reports

10

Brokerresearchreportsprovidealargesourceofdocumentsofclear

relevancetofinancialmarketsandthemacroeconomy.Weanalyseanarchiveof

14brokersfromJune2010untilJune2013,consistingofapproximately100

documentspermonth.Thedocumentsareverylong(upto50pagesinsome

cases),andsowepickuponalargenumberofwords.Visualinspectionofa

sampleofthesedocumentsrevealsthattheyprimarilyfocusonmacroeconomic

developmentsinthemajoreconomies.Wethereforeexpectthesentimentwithin

thesedatatocorrelatemoststronglywithmacroeconomicvariables.Throughout

therestofthepaper,werefertothisdatabaseas‘Brokerreport(BROKER)’.

2.3 Reuters News Archive 

Finally,weusetheThomson‐ReutersNewsarchive,asalsoextensively

studiedbyTuckettetal.(2015)toassessmacroeconomictrends.Atthetimeof

ouranalysis,thearchiveconsistedofover17millionEnglishnewsarticles.For

mostofthispaper,werestrictourattentiontonewspublishedbyReutersin

LondonduringtheperiodbetweenJanuary1996andSeptember2014,inwhich

6,123articleswerepublishedonaverageeachmonth(afterexcludingallarticles

taggedbyReutersas‘Sport’,‘Weather’and/or‘HumanInterest’).Fortherestof

thepaper,werefertothisdatabaseas‘Reuters(RTRS)’.

2.4  Relative Sentiment Shifts 

Asummarystatisticoftwoemotionaltraitsisextractedfromourtextdata

sourcesbyawordcountmethodologydescribedinmoredetailelsewhere

(Tuckett,SmithandNyman,2014).Twolistsofpreviouslyappliedand

experimentallyvalidatedwords(Strauss,2013),eachofapproximatelysize150,

11

areused,onerepresentingexcitementandonerepresentinganxiety.Random

samplesofthesewordscanbefoundinTable1.

Table1:Emotiondictionarysamples

Anxiety Excitement

Jitter Terrors Excited Excels Threatening Worries Incredible Impressively Distrusted Panics Ideal Encouraging Jeopardized Eroding Attract Impress

ForthesummarystatisticofacollectionoftextsT,wecountthenumberof

occurrencesofexcitementwordsandanxietywordsandthenscalethese

numbersbythetotaltextsizeasmeasuredbythenumberofcharacters.3To

arriveatasinglestatistic,highlyrelevanttothetheoryofconvictionnarratives,

wesubtracttheanxietystatisticfromtheexcitementstatistic,sothatanincrease

inthisrelativeemotionscoreisduetoanincreaseinexcitementand/ora

decreaseinanxiety.

| | | |

Wecomputethisonamonthlybasis.Asevidencedbythedefinitionofthe

measure,wedonotcontrolforpossiblenegationsofthesewords(e.g.‘not

anxious’).Wedidhowevercarryoutatest(ontheRTRSdatabase)ifthe

presenceofnegationwordswouldaffectthesentimentseries(followingthe

procedureoutlinedinLoughranandMcDonald,2011)byexcludingallwords

countedtoproducethesentimentscoreiftheywerepreceded(withinawindow

ofthreewords)byeitherofthewords:“no”,“not”,“none”,“neither”,“never”or

“nobody”.The‘negationaware’seriesthusproducedremainedcorrelatedwith3Insomecasesitcouldbemoresuitabletoscalebythenumberofdocuments.However,inthisparticularcase,somedocumentscontainedtablesandothersdidnot,sothenumberofcharactersisamoreappropriatechoice.

12

theoriginalseriesashighlyas0.99,bothinlevelanddifferenceform.Wealso

carriedoutatestoftherobustnessofthemethodologytoanalternative

selectionofsentimentwordlists.Weappliedthesentimentmethodologytothe

‘positive’and‘negative’wordlistsproducedbyLoughranandMcDonaldthatcan

bedownloadedfromtheweb4.TheseriesproducedusingtheRTRSdatabaseis

correlatedwiththeseriesproducedusingourwordlistsby0.84.

Thesimplicityofourmethodisintentionalfortwomainreasons.First,itis

naturaltoconsiderwhethersimpletext‐basedanalysiscanbeinformativebefore

movingtomorecomplexmethods.Second,italsoallowsforaneasier

assessmentoftherobustnessofthemethodology.Inparticular,weapplya

bootstraptechniquetocompute95%confidenceintervalsaroundthesummary

statistic.Wesamplenewweightsforeachwordineachdictionary(sothatthe

sumofweightsequalsthesizeofthedictionary)andre‐computethestatistic.

Repeatingtheproceduregivesadistributionfromwhichtoextractthe

confidenceintervals.5Thistechniquegivesusincreasedconfidencethatthe

meaningofindividualwordsinourtwolistsdoesnotchangeovertime.

3. Results

3.1  The evolution of measures of sentiment WeexploretherelativeemotionseriesextractedfromMCDAILYinFigure1,

annotatingthechartwithkeyeventsrelatingtofinancialstabilityforpurely

illustrativepurposes–inparticular,unlikeeventstudies,wedonottrytoinfer

anythingcausalfromtheeventsthatwedepictonthechartsThegraphmoves

4Thelistsweredownloadedfromthewebsiteastheywereavailablein2011,www3.nd.edu/~McDonald/word_lists.html5Itiseasytoimagineothermethodsofextractingconfidencelevels,e.g.,tosamplewithreplacementfromthecollectionoftexts.

13

broadlyasmightbeexpected.Inparticular,itshowsastableincreaseduringthe

mid‐2000s.Thisisfollowedbyalargeandrapiddeclinefrommid2007,muchof

whichoccursbeforethefailureofBearStearnsinMarch2008–strikingly,

althoughthiswasalreadyaperiodofturmoilinthefinancialsystem,theseries

hitsverylowlevelsbeforetheworstpartsofthecrisisataroundthetimeofthe

LehmanBrothersfailure.

Althoughconvictionnarrativetheoryessentiallyreferstotherelativelevelof

sentiment–excitementminusanxiety–itisalsointerestingtoconsiderthetwo

componentpartsseparately.Figure2showsthatthevariationinanxietylevels

ishigherthanthatinexcitementlevels.Thismayreflectthefactthatfear(ora

lackofit)tendstodrivemovementsinthefinancialsystem,consistentwith

heuristic‐basedapproachestoKnightianuncertainty.

Figure1:RelativesentimentofMCDAILY.They‐axisdisplaysthenormalizedvalueswith0meanand

standarddeviation1

14

Figure2:EmotionalfactorsofMCDAILY;anxiety(red)andexcitement(green).They‐axisdisplays

theindividualaggregatewordfrequenciesscaledbyvolume

Figure3:RelativesentimentofMCDAILY(black),RTRS(green)andBROKER(red).They‐axis

displaysthenormalizedvalueswith0meanandstandarddeviation1

TheMCDAILYseriesiscomparedwiththoseextractedfromtheothertwo

sources,namelyRTRSandBROKERinFigure3.Eachoftheseriesisnormalised

withmeanzeroandstandarddeviationof1tofacilitatecomparison.Thefigure

suggeststhattheseriesshareacommontrend.MCDAILYandBROKERaremore

volatilethanRTRS(duetoamuchlowernumberofstoriespermonth)and

BROKERwasavailabletousonamuchshorterhorizonthantheothertwo

15

archives.TheexactcorrelationsbetweentheseriesarereportedinTable2in

section3.2.

Figures4and5showthetwocomponentpartsofthesentiment,excitement

andanxiety,inRTRSandBROKERrespectively.Againmovementsinanxiety

appeartodrivemuchofthefluctuationinoverallsentiment.Weareprimarily

interestedintheaggregatedifferencebetweentheexcitementandanxiety

components,fortheoreticalreasons.However,theanalysisiscarriedoutusing

thetwocomponentsseparatelyandresultsarepresentedintheappendix.

Figure4:Excitement(green)andAnxiety(red)inRTRS.They‐axisdisplaystheindividualaggregatewordfrequenciesscaledbyvolume

16

Figure5:Excitement(green)andAnxiety(red)inBROKER.They‐axisdisplaystheindividualaggregatewordfrequenciesscaledbyvolume

Thusfarwehaveonlydiscussedhowthestatisticcanbeextractedfroma

genericcollectionoftexts,butitisalsoeasytofilterfortextsmatchingagiven

criteria,forexampletextsrelatingtoaparticulartopicorentity.Toexplorethe

potentialofsuchanapproach,wefilteredforthementionof'property'in

Reuters’newsarchive(Figure6)andthenrantherelativesentimentanalysis

onlyonthematchingsentenceswithinallarticles,withthenumberofsentences

reflectedinthebottompanel(recallthesearearticlespublishedinLondon).Itis

particularlyinterestingtonotethesteadyincreaseandlaterdeclineinvolumeof

articlesthatmatchedthepropertycriterianormedbythetotalnumberof

articlespublishedinLondon,theturningpointoccurringaroundthetimeofthe

bankruptcyofLehmanBrothers.Thepeakoftherelativesentimentseries

(whichhasherebeensmoothedwithparameter 0.3)appearstohave

occurredmuchbeforethis,towardstheendof2006,aftertheserieshad

undergoneasteadyincreaseforatleast4years(anxietyrelativetoexcitement

steadilydroppedoutofthediscussion).Therawrelativesentimentseries

correlateswithRTRSat0.57withnostatisticalevidenceofeitheraleadorlag.

17

Suchfocusedanalysiscouldpotentiallybeofvalueiftryingtomonitorthe

emergenceofexuberanceinpropertymarkets,orindeedchangesinrisk‐taking

sentimentinanyspecificsectoroftheeconomy.Thisparticularexampleseems

toindicatethatthepropertysectorbecameoverlyexuberantpriortothecrisis.

Figure6:Relativesentimentsurrounding'property'inRTRS.They‐axisdisplaysthenormalized

valueswith0meanandstandarddeviation1

3.2  Structural breaks  Importantly,boththeMCDAILYandRTRSshowsharpfallswellinadvanceof

thefinancialcrisis(weonlyhavedataonthethird,BROKER,from2010).For

example,themeanvalueofMCDAILYovertheboomperiodJuly2003through

June2007is0.916,withastandarddeviationof0.567.TheAugust2007value

fellto0.506,andinthesecondhalfof2007,themeanvaluewas0.691.In

January2008,however,therewasasharpfallto‐0.868,3.15standarddeviations

belowthemeanoftheJuly2003toJune2007period,andtheseriescontinuedto

fallwellinadvanceofthefailureofLehmanBrothers.

18

ThebreakintrendsintheRTRSserieswasevenearlier.OvertheJuly2003–

June2007period,thisaveraged1.083withastandarddeviationof0.472.As

earlyasJune2007theRTRSfellto‐0.399,3.14standarddeviationsbelowits

2003‐2007mean.ByAugust2007,itwas6.11standarddeviationsbelow.

Weconductasimpleformalstatisticaltestforstructuralbreaksinthe

sentimentseriesusingthemethodofBaiandPerron(2003).Thenumberof

breakpointsmisestimatedusingBayesianInformationCriterion(BIC)andtheir

positionsareestimatedbyminimisingtheresidualsumofsquaresofthem+1

resultinglinesegments.Wesetamaximumnumberofbreakpointstobefound

to5.ThisyieldsfourstructuralbreaksfortheRTRSseriesonAugust2000,May

2003,May2007andApril2010.WefindthreebreaksfortheMCDAILYserieson

April2003,November2004andDecember2007.Wefindtwobreaksforthe

BROKERseriesonMay2011andOctober2011.

3.3  Comparison with other measures 

Toillustratehowourmeasurescomparewithsomeothermeasuresof

uncertainty,Figure7showsMCDAILYplottedagainsttheVIX,withtheMCDAILY

variableinvertedforeaseofcomparison.Itisclearthatthemeasurestrackeach

19

otherclosely.

Figure7:RelativesentimentofMCDAILY,invertedforconvenience,(black)comparedtotheVIX

(yellow).They‐axisdisplaysthenormalizedvalueswith0meanandstandarddeviation1

Moregenerallywecanlookatcorrelationsbetweenvariables.Toexplorehow

ourmeasurescomparetoarangeofotherfinancialandeconomicindicators,we

conductedasimplecorrelationbasedpairwisecomparisonstudy.The

correlationswiththeMichiganConsumerSentimentindex6(MCI),theVIX7,the

economicpolicyuncertaintyindexofBakeretal.(2013)8(EPU),theBankof

Englandmacroeconomicuncertaintyindex(BoEU–seeHaddowetal.(2013)),

seniorCDSpremia9andPMI10togetherwiththecorrelationsbetweenthe

.6The MCI was created as a means to assess consumers’ ability and willingness to buy. Thesurveyiscarriedoutwithatleast500phoneinterviews,duringaperiodofaround2weeks, inwhich approximately50questionsare asked. Survey results are released twice eachmonthat10.00a.m.EasternTime:preliminary estimates arepublishedusually (variationsoccurduringthewinterseason)onthesecondFridayofeachmonth,andfinalresultsonthefourthFriday.7TheVIX,commonlyknownasthe‘fear’index,isameasureofimpliedvolatilityderivedfromthepriceofS&P500options.WeconsideranaverageofVIX,computedusingclosingpricesofalltradingdaysforagivenmonth.Thusmakingtheseriescomparabletotherelativesentimentseries,whicharealsomonthly‘averages’.8WeusetheUKversionoftheseriesavailableathttp://www.policyuncertainty.com/europe_monthly.html.TheseriesstartsinJanuary1997.9SeniorCDSpremiaaccessiblefromtheBankofEngland’ssetofcoreindicatorsoffinancialstabilityhttp://www.bankofengland.co.uk/financialstability/Pages/fpc/coreindicators.aspx.TheseriesstartsinJanuary2003.

20

individualrelativesentimentseriesarepresentedinTable2.11Tofacilitate

comparisonsandsincethesignofallcorrelationsisasexpected,wegivethe

absolutevaluesofthecorrelations.Itisclearthatourthreesentimentmeasures

arefairlyhighlycorrelatedwithalloftheothermeasures.

Table 2

Correlationsbetweenrelativesentimentseriesandcommonmeasuresof

sentiment,ignoringsigns

MCD RTRS BRO VIX MCI EPU BoEU CDS PMI

MCD 1 0.59 ‐ 0.65 0.26 0.43 0.54 0.67 0.38

RTRS ‐ 1 0.71 0.37 0.54 0.61 0.52 0.71 0.51

BRO ‐ ‐ 1 0.57 0.66 0.06 0.60 0.23 0.42

Lead‐lagcorrelationsbetweenthesentimentseriesandtheeconomicvariables

arepresentedinTable3aand3b.

Table 3a

Correlationsbetweenrelativesentimentseriesandcommonmeasuresof

sentiment,ignoringsigns(‐1ist‐1,+1ist+1)

VIX(‐1)VIX(+1)MCI(‐1)MCI(+1)EPU(‐1)EPU(+1)BoEU(‐1)BoEU(+1)

MCD(t)0.560.650.240.27 0.30 0.410.430.61

RTRS(t)0.26 0.370.49 0.58 0.63 0.630.350.67

BRO(t)0.340.650.34 0.87 0.260.010.060.76

Table 3b

10Businessexpectationssurvey(MarkitPMI).Basedonanswerstothequestionifbusinessactivityisexpectedtobehigher,lowerorstaythesamein12months.TheseriesstartsinApril1997.11Correlationsarecomputedonthefullavailablerangeofoverlappingdata..HereMCD=MCDAILYandBRO=BROKER.SincetheBoEUindexisaquarterlyserieswecreatequarterlyseriesofthethreesentimentindicatorsbyaveragingthevalueswithineachquarter.WedonotdothisfortheVIXandtheMCIasthatisoflessrelevancehere.

21

Correlationsbetweenrelativesentimentseriesandcommonmeasuresof

sentiment,ignoringsigns(‐1ist‐1,+1ist+1)

CDS(‐1) CDS(+1) PMI(‐1) PMI(+1)

MCD(t) 0.63 0.63 0.38 0.43

RTRS(t) 0.67 0.69 0.43 0.57

BRO(t) 0.05 0.22 0.04 0.42

Toformallytestpotentiallead‐lagrelationshipswereporttheresultsofGranger‐

causality tests between the three sentiment series and the various other

indicators considered above. We use the methodology described in Toda and

Yamamoto (1995), as described in theAppendix (section 2). Here,we simply

reportthefinalstepintheprocesswhichprovidestheevidenceontheexistence

orotherwiseofGrangercausality.

Wecarryouttestsusingtheaggregateversionsofeachofthesentimentseries

(ienetbalancebetweenexcitementandanxiety),aswellasthecomponentparts

of each series, namely excitement and anxiety. All tests are carried out on

unsmoothed relative sentiment series. Table 4 below shows results obtained

testing Granger‐causality from the various versions of the RTRS, BROKER and

MCDAILYvariablestoMCI,VIX,BoEU,EPU,CDSandPMI.Table5belowshows

results obtained testing Granger‐causality between the same variables in the

reversedirection,fromMCI,VIX,BoEU,EPU,CDSandPMItothevariousversions

oftheRTRS,BROKERandMCDAILYvariables.12

Table 4

12ThemissingentriesinbothtablescouldnotbedeterminedbecauseofsomeformofVARmisspecification

22

Waldtestp‐valuesofGranger‐causalityfromtherelativesentimentshiftseries

RTRS,BROKERandMCDAILY,andtheircomponentparts,toMCI,VIX,BoEU,

EPU,CDSandPMI

RSSSeries MCI VIX BoEU EPU CDS PMI

RTRSEXC‐ANX 0.005** 0.28 4e‐06** 0.3 0.0002**

RTRSEXC 0.032* 0.044* 0.0013** 0.03* 0.05* 0.05*

RTRSANX 0.003** 0.56 7e‐05** 0.1 0.0004**

MCDAILYEXC‐ANX 0.5 0.09 6e‐05** 0.05* 0.09 0.06

MCDAILYEXC 0.8 0.44 0.13 0.85 0.57 0.57

MCDAILYANX 0.8 0.38 0.001** 0.06 0.12 0.33

BROKEREXC‐ANX 2e‐11** 0.18 0.92 0.6 0.1

BROKEREXC 0.022* 0.84 0.77 0.43 0.82

BROKERANX 3e‐05** 0.12 0.72 0.68 0.03*

Note: *p<0.05;**p<0.01

Table5

Waldtestp‐valuesofGranger‐causalityfromMCI,VIX,BoEU,EPU,CDSandPMI

totherelativesentimentshiftseriesRTRS,BROKERandMCDAILY,andtheir

componentparts

RSSSeries MCI VIX BoEU EPU CDS PMI

RTRSEXC‐ANX 0.29 0.093 0.022* 0.57 0.08

RTRSEXC 0.39 0.22 0.038* 0.62 0.24 0.24

RTRSANX 0.03* 0.0013**0.21 0.85 0.12

MCDAILYEXC‐ANX 0.95 0.39 0.58 0.18 0.89 0.49

MCDAILYEXC 0.94 0.61 0.47 0.76 0.32 0.03*

MCDAILYANX 0.92 0.52 0.69 0.08 0.98 0.95

BROKEREXC‐ANX 0.94 0.16 0.73 0.97 0.41

BROKEREXC 0.22 0.084 0.72 0.001**0.72

BROKERANX 0.72 0.33 0.98 0.78 0.67

Note: *p<0.05;**p<0.01

23

There is some evidence of Granger causality fromour text‐based sentiment

measurestothemetricsweconsiderbutlesscausalityintheoppositedirection.

Inparticular,theRTRSmeasureissignificantinmanyofthetests.Aswemight

expect, RTRS and BROKER, sources more reflective of broad macroeconomic

commentary, appear to relate most closely to the MCI, which is the most

macroeconomicmeasureofcomparison.Bycontrast,MCDAILY,asourcewhich

reflects financialmarketcommentary,exhibitsmuch lowerp‐values inrelation

totheVIXandBoEUmeasures.

Aswellasbeingsuggestiveoftherobustnessandusefulnessofthese

measures,theseresultsareindicativeofthepotentialuseoftherelative

sentimentmeasuresasshort‐termforecastingdevices,aswellastheiruseto

gaugefuturefinancialmarketvolatility,consumerconfidenceandvarious

measuresofuncertainty.TheauthorsshowinNymanetal.(2014)howBROKER

canbeusedtopredict,out‐of‐sample,thechangefromthepreviousfinal

estimateoftheMCItothecurrentpreliminaryestimate.TheadjustedRsquared

ofthepredictionswhenregressedontheactualchangesis0.486comparedto

0.114fortheconsensusforecastsmadebyeconomistsandpublishedinReuters.

Theforecastsareunbiasedastheconstanttermisnotsignificantlydifferent

from0andthecoefficientonthepredictionsisnotsignificantlydifferentfrom1.

Figure8illustratesthedifferenceinaccuracyoftheforecastsmadeusing

BROKERandtheconsensusforecasts.

24

Figure8:ChangeinMCIcomparedtoforecastsofthechangemadeusingBROKERandconsensuseconomistforecasts

 3.4EffectofrelativesentimentontheUKeconomyWefurtherexploretherelationshipbetweenrelativesentimentandeconomic

activityinthecontextofvectorautoregression(VAR).Inthisexerciseweuse

theRTRSseriesextractedfromnewsfortworeasons:1)itisthelongestrelative

sentimentseriesofthethreeand2)emotionsexpressedingeneraleconomic,

financialandbusinessnewsisarguablymorelikelytoberelatedtoeconomic

activitythan,e.g.,financialcommentary.

VARmodelshavebeenusedtoestimatetheeffectofuncertaintyonthe

economy,e.g.Bloom(2009)andHaddow(2013).Itiscommonlyfoundthat

shockstosuch(proxy)measuresofuncertaintyhaveasignificantandnegative

impactoneconomicactivity.

ToestimatetheempiricaleffectofrelativesentimentontheUKeconomywe

usethemodelinHaddowetal.(2013)andreplacetheirmeasureofuncertainty

25

bytheUKRSSRTRSseries(aggregatedquarterlybyaveragingthemonthsina

quarter).ThemodelisspecifiedasthefollowingVAR(2):

,

where isthequarterlyrelativesentimentshiftseriesfortheUK, isthe

quarterlylevelofGDP(inlogdeviationsfromastatisticaltrend), isthe

quarterlylevelofemploymentinhoursworked(inlogdeviationsfroma

statisticaltrend), istheseasonallyadjustedleveloftheconsumerprice

index(inlogdeviationsfromastatisticaltrend), isthelevelofBankRateand

isanindicatorofcreditconditions.Ineachcase,thestatisticaltrendis

estimatedusingaHodrick‐Prescottfilterwithsmoothingparametersetto1600.

WeusethesamedataasusedtoestimatethemodelinHaddowetal.13

WefirstreplicatethemodelusingtheHaddowetal.uncertaintyvariableover

theperiod1989Q2–2012Q2andconfirmthataonestandarddeviationshockto

theuncertaintyindexdoeshaveasignificantandnegativeimpactonactivity.

Thisremainstrue(although,onlyjust)whenweshortentheperiodtostartin

1996Q1(thequarterfromwhichwehaverelativesentimentdatafortheUK).

Whenwereplacetheuncertaintyserieswiththerelativesentimentshift

series,wefindaqualitativelysimilarresponseofactivitytoaonestandard

deviationshock,butitisnotstatisticallysignificant.Figure9showsthe

respectiveimpulseresponsefunctionsofGDPtoshockstouncertaintyandRSS.

13The indexpre‐1995istakenfromFernandez‐CorugedoandMuellbauer(2006)andfrom1995onwardsitisaweightedaverageofinterestratesfacinghouseholdsforcreditcardloans,personalloansandmortgages

26

Note:MGDPrepresentsquarterlylevelofGDP,aslogdeviationsfromastatistical

trend.

However,theGrangercausalityresultsdescribedabovesuggestthatcausality

flowsfromRSStotheHaddowetal.uncertaintyvariable.Weconfirmthisinthe

contextoftheVARmodelsetoutabovebyextendingitincludingbothRSSand

theHaddowetal.uncertaintyvariableinthemodel.Weobservetheimpactof

eachvariableontheotherintheVARmodel.WefindasignificantimpactofRSS

onuncertaintythatpersistsforabout5‐6quartersbutnosignificantimpactin

thereversedirection.Thisresultremainstrueregardlessoftheorderofthetwo

variablesduringmodelestimation.Therespectiveimpulseresponsefunctions

canbeseeninFigure10.

Figure9:ImpactofonestandarddeviationshocksonGDPfromBoEU(left)andRTRSRSS(right)

27

Figure10:ImpactofonestandarddeviationshocksofuncertaintyonRSS(left)andviceversa(right)

Note:HOOrepresentstheuncertaintyvariableofHaddowetalandRSSthe

relativesentimentseries.

Thissuggeststhatitisinfactrelativesentimentthatdrivestheperceptionof

uncertaintyratherthananytrue‘probabilistic’uncertainty,ultimatelyslowing

downtheeconomy.Thismotivatesfurtherresearchintotheeffectsofnarratives

ontheeconomyandfinancialstability.

Wechecktherobustnessoftheabovestatementstotheorderingofthe

variablesintheVAR(placingthevariablesofinterestbothfirstandlastinthe

equation)andtotrending(checkingtheresultsbothwithandwithoutde‐

trendingtherelevantvariablesusingtheHodrick‐Prescottfilter).Therespective

impulseresponsefunctionscanbefoundinthesupplementarymaterial.

4. Measuring consensus 

Weturnnowtooursecond,moreexploratory,lineofinvestigation:canwe

measurestructuralchangesinthevariabilityofnarratives–inparticular,ata

givenpointintime,isthereconsensusoverparticularnarrativesorawide

28

dispersionofnarratives(disagreement)?Theobjectiveistoinvestigateifwecan

detectwhensomenarrativesgrowtobecomedominant,arguablytothe

detrimentofthesmoothfunctioningoffinancialmarketsandpotentiallyhinting

atimpendingdistressifalsoassociatedwithstronglypositiveaggregate

sentiment.Weintroduceanovelmethodologytoexplorethis.

4.1 Methodology 

ForthisinvestigationwefocusedonRTRSasitgenerallyseemstoperform

wellandhasalargersamplethantheothersources,whichishelpfulforthe

techniquesweapply.Tomeasureconsensus,wemakeuseofmodern

informationretrievalmethods.Themainchallengeistofindagoodmethodology

forautomatictopicdetection.Manysuchapproachesexistintheliterature

(Berry,2004),butwerelyonthestraightforwardapproachofclusteringthe

articlesinword‐frequencyspace(afterremovalofcommonplacewords)toform

topicgroups,wherebyeacharticlebelongstoasingledistincttopic.Wethen

measuretheuncertainty(entropy)inthedistributionofthearticlesacrossthe

topicgroups.Weconsideranincreaseintheuncertainty(entropy)ofthetopic

distributionasadecreaseinconsensusandviceversa.Thedetailsand

justificationoftheconstructioncanbefoundintheAppendix(section4).

4.2 Results 

WeplotthenarrativeconsensusfoundinRTRSinFigure11.Thegraphshows

aclearincreaseinconsensus(decreaseinentropy)precedingthecrisisperiod

andmuchmoredisagreementsubsequently.14Havingdecomposedthenarrative

discourseintooneindexmeasuringshiftsinemotion(theprevioussection)and14Wealsoinvestigatedtherobustnessofthenarrativeconsensusmetric,aprocesswhichinvolvessometechnicalanalysisessentiallybaseduponthedegreetowhichdocumentsatapointintimearesimilar.WedescribethisintheAppendix.

29

anothermeasuringstructuralchangesinconsensus(entropy),itappearsfrom

theseresultsthatapredominantlyexcitedconsensusemergedpriorthecrisis,

drivenbylowlevelsofanxiety.Thisseemsconsistentwiththeconvergenceof

beliefsontheideathatanewparadigmcoulddeliverpermanentlyhigher

returnsinthefinancialsystemthanpreviouslywithoutthreateningstability.

Withtheonsetofthecrisis,thiseventuallyshiftedintopredominantlyanxious

disagreement,asmightbeexpectedinanenvironmentoffearanduncertainty.

Interestingly,however,thenarrativeconsensusseriespeaksinmid‐2007,justas

anxietystartstodominate.Exploringsamplearticlesfromthelargesttopic

clusteratthistimerevealsacommonthemeaboutweakcreditconditionsand

economicuncertainty.

Table6showstheoutcomeofWaldtestsofGrangercausalitybetweenthe

entropyseriesandtheVIX,theBankofEnglandUncertaintyIndex(BoEU),the

EPU,CDSandPMI.Weagainusetheproceduredescribedinsection3.Thep‐

valuesofGranger‐causalityintheconversedirection,inthedirectionofthe

entropyseriesfromtheotherthreevariables,canbeseeninTable7.Theonly

significantdirectionsofGranger‐causalityarefromtheentropyvariabletothe

BoEUindexandfromtheentropyvariabletoEPU.

30

Figure11:Relativesentiment(black)andentropy(yellow)inReuters’Londonnews.They‐axis

displaysthenormalizedvalueswith0meanandstandarddeviation1

Table6Waldtestp‐valuesofGranger‐causalityfromEntropytoVIX,BoEU,EPU,CDSand

PMI

RSSSeries VIX BoEU EPU CDS PMI

Entropy 0.12 9.1e‐06** 0.03* 1.0 0.65

Note: *p<0.05;**p<0.01

Table7

Waldtestp‐valuesofGranger‐causalityfromVIX,BoEU,EPU,CDSandPMIto

Entropy

RSSSeries VIX BoEU EPU CDS PMI

Entropy 0.27 0.98 0.52 0.43 0.81

Note: *p<0.05;**p<0.01

Overall,theconsensusseriescapturesboththepresenceofpredominantly

excitedconsensusandpredominantlyanxiousconsensus.Thishighlightshow

31

thetwomeasures,ofemotionandnarrativeconsensus,mighttherefore

beneficiallybeinterpretedsidebyside.

5. Discussion 

Ourresultshighlighthowourmeasuresofsentimentandnarrativeconsensus

correlatewellwith,andinsomecaseseven‘cause’,certaineconomicand

financialvariables.Dependingonthetextsource,someperformbetterwith

financialvariables,somewithmacroeconomicvariables.Atalowerfrequency,

andwiththebenefitofhindsight,themetricsalsoappeartosignalrising

concernspriortotheglobalfinancialcrisis.Inthissection,wefocusonthe

potentialusesofindicatorsforemergingfinancialsystemstress,butwenotethat

thetextsourceslinkedmorecloselytomacroeconomicvariablescouldbeuseful

inforecastingor‘now‐casting’economicactivity(Tuckettetal,2015).

Therearemanydifferentapproachesforidentifyingandmodellingthreatsto

thefinancialsystem,includingtheuseofstresstests,earlywarningmodels,

compositeindicatorsofsystemicrisk,andMerton‐basedmodelsofsystemicrisk

thatusecontingentclaimsanalysis.15Manyauthoritiesuseindicatordashboards

orcobwebs,includingtheEuropeanSystemicRiskBoard,theOfficeofFinancial

ResearchintheUnitedStates,theWorldBank,theReserveBankofNewZealand

andtheNorgesBank.16IntheUnitedKingdom,theFinancialPolicyCommittee

15SeeAikmanetal.(2009),Burrowsetal.(2012)andKapadiaetal.(2013)foradiscussionoftheBankofEngland’sapproachtostresstestingandits“RAMSI”model.Onearlywarningindicatormodels,seeKaminskyandReinhart(1999),Drehmannetal.(2011),BorioandLowe(2002,2004),Barrelletal.(2010)andGieseetal(2014).Oncompositeindicatormodels,seeIllingandLiu(2006)andHollóetal.(2012).Oncontingentclaimsmodels,seeGrayetal.(2008)andGrayandJobst(2011).16FortheUS,seesection3oftheOFRAnnualReport(2012).TheESRB’sRiskDashboard’ispublishedontheweb(seehttp://www.esrb.europa.eu/pub/rd/html/index.en.html).OnthecobwebapproachusedinNewZealandandNorway,seeBedfordandBloor(2009)andDahletal.(2011),respectively.

32

routinelyreviewsasetofcoreindicatorswhichhavebeenhelpfulinidentifying

emergingriskstofinancialstabilityinthepast,andwhichthereforemightbe

usefulindetectingemergingrisks(BankofEngland,2014).17

Recognisingthatnosinglesetofindicatorsormodelscaneverprovidea

perfectguidetosystemicrisk,duetothecomplexityoffinancialinterlinkages

andthetendencyforthefinancialsystemtoevolveovertime,andtimelags

beforerisksbecomeapparent,judgementalsoplaysacrucialroleinspecifying

anypoliciestotacklethreatstothefinancialsystem.Andqualitative

information,includingfrommarketandsupervisoryintelligencetypicallyalso

helpstosupportsuchjudgements.

Aswehaveshowninprevioussections,ourmeasuresofsentimentand

consensus,extractedfromtext‐basedinformation,appeartobeinformativeof,

episodesofemergingsystemicriskandhighmarketvolatility.Assuch,they

offerapotentialmechanismforextractingquantitativemetricsfromqualitative,

text‐basedinformationthatisusedtoinformpolicymakingandmighttherefore

beonecomponentofindicatordashboards,complementingotherapproaches

usedtodetectsystemicrisk.Thesemeasurescouldalsobecalculatedonareal‐

timebasis,offeringthemanimportantadvantageoversomemoreconventional

indicators.Arguably,theyarealsolikelytobemorerobusttotheLucas(1976)

critiquebecausethewritersofindividualdocumentsareveryunlikelyto

respondcollectivelybyadaptingtheirwritingtonesorstylesbecausean

indicatorbasedonvastnumbersofdocumentsisusedasoneguideforhelping

tosetpolicy.

17SeealsoGieseetal.(2014).

33

Atthesametime,itisclearlyimportanttotesttheseindicatorsfurther.For

example,whichparticulartext‐basedsourcesshouldbethefocusofattention,

howgoodarethemetricsindistinguishingsignalfromnoise,andhowdothey

comparewithmoreconventionalindicatorsinthisrespect?Weleavethese

questionsforfurtherwork.

6. Conclusion 

Inthispaper,wehaveexploredthepotentialofusingalgorithmictext

analysis,appliedthroughthelensofconvictionnarrativetheory,toextract

quantitativesummarystatisticsfromnoveldatasources,whichhavelargelyonly

beenusedqualitativelythusfar.Wehavedemonstratedthattheoutcomeofsuch

procedurescanleadtosomeintuitiveandusefulrepresentationsoffinancial

marketsentiment.Theshiftscorrelatewellwithfinancialmarketeventsand

appeartoleadanumberoffinanciallyorientedeconomicindicators.

Wehavealsodevelopedanovelmethodologytomeasureconsensusinthe

distributionofnarratives.Thismetriccanpotentiallybeusedtomeasure

homogenisationamongmarketparticipants.Greaterconsensus,whenviewed

togetherwithanincreaseintherelativesentimentseries,mayalsobe

interpretedasanincreaseofpredominantlyexcitedconsensusofnarratives

priortotheglobalfinancialcrisis.Thus,weappeartohavefoundnovelempirical

evidenceofgroupfeelandthebuild‐upofsystemicbehaviourleadinguptothe

financialcrisis.

Overall,therelativesentimentandconsensussummarystatisticsdeveloped

maybeusefulingaugingriskstofinancialstabilityarisingfromthecollective

behaviourdiscussed.Whilefurtherworkisneededtorefinethesemetrics,

34

includinginrelationtoboththemethodsandthedatainputsused,theyhavethe

potentialtoprovideausefulquantitative,analyticalperspectiveontext‐based

marketinformationwhichcouldhelptocomplementmoretraditionalindicators

ofsystemicrisk.

Acknowledgments  

TheauthorswouldliketothankJonRandattheBankofEngland[andothers]

forsupportingtheprojectandprovidingaccesstodata.Theauthorswouldalso

liketothankChrystiaFreeland,RichardBrownandMaciejPomaleckiof

ThomsonReutersforarrangingaccesstotheReutersNewsarchive.

DavidTuckett’sinvolvementinthisworkwasmadepossiblethroughInstitute

ofNewEconomicThinkingGrantnoIN01100025andagrantfromtheEric

SimenhauerTrustofTheInstituteofPsychoanalysis.

ThanksarealsoduetoOliverBurrows,AndyHaldane,PhilipTreleavenand

KimberlyChongforadviceandsupportaswellastocolleaguesinthePhD

FinancialComputingCentre,thePsychoanalysisUnitatUCLandtotheAnna

FreudCentre.Wehavealsobenefittedgreatlyfromdiscussionattheconferences

andothermeetingsatwhichtheworkhasbeenpresented.

Appendix1. Wordlists 

Table A1 contains a random sample of 40 anxiety words and 40 excitement words. 

Note that when the same word is spelled differently in American and British English 

we have included both variants in the list. 

Table A1 

35

Randomly Drawn Selection of Words indicating excitement (about gain) and anxiety 

(about loss)  

Anxiety    Anxiety    Excitement    Excitement 

Jitter      Terrors     Excited     Excels Threatening    Worries    Incredible     Impressively Distrusted    Panics      Ideal      Encouraging Jeopardized    Eroding    Attract     Impress Jitters      Terrifying    Tremendous     Favoured Hurdles    Doubt      Satisfactorily    Enjoy Fears      Traumatised    Brilliant    Pleasures Feared     Panic      Meritorious    Positive Traumatic    Imperils    Superbly    Unique Fail      Mistrusts    Satisfied    Impressed Erodes     Failings    Perfect     Enhances   Uneasy     Nervousness    Win      Delighted Distressed    Conflicted    Amazes    Energise Unease    Reject      Energizing    Spectacular Disquieted    Doubting    Gush      Enjoyed Perils      Fearing    Wonderful    Enthusiastic Traumas    Dreads     Attracts    Inspiration Alarm      Distrust    Enthusiastically  Galvanized Distrusting    Disquiet    Exceptionally    Amaze Doubtable    Questioned    Encouraged    Excelling 

 

2. Granger causality procedure

We use  the methodology described  in Toda and Yamamoto  (1996).    In outline,  in 

investigating Granger causality between any two series, this is as follows: 

1. Check  the order of  integration of  the  two  series using Augmented Dickey‐

Fuller (Said and Dickey 1984; p‐values are interpolated from Table 4.2, p. 103 

of  Banerjee  et  al.  1993)  and  the  Kwiatowski‐Phillips‐Schmidt‐Shin  (1992) 

tests. Let m be the maximum order of integration found.   

36

2. Specify the VAR model using the data in levelled form, regardless of what was 

found  in  step  1,  to  determine  the  number  of  lags  to  use  with  standard 

method. We use the Akaike Information Criteria  

3. Check the stability of the VAR (we use OLS‐CUSUM plots) 

4. Test for autocorrelation of residuals. If autocorrelation is found, increase the 

number of lags until it goes away.  We use the multivariate Portmanteau‐ and 

Breusch‐Godfrey  tests  for serially correlated errors. Let p be  the number of 

lags then used. 

5. Add m extra lags of each variable to the VAR. 

6. Perform Wald  tests with null being  that  the  first p  lags of  the  independent 

variable have coefficients equal to 0. If this  is rejected, we have evidence of 

Granger‐causality from the independent to dependent variable.  

 

We used the statistical program R to carry out the analysis, and the various packages 

used  to  carry  out  the  above  Toda‐Yamamoto  procedure  are  documented  in  the 

Supplement.  The  details  of  the  specific  results  obtained,  and  a  description  of  the 

various R packages, using the test procedure are available on request in the form of 

a Supplementary Material document. 

 

3. Narrative Consensus   

3.1 Constructing the Narrative Consensus series 

 We proceed as follows, following well‐established methods: 

37

1. Pre‐process all documents by representing them as ‘bags‐of‐words’ in which 

word order is ignored and word‐endings are removed using a standard 

English word stemmer, known as the Porter stemmer (Porter, 1980) 

2. Compute a word by document frequency matrix, with words as rows and 

documents  documents as columns (each entry ij is the frequency of word I in 

document j) 

3. Remove uninformative rows (words at the extremes of the total word 

frequency distribution). We remove words at the top of the cumulative 

distribution (the smallest number of words accounting for a fixed percentage 

of the total word count) and at the bottom (the largest number of words 

accounting for a fixed percentage of the total word count) as the most 

frequent words rarely help us distinguish between topics and the least 

frequent words typically introduce too much noise and fail to show 

consistent patterns. Another commonly used technique is to remove all 

words in a predefined list, so called ‘stopwords’.  

4. Reduce the dimensionality of the document vectors (columns), to d 

dimensions, by the use of Singular Value Decomposition (SVD). In the 

information retrieval literature, the method we use is referred to as Latent 

Semantic Analysis (Deerwester, 1988) and has proved highly successful in a 

wide range of applications. LSA is naturally able to model important language 

structures, such as the similarity between synonyms. 

5. Cluster the document vectors. The clustering algorithm must automatically 

determine the number of clusters used to model the data. There are several 

such algorithms; we pick an extension of the popular K‐means algorithm 

38

known as X‐means (Pelleg and Moore, 2000), which iteratively decides 

whether or not to split one cluster into two using the Bayesian Information 

Criteria (BIC) as a measure of model fit. BIC measures how well a model fits 

data by the level of observed noise given the model while penalised linearly 

for the number of model parameters (i.e., penalising over‐fitting). 

This procedure gives us a distribution of the number of documents in each 

cluster, e.g., 1000 articles on sovereign debt, 100 articles on crude oil, etc., and the 

total number of clusters found.  

Using this distribution, we want our measure of consensus to have two intuitive 

properties: 

If the number of topics (clusters) is reduced while the size of each cluster is 

held fixed and equal ‐ consensus should increase. 

If given a fixed number of topics, any particular topic grows in proportion to 

the others ‐ consensus should increase. 

A measure of the topic distribution, which would give us these properties, is 

information entropy (Shannon, 1948).  

 

Definition: Discrete Entropy 

For a discrete distribution, such as in our particular case, the entropy is simply a 

logarithmically weighted sum of probabilities, 

log log log1

log , 

where   is the number of articles in cluster  ,   is the total number of articles and   

is the number of clusters.  

39

The entropy is maximised (for a fixed number of clusters ) when documents are 

uniformly distributed over the clusters. As the distribution moves away from 

uniformity the entropy will decrease. To better understand how entropy changes 

with k (the number of found clusters), we can simplify the equation as follows (if we 

assume a uniform distribution of documents across the clusters), 

log1

log log log log . 

It is clear from this that entropy is increasing logarithmically as k increases. In 

other words, the entropy is like an inverse consensus measure. Thus, if a narrative 

grows to dominate the news, for example narratives such as sovereign debt, 

structured finance or housing, the narrative entropy will decrease showing an 

increase in consensus. Similarly, if the total number of narratives decrease, all else 

fixed, consensus will increase (again signified by a decrease in narrative entropy).  

We smooth the result using a method known as double exponential smoothing. 

Double exponential smoothing is often chosen as an alternative to the simple single 

exponential smoothing when it is believed that the underlying data contains a trend 

component. 

Definition: Double Exponential Smoothing 

Given series  …  we decompose it into a smoothed series   and a trend 

component   by the procedure 

, 2

 

For  0: 

1  

1  

40

for some  , ∈ 0,1 . In our case we discard   after using it to estimate the 

smoothed series  . 

We run the algorithm across several choices of parameters (the list of model 

parameters, and combinations used (e.g., ‘40, 5, 100’ and ’50, 2, 100’), can be found 

in Table A1) and smooth (using double exponential smoothing, with  0.3) 

and average the results across parameter runs to arrive at the yellow line in Figure 9.  

Table A2: Consensus parameters 

Table A2 Consensus parameter combinations used to construct the Narrative ENTROPY index 

 

  Parameter    Values Considered 

Upper word bound    40,50,50,60,40,50,50,60 Lower word bound    5,2,10,10,5,2,10,10 Vector Dimensionality   100,100,100,100,200,200,200,200 

Note: the considered values were combined in the ordered they are listed, i.e. (40, 5, 100), (50, 2, 100), etc.    

3.2 Constructing Narrative Consensus proxy measures 

To investigate the robustness of the narrative consensus metric we devise two 

further methodologically distinct approaches to capture proxies for narrative 

consensus.  

1. Average document ‘overlap’ 

2. Average document ‘similarity’ 

We compute (1) from the word‐by‐document frequency matrix (after removing the 

‘uninformative’ words) by simply dividing the number of non‐zero entries in the 

matrix by the total number of entries, giving us a comparable time series (in which 

higher document overlap is a proxy for higher narrative consensus). We compute (2) 

by repeatedly sampling pairs of document vectors and computing the angle between 

41

them (a standard similarity metric for document vectors). We repeat the procedure 

1000 times and compute the mean angle. In this case, a mean angle closer to zero is 

a proxy for higher narrative consensus.

References 

Aikman, D, Alessandri, P, Eklund, B, Gai, P, Kapadia, S, Martin, E, Mora, N, Sterne, G and Willison, M (2009), ‘Funding liquidity risk in a quantitative model of systemic stability’, Bank of England Working Paper No. 372. Aikman,D,Nelson,BandTanaka,M(2015),‘Reputation,risk‐takingandmacroprudentialpolicy’,JournalofBankingandFinance,50,pages428‐439.

BaiJ.,PerronP.(2003),ComputationandAnalysisofMultipleStructuralChangeModels,JournalofAppliedEconometrics,18,1‐22.

Baker,M.,Stein,J.S.(2004).MarketLiquidityasaSentimentIndicator.JournalofFinancialMarkets,7(3):pp.271‐299.

Baker,M.,Wurgler,J.(2006).InvestorSentimentandtheCross‐SectionofStockReturns.TheJournalofFinance,61(4):pp.1645‐1680.

Baker,M.,Wurgler,J.(2007).InvestorSentimentintheStockMarket.TheJournalofEconomicPerspectives,21(2):pp.129‐151.

Baker,M.,Wurgler,J.,Yuan,Y.(2012).Global,LocalandContagiousInvestorSentiment.TheJournalofFinancialEconomics,104(2):pp.272‐287.

Baker,S.R.,Bloom,N.,Davis,S.J.(2013).MeasuringEconomicPolicyUncertainty.ChicagoBoothResearchPaperNo13‐02.AvailableatSSRN:http://ssrn.com/abstract=2198490

BankofEngland(2014),‘TheFinancialPolicyCommittee’spowerstosupplementcapitalrequirements:aPolicyStatement’,January.

Barrell,R.,E.Davis,D.Karim,andI.Liadze(2010).“Bankregulation,propertypricesandearlywarningsystemsforbankingcrisesinOECDcountries”,JournalofBankingandFinance,vol.34,no.9,pp.2255‐2264.

Barsalou,L.W.(2008).Groundedcognition.Annu.Rev.Psychol.,59,617‐645.

Baumeister,R.F.,&Masicampo,E.J.(2010).Consciousthoughtisforfacilitatingsocialandculturalinteractions:Howmentalsimulationsservetheanimal‐cultureinterface.PsychologicalReview,117,945‐971.

Berry,M.W.(2004).Surveyoftextmining:Clustering,classification,andretrieval.Springer.

Blinder,A.S.,Ehrmann,M.,Fratzscher,M.,DeHaan,J.,Jansen,D.J.,(2008).CentralBankCommunicationandMonetaryPolicy:ASurveyofTheoryandEvidence.EuropeanCentralBank,WorkingPaperSeriesNo898.

42

Borio,C.,andP.Lowe(2002).“Assessingtheriskofbankingcrises”,BISQuarterlyReview,December,pp43‐54.

Borio,C.,andP.Lowe(2004).Securingsustainablepricestability.Shouldcreditcomebackfromthewilderness?,BISWorkingPapers,no.157.

Brown,G.W.,Cliff,M.T.,(2005).InvestorSentimentandAssetValuation.TheJournalofBusiness,78(2):405‐440.

Bruner, J. (1991). The Narrative Construction of Reality. Critical Inquiry, 1-21.

Burrows,O,Learmonth,DandMcKeown,J(2012).RAMSI:atop‐downstress‐testingmodel.BankofEnglandFinancialStabilityPaperNo.17.

Choi,H.,andVarian,H.(2012).PredictingthePresentwithGoogleTrends.EconomicRecordSpecialIssue:SelectedPapersfromthe40thAustralianConferenceofEconomists.Volume88,IssueSupplements1pp.2–9DOI:10.1111/j.1475‐4932.2012.00809.x

Chong,K.andTuckett,D.(2014)ConstructingConvictionthroughActionandNarrative:HowMoneyManagersManageUncertaintyandtheConsequencesforFinancialMarketFunctioning.Socio‐EconomicReview.1‐26.doi:10.1093/ser/mwu020

DamasioA,CarvalhoG.B.(2013)Thenatureoffeelings:evolutionaryandneurobiologicalorigins.NatureReviews.Neuroscience[2013,14(2):143‐152]

Deerwester,S.etal.(1988).Improvinginformationretrievalwithlatentsemanticindexing.In:Proceedingsofthe51stAnnualMeetingoftheAmericanSocietyforInformationScience,Vol.25,36‐40.

Drehmann,M,Borio,CandTsatsaronis,K(2011),‘Anchoringcountercyclicalcapitalbuffers:theroleofcreditaggregates’,InternationalJournalofCentralBanking,Vol.7,No.4,pages189–240.

Edmans,A.,Garcia,D.,Norli,A.(2007).SportsSentimentandStockReturns.TheJournalofFinance,62(4):1967‐1998.

Giese,J,Andersen,H,BushO,Castro,C,Farag,MandKapadia,S(2014),‘Thecredit‐to‐GDPgapandcomplementaryindicatorsformacroprudentialpolicy:evidencefromtheUK’.InternationalJournalofFinanceandEconomics,19(1),25‐47.

Gray,D.F.,R.C.Merton,andZ.Bodie(2008).Anewframeworkformeasuringandmanagingmacrofinancialriskandfinancialstability,HarvardBusinessSchoolWorkingPaperno.09‐015.

Gray,D.F.,andA.Jobst(2011).“Modelingsystemicfinancialsectorandsovereignrisk”,SverigesRiksbankEconomicReview,September.

Granger,C.W.J.(1969).InvestigatingCausalRelationsbyEconometricModelsandCross‐spectralMethods.Econometrica37(3):pp.424‐438.DOI:10.2307/1912791.

Greenwood,R.,S.Nagel(2009).InexperiencedInvestorsandBubbles.JournalofFinancialEconomics,93(2):239‐258.

43

Haddow,A,Hare,C,Hooley,JandShakir,T(2013),‘Macroeconomicuncertainty;whatisit,howcanwemeasureitandwhydoesitmatter?’,BankofEnglandQuarterlyBulletin,Vol.53,No.2,pages100‐109.

Hansen,S,McMahon,MandPrat,A(2014),“TransparencyandDeliberationwithintheFOMC:aComputationalLinguisticsApproach,”CEPRDiscussionPaper9994.

Holló,D.,M.Kremer,andM.LoDuca(2012).CISS‐acompositeindicatorofsystemicstressinthefinancialsystem,EuropeanCentralBankWorkingPaper,no.1426.

Hubert,P.(2012).DisentanglingQualitativeandQuantitativeCentralBankInfluence.PreliminaryVersion.

Illing,M,andLiu,Y(2006).“Measuringfinancialstressinadevelopedcountry:anapplicationtoCanada”,JournalofFinancialStability,vol.2,no.4,pp.243‐265.

Kaminsky,G.LandReinhart,C.M(1999)Thetwincrises:thecausesofbankingandbalance‐of‐paymentsproblems.AmericanEconomicReview,vol.89,no.3,June,pp.473‐500.

Kapadia,S.,M.Drehmann,J.Elliott,andG.Sterne(2013).“Liquidityrisk,cashflowconstraints,andsystemicfeedbacks”,inJ.G.Haubrich,andA.W.Lo(eds.),QuantifyingSystemicRisk,UniversityofChicagoPress.

Kohonen,Teuvo(1982)."Self‐OrganizedFormationofTopologicallyCorrectFeatureMaps".BiologicalCybernetics43(1):pp.59–69.

Knight,F.H.(1921).Risk,UncertaintyandProfit.Boston,MA:Hart,Schaffner&Marx;HoughtonMifflinCompany.

Lazer,David,RyanKennedy,GaryKing,andAlessandroVespignani(2014).TheParableofGoogleFlu:TrapsinBigDataAnalysis.Science343,no.14March:1203‐1205.Copyathttp://j.mp/1ii4ETo

Loughran,TimandBillMcDonald,2011,“WhenisaLiabilitynotaLiability?TextualAnalysis,Dictionaries,and10‐Ks,”JournalofFinance,66:1,35‐65.

Lucas,Robert(1976)."EconometricPolicyEvaluation:ACritique".InBrunner,K.;Meltzer,A.ThePhillipsCurveandLaborMarkets.Carnegie‐RochesterConferenceSeriesonPublicPolicy1.NewYork:AmericanElsevier.pp.19–46

Lux,T.(2008).SentimentDynamicsandStockReturns:TheCaseoftheGermanStockMarket.KielWorkingPapersNo1470.

Miller,G.A.(1956),TheMagicalNumberSeven,PlusorMinusTwo:SomeLimitsonourCapacityforProcessingInformation.PsychologicalReview,63,81‐97.

Nyman,R.,Ormerod,P.,Smith,R.,Tuckett,D.(2014).BigDataandEconomicForecasting:aTop‐DownApproachusingDirectedAlgorithmicTextAnalysis.Workshoponusingbigdataforforecastingandstatistics.EuropeanCentralBank.7‐8April,2014.

OFRannualreport(2012)https://financialresearch.gov/annual‐reports/files/office‐of‐financial‐research‐annual‐report‐2012.pdf

Ormerod,P.,Nyman,R.,Bentley,R.A,(2014).Nowcastingeconomicandsocialdata:whenandwhysearchenginedatafails,anillustrationusingGoogleFlu

44

Trends.arXiv:1408.0699[physics.soc‐ph]

Pelleg,D.,Moore,A.(2000).X‐means:ExtendingK‐meanswithEfficientEstimationoftheNumberofClusters.ProceedingsoftheSeventeenthInternationalConferenceonMachineLearning.Pp.727‐734.

Porter,M.F.(1980).AnAlgorithmforSuffixStripping.Program,14(3):pp.130‐137.

Pribram,KarlH.;Miller,GeorgeA.;Galanter,Eugene(1960).Plansandthestructureofbehavior.NewYork:Holt,RinehartandWinston

Reinhart,C,andRogoff,K(2009).Thistimeisdifferent:EightCenturiesofFinancialFolly,PrincetonUniversityPress.

Rottman,M.B.,andHastie,R.(2013)ReasoningAboutCausalRelationships:InferencesonCausalNetworks.PsychologicalBulletin.PsycholBull.2014Jan;140(1):109–139.doi:10.1037/a0031903

Shannon,ClaudeE.(1948).AMathematicalTheoryofCommunication.BellSystemTechnicalJournal27(3):379–423.

Sloman,S.AandLagnado,D(2015)CausalityinThought.Annu.Rev.Psychol.2015.66:3.1–3.25

Soo,C.K.(2013).QuantifyingAnimalSpirits:NewsMediaandSentimentintheHousingMarket.TheStephenM.RossSchoolofBusinessattheUniversityofMichiganResearchPaperSeries,Forthcoming.AvailableatSSRN:http://ssrn.com/abstract=2330392

Sturm,J.E.,DeHaan,J.,(2011).Doescentralbankcommunicationreallyleadtobetterforecastsofpolicydecisions?NewevidencebasedonaTaylorrulemodelfortheECB.RevWorldEcon.DOI10.1007/s10290‐010‐0076‐4

Strauss,Viktor,M.(2013).EmotionalValuesofWordsinFinance:AnxietyaboutLossesandExcitementaboutGains.M.Sc.thesisinSocialCognition,UniversityCollegeLondon.

Suddendorf,T.&Corballis,M.C.2007Theevolutionofforesight:whatismentaltimetravelandisituniquetohumans?Behav.BrainSci.30,299–313.(doi:10.1017/S0140525X07001975)

Tetlock,PaulC.2007.“GivingContenttoInvestorSentiment:TheRoleofMediaintheStockMarket.”TheJournalofFinance,62(3):pp.1139–1168.

Tetlock,PaulC.,MaytalSaar‐Tsechansky,andSofusMacskassy.2008.“MorethanWords:QuantifyingLanguagetoMeasureFirms’Fundamentals.”TheJournalofFinance,63(3):pp.1437–1467.

Tetlock,P.C.(2011).AlltheNewsThat’sFittoReprint:DoInvestorsReacttoStaleInformation?ReviewofFinancialStudies24,pp.1481‐1512.

Toda,H.Y.andT.Yamamoto(1995).StatisticalInferenceinVectorAutoregressionswithPossiblyIntegratedProcesses,JournalofEconometrics,Vol.66,pp.225‐250

Tuckett,D.(2011).MindingtheMarkets:AnEmotionalFinanceViewofFinancialInstability.PalgraveMacmillan.

45

Tuckett,D.AandNikolic,M.,(2016)TheRoleofConvictioninDecision‐MakingUnderRadicalUncertainty.PresentedtotheInternationalConferenceonThinking,BrownUniversityRhodeIsland,August4,2016.Availableat:https://www.researchgate.net/publication/307905267_The_Role_of_Conviction_and_Narrative_in_Decision_Making_under_Radical_Uncertainty

Tuckett,D.,Smith,R.E.,&Nyman,R.(2014).Trackingphantasticobjects:Acomputeralgorithmicinvestigationofnarrativeevolutioninunstructureddatasources.SocialNetworks,38,121–133.doi:10.1016/j.socnet.2014.03.001

Tuckett,D.,Ormerod,P.,Nyman,R.,andSmithR.E.(2015)InformationAndEconomics:ANewWayToThinkAboutExpectationsAndToImproveEconomicPrediction.http://206.191.128.204/ideas‐papers/research‐papers/information‐and‐economics‐a‐new‐way‐to‐think‐about‐expectations‐and‐to‐improve‐economic‐predictionPaperpresentedattheOECD/INETconference,Paris,April.