Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 150–155,Jeju, Republic of Korea, 8-14 July 2012. c©2012 Association for Computational Linguistics

Humor as Circuits in Semantic Networks

Igor LabutovCornell University

iil4@cornell.edu

Hod LipsonCornell University

hod.lipson@cornell.edu

Abstract

This work presents a first step to a general im-plementation of the Semantic-Script Theoryof Humor (SSTH). Of the scarce amount ofresearch in computational humor, no researchhad focused on humor generation beyond sim-ple puns and punning riddles. We proposean algorithm for mining simple humorousscripts from a semantic network (Concept-Net) by specifically searching for dual scriptsthat jointly maximize overlap and incongruitymetrics in line with Raskin’s Semantic-ScriptTheory of Humor. Initial results show that amore relaxed constraint of this form is capableof generating humor of deeper semantic con-tent than wordplay riddles. We evaluate thesaid metrics through a user-assessed quality ofthe generated two-liners.

1 Introduction

While of significant interest in linguistics and phi-losophy, humor had received less attention in thecomputational domain. And of that work, most re-cent is predominately focused on humor recognition.See (Ritchie, 2001) for a good review. In this pa-per we focus on the problem of humor generation.While humor/sarcasm recognition merits direct ap-plication to the areas such as information retrieval(Friedland and Allan, 2008), sentiment classifica-tion (Mihalcea and Strapparava, 2006), and human-computer interaction (Nijholt et al., 2003), the ap-plication of humor generation is not any less sig-nificant. First, a good generative model of humorhas the potential to outperform current discrimina-tive models for humor recognition. Thus, ability to

!

Figure 1: Semantic circuit

generate humor will potentially lead to better humordetection. Second, a computational model that con-forms to the verbal theory of humor is an accessi-ble avenue for verifying the psycholinguistic theory.In this paper we take the Semantic Script Theoryof Humor (SSTH) (Attardo and Raskin, 1991) - awidely accepted theory of verbal humor and build agenerative model that conforms to it.

Much of the existing work in humor generationhad focused on puns and punning riddles - hu-mor that is centered around wordplay. And whilemore recent of such implementations (Hempelmannet al., 2006) take a knowledge-based approach thatis rooted in the linguistic theory (SSTH), the con-straint, nevertheless, significantly limits the poten-tial of SSTH. To our knowledge, our work is the firstattempt to instantiate the theory at the fundamentallevel, without imposing constraints on phonologicalsimilarity, or a restricted set of domain oppositions.

150

1.1 Semantic Script Theory of Humor

The Semantic Script Theory of Humor (SSTH) pro-vides machinery to formalize the structure of mosttypes of verbal humor (Ruch et al., 1993). SSTHposits an existence of two underlying scripts, one ofwhich is more obvious than the other. To be humor-ous, the underlying scripts must satisfy two condi-tions: overlap and incongruity. In the setup phase ofthe joke, instances of the two scripts are presentedin a way that does not give away the less obviousscript (due to their overlap). In the punchline (res-olution), a trigger expression forces the audienceto switch their interpretation to the alternate (lesslikely) script. The alternate script must differ sig-nificantly in meaning (be incongruent with the firstscript) for the switch to have a humorous effect. Anexample below illustrates this idea (S1 is the obvi-ous script, and S2 is the alternate script. Bracketedphrases are labeled with the associated script).

‘‘Is the [doctor]S1 at home?’’

the [patient]S1asked in his

[bronchial]S1[whisper]S2

. ‘‘No,’’

the [doctor’s]S1[young and pretty

wife]S2 [whispered]S2 in reply.

[‘‘Come right in.’’]S2 (Raskin, 1985)

2 Related Work

Of the early prototypes of pun-generators, JAPE(Binsted and Ritchie, 1994), and its successor,STANDUP (Ritchie et al., 2007), produced ques-tion/answer punning riddles from general non-humorous lexicon. While humor in the generatedpuns could be explained by SSTH, the SSTH modelitself was not employed in the process of generation.Recent work of Hempelmann (2006) comes closerto utilizing SSTH. While still focused on generatingpuns, they do so by explicitly defining and applyingscript opposition (SO) using ontological semantics.Of the more successful pun generators are systemsthat exploit lexical resources. HAHAcronym (Stockand Strapparava, 2002), a system for generating hu-morous acronyms, for example, utilizes WordNet-Domains to select phonologically similar conceptsfrom semantically disparate domains. While the de-gree of humor sophistication from the above systems

varies with the sophistication of the method (lexi-cal resources, surface realizers), they all, without ex-ception, rely on phonological constraints to producescript opposition, whereas a phonological constraintis just one of the many ways to generate script op-position.

3 System overview

ConceptNet (Liu and Singh, 2004) lends itself as anideal ontological resource for script generation. As anetwork that connects everyday concepts and eventswith a set of causal and spatial relationships, the re-lational structure of ConceptNet parallels the struc-ture of the fabula model of story generation - namelythe General Transition Network (GTN) (Swartjesand Theune, 2006). As such, we hypothesize thatthere exist paths within the ConceptNet graph thatcan be represented as feasible scripts in the sur-face form. Moreover, multiple paths between twogiven nodes represent overlapping scripts - a nec-essary condition for verbal humor in SSTH. Givena semantic network hypergraph G = (V,L) whereV ∈ Concepts, L ∈ Relations, we hypothesizethat it is possible to search for script-pairs as seman-tic circuits that can be converted to a surface formof the Question/Answer format. We define a circuitas two paths from root A that terminate at a commonnode B. Our approach is composed of three stages -(1) we build a script model (SM) that captures likelytransitions between concepts in a surface-realizablesequence, (2) The script model (SM) is then em-ployed to generate a set of feasible circuits from auser-specified root node through spreading activa-tion, producing a set of ranked scripts. (3) Rankedscripts are converted to surface form by aligning asubset of its concepts to natural language templatesof the Question/Answer form. Alignment is per-formed through a scoring heuristic which greedilyoptimizes for incongruity of the surface form.

3.1 Script model

We model a script as a first order Markov chain ofrelations between concepts. Given a seed concept,depth-first search is performed starting from the rootconcept, considering all directed paths terminatingat the same node as candidates for feasible scriptpairs. Most of the found semantic circuits, however,

151

do not yield a meaningful surface form and needto be pruned. Feasible circuits are learned in a su-pervised way, where binary labels assign each can-didate circuit one of the two classes {feasible,infeasible} (we used 8 seed concepts, with 300generated circuits for each concept). Learned tran-sition probabilities are capable of capturing primi-tive stories with events, consequences, as well asappropriate qualifiers of certainty, time, size, loca-tion. Given a chain of concepts S (from hereon re-ferred to as a script) c1, c2...cn, we obtain its likeli-hood Pr(S) =

∏Pr(rij |rjk), where rij and rjk are

directed relations joining concepts < ci, cj >, and< cj , ck > respectively, and the conditionals arecomputed from the maximum likelihood estimate ofthe training data.

3.2 Semantic overlap and spreading activationWhile the script model is able to capture seman-tically meaningful transitions in a single script, itdoes not capture inter-script measures such as over-lap and incongruity. We employ a modified formof spreading activation with fan-out and path con-straints to find semantic circuits while maximizingtheir semantic overlap. Activation starts at the user-specified root concept and radiates along outgoingedges. Edge pairs are weighted with their respectivetransition probabilities Pr(rij |rjk) and a decay fac-tor γ < 1 to penalize for long scripts. An additionalfan-out constraint penalizes nodes with a large num-ber of outgoing edges (concepts that are too gen-eral to be interesting). The weight of a current nodew(ci) is given by:

w(ci) =∑

ck∈fin(cj)

∑cj∈fin(ci)

Pr(rij |rjk)|fout(ci)|

γw(cj) (1)

Termination condition is satisfied when the activa-tion weights fall below a threshold (loop checkingis performed to prevent feedback). Upon termina-tion, nodes are ranked by their activation weight, andfor each node above a specified rank, a set of paths(scripts) Sk ∈ S is scored according to:.

φk = |Sk| log γ +

|Sk|∑i

log Prk(ri+1|ri) (2)

where φk is decay-weighted log-likelihood of scriptSk in a given circuit and |Sk| is the length of script

A

Q

Q

Q

S1

S2

C1

C2

Figure 2: Question(Q) and Answer(A) concepts withinthe semantic circuit. Areas C1 and C2 represent differ-ent semantic clusters. Note that the answer(A) concept ischosen from a different cluster than the question concepts

Sk (number of nodes in the kth chain). A set ofscripts S with the highest scores in the highest rank-ing circuits represent scripts that are likely to be fea-sible and display a significant amount of semanticoverlap within the circuit.

3.3 Incongruity and surface realization

The task is to select a script pair {Si, Sj i 6= j} ∈S × S and a set of concepts C ∈ Si ∪ Sj that willalign with some surface template, while maximiz-ing inter-script incongruity. As a measure of con-cept incongruity, we hierarchically cluster the entireConceptNet using a Fast Community Detection al-gorithm (Clauset et al., 2004). We observe that clus-ters are generated for related concepts, such as reli-gion, marriage, computers. Each template presentsup to two concepts {c1 ∈ Si, c2 ∈ Sj i 6= j} in thequestion sentence (Q in Figure 2), and one conceptc3 ∈ Si ∪ Sj in the answer sentence (A in Figure2). The motivation of this approach is that the twoconcepts in the question are selected from two dif-ferent scripts but from the same cluster, while the an-swer concept is selected from one of the two scriptsand from a different cluster. The effect the generatedtwo-liner produces is that of a setup and resolution(punchline), where the question intentionally sets uptwo parallel and compatible scripts, and the answertriggers the script switch. Below are the top-rankingtwo-liners as rated by a group of fifteen subjects(testing details in the next section). Each conceptis indicated in brackets and labeled with the scriptfrom which the concept had originated:

Why does the [priest]root [kneel]S1in

[church]S2? Because the [priest]root

wants to [propose woman]S1

152

Why does the [priest]root [drink

coffee]S1and [believe god]S2

?

Because the [priest]root wants to

[wake up]S1

Why is the [computer]root [hot]S1in

[mit]S2? Because [mit]S2 is [hell]S2

Why is the [computer]root in

[hospital]S1? Because the

[computer]root has [virus]S2

4 Results

We evaluate the generated two-liners by presentingthem as human-generated to remove possible bias.Fifteen subjects (N = 15, 12 male, 3 female - grad-uate students in Mechanical Engineering and Com-puter Science departments) were presented 48 high-est ranking two-liners, and were asked to rate eachjoke on the scale of 1 to 4 according to four cat-egories: hilarious (4), humorous (3), not humor-ous (2), nonsense(1). Each two-liner was generatedfrom one of the three root categories (12 two-linersin each): priest, woman, computer, robot, and tonormalize against individual humor biases, human-made two-liners were mixed in in the same cate-gories. Two-liners generated by three different al-gorithms were evaluated by each subject:

Script model + Concept clustering (SM+CC)Both script opposition and incongruity arefavored through spreading activation andconcept clustering.

Script model only (SM) No concept clustering isemployed. Adherence of scripts to the scriptmodel is ensured through spreading activation.

Baseline Loops are generated from a user-specifiedroot using depth first search. Loops are prunedonly to satisfy surface templates.

We compare the average scores between the two-liners generated using both the script model and con-cept clustering (SM+CC) (MEAN=1.95, STD=0.27)and the baseline (MEAN=1.06, STD=0.58). Weobserve that SM+CC algorithm yields significantlyhigher-scoring two-liners (one-sided t-test) with95% confidence.

0

20

40

60

80

100

Baseline SM SM+CC Human

% (N

=15)

Nonsense

Non-humorous Humorous

Hilarious

Figure 3: Human blind evaluation of generated two-liners

We observe that the fraction of non-humorous andnonsensical two-liners generated is still significant.Many non-humorous (but semantically sound) two-liners were formed due to erroneous labels on theconcept clusters. While clustering provides a fun-damental way to generate incongruity, noise in theConceptNet often leads of cluster overfitting, and as-signs related concepts into separate clusters.

Nonsensical two-liners are primarily due to the in-consistencies in POS with relation types within theConceptNet. Because our surface form templatesassume a part of speech, or a phrase type from theConceptNet specification, erroneous entries producenonsensical results. We partially address the prob-lem by pruning low-scoring concepts (ConceptNetfeatures a SCORE attribute reflecting the number ofuser votes for the concept), and all terminal nodesfrom consideration (nodes that are not expanded byusers often indicate weak relationships).

5 Future Work

Through observation of the generated semanticpaths, we note that more complex narratives, beyondquestions/answer forms can be produced from theConceptNet. Relaxing the rigid template constraintof the surface realizer will allow for more diversetypes of generated humor. To mitigate the fragilityof concept clustering, we are augmenting the Con-ceptNet with additional resources that provide do-main knowledge. Resources such as SenticNet(WordNet-Affect aligned with ConceptNet) (Cam-bria et al., 2010b), and WordNet-Domains (Kolteand Bhirud, 2008) are both viable avenues for robustconcept clustering and incongruity generation.

153

AcknowledgementThis paper is for my Babishan - the most importantperson in my life.Huge thanks to Max Kelner - those everyday teas atMattins and continuous inspiration.This work was supported in part by NSF CDI GrantECCS 0941561. The content of this paper is solelythe responsibility of the authors and does not neces-sarily represent the official views of the sponsoringorganizations.

References

S. Attardo and V. Raskin. 1991. Script theory revis (it)ed: Joke similarity and joke representation model. Hu-mor: International Journal of Humor Research; Hu-mor: International Journal of Humor Research.

K. Binsted and G. Ritchie. 1994. A symbolic descriptionof punning riddles and its computer implementation.Arxiv preprint cmp-lg/9406021.

K. Binsted, A. Nijholt, O. Stock, C. Strapparava,G. Ritchie, R. Manurung, H. Pain, A. Waller, andD. O’Mara. 2006. Computational humor. IntelligentSystems, IEEE, 21(2):59–69.

K. Binsted. 1996. Machine humour: An implementedmodel of puns.

E. Cambria, A. Hussain, C. Havasi, and C. Eckl. 2010a.Senticspace: visualizing opinions and sentiments ina multi-dimensional vector space. Knowledge-Basedand Intelligent Information and Engineering Systems,pages 385–393.

E. Cambria, R. Speer, C. Havasi, and A. Hussain. 2010b.Senticnet: A publicly available semantic resource foropinion mining. In Proceedings of the 2010 AAAI FallSymposium Series on Commonsense Knowledge.

A. Clauset, M.E.J. Newman, and C. Moore. 2004. Find-ing community structure in very large networks. Phys-ical review E, 70(6):066111.

F. Crestani. 1997. Retrieving documents by constrainedspreading activation on automatically constructed hy-pertexts. In EUFIT 97-5th European Congress on In-telligent Techniques and Soft Computing. Germany.Citeseer.

L. Friedland and J. Allan. 2008. Joke retrieval: recogniz-ing the same joke told differently. In Proceeding of the17th ACM conference on Information and knowledgemanagement, pages 883–892. ACM.

C.F. Hempelmann, V. Raskin, and K.E. Triezenberg.2006. Computer, tell me a joke... but please make itfunny: Computational humor with ontological seman-tics. In Proceedings of the Nineteenth International

Florida Artificial Intelligence Research Society Con-ference, Melbourne Beach, Florida, USA, May 11, vol-ume 13, pages 746–751.

S.G. Kolte and S.G. Bhirud. 2008. Word sense disam-biguation using wordnet domains. In Emerging Trendsin Engineering and Technology, 2008. ICETET’08.First International Conference on, pages 1187–1191.IEEE.

H. Liu and P. Singh. 2004. Conceptneta practical com-monsense reasoning tool-kit. BT technology journal,22(4):211–226.

R. Mihalcea and C. Strapparava. 2006. Learning to laugh(automatically): Computational models for humorrecognition. Computational Intelligence, 22(2):126–142.

M.E.J. Newman. 2006. Modularity and communitystructure in networks. Proceedings of the NationalAcademy of Sciences, 103(23):8577–8582.

A. Nijholt, O. Stock, A. Dix, and J. Morkes. 2003. Hu-mor modeling in the interface. In CHI’03 extended ab-stracts on Human factors in computing systems, pages1050–1051. ACM.

V. Raskin. 1998. The sense of humor and the truth. TheSense of Humor. Explorations of a Personality Char-acteristic, Berlin: Mouton De Gruyter, pages 95–108.

G. Ritchie, R. Manurung, H. Pain, A. Waller, R. Black,and D. OMara. 2007. A practical application of com-putational humour. In Proceedings of the 4th. Inter-national Joint Workshop on Computational Creativity,London, UK.

G. Ritchie. 2001. Current directions in computationalhumour. Artificial Intelligence Review, 16(2):119–135.

W. Ruch, S. Attardo, and V. Raskin. 1993. Toward anempirical verification of the general theory of verbalhumor. Humor: International Journal of Humor Re-search; Humor: International Journal of Humor Re-search.

J. Savoy. 1992. Bayesian inference networks and spread-ing activation in hypertext systems. Information pro-cessing & management, 28(3):389–406.

S. Spagnola and C. Lagoze. 2011. Edge dependentpathway scoring for calculating semantic similarity inconceptnet. In Proceedings of the Ninth InternationalConference on Computational Semantics, pages 385–389. Association for Computational Linguistics.

O. Stock and C. Strapparava. 2002. Hahacronym:Humorous agents for humorous acronyms. Stock,Oliviero, Carlo Strapparava, and Anton Nijholt. Eds,pages 125–135.

I. Swartjes and M. Theune. 2006. A fabula model foremergent narrative. Technologies for Interactive Digi-tal Storytelling and Entertainment, pages 49–60.

154

J.M. Taylor and L.J. Mazlack. 2004. Humorous word-play recognition. In Systems, Man and Cybernetics,2004 IEEE International Conference on, volume 4,pages 3306–3311. IEEE.

J. Taylor and L. Mazlack. 2005. Toward computationalrecognition of humorous intent. In Proceedings ofCognitive Science Conference, pages 2166–2171.

J.M. Taylor. 2009. Computational detection of humor: Adream or a nightmare? the ontological semantics ap-proach. In Proceedings of the 2009 IEEE/WIC/ACMInternational Joint Conference on Web Intelligenceand Intelligent Agent Technology-Volume 03, pages429–432. IEEE Computer Society.

155