syntactic patterns versus word alignment: extracting ... antho/p/p13/p13-1172.pdf · pdf...

Click here to load reader

Post on 22-Mar-2020




0 download

Embed Size (px)


  • Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1754–1763, Sofia, Bulgaria, August 4-9 2013. c©2013 Association for Computational Linguistics

    Syntactic Patterns versus Word Alignment: Extracting Opinion Targets from Online Reviews

    Kang Liu, Liheng Xu and Jun Zhao National Laboratory of Pattern Recognition

    Institute of Automation, Chinese Academy of Sciences {kliu, lhxu, jzhao}

    Abstract Mining opinion targets is a fundamen- tal and important task for opinion min- ing from online reviews. To this end, there are usually two kinds of methods: syntax based and alignment based meth- ods. Syntax based methods usually ex- ploited syntactic patterns to extract opin- ion targets, which were however prone to suffer from parsing errors when dealing with online informal texts. In contrast, alignment based methods used word align- ment model to fulfill this task, which could avoid parsing errors without using pars- ing. However, there is no research fo- cusing on which kind of method is more better when given a certain amount of re- views. To fill this gap, this paper empiri- cally studies how the performance of these two kinds of methods vary when chang- ing the size, domain and language of the corpus. We further combine syntactic pat- terns with alignment model by using a par- tially supervised framework and investi- gate whether this combination is useful or not. In our experiments, we verify that our combination is effective on the corpus with small and medium size.

    1 Introduction

    With the rapid development of Web 2.0, huge amount of user reviews are springing up on the Web. Mining opinions from these reviews be- come more and more urgent since that customers expect to obtain fine-grained information of prod- ucts and manufacturers need to obtain immediate feedbacks from customers. In opinion mining, ex- tracting opinion targets is a basic subtask. It is to extract a list of the objects which users express their opinions on and can provide the prior infor- mation of targets for opinion mining. So this task

    has attracted many attentions. To extract opin- ion targets, pervious approaches usually relied on opinion words which are the words used to ex- press the opinions (Hu and Liu, 2004a; Popescu and Etzioni, 2005; Liu et al., 2005; Wang and Wang, 2008; Qiu et al., 2011; Liu et al., 2012). In- tuitively, opinion words often appear around and modify opinion targets, and there are opinion re- lations and associations between them. If we have known some words to be opinion words, the words which those opinion words modify will have high probability to be opinion targets.

    Therefore, identifying the aforementioned opin- ion relations between words is important for ex- tracting opinion targets from reviews. To fulfill this aim, previous methods exploited the words co-occurrence information to indicate them (Hu and Liu, 2004a; Hu and Liu, 2004b). Obviously, these methods cannot obtain precise extraction be- cause of the diverse expressions by reviewers, like long-span modified relations between words, etc. To handle this problem, several methods exploited syntactic information, where several heuristic pat- terns based on syntactic parsing were designed (Popescu and Etzioni, 2005; Qiu et al., 2009; Qiu et al., 2011). However, the sentences in online reviews usually have informal writing styles in- cluding grammar mistakes, typos, improper punc- tuation etc., which make parsing prone to gener- ate mistakes. As a result, the syntax-based meth- ods which heavily depended on the parsing per- formance would suffer from parsing errors (Zhang et al., 2010). To improve the extraction perfor- mance, we can only employ some exquisite high- precision patterns. But this strategy is likely to miss many opinion targets and has lower recall with the increase of corpus size. To resolve these problems, Liu et al. (2012) formulated identifying opinion relations between words as an monolin- gual alignment process. A word can find its cor- responding modifiers by using a word alignment


  • Figure 1: Mining Opinion Relations between Words using Partially Supervised Alignment Model

    model (WAM). Without using syntactic parsing, the noises from parsing errors can be effectively avoided. Nevertheless, we notice that the align- ment model is a statistical model which needs suf- ficient data to estimate parameters. When the data is insufficient, it would suffer from data sparseness and may make the performance decline.

    Thus, from the above analysis, we can observe that the size of the corpus has impacts on these two kinds of methods, which arises some impor- tant questions: how can we make selection be- tween syntax based methods and alignment based method for opinion target extraction when given a certain amount of reviews? And which kind of methods can obtain better extraction performance with the variation of the size of the dataset? Al- though (Liu et al., 2012) had proved the effective- ness of WAM, they mainly performed experiments on the dataset with medium size. We are still curi- ous about that when the size of dataset is larger or smaller, can we obtain the same conclusion? To our best knowledge, these problems have not been studied before. Moreover, opinions may be expressed in different ways with the variation of the domain and language of the corpus. When the domain or language of the corpus is changed, what conclusions can we obtain? To answer these ques- tions, in this paper, we adopt a unified framework to extract opinion targets from reviews, in the key component of which we vary the methods between syntactic patterns and alignment model. Then we run the whole framework on the corpus with dif- ferent size (from #500 to #1, 000, 000), domain (three domains) and language (Chinese and En- glish) to empirically assess the performance varia- tions and discuss which method is more effective.

    Furthermore, this paper naturally addresses an- other question: is it useful for opinion targets ex- traction when we combine syntactic patterns and word alignment model into a unified model? To

    this end, we employ a partially supervised align- ment model (PSWAM) like (Gao et al., 2010; Liu et al., 2013). Based on the exquisitely designed high-precision syntactic patterns, we can obtain some precisely modified relations between words in sentences, which provide a portion of links of the full alignments. Then, these partial alignment links can be regarded as the constrains for a stan- dard unsupervised word alignment model. And each target candidate would find its modifier un- der the partial supervision. In this way, the er- rors generated in standard unsupervised WAM can be corrected. For example in Figure 1, “kindly” and “courteous” are incorrectly regarded as the modifiers for “foods” if the WAM is performed in an whole unsupervised framework. However, by using some high-precision syntactic patterns, we can assert “courteous” should be aligned to “services”, and “delicious” should be aligned to “foods”. Through combination under partial su- pervision, we can see “kindly” and “courteous” are correctly linked to “services”. Thus, it’s rea- sonable to expect to yield better performance than traditional methods. As mentioned in (Liu et al., 2013), using PSWAM can not only inherit the advantages of WAM: effectively avoiding noises from syntactic parsing errors when dealing with informal texts, but also can improve the mining performance by using partial supervision. How- ever, is this kind of combination always useful for opinion target extraction? To access this problem, we also make comparison between PSWAM based method and the aforementioned methods in the same corpora with different size, language and do- main. The experimental results show the combina- tion by using PSWAM can be effective on dataset with small and medium size.


  • 2 Related Work

    Opinion target extraction isn’t a new task for opin- ion mining. There are much work focusing on this task, such as (Hu and Liu, 2004b; Ding et al., 2008; Li et al., 2010; Popescu and Etzioni, 2005; Wu et al., 2009). Totally, previous studies can be divided into two main categories: supervised and unsupervised methods.

    In supervised approaches, the opinion target ex- traction task was usually regarded as a sequence labeling problem (Jin and Huang, 2009; Li et al., 2010; Ma and Wan, 2010; Wu et al., 2009; Zhang et al., 2009). It’s not only to extract a lexicon or list of opinion targets, but also to find out each opin- ion target mentions in reviews. Thus, the contex- tual words are usually selected as the features to indicate opinion targets in sentences. And classi- cal sequence labeling models are used to train the extractor, such as CRFs (Li et al., 2010), HMM (Jin and Huang, 2009) etc.. Jin et al. (2009) pro- posed a lexicalized HMM model to perform opin- ion mining. Both Li et al. (2010) and Ma et al. (2010) used CRFs model to extract opinion tar- gets in reviews. Specially, Li et al. proposed a Skip-Tree CRF model for opinion target extrac- tion, which exploited three structures including linear-chain structure, syntactic structure, and con- junction structure. However, the main limitation of these supervised methods is the need of labeled training data. If the labeled training data is insuf- ficient, the trained model would have unsatisfied extraction performance. Labeling sufficient train- ing data is time and labor consuming. And for dif- ferent domains, we need label data independently, which is obviously impracticable.

    Thus, many researches focused on unsupervised methods, which are mainly to extract a list of opin- ion targets from reviews. Similar to ours, most ap- proaches regarded opinion wor

View more