from syntactic-semantic tagging to knowledge discovery in medical texts

International Journal of Medical Informatics 52 (1998) 149–157

From syntactic-semantic tagging to knowledge discovery inmedical texts

W. Ceusters a,b,*, P. Spyns b, G. De Moor b

a Language and Computing NV, Het Moorhof, Hazenakkerstraat 20, B-9520 Zonnegem, Belgiumb RAMIT VZW, Uni6ersity Hospital, De Pintelaan 185, B-9000 Gent, Belgium

Abstract

In the GALEN project, the syntactic-semantic tagger MultiTALE is upgraded to extract knowledge from naturallanguage surgical procedure expressions. In this paper, we describe the methodology applied and show that out of arandomly selected sample of such expressions coming from the procedure axis of Snomed International, 81% couldbe analysed correctly. The problems encountered fall in three different categories: unusual grammatical configurationswithin the Snomed terms, insufficient domain knowledge and different categorisation of concepts and semantic linksin the domain and linguistic models used. It is concluded that the Multi-TALE system can be used to attach meaningto words that not have been encountered previously, but that an interface ontology mediating between domainmodels and linguistic models is needed to arrive at a higher level of independence from both particular languages andfrom particular domains. © 1998 Elsevier Science Ireland Ltd. All rights reserved.

Keywords: Natural language processing; Knowledge acquisition; Concept representation

1. Introduction

The purpose of the GALEN project is todevelop language independent concept repre-sentation systems as the foundations for thenext generation of multilingual coding sys-tems [1]. At the heart of the project is thedevelopment of a reference model for medicalconcepts (CORE) supported by a formal lan-

guage for medical concept representation(GRAIL) [2]. A particular characteristic ofthe approach is the clear separation of thepure conceptual knowledge from other typesof knowledge, including linguistic knowledge[3], in order to arrive in the future to applica-tion-independent medical terminologies [4].Although on a theoretical basis the feasibilityof these objectives is debatable [5], actualwork within the GALEN-IN-USE projectshows that on a relatively concise domainsuch as surgical procedures, distributed col-

* Corresponding author. Tel.: +32 53 629545; fax: +32 53629555; e-mail: [email protected]

1386-5056/98/$ - see front matter © 1998 Elsevier Science Ireland Ltd. All rights reserved.

PII S1386-5056(98)00134-8

W. Ceusters et al. / International Journal of Medical Informatics 52 (1998) 149–157150

Fig. 1. MultiTALE analysis of the sentence ‘closed reductionof fracture of zygoma or zygomatic arch’.

laborative modelling can be achieved overlinguistic borders. As could be expected, theprocess is however extremely slow. Formal‘naming’ and subcategorisation of new con-cepts at the one hand and (in)consistent mod-elling of natural language expressions usingthe building blocks of the CORE that alreadyare available, turn out to be the most frequentreasons for discussion.

Given the promising results of the Multi-TALE semantic tagger for neurosurgical pro-cedure reports [6–8], it was investigatedwhether or not this manual modelling workcould be speeded up by using MultiTALE asan automatic modelling device.

2. Material and methods

A total of 100 English surgical procedureexpressions were randomly selected from theSNOMED International V3.1 procedure, ex-cluding generic (codes P1-0xxxx) and anaes-thetic (codes P1-Cxxxx) procedures. Theseexpressions were then processed by the originalMultiTALE tagger. The results were analysedto identify possible shortcomings at the levelof the lexicon, the syntactic-semantic grammarand the desired format of the output, i.e.GALEN templates [9,10]. Based on this anal-ysis, a stepwise lingware refinement methodol-ogy was adopted, until a satisfactory numberof expressions could correctly be processed.

The purpose of this study was then toinvestigate: (1) whether the high level ontologyof GALEN and the representation power ofthe GALEN surgical procedure templateswere sufficiently elaborated for use in naturallanguage understanding; (2) to identify whatadditional linguistic knowledge was neededto improve the results; and (3) to investigatewhether the SNOMED expressions them-selves could unambiguously be understoodusing the available conceptual and linguisticknowledge.

3. From MultiTALE to MultiTALE II

Multi-TALE is a syntactic-semantic taggerfor English and Dutch neurosurgery reports[11]. The function of a tagger is to perform thefirst and essential pre-processing stage for anynatural language processing application, thelabelling of words with their grammaticalcategories, only taking into account a limitedcontext. In addition to this syntactic labelling,the Multi-TALE tagger also provide semantictags to words and word groups on the basis ofthe model for surgical procedures as describedin CEN/ENV 1828:1995 [12].

Prior to any modification, MultiTALEanalysed the expression

P1-11E52: closed reduction of fracture ofzygoma or zygomatic arch (s1)

W. Ceusters et al. / International Journal of Medical Informatics 52 (1998) 149–157 151

Fig. 2. MultiTALE II syntactic output of the expression ‘closed reduction of fracture of zygoma or zygomatic arch’.

as an action of type repair which has as directobject a pathology, namely a fracture ofzygoma or zygomatic arch (Fig. 1). The se-mantic links discovered (action and do), aswell as the semantic types (repair, path, anat)have their origin in CEN/ENV 1828:1995. Inaddition, for the individual concepts discov-ered, the SNOMED International code isgiven.

Notice that the correct final results given inlines 1 and 3 of Fig. 1 originate from anerroneous intermediate processing at lines 8and 9 where the co-ordination is attributed atthe wrong constituents. This is entirely due tothe tagging nature of MultiTALE (as op-posed to traditional parsers) according towhich only the segmentation at the highestlevel matters.

With the objectives of GALEN in mind,this approach was no longer feasible as amore detailed analysis was required. Multi-TALE was upgraded to MultiTALE II whichproduces the output of the same sentence (s1)as given in Fig. 2 and Fig. 3.

In order to achieve these results, the fol-lowing changes to the original system wereneeded.

3.1. Implementation of a refined model forsurgical procedures

According to CEN/ENV 1828:1995, a sur-gical procedure is conceptually composed of

a surgical deed (i.e. deed that can be done bythe operator to the patient’s body during thesurgical procedure) which is semanticallylinked to the concept fields human anatomy,pathology and interventional equipment. Po-tential semantic links are direct object (refer-ring to that on which the surgical deed iscarried out), the indirect object (referring tothe site of the surgical deed) and the means(referring to the means with which the deed iscarried out).

Although the standard was developed forthe description of elementary surgical proce-dure expressions as they can be found inclassification and coding systems, one of thehypotheses of the MultiTALE project wasthat the same structure could be used torepresent the particular tasks described in fulltext reports. This turned out to be feasible,though not unproblematic. Especially thelinks indirect object and direct object wererecognised to be underspecified for being use-ful within a natural language understandingenvironment and lead to ‘non-monotoniclike’ semantic analyses. See for instance pat-terns such as:

Injection (deed) of antibiotic (direct object)(s2)

Injection (deed) of cyst (direct object) (s3)

Injection (deed) of antibiotic (direct object)

in cyst (indirect object) (s4)


Fig. 3. MultiTALE II semantic analysis of the expression ‘closed reduction of fracture of zygoma or zygomatic arch’,presented in GALEN-template format.

Irrigation (deed) of cyst (direct object)

with antibiotic (means) (s5)

For this reason, more refined links had tobe foreseen such as has–location, has–source, has–target, has–recipient. As inter-nally in MultiTALE II these links stand in an-to-1 relationship to the original links ofCEN/ENV 1828:1995, output can still begiven according to the specifications of theENV.

3.2. Implementation of a concept hierarchy

The MultiTALE tagger was directly basedon the ‘flat’ concept model of ENV1828:1995, lexemes being encoded directly assurgical–deed, anatomy, pathology or instru-ment. To resolve certain linguistic ambigui-ties, a hierarchical model was needed. Therelevant parts of the hierarchy needed toanalyse the sentence of Fig. 3 and the restric-tional constraints on how some concepts maybe linked, are outlined in Fig. 4. However, inorder not to duplicate the work of the mod-ellers in the GALEN-IN-USE project, theconceptual model was not more enhancedthan needed for an unambiguous interpreta-tion of the expressions, leaving out the detailsrequired for generation purposes. In addition,only that part of the GALEN ontology thatsurfaces grammatically in the expressions,was incorporated [13]. A complete correspon-dence with the genuine GALEN model couldhowever not be maintained as quite often thecategorisation of concepts from a medical

perspective does not follow the same seman-tic principles that underly sentence forma-tion. Given the nature of surgical procedureswhere most often anatomical structures arebeing displaced or worked upon, we optedfor a linguistically inspired categorisationwhere events are distinguished from entitiesand further subdivided into states, acts, in-choatives and resultatives ([14] pp 183–184).For each of those, motional events are distin-guished from non-motional events. As a con-sequence, procedures most often are motionalacts involving thematic roles such asTHEME, DESTINATION, SOURCE, LO-CATION and PATH. Most of these thematicroles are then mapped to GALEN semanticlinks though not in a one-to-one fashion.

3.3. Implementation of mechanisms forknowledge disco6ery

As MultiTALE II is designed to enrich theGALEN CORE and linguistic annotationmodules (semi)automatically, mechanismshad to be foreseen for dealing with unknownwords in the input. This was achieved using abottom-up parsing strategy where both syn-tactic and semantic configurations limit eachothers possible interpretations. In Fig. 5, thesentence

injection of xyz (s6)

(were xyz obviously is an unknown word) isanalysed by MultiTALE II with one possiblesyntactic solution (xyz being a noun) andfour possible semantic interpretations. First,


Fig. 4. Relevant part of the concept hierarchy for the sentence closed reduction of fracture of zygoma or zygomaticarch.

xyz might be a body part, body region orpathology in which a not specified chemical isinjected, as in

P1-10542: injection of ligament (s7)

In these three cases, the HAS–DESTINA-TION semantic link applies. Next, xyz mightbe the chemical itself, with no destinationspecified, as in

P1-05027: injection of gas (s8)

The various possibilities with respect to themeaning of xyz and the associated semanticlinks, are derived both from the internal con-cept hierarchy of MultiTALE-II (some rele-vant parts of which being shown in Fig. 6)and from the possible syntactic configura-tions.

After dictionary lookup and grammaticalanalysis, the only possible syntactic configu-ration for the sentence is ‘noun-preposition-noun’. The noun ‘injection’ is found to haveone possible meaning, namely an install pro-cedure with as THEME a chemical. Thepreposition ‘of’ can have several meanings,indicating a genitive semantic link (LOCA-TION, COMPONENT,…), an objective link(THEME, PATIENT, EXPERIENCER,…)

or a receptive link (DESTINATION). Incombination with ‘injection’ only the objec-tive and receptive interpretation are possible.The objective reading fixes the chemical asTHEME, while in the receptive reading, bodyparts are candidate destinations. When finallythe unknown word ‘xyz’ is taken into consid-eration, it can unify with both the chemicaland body-part readings.

4. Results

Out of the 100 randomly selected expres-sions, 10 could not be processed by Multi-TALE-II. For seven of them, the requiredconcepts or links were not yet available in theGALEN template-formalism, clearly a reasonfor failure outside the responsibility of Multi-TALE. Of the remaining three, two showedpeculiar (a)grammatical configurations:

P1-37865: Femoral-popliteal artery bypassgraft with other than vein (s9)

P1-41335: Excision of benign lesion of scalpand neck, lesion diameter 3.1 to 4.0 cm)

(s10)


Fig. 5. Syntactic and semantic analysis of the sentence ‘injection of xyz’.

while the other one contained deictic refer-ences and ellipsis, linguistic phenomena forwhich no grammar rules are currentlyimplemented

P1-B9846: Bilateral repair of inguinalhernia, one direct and one indirect (s11)

Of the 90 expressions that could be pro-cessed, 73 (81%) were analysed correctly, giv-ing the only one possible interpretation, 58 ofwhich by using exclusively the links foreseenin the GALEN template formalism, an inter-mediate representation developed in ordernot to confront the domain modelling expertswith the complexity of the GRAIL language[15], while for the remaining 15, additionalsemantic links were introduced. It wouldhave been possible to map these extra links to

the ‘garbage’-link HAS–-OTHER–-FEA-TURE that is allowed in the templates, butwe choose deliberately for not doing so inorder to preserve the depth of the interpreta-tion. 17 expressions led to multiple interpre-tations, 48 all together. Of those 48, 75%could be judged being correct.

5. Discussion

The results presented in this paper reflectnot the final desired outcome of MultiTALE-II, but are rather to be seen as a first evalua-tion of the actual stage of the system, withfurther improvement in mind.

Ambiguities in the input phrases was themost important reason for multiple interpre-tations e.g.

P1-A1122: Decompression of orbit only bytranscranial approach (s12)

where ‘only’ can refer to the orbit (nothingelse being decompressed), to the decompres-sion (nothing else than a decompression be-ing done on the orbit), or to the approach(no other approach allowed for giving thiscode).

Coordination also led often to multipleinterpretations, though the semantic con-straints prevented all possible syntactic com-binations, as can be seen in Fig. 2, wheresyntactically a possible bracketing wouldhave been: {{Closed reduction} of {{fractureof zygoma} or zygomatic arch}}. This possi-

Fig. 6. Relevant part of the concept hierarchy of Multi-TALE-II with respect to the sentence ‘injection of xyz’.


ble syntactic solution is however not retainedon semantic grounds.

Failure to reach an adequate interpretationwas due to one of three reasons. For fewsentences, the representational power of theGALEN-templates was not sufficient. It isfor instance not yet possible to representco-ordination amongst different semanticlinks that apply at the same time to oneconcept, e.g.

P1-2682B: repair of internal or complexfistula of trachea (s13)

where ‘internal’ and ‘complex’ specify twodifferent features of ‘fistula’. Also, theGALEN templates allow numbers to belinked to concepts using the HAS–NUM-BER link, but quite often, an exact numbercannot be deduced from the expression, asjust a plural is given as in (s14) where one canonly infer that there must be more than oneadhesion.

P1-7AC34: Lysis of adhesions of spermaticcord (s14)

For some other sentences, specific surfacelinguistic constructs turned out to be prob-lematic e.g. in

P1-19B05: Primary suture of rupturedligament of ankle, collateral (s15)

’Collateral’ obviously specifies ‘ligament’,but no grammar rule could yet be imple-mented in such a way that this sentencewould be analysed correctly without intro-ducing erroneous output for other sentencessuch as in (s16).

P1-40141: Incision and drainage ofhematoma, complicated (s16)

Similar difficulties are caused by co-ordi-nated multiword units upon which ellipsis isapplied, as in

P1-21A08:…rhinoplasty with lateral andalar cartilage’s… (s17)

A third reason for incorrect results, is thelack of detailed anatomical knowledge suchas the one required for correctly parsing(s18).

P1-17A26: Tenodesis for proximalinterphalangeal finger joint stabilization

(s18)

In this case, the system must know that‘interphalangeal’ refers to ‘joint’ and not to‘finger’, in contrast with ‘abdominal wallmass’ were ‘abdominal’ refers to ‘wall’.

6. Conclusion

The main conclusion of this work is that itis indeed feasible to develop a syntactic-se-mantic parser that quite satisfactorily trans-lates natural language expressions into apredefined formalism for further processing.However, in order to be able to extract newknowledge from texts, a certain amount ofbackground knowledge, both conceptual andlinguistic, must be available. All knowledgebased approaches rely on an ontology, amore or less formal representation—to beused in computer systems—of what conceptsexist in the world and how they relate to oneanother. The GALEN ontology is viewed asa strictly language independent model of theworld [16]. Meanwhile, the need for an ontol-ogy in natural language processing applica-tions is generally accepted [17] as well. This isnot to say that knowledge structuring basedon a linguistic approach leads to the sameresult as when opting for a conceptual ap-proach. A typical example is the ontologicaldistinction between nominal and naturalkinds [18], that in no language is grammati-calised just because the difference is pure


definitional [19]. This again does not meanthat such distinctions are not useful in anatural language processing applications. InMultiTALE-II for instance is the distinctionbetween natural and nominal kinds used toanalyse correctly expressions such as ‘capsu-lotomy of wrist’ where at the surface ‘wrist’ isconnected to ‘capsulotomy’ by an objectivereading of ‘of’, while at deep level, the realTHEME is the ‘capsule’ that is located in the‘wrist’. Also it has been stated that whensituated ontologies (i.e. ontologies that aredeveloped for solving particular problems inknowledge based applications [20]) have tooperate in natural language processing appli-cations, they are better suited to assist lan-guage understanding when the concepts andrelationships they are built upon, are linguis-tically motivated [21].

A second conclusion of this work is thatautomated knowledge acquisition fromnomenclatures such as SNOMED Interna-tional, using natural language processingtechniques, could be improved when con-trolled language principles would be appliedfor term formation in such nomenclatures.Such principles reduce ambiguity [22].

Based on these observations, future devel-opments around Multi-TALE will concen-trate on the design of an interface ontologymediating between situated ontologies andlinguistic ontologies. This is expected to bringthe Multi-TALE system to a sufficient levelof independence from both particular lan-guages and from particular domains.

References

[1] A.L. Rector, W.A. Nowlan, A. Glowinski, Goalsfor concept representation in the GALEN project,in: C. Safran (Ed.), SCAMC ‘93 Proceedings, Mc-Graw-Hill, New York, 1993, pp. 414–418.

[2] A.L. Rector, A. Glowinski, W.A. Nowlan, A.Rossi-Mori, Medical concept models and medical

records: an approach based on GALEN and PENand PAD, J. Am. Med. Inform. Assoc. 2 (1995)19–35.

[3] A.L. Rector, W.A. Nowlan, S. Kay, Conceptualknowledge: the core of medical information sys-tems, in: K.C. Lun, P. Degoulet, T.E. Piemme, O.Rienhoff (Eds.), MEDINFO ‘92 Proceedings,North-Holland, Amsterdam 1992, pp. 1420–1426.

[4] A.L. Rector, Compositional models of medicalconcepts: towards re-usable application indepen-dent medical terminologies, in: P. Barahona, J.P.Christensen (Eds.), Knowledge and Decisions inHealth Telematics, IOS Press, Amsterdam, 1994,pp. 133–142.

[5] W. Ceusters, G. Deville, F. Buekens, The chimeraof purpose- and language-independent concept sys-tems in healthcare, in: P. Barahona, M. Veloso, J.Bryant (Eds.), MIE ‘94 Proceedings 1994, pp.208–212.

[6] W. Ceusters, G. Deville, G. De Moor, Automatedextraction of neurosurgical procedure expressionsfrom full text reports: the Multi-TALE experience,in: J. Brender, J.P. Christensen, J.-R. Scherrer, P.McNair (Eds.), MIE ‘96 Proceedings, IOS Press,Amsterdam 1996, pp. 154–158.

[7] W. Ceusters, G. Deville, A mixed syntactic-seman-tic grammar for the analysis of neurosurgical pro-cedure reports: the Multi-TALE experience, in: C.Sevens, G. De Moor (Eds.), MIC ‘96 Proceedings,1996, pp. 59–68.

[8] W. Ceusters, C. Lovis, A. Rector, R. Baud, Natu-ral language processing tools for the computerisedpatient record: present and future, in: P. Waege-mann (Ed.), Toward an Electronic Health RecordEurope ’96 Proceedings, 1996, pp. 294–300.

[9] GALEN Consortium, Guidelines and Recipes forCompleting templates. Internal documentVUM02/96 version 1.0.

[10] GALEN Consortium, Links and Templates Sum-mary. Internal document VUM/03/96 version 1.0.

[11] W. Ceusters, P. Spyns, G. De Moor, W. Martin(Eds.) Syntactic-Semantic Tagging of MedicalTexts: the Multi-TALE Project. Studies in HealthTechnologies and Informatics, IOS Press, Amster-dam, 1998.

[12] CEN ENV 1828:1995. Medical Informatics-Struc-ture for classification and coding of surgical proce-dures.

[13] W. Ceusters, F. Buekens, G. De Moor, A. Waag-meester, The Distinction between linguistic andconceptual semantics in medical terminology andits implications for NLP-based knowledge acquisi-tion, in: Proceedings of IMIA WG6 Conference on


Natural Language and Medical Concept Represen-tation. Jacksonville 19-22/01/97, pp. 71–80.

[14] W. Frawley, Linguistic Semantics, Lawrence Erl-baum Associates, Hillsdale, NJ, 1992.

[15] J. Rogers, W. Solomon, A. Rector, P. Pole, P.Zanstra, E. van der Haring, Rubrics to dissectionsto GRAIL to classifications, in: C. Pappas, N.Maglaveras, J.-R. Scherrer (Eds.), Medical Infor-matics Europe ’97, IOS Press, Amsterdam, 1997,pp. 241–245.

[16] A.L. Rector, J.E. Rogers, P. Pole, The GALENhigh level ontology, in: J. Brender, J.P. Chris-tensen, J.-R. Scherrer, P. McNair (Eds.), MIE ‘96Proceedings, IOS Press, Amsterdam 1996, pp.174–178.

[17] J.A. Bateman, Ontology construction and naturallanguage, in: Proceedings of the InternationalWorkshop on Formal Ontology, Padua, Italy,1993, pp. 83–93.

[18] S. Kripke, Naming and necessity, in: D. Davidson,G. Harman (Eds.), Semantics of Natural Lan-

guage, Reidel, Dordrecht, 1972, pp. 253–355.[19] C. Welsh, On the non-existence of natural kind

terms as a linguistically relevant category. Paperpresented at the Liguistic Society of America, NewOrleans, LA, 1988.

[20] K. Mahesh, S. Nirenburg, A situated ontology forpractical NLP, in: Proceedings of the Workshopon Basic Ontological Issues in Knowledge Sharing,IJCAI-95. Montreal, Canada, 1995.

[21] G. Deville, W. Ceusters, A multi-dimensional viewon natural language modelling in medicine: identi-fying key-features for successful applications. Sup-plementary paper in Proceedings of the ThirdInternational Working Conference of IMIA WG6,Geneva, 1994.

[22] W. Ceusters, F. Steurs, P. Zanstra, E. van derHaring, J. Rogers, From a time standard for med-ical informatics to a controlled language forhealth. IMIA WG16 conference, Bermuda, 11–13/09/97 (unnumbered. Also to appear in Interna-tional Journal of Medical Informatics).

.

from syntactic-semantic tagging to knowledge discovery in medical texts

Documents