Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics-System Demonstrations, pages 111–115Melbourne, Australia, July 15 - 20, 2018. c©2018 Association for Computational Linguistics

111

Praaline: An Open-Source System for Managing, Annotating,Visualising and Analysing Speech Corpora

George ChristodoulidesMetrology and Language Sciences Unit, University of Mons

Place du Parc 18, B-7000 Mons, Belgiumgeorge@mycontent.gr

Abstract

In this system demonstration we presentthe latest developments of Praaline, anopen-source software system for constitut-ing and managing, manually and automati-cally annotating, visualising and analysingspoken language and multimodal corpora.We review the system’s functionality anddesign architecture, present current usecases and directions for future develop-ment.

1 Introduction

In this system demonstration, we present Praaline.Praaline is an open-source application for creatingand managing speech and multi-modal corpora,annotating them manually or automatically, visu-alising the speech signal and annotations, query-ing the data and creating concordances, and per-forming statistical analyses. The system was firstpresented in (Christodoulides, 2014). Praaline isa cross-platform standalone application that runson Windows, Mac OSX and Linux. It is writ-ten in C++ using the Qt framework. The ap-plication is available for download at https://www.praaline.org and the source code isavailable on GitHub (https://github.com/praaline), under the GPL version 3 license.

Several tools are being used currently in spo-ken language research. In the phonetics andprosody domain, the most widely used tool isPraat (Boersma and Weenink, 2018); in the prag-matics and discourse studies community Exmar-alda (Schmidt and Worner, 2009) is appreciatedfor its concordancing functionality; for multi-modal and signed language corpora, ELAN (Brug-man and Russel, 2004) is often preferred; andlarge corpora have been transcribed using Tran-scriberAG (Barras et al., 1998).

We have focused on creating a tool that willcover the needs of researchers working simultane-ously on speech (phonetics, phonology, prosody)and discourse (syntax, semantics, pragmatics).Praaline should help an individual researcher or-ganise their data using the best practices in corpusdesign and data management, and allow them towork independently. At the same time, our systemaims to facilitate depositing the corpus data to acentral repository at the end of a research project,using a structure that will facilitate long-term stor-age and reuse. For these reasons, we have decidedto use an SQL relational database for data stor-age: an SQLite database for local projects, or aMySQL or PostgreSQL database for using Praa-line in a network environment. Furthermore thesystem provides extensive support for importingand exporting data between Praaline and otherannotation tools (Praat, Exmaralda, ELAN, Tran-sriberAG, other formats), as well as support forimporting and exporting annotation data in theXML and JSON formats.

In the following, we will present the main func-tionality of Praaline, describe its design architec-ture and conclude with our plans for future devel-opment.

2 Features

2.1 Corpus Management

A collection of recordings (audio and video) con-stitutes a Corpus, which can be stored in a Cor-pus Repository. A repository provides a com-mon structure for metadata and annotations (seenext section). A Corpus contains a collection ofCommunications and Speakers. Each Communi-cation consists of one or more Recordings and cor-responding Annotation documents. Speakers arelinked to the Communications in which they areinvolved though Participation objects. The six ba-

112

sic objects can all be described using a set of meta-data. Praaline does not impose any schema formetadata: the user can define their own metadataschema per Repository. The Corpus Explorer (seeFigure 1) is used to constitute the corpus and toedit its metadata: corpus objects can be browsedin a tree (possibly grouped based on metadata in-formation); the user can also edit metadata using aspreadsheet-like display.

Figure 1: Corpus Explorer

2.2 Metadata and Annotation StructureEach Corpus Repository will have a defined datastructure for its metadata and annotations. Tem-plates are provided to help individual researchers;it is also possible to standardise these struc-tures when multiple users are accessing the samedatabase. Praaline includes an editor for metadataand annotation structures: the editors perform datadefinition language (DDL) queries to change theSQL database structure. The metadata structureeditor allows the user to define the fields that willbe stored for each of the six main objects describedin the previous section. Annotations are organisedin Levels and Attributes: a Level describes a unitof linguistic analysis (e.g. a syllable, a token, anutterance etc) and contains a label and an arbitrarynumber of Attributes. Each Annotation Level cor-responds to a table in the SQL database and At-tributes correspond to columns. The data structureeditor for annotations is shown in Figure 2).

The metadata and annotation structure editorsallow the user to define fields having any of thestandard SQL data types (integers, real numbers,strings, boolean values etc.). Furthermore, a sys-tem for defining and importing/exporting name-value lists is available. A name-value list (NVL)can be associated with any number of metadatafields or annotation attributes. The NVL subsys-tem can improve data quality by enforcing ref-

Figure 2: Annotation Structure and Vocabularies

erential integrity rules and help users in annotat-ing data based on a closed vocabulary (e.g. a de-fined set of part-of-speech tags, dependency re-lations, discourse relations, phoneme labels etc.).The metadata and annotation structure of a corpusrepository can be exported as an XML or JSONfile; when these files are imported on anotherrepository, Praaline recreates the correspondingdatabase structures and NVL data.

2.3 Annotation

Annotations can be added to the corpus using oneof the Manual Annotation editors: a spreadsheet-like editor that can combine multiple levels of an-notation; a transcriber interface; an editor for se-quences or for relations. The tabular editor is showin Figure 3.

Figure 3: Manual Annotation Editor

The annotation tools offered attempt to coverthe entire workflow of constructing a speech cor-pus: collecting recordings, transcribing them orre-using existing transcriptions, speech-text align-ment, enrichment of the annotations with addi-tional levels and feature extraction (before pro-ceeding with analysing the data). The Transcribermodule provides a user interface for quick manualtranscription, especially in the presence of multi-

113

ple speakers. The Long Sound Aligner module al-lows the user to reuse existing, possibly imperfect,transcriptions of the corpus materials. The LongSound Aligner uses the output of a speech recog-nition engine and the transcription text to pro-duce iteratively refined alignments of transcrip-tion utterances to the speech signal (similar toother long sound alignment tools, e.g. Katsama-nis et al. (2011)). The Forced Alignment mod-ule allows the user to produce alignments at thephone, syllable and token level, based on a pro-nunciation dictionary or other system for phoneti-sation, and an ASR engine. The currently sup-ported ASR engines in Praaline are HTK(Younget al., 2006), PocketSphinx (Walker et al., 2004)and Kaldi (Povey et al., 2011).

The annotation framework in Praaline is lan-guage independent: annotations are stored inUnicode format and no assumptions are madeabout language. However, several tasks requirelanguage-specific resources: tokenisation rules,pronunciation dictionaries, acoustic models andlanguage models for the ASR engine. A collectionof open resources is available, and the develop-ment of language resources for the back-end toolsin the context of other open source projects can beharnessed.

Much of the functionality in Praaline comesfrom its automatic annotation plug-ins. The usercan run a cascade of automatic annotation plugins,after setting the relevant parameters (each plugindefines its own set) on the entire corpus or on asubset of corpus items. The user interface for theseoperations is shown in Figure 4.

Figure 4: Automatic Annotation Plugins

Currently, plugins are available for the follow-ing tasks:

• Part-of-speech tagging and syntactical analy-sis of spoken language, in several languages,

using the DisMo plug-in (Christodoulideset al., 2014). Statistical models for manylanguages are provided, based on the workby the Universal Dependencies project (Nivreet al., 2016).

• Prosodic feature extraction and pitch styli-sation, using either the Prosogram (Mertens,2004) system, or the INTSINT/MoMel sys-tem (Hirst, 2007) for intonation annotation.

• Automatic detection and annotation ofprosodic events, with language-specificstatistical models extracted from manuallyannotated corpora.

2.4 VisualisationThe Visualisation module of Praaline reuses andextends the code of Sonic Visualiser (Cannamet al., 2010), which is also an open-source (GPL)project written in C++/Qt and used in the fieldof musicology. The user can create the visualisa-tion that best suits their needs by combining panesand layers, containing: annotation tiers, points,curves, histograms, colour-coded regions, spec-trograms etc. Extensions have been added forvisualising intonation information, for studies inprosody. Visualisation templates can be saved inXMl and JSON formats; a collection of availablevisualisation templates is presented to the user. Anexample of a visualisation is show in Figure 5.

Figure 5: Visualisation

An annotation editor can be combined with thevisualisation user interface, to facilitate use caseswhere the user codes a linguistic phenomenonwith the aid of a visual representation. Visuali-sations can be exported in image formats for usein presentations and publications. A system ofBookmarks allows the user to save points of in-terest in the corpus: the identification data of theCommunication, Recording and Annotation along

114

with a time-code constitute a Bookmark that canbe stored in collections of bookmarks for easy ac-cess. The results of a concordance search (see nextsection) can also be exported as bookmarks.

2.5 Queries and Concordances

The user can perform queries on any of the an-notation levels or a combination of these levels.The results are displayed in keyword-in-context(KWIC) concordances and can be exported forfurther analysis. It is possible to search for se-quences and patterns. The Concordancer resultscan be exported as Bookmarks for immediate ac-cess in the Annotation editors and in the Visu-aliser. The Concordancer is show in Figure 6.

Figure 6: Concordancer

Furthermore, an interface is provided for cre-ating datasets. Using a dataset, information frommultiple annotation levels can be combined, a setof basic statistic operations can be applied andthe results can be saved for further analysis. Asan example, in a corpus annotated on four lev-els (phones, syllables, tokens, discourse units), theuser may export a dataset containing: all the syl-lables in the corpus, their acoustic features (ex-tracted from upstream plug-ins), the correspond-ing tokens and associated attributes (e.g. POStags), the corresponding discourse units and asso-ciated attributes, and some metadata attributes. Adataset has a minimum unit (in the previous ex-ample, the syllable) and lower-level (grouped) orhigher-level (associated) data.

2.6 Statistics

Praaline includes a set of statistical analysis plu-gins, covering a range of common analyses inspeech research: basic counts; temporal anal-ysis (pauses, speech rate, dialogue dynamics);prosodic and disfluency analysis; clustering cor-pus items using principal component analysis. The

results of the analysis can be exported, and thebasic graphics can be immediately accessed fromwithin Praaline. For more advanced statisticalanalysis of corpus data, users can use R scripts (RCore Team, 2018) that take advantage of the factthat the corpus data and annotations are stored inSQL format. An example of a PCA analysis isshown in Figure 7.

Figure 7: Statistics (PCA plot)

3 System Architecture and Extensibility

The system is following a modular design. A corelibrary (Praaline Core) contains all the basic func-tionality needed to access and manage Praalinecorpus metadata and annotations. The library canbe reused in other software (see also next section).At the next level, a set of modules (Praaline Li-braries) contain the functionality needed for auto-matic speech recognition, the application of ma-chine learning algorithms, feature extraction etc.These libraries are often wrappers of existing andwell-established open-source NLP tools, such asSphinx, OpenSmile, HTK, CRF++ etc. An addi-tional module contains the user interface elements(e.g. widgets for visualisation). All these modulesare brought together in the main application.

An API for the Core library and for interfacingwith the main application allows for the creationof plugins. Plugins add functionality to Praalineand include implementations of automatic annota-tion algorithms, statistical analysis plugins or newvisualisation elements.

Examples of plug-in development are providedon the project’s website: a skeleton C++ project isgiven, and Python scripts using the Praaline Corelibrary, or other modules of the Praaline Librariesillustrate the use of these resources for commonNLP tasks. The Praaline Core library can be usedas a tool for input-output and interfacing with an-

115

notation information without using Praaline’s userinterface.

4 Future Development

Praaline is under active development. The mainpriorities for adding features are as follows:

• A signal annotation editor strongly resem-bling Praat. We have found that researchersin some communities are strongly attachedto this user interface, and this addition willfacilitate the acceptance of a new system ontheir part.

• Stabilise the API of the Praaline Core libraryand provide Python and R bindings.

• Finalise the development of PraalineWeb, asystem that allows the publication of corporaon the web, using Django.

• Add functionality for editing dependency re-lations.

The development of Praaline has been mainlydriven by the expectations and needs of its users.It is hoped that this system demonstration will pro-vide additional feedback.

ReferencesClaude Barras, Edouard Geoffrois, Zhibiao Wu, and

Mark Liberman. 1998. Transcriber: a free toolfor segmenting, labeling and transcribing speech.In LREC 1998 – 1st International Conference onLanguage Resources and Evaluation, 28–30 May,Granada, Spain Proceedings, pages 1373–1376.

Paul Boersma and David Weenink. 2018. Praat: doingphonetics by computer, ver. 6.0.37.

Hennie Brugman and Albert Russel. 2004. Annotatingmultimedia and multi-modal resources with elan. InLREC 2004 – 4th International Conference on Lan-guage Resources and Evaluation, May 26–28, Paris,France, Proceedings, pages 2065–2068.

Chris Cannam, Christian Landone, and Sandler Mark.2010. Sonic visualiser: An open source applicationfor viewing, analysing, and annotating music audiofiles. In Proceedings of the ACM Multimedia 2010International Conference, pages 1467–1468.

George Christodoulides. 2014. Praaline: Integratingtools for speech corpus research. In LREC 2014– 9th International Conference on Language Re-sources and Evaluation, May 26–31, Reykjavik, Ice-land, Proceedings, pages 31–34.

George Christodoulides, Mathieu Avanzi, andJean Philippe Goldman. 2014. DisMo: A mor-phosyntactic, disfluency and multi-word unitannotator. In LREC 2014 – 9th International Con-ference on Language Resources and Evaluation,May 26–31, Reykjavik, Iceland, Proceedings, pages3902–3907.

Daniel Hirst. 2007. A praat plugin for momel andintsint with improved algorithms for modelling andcoding intonation.

Athanasios Katsamanis, Matthew P. Black, Panayio-tis G. Georgiou, Louis Goldstein, and Shrikanth S.Narayanan. 2011. SailAlign: Robust long speech-text alignment. In Proceedings of the Workshop onNew Tools and Methods for Very-Large Scale Pho-netics Research.

Piet Mertens. 2004. The Prosogram: Semi-automatictranscription of prosody based on a tonal perceptionmodel. In Proc. of Speech Prosody 2004, March 23–26, Nara, Japan, pages 549–552.

Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin-ter, Yoav Goldberg, Jan Hajic, Christopher D. Man-ning, Ryan McDonald, Slav Petrov, Sampo Pyysalo,Natalia Silveira, and et al. 2016. Universal depen-dencies v1: A multilingual treebank collection. InProceedings of the 10th International Conference onLanguage Resources and Evaluation (LREC 2016).

Daniel Povey, Arnab Ghoshal, Gilles Boulianne, LukasBurget, Ondrej Glembek, Nagendra Goel, MirkoHannemann, Petr Motlicek, Yanmin Qian, PetrSchwarz, Jan Silovsky, Georg Stemmer, and KarelVesely. 2011. The kaldi speech recognition toolkit.In IEEE 2011 Workshop on Automatic SpeechRecognition and Understanding. IEEE Signal Pro-cessing Society. IEEE Catalog No.: CFP11SRW-USB.

R Core Team. 2018. R: A language and environmentfor statistical computing. R Foundation for Statisti-cal Computing, Vienna, Austria.

Thomas Schmidt and Kai Worner. 2009. EXMAR-aLDA – creating, analysing and sharing spoken lan-guage corpora for pragmatic research. Pragmatics,19(4):565–582.

Willie Walker, Paul Lamere, Philip Kwok, BhikshaRaj, Rita Singh, Evandro Gouvea, Peter Wolf, andJoe Woelfel. 2004. Sphinx-4: A flexible open sourceframework for speech recognition. Technical report,Mountain View, CA, USA.

Steve J. Young, D. Kershaw, J. Odell, D. Ollason,V. Valtchev, and P. Woodland. 2006. The HTK BookVersion 3.4. Cambridge University Press.