multi-view contrastive self-supervised learning of

Multi-view Contrastive Self-Supervised Learning ofAccounting Data Representations for Downstream Audit Tasks

Marco SchreyerUniversity of St. GallenSt. Gallen, Switzerland

[email protected]

Timur SattarovDeutsche Bundesbank

Frankfurt am Main, [email protected]

Damian BorthUniversity of St. GallenSt. Gallen, [email protected]

ABSTRACTInternational audit standards require the direct assessment of afinancial statement’s underlying accounting transactions, referredto as journal entries. Recently, driven by the advances in artificialintelligence, deep learning inspired audit techniques have emergedin the field of auditing vast quantities of journal entry data. Nowa-days, the majority of such methods rely on a set of specializedmodels, each trained for a particular audit task. At the same time,when conducting a financial statement audit, audit teams are con-fronted with (i) challenging time-budget constraints, (ii) extensivedocumentation obligations, and (iii) strict model interpretabilityrequirements. As a result, auditors prefer to harness only a singlepreferably ‘multi-purpose’ model throughout an audit engagement.We propose a contrastive self-supervised learning framework de-signed to learn audit task invariant accounting data representationsto meet this requirement. The framework encompasses deliberateinteracting data augmentation policies that utilize the attributecharacteristics of journal entry data. We evaluate the framework ontwo real-world datasets of city payments and transfer the learnedrepresentations to three downstream audit tasks: anomaly detec-tion, audit sampling, and audit documentation. Our experimentalresults provide empirical evidence that the proposed frameworkoffers the ability to increase the efficiency of audits by learning richand interpretable ‘multi-task’ representations.

CCS CONCEPTS• Computing methodologies → Machine learning; Multi-tasklearning; Dimensionality reduction and manifold learning; • Infor-mation systems→ Enterprise resource planning.

KEYWORDSrepresentation learning, self-supervised learning, multi-task learn-ing, audit, computer assisted audit techniques, accounting informa-tion systems, enterprise resource planning systemsACM Reference Format:Marco Schreyer, Timur Sattarov, and Damian Borth. 2021. Multi-view Con-trastive Self-Supervised Learning of Accounting Data Representations forDownstream Audit Tasks. In Proceedings of (Preprint) , 8 pages.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected], currently under review.© 2021 Association for Computing Machinery.ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00

3

Data

base

Accounting Information System

(AIS)

Company Entry ID Fiscal Year Type Date Currency Reference Text Transaction

AAA 100011 2017 SA 31.10.2016 USD 000349 Annual... SA01

AAA 100012 2017 MZ 31.10.2016 USD - Material... MM23

BBB 900124 2017 IN 01.02.2017 USD 148292 Invoice... IN04

... ... ... ... ... … … … …

Journal Entry Headers Table

Journal Entry Segments Table

Company Entry ID Sub-ID Currency Amount D/C Ledger Vendor Customer

AAA 100011 0001 USD 1’000.00 D 1000129 - -

AAA 100011 0002 USD 1’000.00 C 4010111 5890 -

BBB 900124 0001 USD 2’232.00 D 1022001 - 82304

... ... ... ... ... ... … … …

Acco

untin

gPr

oces

s

GoodReceived

IncomingInvoice

Outgoing Payment

PurchaseRequest

Vendor Master Data

Expenses Liabilities BankCreditDebit CreditDebit CreditDebit

PurchaseRequest

GoodReceived

GoodReceived

PurchaseOrder

$ 1,000$ 1,000 $ 1,000$ 1,000

Figure 1: Hierarchical view of an Accounting InformationSystem (AIS) that records distinct layer of abstractions,namely (1) the business process, (2) the accounting and (3)technical journal entry information in designated tables.

1 INTRODUCTIONIn general, auditors are mandated to collect reasonable assurancethat an issued financial statement is free from material misstate-ment (’true and fair presentation‘ ) [4, 26]. Traditionally, auditinghas been viewed as one of the cornerstones of trustworthy financialstatements. Independent financial audits seek to reduce informationasymmetries between management and stakeholders. As a result,the audit opinion of external auditors plays a fundamental role inthe economic decision making of investors [23]. To detect possiblemisstatements, international audit standards demand the directassessment of a statement’s underlying accounting transactions,usually referred to as ’journal entries‘ [1]. Nowadays, organizationsrecord vast quantities of such journal entries in Accounting Infor-mation Systems (AIS) or more general Enterprise Resource Planning(ERP) systems [18]. Figure 1 depicts an exemplary hierarchical viewof the journal entry recording process in designated database tablesof an ERP system.

When auditing detailed journal entry data, auditors are con-stantly forced to trade off the time dedicated to the audit, and itsquality [9]. In recent years accounting firms have responded withtechnological innovations to increase audit efficiency and mitigatethe audit’s time pressure [38]. Driven by such efforts and the rapidtechnological advances of artificial intelligence, deep learning [39]techniques are applied in external and internal financial audits[42, 54]. Nowadays, the majority of deep learning-inspired audittechniques rely on several models, each highly specialized towardsa particular audit task, e.g., the detection of anomalous journal

arX

iv:2

109.

1120

1v1

[cs

.LG

] 2

3 Se

p 20

21

Preprint, currently under review. Schreyer and Sattarov, et al.

entries [48] or to learn a representative audit sample [49]. Causedby the characteristics of modern audit engagements, such as (i)challenging time-budget constraints, (ii) extensive documentationobligations, and (iii) strict model interpretability requirements, audi-tors often desire to train and apply only a single (or limited numberof) model(s) to audit vast quantities of accounting records. Thisobservation motivates the design of techniques that learn invariantaccounting data representations transferable across different audittasks. Learning a single set of ‘general-purpose’ representationspotentially provides the ability to generalize better on a single task.

Learning task invariant representations without human supervi-sion is a long-standing challenge in computer vision. Recently, theidea of contrastive self-supervised learning [10, 20, 41, 45, 57] hasemerged as a promising avenue of general-purpose representationlearning, achieving state-of-the-art performance on the ImageNetclassification challenge [13] and closing the gap between supervisedand unsupervised training. Self-supervised learning (SSL) [2, 14, 60]exploits labels that can be ‘freely’ derived from the data and usesthem as intrinsic reward signals to learn transferable representa-tions. Contrastive learning (CL) [19] retains ‘rich’ representationsfrom multiple views of a single data sample by letting them attracteach other while at the same time repelling them from other sam-ples. Such views are derived from different data augmentationsin order to generalize to a variety of downstream tasks [17], e.g.,image classification or image segmentation.

In this work, inspired by the recent successes of contrastive self-supervised learning, we investigate whether such techniques canbe utilized to learn task invariant representations of accountingrecords? Moreover, are the learned representations transferableto different downstream audit tasks? In summary, we present thefollowing contributions:

• Trainability - We propose a learning framework that ex-tracts information from intra-attribute accounting data char-acteristics and exploits inter-attribute correlations.

• Transferability - We demonstrate that the pre-trained rep-resentations can be transferred to different downstream audittasks and achieve high performance.

• Auditability -We illustrate that the learned latent inter- andintra-attribute semantic disentanglement allows for a visualinspection and high interpretability by human auditors.

We believe that learning general-purpose representations ofaccounting data for downstream audit tasks is an essential butunder-explored domain in the context of auditing. We regard thiswork to be an initial but promising step towards the adoption ofself-supervised learning techniques in internal and external audits.

The remainder of this work is structured as follows: In Section2, we provide an overview of related work. Section 3 follows witha description of the proposed methodology to learn transferableaccounting data representations from vast quantities of journalentries. The experimental setup and results are outlined in Section4 and Section 5. In Section 6, the paper concludes with a summaryof the current work and highlights future research directions. Areference implementation of the proposed methodology will bemade available via [url redacted due to double blind review].

2 RELATEDWORKIn recent years, techniques that build on (deep) machine learninghave been gradually applied to different audit tasks [11, 54]. At thesame time, the idea of contrastive self-supervised learning triggereda sizable body of research by academia [28, 32]. In this work, wefocus our literature review on (i) learning representations of real-world accounting data and (ii) contrastive self-supervised deeplearning techniques.

2.1 Accounting Data Representation LearningSince the advent of AIS systems, the design of relevant data featureshas been of central relevance in auditing. Various techniques havebeen proposed that depend on human-engineered data representa-tions, usually referred to as features, to audit structured account-ing data. Such representations have been derived from: (i) specifictransaction attribute characteristics [6–8, 30, 40, 55], (ii) transactionamount distributions [12, 53], (iii) transaction or audit log infor-mation [27, 33–35], or (iv) business process event logs [29, 31, 62].With the advent of deep learning, novel representation learningtechniques have emerged in auditing. Such methods rely on ‘end-to-end’ machine-learned accounting data representations ratherthan human-engineered features. Such techniques encompass: (i)autoencoder neural networks [43, 48, 52], (ii) variational autoen-coders [66], and (iii) adversarial autoencoders [50]. Lately, vectorquantized variational autoencoders have been applied to learn rep-resentative audit sampling [49]. Concluding from the reviewedliterature, most references either draw from human-engineered rep-resentations or learned representations highly optimized towards aparticular audit task.

2.2 Self-Supervised Representation LearningThe convention of self-supervised learning (SSL) [10, 20, 41, 45, 57]is to solve a manually-defined ‘pretext’ task for task invariant rep-resentation learning. In computer vision, predicting the relativelocation of image patches has shown to be a successful pretext task[14, 15, 44]. Recently, contrastive learning (CL) algorithms havebeen successfully employed in pretext training phases to learn in-variant representations of data observations. This is implementedby minimizing a contrastive loss [45, 51, 61] evaluated on pairsof augmented feature vectors derived from a single data observa-tion. In computer vision, multiple image augmentation strategieshave been proposed [10, 20, 41, 56, 63, 65], e.g., rotation, cropping,random greyscale, and color jittering. Similarly, in speech analy-sis, several audio augmentation methods have been successfullyapplied [36, 46, 59], e.g., noise augmentation, band-reject filtering,and time masking. In summary, none of the reviewed literaturefocuses on applying contrastive self-supervised learning in internaland external financial audits.

To the best of our knowledge, this work presents a first steptowards learning general purpose representations of structuredaccounting data for downstream audit tasks.

3 METHODOLOGYIn this section, we describe the architectural components, dataaugmentation techniques, and learning objective applied to learntask invariant accounting data representations.

Multi-view Contrastive Self-Supervised Learning ofAccounting Data Representations for Downstream Audit Tasks Preprint, currently under review.

1

§ Company Code

§ Posting Key

§ Date

§ Account

§ Debit / Credit

§ Amount

§ Vendor ID

§ T-Code

Journal Entry

…

f✓<latexit sha1_base64="XzBH6XintChx/rhteU50ZISc76s=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lqQY8FLx4r2A9oQ9lsN+3SzSbuToQS+ie8eFDEq3/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1gNOE+xEdKREKRtFK3XDQxzFHOihX3Kq7AFknXk4qkKM5KH/1hzFLI66QSWpMz3MT9DOqUTDJZ6V+anhC2YSOeM9SRSNu/Gxx74xcWGVIwljbUkgW6u+JjEbGTKPAdkYUx2bVm4v/eb0Uwxs/EypJkSu2XBSmkmBM5s+TodCcoZxaQpkW9lbCxlRThjaikg3BW315nbRrVe+qWruvVxr1PI4inME5XIIH19CAO2hCCxhIeIZXeHMenRfn3flYthacfOYU/sD5/AEao4/3</latexit>




...

⌧+1

<latexit sha1_base64="bGPBBJmE/LAyGAarEV+HHryQmkk=">AAAB83icbVDLSsNAFL2pr1pfVZduBosgCCWpgi4LblxWsA9oYplMJ+3QySTMQyghv+HGhSJu/Rl3/o3TNgttPXDhcM693HtPmHKmtOt+O6W19Y3NrfJ2ZWd3b/+genjUUYmRhLZJwhPZC7GinAna1kxz2kslxXHIaTec3M787hOViiXiQU9TGsR4JFjECNZW8n2NzWN2kQ8yLx9Ua27dnQOtEq8gNSjQGlS//GFCTEyFJhwr1ffcVAcZlpoRTvOKbxRNMZngEe1bKnBMVZDNb87RmVWGKEqkLaHRXP09keFYqWkc2s4Y67Fa9mbif17f6OgmyJhIjaaCLBZFhiOdoFkAaMgkJZpPLcFEMnsrImMsMdE2pooNwVt+eZV0GnXvst64v6o13SKOMpzAKZyDB9fQhDtoQRsIpPAMr/DmGOfFeXc+Fq0lp5g5hj9wPn8A9juRkw==</latexit>

⌧+2

<latexit sha1_base64="ma9tYmqh06zWAnqtzcbM/VBViGc=">AAAB83icbVDLSsNAFL2pr1pfVZduBosgCCWpgi4LblxWsA9oYplMJ+3QySTMQyghv+HGhSJu/Rl3/o3TNgttPXDhcM693HtPmHKmtOt+O6W19Y3NrfJ2ZWd3b/+genjUUYmRhLZJwhPZC7GinAna1kxz2kslxXHIaTec3M787hOViiXiQU9TGsR4JFjECNZW8n2NzWN2kQ+yRj6o1ty6OwdaJV5BalCgNah++cOEmJgKTThWqu+5qQ4yLDUjnOYV3yiaYjLBI9q3VOCYqiCb35yjM6sMUZRIW0Kjufp7IsOxUtM4tJ0x1mO17M3E/7y+0dFNkDGRGk0FWSyKDEc6QbMA0JBJSjSfWoKJZPZWRMZYYqJtTBUbgrf88irpNOreZb1xf1VrukUcZTiBUzgHD66hCXfQgjYQSOEZXuHNMc6L8+58LFpLTjFzDH/gfP4A98CRlA==</latexit>

x̂+2

<latexit sha1_base64="JgcCDputMOBxkKZUhUFt5mb0hZU=">AAAB+HicbVBNS8NAEN34WetHox69LBZBEEpSBT0WvHisYD+gjWGz3bRLN5uwOxFryC/x4kERr/4Ub/4bt20O2vpg4PHeDDPzgkRwDY7zba2srq1vbJa2yts7u3sVe/+greNUUdaisYhVNyCaCS5ZCzgI1k0UI1EgWCcYX0/9zgNTmsfyDiYJ8yIylDzklICRfLvSHxHIHnM/q+f32Vnu21Wn5syAl4lbkCoq0PTtr/4gpmnEJFBBtO65TgJeRhRwKlhe7qeaJYSOyZD1DJUkYtrLZofn+MQoAxzGypQEPFN/T2Qk0noSBaYzIjDSi95U/M/rpRBeeRmXSQpM0vmiMBUYYjxNAQ+4YhTExBBCFTe3YjoiilAwWZVNCO7iy8ukXa+557X67UW14RRxlNAROkanyEWXqIFuUBO1EEUpekav6M16sl6sd+tj3rpiFTOH6A+szx8KCpNG</latexit>

x̂+1

<latexit sha1_base64="JqjzL4gg4fezNAe9c8GcNHSUZy8=">AAAB+HicbVBNS8NAEN34WetHox69LBZBEEpSBT0WvHisYD+gjWGz3bRLN5uwOxFryC/x4kERr/4Ub/4bt20O2vpg4PHeDDPzgkRwDY7zba2srq1vbJa2yts7u3sVe/+greNUUdaisYhVNyCaCS5ZCzgI1k0UI1EgWCcYX0/9zgNTmsfyDiYJ8yIylDzklICRfLvSHxHIHnM/c/P77Cz37apTc2bAy8QtSBUVaPr2V38Q0zRiEqggWvdcJwEvIwo4FSwv91PNEkLHZMh6hkoSMe1ls8NzfGKUAQ5jZUoCnqm/JzISaT2JAtMZERjpRW8q/uf1UgivvIzLJAUm6XxRmAoMMZ6mgAdcMQpiYgihiptbMR0RRSiYrMomBHfx5WXSrtfc81r99qLacIo4SugIHaNT5KJL1EA3qIlaiKIUPaNX9GY9WS/Wu/Uxb12xiplD9AfW5w8IgZNF</latexit>

xqi

<latexit sha1_base64="JlNu1CPmm7thQMRscaXQ+44cRhc=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBU9mtgh4LXjxWsB/SriWbZtvQJLsmWbEs+yu8eFDEqz/Hm//GtN2Dtj4YeLw3w8y8IOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpaNEEdokEY9UJ8CaciZp0zDDaSdWFIuA03Ywvpr67UeqNIvkrZnE1Bd4KFnICDZWunvqpyy7Tx+yfrniVt0Z0DLxclKBHI1++as3iEgiqDSEY627nhsbP8XKMMJpVuolmsaYjPGQdi2VWFDtp7ODM3RilQEKI2VLGjRTf0+kWGg9EYHtFNiM9KI3Ff/zuokJL/2UyTgxVJL5ojDhyERo+j0aMEWJ4RNLMFHM3orICCtMjM2oZEPwFl9eJq1a1Tur1m7OK3U3j6MIR3AMp+DBBdThGhrQBAICnuEV3hzlvDjvzse8teDkM4fwB87nD3I3kMU=</latexit>

... ...





⌧�1<latexit sha1_base64="TaiPe4QimArwPs3p0+5XI3AI4/M=">AAAB83icbVDLSsNAFL2pr1pfVZduBovgxpJUQZcFNy4r2Ac0sUymk3boZBLmIZSQ33DjQhG3/ow7/8Zpm4W2HrhwOOde7r0nTDlT2nW/ndLa+sbmVnm7srO7t39QPTzqqMRIQtsk4YnshVhRzgRta6Y57aWS4jjktBtObmd+94lKxRLxoKcpDWI8EixiBGsr+b7GZpB5+WN2kQ+qNbfuzoFWiVeQGhRoDapf/jAhJqZCE46V6ntuqoMMS80Ip3nFN4qmmEzwiPYtFTimKsjmN+fozCpDFCXSltBorv6eyHCs1DQObWeM9VgtezPxP69vdHQTZEykRlNBFosiw5FO0CwANGSSEs2nlmAimb0VkTGWmGgbU8WG4C2/vEo6jbp3WW/cX9WabhFHGU7gFM7Bg2towh20oA0EUniGV3hzjPPivDsfi9aSU8wcwx84nz/5YZGV</latexit>

⌧�2<latexit sha1_base64="cIezIZyHMzdsbmhBIPc4/Hj9hRs=">AAAB83icbVDLSsNAFL2pr1pfVZduBovgxpJUQZcFNy4r2Ac0sUymk3boZBLmIZSQ33DjQhG3/ow7/8Zpm4W2HrhwOOde7r0nTDlT2nW/ndLa+sbmVnm7srO7t39QPTzqqMRIQtsk4YnshVhRzgRta6Y57aWS4jjktBtObmd+94lKxRLxoKcpDWI8EixiBGsr+b7GZpA18sfsIh9Ua27dnQOtEq8gNSjQGlS//GFCTEyFJhwr1ffcVAcZlpoRTvOKbxRNMZngEe1bKnBMVZDNb87RmVWGKEqkLaHRXP09keFYqWkc2s4Y67Fa9mbif17f6OgmyJhIjaaCLBZFhiOdoFkAaMgkJZpPLcFEMnsrImMsMdE2pooNwVt+eZV0GnXvst64v6o13SKOMpzAKZyDB9fQhDtoQRsIpPAMr/DmGOfFeXc+Fq0lp5g5hj9wPn8A+uqRlg==</latexit>

⌧�3<latexit sha1_base64="fRQZGBI3XDwO4fmcN2W67iE/SyI=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBiyVpBT0WvHisYGuhiWWz3bRLN5uwOyuUkL/hxYMiXv0z3vw3btsctPXBwOO9GWbmhangGlz32ymtrW9sbpW3Kzu7e/sH1cOjrk6MoqxDE5GoXkg0E1yyDnAQrJcqRuJQsIdwcjPzH56Y0jyR9zBNWRCTkeQRpwSs5PtAzCBr5o/ZRT6o1ty6OwdeJV5BaqhAe1D98ocJNTGTQAXRuu+5KQQZUcCpYHnFN5qlhE7IiPUtlSRmOsjmN+f4zCpDHCXKlgQ8V39PZCTWehqHtjMmMNbL3kz8z+sbiK6DjMvUAJN0sSgyAkOCZwHgIVeMgphaQqji9lZMx0QRCjamig3BW355lXQbda9Zb9xd1lpuEUcZnaBTdI48dIVa6Ba1UQdRlKJn9IreHOO8OO/Ox6K15BQzx+gPnM8f/HORlw==</latexit>

x̂�2

<latexit sha1_base64="p2687myUEOQKEOGXJ7Yz0lNmRis=">AAAB+HicbVBNS8NAEN34WetHox69LBbBiyWpgh4LXjxWsB/QxrDZbtqlm03YnYg15Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QSK4Bsf5tlZW19Y3Nktb5e2d3b2KvX/Q1nGqKGvRWMSqGxDNBJesBRwE6yaKkSgQrBOMr6d+54EpzWN5B5OEeREZSh5ySsBIvl3pjwhkj/l9dpb7WT337apTc2bAy8QtSBUVaPr2V38Q0zRiEqggWvdcJwEvIwo4FSwv91PNEkLHZMh6hkoSMe1ls8NzfGKUAQ5jZUoCnqm/JzISaT2JAtMZERjpRW8q/uf1UgivvIzLJAUm6XxRmAoMMZ6mgAdcMQpiYgihiptbMR0RRSiYrMomBHfx5WXSrtfc81r99qLacIo4SugIHaNT5KJL1EA3qIlaiKIUPaNX9GY9WS/Wu/Uxb12xiplD9AfW5w8M/JNI</latexit>

x̂�3

<latexit sha1_base64="vBnwcMEkkdnStQ5Zfs5F7gd4M6g=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIXixJK+ix4MVjBfsBbQyb7bZdutmE3Y1YQ36JFw+KePWnePPfuG1z0NYHA4/3ZpiZF8ScKe0431ZhbX1jc6u4XdrZ3dsv2weHbRUlktAWiXgkuwFWlDNBW5ppTruxpDgMOO0Ek+uZ33mgUrFI3OlpTL0QjwQbMoK1kXy73B9jnT5m9+l55qf1zLcrTtWZA60SNycVyNH07a/+ICJJSIUmHCvVc51YeymWmhFOs1I/UTTGZIJHtGeowCFVXjo/PEOnRhmgYSRNCY3m6u+JFIdKTcPAdIZYj9WyNxP/83qJHl55KRNxoqkgi0XDhCMdoVkKaMAkJZpPDcFEMnMrImMsMdEmq5IJwV1+eZW0a1W3Xq3dXlQaTh5HEY7hBM7AhUtowA00oQUEEniGV3iznqwX6936WLQWrHzmCP7A+vwBDoGTSQ==</latexit>

0 1 0 … 0

1 0 0 … 0

0 1 0 … 0

0 0 0 … 0

0 0 1 … 0

0 0 0 … 1

0 0 0 … 1

0 1 0 … 0...

...

xi 2 X<latexit sha1_base64="fNm0YNGpN4McSFazWoKLcwez5yg=">AAAB8HicbVBNSwMxEJ3Ur1q/qh69BIvgqexWQY8FLx4r2A9pl5JNs21okl2SrFiW/govHhTx6s/x5r8xbfegrQ8GHu/NMDMvTAQ31vO+UWFtfWNzq7hd2tnd2z8oHx61TJxqypo0FrHuhMQwwRVrWm4F6ySaERkK1g7HNzO//ci04bG6t5OEBZIMFY84JdZJD099jntc4U6/XPGq3hx4lfg5qUCORr/81RvENJVMWSqIMV3fS2yQEW05FWxa6qWGJYSOyZB1HVVEMhNk84On+MwpAxzF2pWyeK7+nsiINGYiQ9cpiR2ZZW8m/ud1UxtdBxlXSWqZootFUSqwjfHsezzgmlErJo4Qqrm7FdMR0YRal1HJheAvv7xKWrWqf1Gt3V1W6rU8jiKcwCmcgw9XUIdbaEATKEh4hld4Qxq9oHf0sWgtoHzmGP4Aff4A/jyP0w==</latexit>

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.1 0.9 0.3 … 0.1

0.9 0.1 0.2 … 0.1

0.1 0.3 0.3 … 1.1

0.1 0.1 0.2 … 0.9

...

0.2 0.9 0.1 … 0.1

1.1 0.0 0.2 … 0.3

0.1 0.3 0.3 … 1.1

0.3 0.1 0.2 … 0.9

...

0.1 1.1 0.2 … 0.1

0.9 0.2 0.1 … 0.0

0.1 0.3 0.3 … 0.9

0.1 0.3 0.2 … 1.2

...

§ Company Code

§ Posting Key

§ Date

§ Account

§ Debit / Credit

§ Amount

§ Vendor ID

§ T-Code

Journal Entry

…

0 1 0 … 0

1 0 0 … 0

0 1 0 … 0

0 0 0 … 0

0 0 1 … 0

0 0 0 … 1

0 0 0 … 1

0 1 0 … 0...

...


...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

0.0 1.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

0.0 0.0 1.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

Positive (intra-attribute) data augmentations Negative (inter-attribute) data augmentations

Encodedentry

Encodedentry

eqi

<latexit sha1_base64="FB754kikMRC56ik+DTtE699vGuw=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkV9Fjw4rGC/ZA2ls122i7dTeLuRighv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLYsG1cd1vZ2V1bX1js7BV3N7Z3dsvHRw2dZQohg0WiUi1A6pR8BAbhhuB7VghlYHAVjC+nvqtJ1SaR+GdmcToSzoM+YAzaqx0jw/pY9ZLedYrld2KOwNZJl5OypCj3it9dfsRSySGhgmqdcdzY+OnVBnOBGbFbqIxpmxMh9ixNKQStZ/ODs7IqVX6ZBApW6EhM/X3REql1hMZ2E5JzUgvelPxP6+TmMGVn/IwTgyGbL5okAhiIjL9nvS5QmbExBLKFLe3EjaiijJjMyraELzFl5dJs1rxzivV24tyrZrHUYBjOIEz8OASanADdWgAAwnP8ApvjnJenHfnY9664uQzR/AHzucPVYmQtA==</latexit>

e+1

<latexit sha1_base64="E2emIGdb0Ryhn6J2TwjhX5B15r0=">AAAB8HicbVBNS8NAEJ34WetX1aOXYBEEoSRV0GPBi8cK9kPaWjbbSbt0dxN2N0IJ+RVePCji1Z/jzX/jts1BWx8MPN6bYWZeEHOmjed9Oyura+sbm4Wt4vbO7t5+6eCwqaNEUWzQiEeqHRCNnElsGGY4tmOFRAQcW8H4Zuq3nlBpFsl7M4mxJ8hQspBRYqz0gI/pedZP/axfKnsVbwZ3mfg5KUOOer/01R1ENBEoDeVE647vxaaXEmUY5ZgVu4nGmNAxGWLHUkkE6l46OzhzT60ycMNI2ZLGnam/J1IitJ6IwHYKYkZ60ZuK/3mdxITXvZTJODEo6XxRmHDXRO70e3fAFFLDJ5YQqpi91aUjogg1NqOiDcFffHmZNKsV/6JSvbss16p5HAU4hhM4Ax+uoAa3UIcGUBDwDK/w5ijnxXl3PuatK04+cwR/4Hz+AJTskDY=</latexit>

e+2

<latexit sha1_base64="ETE8hwTEKN292J0Tx2WuTWOTO1Q=">AAAB8HicbVBNS8NAEN34WetX1aOXxSIIQkmioMeCF48V7Ie0sWy2k3bp7ibsboQS8iu8eFDEqz/Hm//GbZuDtj4YeLw3w8y8MOFMG9f9dlZW19Y3Nktb5e2d3b39ysFhS8epotCkMY9VJyQaOJPQNMxw6CQKiAg5tMPxzdRvP4HSLJb3ZpJAIMhQsohRYqz0AI/Zed7P/Lxfqbo1dwa8TLyCVFGBRr/y1RvENBUgDeVE667nJibIiDKMcsjLvVRDQuiYDKFrqSQCdJDNDs7xqVUGOIqVLWnwTP09kRGh9USEtlMQM9KL3lT8z+umJroOMiaT1ICk80VRyrGJ8fR7PGAKqOETSwhVzN6K6YgoQo3NqGxD8BZfXiYtv+Zd1Py7y2rdL+IooWN0gs6Qh65QHd2iBmoiigR6Rq/ozVHOi/PufMxbV5xi5gj9gfP5A5ZxkDc=</latexit>

e+`1

<latexit sha1_base64="hf5NTc4aQuGht7nzGcTrSrzTkr0=">AAAB+XicbVBNS8NAEN3Ur1q/oh69LBZBEEpSBT0WvHisYGuhjWGznbZLN5uwuymUJf/EiwdFvPpPvPlv3LY5aOuDgcd7M8zMi1LOlPa8b6e0tr6xuVXeruzs7u0fuIdHbZVkkkKLJjyRnYgo4ExASzPNoZNKIHHE4TEa3878xwlIxRLxoKcpBDEZCjZglGgrha4LT+YiD00POA+Nn+ehW/Vq3hx4lfgFqaICzdD96vUTmsUgNOVEqa7vpTowRGpGOeSVXqYgJXRMhtC1VJAYVGDml+f4zCp9PEikLaHxXP09YUis1DSObGdM9EgtezPxP6+b6cFNYJhIMw2CLhYNMo51gmcx4D6TQDWfWkKoZPZWTEdEEqptWBUbgr/88ipp12v+Za1+f1Vt1Is4yugEnaJz5KNr1EB3qIlaiKIJekav6M0xzovz7nwsWktOMXOM/sD5/AGk0pOd</latexit>

z+`1

<latexit sha1_base64="Mcs16skclmUbtDIhPWqcQAxt7TM=">AAAB+XicbVDLSgNBEOyNrxhfqx69DAZBEMJuFPQY8OIxgnlAsi6zk9lkyOyDmdlAHPZPvHhQxKt/4s2/cZLsQRMLGoqqbrq7gpQzqRzn2yqtrW9sbpW3Kzu7e/sH9uFRWyaZILRFEp6IboAl5SymLcUUp91UUBwFnHaC8e3M70yokCyJH9Q0pV6EhzELGcHKSL5tPz3qi9zXfcq5r9089+2qU3PmQKvELUgVCjR9+6s/SEgW0VgRjqXsuU6qPI2FYoTTvNLPJE0xGeMh7Rka44hKT88vz9GZUQYoTISpWKG5+ntC40jKaRSYzgirkVz2ZuJ/Xi9T4Y2nWZxmisZksSjMOFIJmsWABkxQovjUEEwEM7ciMsICE2XCqpgQ3OWXV0m7XnMva/X7q2qjXsRRhhM4hXNw4RoacAdNaAGBCTzDK7xZ2nqx3q2PRWvJKmaO4Q+szx/F4ZOy</latexit>

z+2

<latexit sha1_base64="HcqiHJqGvKouJP3wsjBgcAtsL5s=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRZBEMpuFfRY8OKxgv2Qdi3ZNNuGJtklyQp12V/hxYMiXv053vw3pu0etPXBwOO9GWbmBTFn2rjut1NYWV1b3yhulra2d3b3yvsHLR0litAmiXikOgHWlDNJm4YZTjuxolgEnLaD8fXUbz9SpVkk78wkpr7AQ8lCRrCx0v3TQ3qW9dNa1i9X3Ko7A1omXk4qkKPRL3/1BhFJBJWGcKx113Nj46dYGUY4zUq9RNMYkzEe0q6lEguq/XR2cIZOrDJAYaRsSYNm6u+JFAutJyKwnQKbkV70puJ/Xjcx4ZWfMhknhkoyXxQmHJkITb9HA6YoMXxiCSaK2VsRGWGFibEZlWwI3uLLy6RVq3rn1drtRaVey+MowhEcwyl4cAl1uIEGNIGAgGd4hTdHOS/Ou/Mxby04+cwh/IHz+QO27ZBM</latexit>

z+1

<latexit sha1_base64="DnTBrQ2Ljapou3nBKDjL9wxXBnA=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRZBEMpuFfRY8OKxgv2Qdi3ZNNuGJtklyQp12V/hxYMiXv053vw3pu0etPXBwOO9GWbmBTFn2rjut1NYWV1b3yhulra2d3b3yvsHLR0litAmiXikOgHWlDNJm4YZTjuxolgEnLaD8fXUbz9SpVkk78wkpr7AQ8lCRrCx0v3TQ3qW9VMv65crbtWdAS0TLycVyNHol796g4gkgkpDONa667mx8VOsDCOcZqVeommMyRgPaddSiQXVfjo7OEMnVhmgMFK2pEEz9fdEioXWExHYToHNSC96U/E/r5uY8MpPmYwTQyWZLwoTjkyEpt+jAVOUGD6xBBPF7K2IjLDCxNiMSjYEb/HlZdKqVb3zau32olKv5XEU4QiO4RQ8uIQ63EADmkBAwDO8wpujnBfn3fmYtxacfOYQ/sD5/AG1aJBL</latexit>

zqi

<latexit sha1_base64="cGCK5ea21ze0PrCl1MV1gPEv/2M=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBU9mtgh4LXjxWsB/SriWbZtvQJLsmWaEu+yu8eFDEqz/Hm//GtN2Dtj4YeLw3w8y8IOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpaNEEdokEY9UJ8CaciZp0zDDaSdWFIuA03Ywvpr67UeqNIvkrZnE1Bd4KFnICDZWunu6Tx+yfsqyfrniVt0Z0DLxclKBHI1++as3iEgiqDSEY627nhsbP8XKMMJpVuolmsaYjPGQdi2VWFDtp7ODM3RilQEKI2VLGjRTf0+kWGg9EYHtFNiM9KI3Ff/zuokJL/2UyTgxVJL5ojDhyERo+j0aMEWJ4RNLMFHM3orICCtMjM2oZEPwFl9eJq1a1Tur1m7OK/VaHkcRjuAYTsGDC6jDNTSgCQQEPMMrvDnKeXHenY95a8HJZw7hD5zPH3YFkMk=</latexit>

⌧+`i<latexit sha1_base64="mzVLqDCZm97VVRcAufcQUYKHnpI=">AAAB/HicbVDLSsNAFJ3UV62vaJduBosgCCWpgi4LblxWsA9oYphMJ+3QyYOZGyGE+CtuXCji1g9x5984bbPQ1gMXDufcy733+IngCizr26isrW9sblW3azu7e/sH5uFRT8WppKxLYxHLgU8UEzxiXeAg2CCRjIS+YH1/ejPz+49MKh5H95AlzA3JOOIBpwS05Jl1B0j6kJ8XXu4wIbycF4VnNqymNQdeJXZJGqhExzO/nFFM05BFQAVRamhbCbg5kcCpYEXNSRVLCJ2SMRtqGpGQKTefH1/gU62McBBLXRHgufp7IiehUlno686QwEQtezPxP2+YQnDt5jxKUmARXSwKUoEhxrMk8IhLRkFkmhAqub4V0wmRhILOq6ZDsJdfXiW9VtO+aLbuLhttq4yjio7RCTpDNrpCbXSLOqiLKMrQM3pFb8aT8WK8Gx+L1opRztTRHxifP2TOlTI=</latexit> x̂+

`i<latexit sha1_base64="qWaxSKiuuk3bbCCJzyre5Io4bXE=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWARBKEkVdBlwY3LCvYBTQyT6aQdOpmEmYlYxiz8FTcuFHHrb7jzb5y2WWjrgQuHc+7l3nvClFGpHOfbKi0tr6yuldcrG5tb2zv27l5bJpnApIUTlohuiCRhlJOWooqRbioIikNGOuHoauJ37omQNOG3apwSP0YDTiOKkTJSYB94Q6T0Q36nT/NAe4SxQNM8D+yqU3OmgIvELUgVFGgG9pfXT3AWE64wQ1L2XCdVvkZCUcxIXvEySVKER2hAeoZyFBPp6+n9OTw2Sh9GiTDFFZyqvyc0iqUcx6HpjJEaynlvIv7n9TIVXfqa8jRThOPZoihjUCVwEgbsU0GwYmNDEBbU3ArxEAmElYmsYkJw519eJO16zT2r1W/Oqw2niKMMDsEROAEuuAANcA2aoAUweATP4BW8WU/Wi/VufcxaS1Yxsw/+wPr8AQlwlrM=</latexit>

z�1<latexit sha1_base64="tw7dZysvZuvwrVYTq3V8Dj65DlQ=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBi2W3CnosePFYwX5Iu5Zsmm1Dk+ySZIW67K/w4kERr/4cb/4b03YP2vpg4PHeDDPzgpgzbVz32ymsrK6tbxQ3S1vbO7t75f2Dlo4SRWiTRDxSnQBrypmkTcMMp51YUSwCTtvB+Hrqtx+p0iySd2YSU1/goWQhI9hY6f7pIT3L+qmX9csVt+rOgJaJl5MK5Gj0y1+9QUQSQaUhHGvd9dzY+ClWhhFOs1Iv0TTGZIyHtGupxIJqP50dnKETqwxQGClb0qCZ+nsixULriQhsp8BmpBe9qfif101MeOWnTMaJoZLMF4UJRyZC0+/RgClKDJ9Ygoli9lZERlhhYmxGJRuCt/jyMmnVqt55tXZ7UanX8jiKcATHcAoeXEIdbqABTSAg4Ble4c1Rzovz7nzMWwtOPnMIf+B8/gC4epBN</latexit>

z�2<latexit sha1_base64="B7qfbQWYCPmO42NLYCRJiHKcSUc=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBi2W3CnosePFYwX5Iu5Zsmm1Dk+ySZIW67K/w4kERr/4cb/4b03YP2vpg4PHeDDPzgpgzbVz32ymsrK6tbxQ3S1vbO7t75f2Dlo4SRWiTRDxSnQBrypmkTcMMp51YUSwCTtvB+Hrqtx+p0iySd2YSU1/goWQhI9hY6f7pIT3L+mkt65crbtWdAS0TLycVyNHol796g4gkgkpDONa667mx8VOsDCOcZqVeommMyRgPaddSiQXVfjo7OEMnVhmgMFK2pEEz9fdEioXWExHYToHNSC96U/E/r5uY8MpPmYwTQyWZLwoTjkyEpt+jAVOUGD6xBBPF7K2IjLDCxNiMSjYEb/HlZdKqVb3zau32olKv5XEU4QiO4RQ8uIQ63EADmkBAwDO8wpujnBfn3fmYtxacfOYQ/sD5/AG5/5BO</latexit>

z�3<latexit sha1_base64="kd8/0YxYb2EFs7klNBFHT428jYM=">AAAB8HicbVBNSwMxEJ31s9avqkcvwSJ4sey2gh4LXjxWsB/SriWbZtvQJLskWaEu+yu8eFDEqz/Hm//GtN2Dtj4YeLw3w8y8IOZMG9f9dlZW19Y3Ngtbxe2d3b390sFhS0eJIrRJIh6pToA15UzSpmGG006sKBYBp+1gfD31249UaRbJOzOJqS/wULKQEWysdP/0kJ5n/bSW9Utlt+LOgJaJl5My5Gj0S1+9QUQSQaUhHGvd9dzY+ClWhhFOs2Iv0TTGZIyHtGupxIJqP50dnKFTqwxQGClb0qCZ+nsixULriQhsp8BmpBe9qfif101MeOWnTMaJoZLMF4UJRyZC0+/RgClKDJ9Ygoli9lZERlhhYmxGRRuCt/jyMmlVK16tUr29KNereRwFOIYTOAMPLqEON9CAJhAQ8Ayv8OYo58V5dz7mrStOPnMEf+B8/gC7hJBP</latexit>

z�`2<latexit sha1_base64="0XaJpSwohnGu3l/b4DYTgqzV65I=">AAAB9XicbVBNS8NAEJ34WetX1aOXxSJ4sSRR0GPBi8cK9gPaNGy2m3bpZhN2N0oN+R9ePCji1f/izX/jts1BWx8MPN6bYWZekHCmtG1/Wyura+sbm6Wt8vbO7t5+5eCwpeJUEtokMY9lJ8CKciZoUzPNaSeRFEcBp+1gfDP12w9UKhaLez1JqBfhoWAhI1gbqf/Uz85zP+tRzn039ytVu2bPgJaJU5AqFGj4la/eICZpRIUmHCvVdexEexmWmhFO83IvVTTBZIyHtGuowBFVXja7OkenRhmgMJamhEYz9fdEhiOlJlFgOiOsR2rRm4r/ed1Uh9dexkSSairIfFGYcqRjNI0ADZikRPOJIZhIZm5FZIQlJtoEVTYhOIsvL5OWW3Muau7dZbXuFnGU4BhO4AwcuII63EIDmkBAwjO8wpv1aL1Y79bHvHXFKmaO4A+szx+E+ZJ4</latexit>

e�`2<latexit sha1_base64="rKHZH/1spwJK6hgsPYenq4NHARs=">AAAB9XicbVBNS8NAEN3Ur1q/qh69LBbBiyWJgh4LXjxWsB/QpmGznbRLN5uwu1FKyP/w4kERr/4Xb/4bt20O2vpg4PHeDDPzgoQzpW372yqtrW9sbpW3Kzu7e/sH1cOjtopTSaFFYx7LbkAUcCagpZnm0E0kkCjg0AkmtzO/8whSsVg86GkCXkRGgoWMEm2kAQyyi9zP+sC57+Z+tWbX7TnwKnEKUkMFmn71qz+MaRqB0JQTpXqOnWgvI1IzyiGv9FMFCaETMoKeoYJEoLxsfnWOz4wyxGEsTQmN5+rviYxESk2jwHRGRI/VsjcT//N6qQ5vvIyJJNUg6GJRmHKsYzyLAA+ZBKr51BBCJTO3YjomklBtgqqYEJzll1dJ2607l3X3/qrWcIs4yugEnaJz5KBr1EB3qIlaiCKJntErerOerBfr3fpYtJasYuYY/YH1+QNkFJJj</latexit>

e�3<latexit sha1_base64="h6414PvC1Z+OSNH8hwPGhh/+OeY=">AAAB8HicbVBNSwMxEM3Wr1q/qh69BIvgxbLbCnosePFYwX5Iu5ZsOtuGJtklyQpl2V/hxYMiXv053vw3pu0etPXBwOO9GWbmBTFn2rjut1NYW9/Y3Cpul3Z29/YPyodHbR0likKLRjxS3YBo4ExCyzDDoRsrICLg0AkmNzO/8wRKs0jem2kMviAjyUJGibHSAzymF9kgrWeDcsWtunPgVeLlpIJyNAflr/4wookAaSgnWvc8NzZ+SpRhlENW6icaYkInZAQ9SyURoP10fnCGz6wyxGGkbEmD5+rviZQIracisJ2CmLFe9mbif14vMeG1nzIZJwYkXSwKE45NhGff4yFTQA2fWkKoYvZWTMdEEWpsRiUbgrf88ipp16pevVq7u6w0ankcRXSCTtE58tAVaqBb1EQtRJFAz+gVvTnKeXHenY9Fa8HJZ47RHzifP5sIkDo=</latexit>

e�2<latexit sha1_base64="nEldZdCZvOYWMR08Pc7Si9O/udM=">AAAB8HicbVBNS8NAEN34WetX1aOXxSJ4sSRR0GPBi8cK9kPaWDbbSbt0dxN2N0IJ+RVePCji1Z/jzX/jts1BWx8MPN6bYWZemHCmjet+Oyura+sbm6Wt8vbO7t5+5eCwpeNUUWjSmMeqExINnEloGmY4dBIFRIQc2uH4Zuq3n0BpFst7M0kgEGQoWcQoMVZ6gMfsPO9nft6vVN2aOwNeJl5BqqhAo1/56g1imgqQhnKidddzExNkRBlGOeTlXqohIXRMhtC1VBIBOshmB+f41CoDHMXKljR4pv6eyIjQeiJC2ymIGelFbyr+53VTE10HGZNJakDS+aIo5djEePo9HjAF1PCJJYQqZm/FdEQUocZmVLYheIsvL5OWX/Muav7dZbXuF3GU0DE6QWfIQ1eojm5RAzURRQI9o1f05ijnxXl3PuatK04xc4T+wPn8AZmDkDk=</latexit>

e�1<latexit sha1_base64="ssdABDkwftsVQYlv4irnHsfDH3A=">AAAB8HicbVBNS8NAEJ34WetX1aOXYBG8WJIq6LHgxWMF+yFtLZvtpF26uwm7G6GE/AovHhTx6s/x5r9x2+agrQ8GHu/NMDMviDnTxvO+nZXVtfWNzcJWcXtnd2+/dHDY1FGiKDZoxCPVDohGziQ2DDMc27FCIgKOrWB8M/VbT6g0i+S9mcTYE2QoWcgoMVZ6wMf0POunftYvlb2KN4O7TPyclCFHvV/66g4imgiUhnKidcf3YtNLiTKMcsyK3URjTOiYDLFjqSQCdS+dHZy5p1YZuGGkbEnjztTfEykRWk9EYDsFMSO96E3F/7xOYsLrXspknBiUdL4oTLhrInf6vTtgCqnhE0sIVcze6tIRUYQam1HRhuAvvrxMmtWKf1Gp3l2Wa9U8jgIcwwmcgQ9XUINbqEMDKAh4hld4c5Tz4rw7H/PWFSefOYI/cD5/AJf+kDg=</latexit>

x̂�`2

<latexit sha1_base64="N/pZdnUX8qfOdSYuVbtc7+28UTY=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5CRbBjSWpgi4LblxWsA9oYphMb9qhk0mYmYg1BH/FjQtF3Pof7vwbp20W2nrgwuGce7n3niBhVCrb/jZKS8srq2vl9crG5tb2jrm715ZxKgi0SMxi0Q2wBEY5tBRVDLqJABwFDDrB6Grid+5BSBrzWzVOwIvwgNOQEqy05JsH7hCr7CG/y05zP3OBMb+e+2bVrtlTWIvEKUgVFWj65pfbj0kaAVeEYSl7jp0oL8NCUcIgr7iphASTER5AT1OOI5BeNr0+t4610rfCWOjiypqqvycyHEk5jgLdGWE1lPPeRPzP66UqvPQyypNUASezRWHKLBVbkyisPhVAFBtrgomg+laLDLHAROnAKjoEZ/7lRdKu15yzWv3mvNqwizjK6BAdoRPkoAvUQNeoiVqIoEf0jF7Rm/FkvBjvxsestWQUM/voD4zPH+WJlXI=</latexit>

x̂�1

<latexit sha1_base64="TLZeyAhnbzlkRS0Xmis0XM67WhM=">AAAB+HicbVBNS8NAEN34WetHox69LBbBiyWpgh4LXjxWsB/QxrDZbtqlm03YnYg15Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QSK4Bsf5tlZW19Y3Nktb5e2d3b2KvX/Q1nGqKGvRWMSqGxDNBJesBRwE6yaKkSgQrBOMr6d+54EpzWN5B5OEeREZSh5ySsBIvl3pjwhkj/l9dpb7mZv7dtWpOTPgZeIWpIoKNH37qz+IaRoxCVQQrXuuk4CXEQWcCpaX+6lmCaFjMmQ9QyWJmPay2eE5PjHKAIexMiUBz9TfExmJtJ5EgemMCIz0ojcV//N6KYRXXsZlkgKTdL4oTAWGGE9TwAOuGAUxMYRQxc2tmI6IIhRMVmUTgrv48jJp12vuea1+e1FtOEUcJXSEjtEpctElaqAb1EQtRFGKntErerOerBfr3fqYt65Yxcwh+gPr8wcLd5NH</latexit>

⌧�`2<latexit sha1_base64="BT8zPJxaxf9iZjJzdZ4vHzIEPY8=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16CRbBiyWpgh4LXjxWsB/QxLDZbtqlm03Y3QghxL/ixYMiXv0h3vw3btsctPXBwOO9GWbmBQmjUtn2t1FZW9/Y3Kpu13Z29/YPzMOjnoxTgUkXxywWgwBJwignXUUVI4NEEBQFjPSD6c3M7z8SIWnM71WWEC9CY05DipHSkm/WXYXSh/y88HOXMObnraLwzYbdtOewVolTkgaU6PjmlzuKcRoRrjBDUg4dO1FejoSimJGi5qaSJAhP0ZgMNeUoItLL58cX1qlWRlYYC11cWXP190SOIimzKNCdEVITuezNxP+8YarCay+nPEkV4XixKEyZpWJrloQ1ooJgxTJNEBZU32rhCRIIK51XTYfgLL+8SnqtpnPRbN1dNtp2GUcVjuEEzsCBK2jDLXSgCxgyeIZXeDOejBfj3fhYtFaMcqYOf2B8/gAUJJT9</latexit>

x̂i<latexit sha1_base64="HUIrEVPvsHPJa1WvoXaPzoxNwYI=">AAAB8nicbVBNS8NAEJ34WetX1aOXxSJ4KkkV9Fjw4rGC/YA0lM120y7dbMLuRCyhP8OLB0W8+mu8+W/ctjlo64OBx3szzMwLUykMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3Q2q4FIq3UKDk3VRzGoeSd8Lx7czvPHJtRKIecJLyIKZDJSLBKFrJ740o5k/Tfi6m/UrVrblzkFXiFaQKBZr9yldvkLAs5gqZpMb4nptikFONgkk+Lfcyw1PKxnTIfUsVjbkJ8vnJU3JulQGJEm1LIZmrvydyGhsziUPbGVMcmWVvJv7n+RlGN0EuVJohV2yxKMokwYTM/icDoTlDObGEMi3srYSNqKYMbUplG4K3/PIqaddr3mWtfn9VbbhFHCU4hTO4AA+uoQF30IQWMEjgGV7hzUHnxXl3Phata04xcwJ/4Hz+APW5kaM=</latexit>


g�<latexit sha1_base64="xxiL0qU5I/x8nNpZUQf/SWqCaAY=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBg5SkCnosePFYwX5AG8pmu2mXbjZxdyKU0D/hxYMiXv073vw3btsctPXBwOO9GWbmBYkUBl332ymsrW9sbhW3Szu7e/sH5cOjlolTzXiTxTLWnYAaLoXiTRQoeSfRnEaB5O1gfDvz209cGxGrB5wk3I/oUIlQMIpW6gz7WS8ZiWm/XHGr7hxklXg5qUCORr/81RvELI24QiapMV3PTdDPqEbBJJ+WeqnhCWVjOuRdSxWNuPGz+b1TcmaVAQljbUshmau/JzIaGTOJAtsZURyZZW8m/ud1Uwxv/EyoJEWu2GJRmEqCMZk9TwZCc4ZyYgllWthbCRtRTRnaiEo2BG/55VXSqlW9y2rt/qpSv8jjKMIJnMI5eHANdbiDBjSBgYRneIU359F5cd6dj0VrwclnjuEPnM8fTyCQEw==</latexit>








Query entry

AttractRepel

Figure 2: The adapted SimCLR contrastive learning framework [10], applied to learn audit task invariant accounting journalentry representations. Given an encoded query entry 𝑥𝑞

𝑖, a task invariant representation 𝑧

𝑞

𝑖is learnt by mapping positive

(intra-attribute) augmentations 𝑥+𝑖of the same journal entry to nearby points in a latent space, while pushing negative (inter-

attribute) augmentations 𝑥−𝑖to apart points.

3.1 Accounting Journal EntriesLet 𝑋 = {𝑥1, 𝑥2, ..., 𝑥𝑁 } be a set of 𝑖 = 1, 2, ..., 𝑁 journal entries.Each journal entry 𝑥𝑖 = {𝑥1

𝑖, 𝑥2𝑖, ..., 𝑥𝑀

𝑖;𝑥1𝑖, 𝑥2𝑖, ..., 𝑥𝐾

𝑖} consists of

𝑗 = 1, 2, ..., 𝑀 categorical accounting attributes and 𝑙 = 1, 2, ..., 𝐾numerical accounting attributes. The individual attributes describethe journal entries details, e.g., the entries’ fiscal year, posting type,posting date, amount, general-ledger. Furthermore, the followingpre-processing is applied to the distinct attribute classes:

• The categorical attribute values 𝑥 𝑗𝑖are converted into ‘one-

hot’ numerical tuples of bits 𝑥 𝑗𝑖∈ {0, 1}𝜐 , where 𝜐 denotes the

number of unique attribute values in 𝑥 𝑗 .• The numerical attribute values 𝑥𝑙

𝑖are scaled, according to

𝑠𝑐𝑎𝑙𝑒 (𝑥𝑙𝑖) = (𝑥𝑙

𝑖−min(𝑥𝑙 )/(max(𝑥𝑙 ) −min(𝑥𝑙 )), wheremin (max)

denotes the minimum (maximum) attribute value in 𝑥𝑙 .We denote the pre-processed journal entries 𝑥 ∈ 𝑋 , where the pre-processed attributes are defined as𝑥𝑖 = {𝑥1

𝑖, 𝑥2𝑖, ..., 𝑥𝑀

𝑖;𝑥1𝑖, 𝑥2𝑖, ..., 𝑥𝐾

𝑖}.

3.2 Contrastive Learning FrameworkThe objective of contrastive SSL is to learn task invariant represen-tations by maximizing the similarity and dissimilarity over datasamples which are organized in pairs. In the context of this work,pairs are created by augmenting an encoded reference journal en-try 𝑥𝑖 , denoted as query 𝑥𝑞

𝑖, into: (i) one similar positive key 𝑥+

𝑖and (ii) several dissimilar negative keys 𝑥−

𝑖. The contrastive SSL

framework is designed to maximize the mutual agreement betweenthe positive pair (𝑥𝑞

𝑖, 𝑥+𝑖) and minimize the agreement of the neg-

ative pairs (𝑥𝑞𝑖, 𝑥−𝑖) of the same journal entry. To establish such a

learning framework we utilize the recently proposed SimCLR ar-chitecture [10]. The architecture, illustrated in Fig. 2, encompassesthe following components:

• A stochastic data augmentation module. The module trans-forms a given query entry 𝑥𝑞

𝑖into ℓ1 positive (ℓ2 negative)

augmentations T + (𝑥𝑞𝑖) = {𝑥+𝜔 }

ℓ1𝜔=1 (corresp. T− (𝑥𝑞

𝑖) = {𝑥−

_}ℓ2_=1)

of the same entry considered as positive (negative) pairs.

• A neural network base encoder 𝑓\ (·) with parameters \ thatextracts a representation 𝑧𝑖 from a given encoded and aug-mented journal entry 𝑥𝑖 . Commonly a feed-forward networkis used to map 𝑥𝑖 to a latent space 𝑍 ∈ R𝑑1 .

• A neural network projection head 𝑔𝜙 (·) with parameters 𝜙which maps a representation 𝑧𝑖 to an embedding 𝑒𝑖 of anormalized space 𝐻 ∈ R𝑑2 , where 𝑑2 < 𝑑1. 𝐻 is used tocompute a contrastive objective of the learning task.

• A contrastive learning objective that, given a query embed-ding 𝑒𝑞

𝑖, aims to identify the positive key over a set of em-

beddings {𝑒𝑘 }ℓ2+1𝑖=1 = 𝑒+𝑖∪ {𝑒−

_}ℓ2_=1 consisting of the positive

embedding 𝑒+𝑖and a set of ℓ2 negative embeddings {𝑒−

_}ℓ2_=1 .

3.3 Contrastive Learning ObjectiveA common contrastive learning objective is the InfoNCE [45] losswhich maps representations of positive pairs to nearby points in theembedding space𝑍 and representations of negative pairs to far apartpoints. To compute the loss, random mini-batches of 𝑁 examplesare drawn, resulting in 2𝑁 pairwise data points per batch. Given apositive pair of embeddings (𝑒𝑞

𝑖, 𝑒+𝑖), all other 2(𝑁 − 1) augmented

examples within the mini-batch are considered as negative exam-ples. For each positive embedding pair of query 𝑒𝑞

𝑖= 𝑔𝜙 (𝑓\ (𝑥

𝑞

𝑖))

and key 𝑒+𝑖,𝜔

= 𝑔𝜙 (𝑓\ (𝑥+𝑖,𝜔 )), the InfoNCE loss is defined as:

L𝜔𝑁𝐶𝐸

(𝑒𝑞𝑖, 𝑒+𝑖,𝜔 , {𝑒𝑘 }) = − log

exp(𝑠𝑖𝑚(𝑒𝑞𝑖, 𝑒+𝑖,𝜔

)/𝜏)∑2𝑁𝑘=1 1[𝑘≠𝑖 ] exp(𝑠𝑖𝑚(𝑒𝑞

𝑖, 𝑒𝑘 )/𝜏)

, (1)

where 1 denotes an indicator function evaluating to 1 if 𝑘 ≠ 𝑖 , 𝜔denotes an applied positive augmentation, and 𝜏 is a temperatureparameter. Furthermore, 𝑠𝑖𝑚(𝑢, 𝑣) = 𝑢𝑇 𝑣/∥𝑢∥∥𝑣 ∥ denotes a nor-malized dot product between 𝑢 and 𝑣 (i.e. the cosine similarity).Recently, it was shown in [56] that the task invariance of encodedinformation increases proportional to the pair-wise comparison ofpositive pairs. Following this observation we derive the InfoNCE


loss for each query embedding 𝑒𝑞𝑖over all its positive embeddings,

as defined by:

L𝑃𝑎𝑖𝑟𝑁𝐶𝐸 (𝑒𝑞𝑖 , {𝑒+𝑖,𝜔 }

ℓ1𝜔=1, {𝑒𝑘 }) =

∑︁ℓ1

𝜔=1L𝜔

𝑁𝐶𝐸(𝑒𝑞𝑖, 𝑒+𝑖,𝜔 , {𝑒𝑘 }), (2)

where {𝑒+𝜔,𝑖

}ℓ1𝜔=1 denotes the set of ℓ1 positive embeddings, {𝑒𝑘 },

and the set of all embedding keys and 𝜏 is a temperature parameter.

3.4 Accounting Data AugmentationsLearning transferable representations that result in a good down-stream task performance highly relies on the applied data augmen-tations [47]. As shown in Fig. 2, the InfoNCE objective guides thelearning framework to map positive pairs (𝑥𝑞

𝑖, 𝑥+𝑖) to nearby loca-

tions in 𝑍 , and negative pairs (𝑥𝑞𝑖, 𝑥−𝑖) further apart. To enable the

learning of such spatial relationships, we propose a data augmenta-tion technique comprised of a positive and a negative augmentationpolicy, that acts directly on the unique attribute characteristics ofjournal entry data.Negative augmentation policy: The policy, denoted by T−, de-liberately dismisses the semantic content of an entry. It is designedto create inter-attribute augmentations resulting in negative pairs(𝑥𝑞𝑖, 𝑥−𝑖). For a given encoded query entry 𝑥𝑞

𝑖, the following aug-

mentation steps are applied to obtain a set of negative keys:

• First, the query entry 𝑥𝑞𝑖is replicated _ times resulting in _

negative keys {𝑥−𝑖}_𝑖=1 of the original query entry.

• Second, a random journal entry attribute 𝑗∗ is sampled fromthe population of categorical accounting attributes.

• Third, for each key 𝑥−𝑖the high bit ‘1’ of the attribute 𝑗∗ is

swapped with another random ‘0‘ bit of the encoding.Positive augmentation policy: The policy, denoted by T +, pre-serves the semantic content of an entry. It is designed to createintra-attribute augmentations resulting in positive pairs (𝑥𝑞

𝑖, 𝑥+𝑖).

For a given encoded query entry 𝑥𝑞𝑖, the following augmentations

are applied to obtain a set of positive keys:• Random-Noise Augmentation, inwhich a given encoded queryentry 𝑥𝑞

𝑖is perturbed by the addition of random noise 𝜌 ∼

N(`𝑅, 𝜎𝑅), formally denoted by 𝑥𝑞𝑖+ 𝜌 .

• Random-Cut Augmentation, in which the high ‘1’ bits of agiven encoded query entry 𝑥𝑞

𝑖are randomly multiplied by

𝛿 ∼ U(`𝐶, 1), formally denoted by 𝑥𝑞𝑖∗ 𝛿 .

• Gaussian-Kernel Augmentation, in which a given encodedquery entry 𝑥𝑞

𝑖is convolved by a Gaussian kernel (‘blurring’),

formally denoted by 𝑥𝑖 ⊛ N(`𝐺 , 𝜎𝐺 ).Given an encoded journal entry 𝑥𝑖 , both augmentation polices aresequentially applied as illustrated in Fig. 3. First, the negative policyT− is used to derive a set of ℓ2 inter-attribute (negative) augmen-tations {𝑥−

𝑖,_}ℓ2_=1 of the entry. Second, the positive policy T + is ap-

plied to augment each 𝑥−𝑖,_

∈ {𝑥−𝑖,_

}ℓ2_=1. Resulting in ℓ1 intra-attribute

(positive) augmentations {𝑥+𝑖,_,𝜔

}ℓ1𝜔=1 for a given inter-attribute aug-

mentation. Subsequently, for each positive augmentation 𝜔 a set ofaugmentation pairs is formed, formally defined by:

𝜗𝜔 = {𝑥−𝑖,_

}ℓ2_=1 ∪ {𝑥+

𝑖,_,𝜔}ℓ2_=1, (3)

where {(𝑥−𝑖,_1

, 𝑥+𝑖,_2,𝜔

) | _1 = _2}ℓ1_=1 defines the sets’ 𝜗𝜔 positive pairs,{(𝑥−

𝑖,_1, 𝑥+𝑖,_2,𝜔

) | _1 ≠ _2}ℓ1_=1 the sets’ negative pairs, and 𝑥−𝑖,_1

serves aseach pair’s query entry. The pair-wise InfoNCE loss, as defined byEq. 2, is then derived for each set of augmentation pairs {𝜗𝜔 }ℓ1𝜔=1.The loss forces the contrastive learning framework to map the intra-attribute (positive) pairs of each set 𝜗𝜔 to nearby locations in 𝑍 ,and the inter-attribute (negative) pairs further apart.

4 EXPERIMENTAL SETUPIn this section, we describe the experimental details of the self-supervised pre-training and downstream audit task evaluation1.

4.1 Datasets and Data PreparationTo evaluate the invariant representation learning capabilities of thecontrastive SSL framework we use two publicly available datasetsof real-world financial payment data that exhibit high similarity toreal-world accounting data. The datasets are referred to as datasetA and dataset B in the following. Dataset A corresponds to the Cityof Philadelphia payment data of the fiscal year 2017 2. It representsthe city’s nearly $4.2 billion in payments obtained from almost60 city offices, departments, boards and committees. Dataset Bcorresponds to vendor payments of the City of Chicago rangingfrom 1996 to 2020 3. The data is collected from the city’s ’Vendor,Contract and Payment Search’ and encompasses the procurementof goods and services. The majority of attributes recorded in bothdatasets (similar to real-world ERP data) correspond to categorical(discrete) variables, e.g., posting date, department, vendor name,document type. We pre-process the original payment line-itemattributes to (i) remove of semantically redundant attributes and (ii)obtain an encoded representation of each payment. The followingdescriptive statistics summarise both datasets upon successful datapre-processing:

• Dataset A: The ‘City of Philadelphia’ payments encompassa total of 238,894 payments comprised of 10 categorical andone numerical attribute. The encoding resulted in a totalof 8,565 encoded dimensions for each of the city’s vendorpayment record 𝑥𝑖 ∈ R8,565.

• Dataset B: The ‘City of Chicago’ payments encompass atotal of 72,814 payments comprised of 7 categorical and onenumerical attribute. The encoding resulted in a total of 2,354encoded dimensions for each of the city’s vendor paymentrecord 𝑥𝑖 ∈ R2,354.

4.2 Contrastive Self-Supervised Pre-TrainingOur architectural setup follows the SimCLR architecture [10] com-prised of an encoder 𝑓\ and a projection-head 𝑔𝜙 as shown in Fig. 2.The encoder uses Leaky-ReLU activation functions [64] with scal-ing factor 𝛼 = 0.4 except in the last layer where no non-linearityis applied. In the projection-head only linear transformations areapplied. Table 1 depicts the architectural details of both networks.

1Due to the general confidentiality of journal entry data, we evaluate the proposedmethodology based on two public available real-world datasets to allow for the repro-ducibility of our results.2https://www.phila.gov/2019-03-29-philadelphias-initial-release-of-city-payments-data/3https://data.cityofchicago.org/Administration-Finance/Payments/s4vu-giwb/

https://www.phila.gov/2019-03-29-philadelphias-initial-release-of-city-payments-data/

https://data.cityofchicago.org/Administration-Finance/Payments/s4vu-giwb/


4

...

Augmentation Set§ Company Code

§ Posting Key

§ Date

§ Account

§ Debit / Credit

§ Amount

§ Vendor ID

§ T-Code

Journal Entry

…

0 1 0 … 0

0 1 0 … 0

0 0 0 … 0

0 0 1 … 0

0 0 0 … 1

0 0 0 … 1

0 1 0 … 0

...

...



1 0 0 … 0

0 1 0 … 0

0 1 0 … 0

0 0 0 … 1

...

1 0 0 … 0

.12 .91 .14 … .09

.13 .94 .17 … .06

.11 .13 .14 … .93

...

.92 .08 .11 … .12

...

...

…

…...

......

…

x̂�i,�=2

<latexit sha1_base64="f2aonEGGKdHGZgQyhwPtct3yI4E=">AAACA3icbVDLSsNAFJ3UV62vqDvdDBbBhZakCroRCm5cVrAPaGKYTCbt0MkkzEzEEgJu/BU3LhRx60+482+ctllo64GBwzn3cOceP2FUKsv6NkoLi0vLK+XVytr6xuaWub3TlnEqMGnhmMWi6yNJGOWkpahipJsIgiKfkY4/vBr7nXsiJI35rRolxI1Qn9OQYqS05Jl7zgCp7CH3MnoMHaaDAbqs53fZSe6ZVatmTQDniV2QKijQ9MwvJ4hxGhGuMENS9mwrUW6GhKKYkbzipJIkCA9Rn/Q05Sgi0s0mN+TwUCsBDGOhH1dwov5OZCiSchT5ejJCaiBnvbH4n9dLVXjhZpQnqSIcTxeFKYMqhuNCYEAFwYqNNEFYUP1XiAdIIKx0bRVdgj178jxp12v2aa1+c1ZtWEUdZbAPDsARsME5aIBr0AQtgMEjeAav4M14Ml6Md+NjOloyiswu+APj8weN8Zdl</latexit>

x̂�i,�=1

<latexit sha1_base64="bv8km5+5rZiHO8mGBLlobn/rO+Q=">AAACA3icbVDLSsNAFJ3UV62vqDvdDBbBhZakCroRCm5cVrAPaGKYTCbt0MkkzEzEEgJu/BU3LhRx60+482+ctllo64GBwzn3cOceP2FUKsv6NkoLi0vLK+XVytr6xuaWub3TlnEqMGnhmMWi6yNJGOWkpahipJsIgiKfkY4/vBr7nXsiJI35rRolxI1Qn9OQYqS05Jl7zgCp7CH3MnoMHaaDAbq087vsJPfMqlWzJoDzxC5IFRRoeuaXE8Q4jQhXmCEpe7aVKDdDQlHMSF5xUkkShIeoT3qachQR6WaTG3J4qJUAhrHQjys4UX8nMhRJOYp8PRkhNZCz3lj8z+ulKrxwM8qTVBGOp4vClEEVw3EhMKCCYMVGmiAsqP4rxAMkEFa6toouwZ49eZ606zX7tFa/Oas2rKKOMtgHB+AI2OAcNMA1aIIWwOARPINX8GY8GS/Gu/ExHS0ZRWYX/IHx+QOMaJdk</latexit>

x̂�i,�=`2

<latexit sha1_base64="H3+sO2xutxWavoDv/EBz+VVPyq8=">AAACCnicbVDLSgMxFM3UV62vUZduokVwoWWmCroRCm5cVrAP6IxDJpNpQzMPkoxYwqzd+CtuXCji1i9w59+YtrPQ1gOBwzn3cHOPnzIqpGV9G6WFxaXllfJqZW19Y3PL3N5piyTjmLRwwhLe9ZEgjMakJalkpJtygiKfkY4/vBr7nXvCBU3iWzlKiRuhfkxDipHUkmfuOwMk1UPuKXoMHaaDAbp0CGOequf5nTrJPbNq1awJ4DyxC1IFBZqe+eUECc4iEkvMkBA920qlqxCXFDOSV5xMkBThIeqTnqYxiohw1eSUHB5qJYBhwvWLJZyovxMKRUKMIl9PRkgOxKw3Fv/zepkML1xF4zSTJMbTRWHGoEzguBcYUE6wZCNNEOZU/xXiAeIIS91eRZdgz548T9r1mn1aq9+cVRtWUUcZ7IEDcARscA4a4Bo0QQtg8AiewSt4M56MF+Pd+JiOlowiswv+wPj8AVVJmps=</latexit>

x̂+i,�=`2,!=`1

<latexit sha1_base64="DXwsMpQrAgEKKovIqrYA08YY25c=">AAACG3icbVDLSsNAFJ3UV62vqks3g0UQlJJEQTdCwY3LCvYBTQyTyW07dPJgZiKWkP9w46+4caGIK8GFf+P0IWjrgYHDOfdw5x4/4Uwq0/wyCguLS8srxdXS2vrG5lZ5e6cp41RQaNCYx6LtEwmcRdBQTHFoJwJI6HNo+YPLkd+6AyFZHN2oYQJuSHoR6zJKlJa8su30icrucy9jx9jhOhiQCwc49zI710ocQu9HsPL8NjvKvXLFrJpj4HliTUkFTVH3yh9OENM0hEhRTqTsWGai3IwIxSiHvOSkEhJCB6QHHU0jEoJ0s/FtOT7QSoC7sdAvUnis/k5kJJRyGPp6MiSqL2e9kfif10lV99zNWJSkCiI6WdRNOVYxHhWFAyaAKj7UhFDB9F8x7RNBqNJ1lnQJ1uzJ86RpV62Tqn19WqmZ0zqKaA/to0NkoTNUQ1eojhqIogf0hF7Qq/FoPBtvxvtktGBMM7voD4zPb/14oVI=</latexit>

x̂+i,�=`2,!=1

<latexit sha1_base64="j3QF7qyPcpSiMctg64LmICDzBkc=">AAACFHicbVC7SgNBFJ31bXytWtoMBkGIhN0oaBMI2FgqmETIxuXu5CYZMvtgZlYMy36Ejb9iY6GIrYWdf+MkptDEAwOHc+7hzj1BIrjSjvNlzc0vLC4tr6wW1tY3Nrfs7Z2GilPJsM5iEcubABQKHmFdcy3wJpEIYSCwGQzOR37zDqXicXSthwm2Q+hFvMsZaCP5dsnrg87ucz/jR9QTJtiBqodC+FklN0ocYg+qbn6blXLfLjplZww6S9wJKZIJLn370+vELA0x0kyAUi3XSXQ7A6k5E5gXvFRhAmwAPWwZGkGIqp2Nj8rpgVE6tBtL8yJNx+rvRAahUsMwMJMh6L6a9kbif14r1d2zdsajJNUYsZ9F3VRQHdNRQ7TDJTIthoYAk9z8lbI+SGDa9FgwJbjTJ8+SRqXsHpcrVyfFmjOpY4XskX1ySFxySmrkglySOmHkgTyRF/JqPVrP1pv1/jM6Z00yu+QPrI9vCtKeHA==</latexit>

x̂+i,�=`2,!=2

<latexit sha1_base64="BGEEyRPBAI3v8A9wvom1XMi3hUc=">AAACFHicbVDLSsNAFJ34tr6qLt0MFkFQShIF3RQKblwq2FZoYriZ3LaDkwczE7GEfIQbf8WNC0XcunDn3zitXfg6MHA45x7u3BNmgitt2x/W1PTM7Nz8wmJlaXllda26vtFWaS4ZtlgqUnkZgkLBE2xprgVeZhIhDgV2wuuTkd+5Qal4mlzoYYZ+DP2E9zgDbaSguucNQBe3ZVDwfeoJE4yg4aEQQeGWRklj7EPDLa+KvTKo1uy6PQb9S5wJqZEJzoLquxelLI8x0UyAUl3HzrRfgNScCSwrXq4wA3YNfewamkCMyi/GR5V0xygR7aXSvETTsfo9UUCs1DAOzWQMeqB+eyPxP6+b696xX/AkyzUm7GtRLxdUp3TUEI24RKbF0BBgkpu/UjYACUybHiumBOf3yX9J2607B3X3/LDWtCd1LJAtsk12iUOOSJOckjPSIozckQfyRJ6te+vRerFev0anrElmk/yA9fYJDFueHQ==</latexit>

x̂+i,�=2,!=2

<latexit sha1_base64="7ujFznx7twKVHBkDzwFRCWskbR4=">AAACDXicbZDLSsNAFIYn9VbrLerSzWAVBKUkVdCNUHDjsoK9QBPDZDJph04yYWYilpAXcOOruHGhiFv37nwbp20W2vrDwMd/zuHM+f2EUaks69soLSwuLa+UVytr6xubW+b2TlvyVGDSwpxx0fWRJIzGpKWoYqSbCIIin5GOP7wa1zv3REjK41s1SogboX5MQ4qR0pZnHjgDpLKH3MvoCXSYHgzQZV0jj0hfU36XHeeeWbVq1kRwHuwCqqBQ0zO/nIDjNCKxwgxJ2bOtRLkZEopiRvKKk0qSIDxEfdLTGKOISDebXJPDQ+0EMORCv1jBift7IkORlKPI150RUgM5Wxub/9V6qQov3IzGSapIjKeLwpRBxeE4GhhQQbBiIw0IC6r/CvEACYSVDrCiQ7BnT56Hdr1mn9bqN2fVhlXEUQZ7YB8cARucgwa4Bk3QAhg8gmfwCt6MJ+PFeDc+pq0lo5jZBX9kfP4AJxea5w==</latexit>

x̂+i,�=2,!=1

<latexit sha1_base64="sISGquhTP/qiYKBo3NiRCVpLR/U=">AAACDXicbVDLSgMxFM3UV62vUZduglUQlDJTBd0UCm5cVrAPaMfhTpq2ocnMkGTEMswPuPFX3LhQxK17d/6Nae1CWw8EDufcw809QcyZ0o7zZeUWFpeWV/KrhbX1jc0te3unoaJEElonEY9kKwBFOQtpXTPNaSuWFETAaTMYXo795h2VikXhjR7F1BPQD1mPEdBG8u2DzgB0ep/5KTvBHW6CXaiUDY0E7UPFzW7T48y3i07JmQDPE3dKimiKmm9/droRSQQNNeGgVNt1Yu2lIDUjnGaFTqJoDGQIfdo2NARBlZdOrsnwoVG6uBdJ80KNJ+rvRApCqZEIzKQAPVCz3lj8z2snunfhpSyME01D8rOol3CsIzyuBneZpETzkSFAJDN/xWQAEog2BRZMCe7syfOkUS65p6Xy9Vmx6kzryKM9tI+OkIvOURVdoRqqI4Ie0BN6Qa/Wo/VsvVnvP6M5a5rZRX9gfXwDJY6a5g==</latexit>

x̂+i,�=2,!=`1

<latexit sha1_base64="mED9ylpnc/2WR94ld17w+5CsU2Q=">AAACFHicbVC7SgNBFJ31bXytWtoMBkGIhN0oaBMI2FgqmETIxuXu5CYZMvtgZlYMy36Ejb9iY6GIrYWdf+MkptDEAwOHc+7hzj1BIrjSjvNlzc0vLC4tr6wW1tY3Nrfs7Z2GilPJsM5iEcubABQKHmFdcy3wJpEIYSCwGQzOR37zDqXicXSthwm2Q+hFvMsZaCP5dsnrg87ucz/jR9QTJtiBasXQOMQeVD0Uws/cPL/NSrlvF52yMwadJe6EFMkEl7796XViloYYaSZAqZbrJLqdgdScCcwLXqowATaAHrYMjSBE1c7GR+X0wCgd2o2leZGmY/V3IoNQqWEYmMkQdF9NeyPxP6+V6u5ZO+NRkmqM2M+ibiqojumoIdrhEpkWQ0OASW7+SlkfJDBteiyYEtzpk2dJo1J2j8uVq5NizZnUsUL2yD45JC45JTVyQS5JnTDyQJ7IC3m1Hq1n6816/xmdsyaZXfIH1sc3A1GeHA==</latexit>

x̂+i,�=1,!=`1

<latexit sha1_base64="L+/oC1sjhFDAPeNQQzwDNPCiZS4=">AAACFHicbVC7SgNBFJ31GeMramkzGARBCbtR0EYQbCwjmAdk1+Xu5CYZnH0wMyuGZT/Cxl+xsVDE1sLOv3GSbOHrwMDhnHu4c0+QCK60bX9aM7Nz8wuLpaXy8srq2nplY7Ol4lQybLJYxLITgELBI2xqrgV2EokQBgLbwc352G/folQ8jq70KEEvhEHE+5yBNpJf2XeHoLO73M/4AXWFCfbg1DE0DnEApy4K4WdOnl9n+7lfqdo1ewL6lzgFqZICDb/y4fZiloYYaSZAqa5jJ9rLQGrOBOZlN1WYALuBAXYNjSBE5WWTo3K6a5Qe7cfSvEjTifo9kUGo1CgMzGQIeqh+e2PxP6+b6v6Jl/EoSTVGbLqonwqqYzpuiPa4RKbFyBBgkpu/UjYECUybHsumBOf3yX9Jq15zDmv1y6PqmV3UUSLbZIfsEYcckzNyQRqkSRi5J4/kmbxYD9aT9Wq9TUdnrCKzRX7Aev8CAbeeGw==</latexit>

x̂+i,�=1,!=1

<latexit sha1_base64="GZG2Z8XsBtwGJ04xmZRErdlcyZM=">AAACDXicbZDLSsNAFIYn9VbrLerSzWAVBKUkVdCNUHDjsoK9QBPDZDJph84kYWYilpAXcOOruHGhiFv37nwbp20W2vrDwMd/zuHM+f2EUaks69soLSwuLa+UVytr6xubW+b2TlvGqcCkhWMWi66PJGE0Ii1FFSPdRBDEfUY6/vBqXO/cEyFpHN2qUUJcjvoRDSlGSlueeeAMkMoeci+jJ9BhejBAl7bGmJO+pvwuO849s2rVrIngPNgFVEGhpmd+OUGMU04ihRmSsmdbiXIzJBTFjOQVJ5UkQXiI+qSnMUKcSDebXJPDQ+0EMIyFfpGCE/f3RIa4lCPu606O1EDO1sbmf7VeqsILN6NRkioS4emiMGVQxXAcDQyoIFixkQaEBdV/hXiABMJKB1jRIdizJ89Du16zT2v1m7NqwyriKIM9sA+OgA3OQQNcgyZoAQwewTN4BW/Gk/FivBsf09aSUczsgj8yPn8AI/ua5Q==</latexit>

x̂+i,�=1,!=2

<latexit sha1_base64="v1baEepS/EoQ7W2r8KPKoDdk5tE=">AAACDXicbVDLSgMxFM3UV62vUZduglUQlDJTBd0UCm5cVrAPaMfhTpq2ocnMkGTEMswPuPFX3LhQxK17d/6Nae1CWw8EDufcw809QcyZ0o7zZeUWFpeWV/KrhbX1jc0te3unoaJEElonEY9kKwBFOQtpXTPNaSuWFETAaTMYXo795h2VikXhjR7F1BPQD1mPEdBG8u2DzgB0ep/5KTvBHW6CXai4hkaC9qFSzm7T48y3i07JmQDPE3dKimiKmm9/droRSQQNNeGgVNt1Yu2lIDUjnGaFTqJoDGQIfdo2NARBlZdOrsnwoVG6uBdJ80KNJ+rvRApCqZEIzKQAPVCz3lj8z2snunfhpSyME01D8rOol3CsIzyuBneZpETzkSFAJDN/xWQAEog2BRZMCe7syfOkUS65p6Xy9Vmx6kzryKM9tI+OkIvOURVdoRqqI4Ie0BN6Qa/Wo/VsvVnvP6M5a5rZRX9gfXwDJYSa5g==</latexit>

... . . .

.01 0.96 .21 … .04

1.0 0.0 0.0 … .06

0.0 0.0 0.0 … .03

0.0 0.0 0.0 … .12

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

.01 .98 .12 … .14

.94 .14 .07 … .03

.03 .15 .06 … .89

.11 .23 .02 … .87

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

x̂�i,�=1


x̂+i,�=1,!=1

<latexit sha1_base64="GZG2Z8XsBtwGJ04xmZRErdlcyZM=">AAACDXicbZDLSsNAFIYn9VbrLerSzWAVBKUkVdCNUHDjsoK9QBPDZDJph84kYWYilpAXcOOruHGhiFv37nwbp20W2vrDwMd/zuHM+f2EUaks69soLSwuLa+UVytr6xubW+b2TlvGqcCkhWMWi66PJGE0Ii1FFSPdRBDEfUY6/vBqXO/cEyFpHN2qUUJcjvoRDSlGSlueeeAMkMoeci+jJ9BhejBAl7bGmJO+pvwuO849s2rVrIngPNgFVEGhpmd+OUGMU04ihRmSsmdbiXIzJBTFjOQVJ5UkQXiI+qSnMUKcSDebXJPDQ+0EMIyFfpGCE/f3RIa4lCPu606O1EDO1sbmf7VeqsILN6NRkioS4emiMGVQxXAcDQyoIFixkQaEBdV/hXiABMJKB1jRIdizJ89Du16zT2v1m7NqwyriKIM9sA+OgA3OQQNcgyZoAQwewTN4BW/Gk/FivBsf09aSUczsgj8yPn8AI/ua5Q==</latexit>

x̂�i,�=1


x̂+i,�=1,!=2

<latexit sha1_base64="v1baEepS/EoQ7W2r8KPKoDdk5tE=">AAACDXicbVDLSgMxFM3UV62vUZduglUQlDJTBd0UCm5cVrAPaMfhTpq2ocnMkGTEMswPuPFX3LhQxK17d/6Nae1CWw8EDufcw809QcyZ0o7zZeUWFpeWV/KrhbX1jc0te3unoaJEElonEY9kKwBFOQtpXTPNaSuWFETAaTMYXo795h2VikXhjR7F1BPQD1mPEdBG8u2DzgB0ep/5KTvBHW6CXai4hkaC9qFSzm7T48y3i07JmQDPE3dKimiKmm9/droRSQQNNeGgVNt1Yu2lIDUjnGaFTqJoDGQIfdo2NARBlZdOrsnwoVG6uBdJ80KNJ+rvRApCqZEIzKQAPVCz3lj8z2snunfhpSyME01D8rOol3CsIzyuBneZpETzkSFAJDN/xWQAEog2BRZMCe7syfOkUS65p6Xy9Vmx6kzryKM9tI+OkIvOURVdoRqqI4Ie0BN6Qa/Wo/VsvVnvP6M5a5rZRX9gfXwDJYSa5g==</latexit>

x̂�i,�=1


x̂+i,�=1,!=`1

<latexit sha1_base64="L+/oC1sjhFDAPeNQQzwDNPCiZS4=">AAACFHicbVC7SgNBFJ31GeMramkzGARBCbtR0EYQbCwjmAdk1+Xu5CYZnH0wMyuGZT/Cxl+xsVDE1sLOv3GSbOHrwMDhnHu4c0+QCK60bX9aM7Nz8wuLpaXy8srq2nplY7Ol4lQybLJYxLITgELBI2xqrgV2EokQBgLbwc352G/folQ8jq70KEEvhEHE+5yBNpJf2XeHoLO73M/4AXWFCfbg1DE0DnEApy4K4WdOnl9n+7lfqdo1ewL6lzgFqZICDb/y4fZiloYYaSZAqa5jJ9rLQGrOBOZlN1WYALuBAXYNjSBE5WWTo3K6a5Qe7cfSvEjTifo9kUGo1CgMzGQIeqh+e2PxP6+b6v6Jl/EoSTVGbLqonwqqYzpuiPa4RKbFyBBgkpu/UjYECUybHsumBOf3yX9Jq15zDmv1y6PqmV3UUSLbZIfsEYcckzNyQRqkSRi5J4/kmbxYD9aT9Wq9TUdnrCKzRX7Aev8CAbeeGw==</latexit>



e�

i,�=

2̀

<latexit sha1_base64="niAuQQfmnpmuDy/NhsLv3qM46js=">AAACBHicbVC7SgNBFJ2NrxhfUcs0g0FIoWE3CtoIARvLCOYB2XWZnb2bDJl9MDMrhGULG3/FxkIRWz/Czr9x8ig08cDA4Zx7uHOPl3AmlWl+G4WV1bX1jeJmaWt7Z3evvH/QkXEqKLRpzGPR84gEziJoK6Y49BIBJPQ4dL3R9cTvPoCQLI7u1DgBJySDiAWMEqUlt1wBN2Mn2OY64pMrGzh3s0ae32enuVuumnVzCrxMrDmpojlabvnL9mOahhApyomUfctMlJMRoRjlkJfsVEJC6IgMoK9pREKQTjY9IsfHWvFxEAv9IoWn6u9ERkIpx6GnJ0OihnLRm4j/ef1UBZdOxqIkVRDR2aIg5VjFeNII9pkAqvhYE0IF03/FdEgEoUr3VtIlWIsnL5NOo26d1Ru359VmbV5HEVXQEaohC12gJrpBLdRGFD2iZ/SK3own48V4Nz5mowVjnjlEf2B8/gAleJez</latexit>

e�

i,�=1

<latexit sha1_base64="LyMFPGBN9LEEv5uPkyVX5DGXhX4=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSxCF1qSKuhGKLhxWcE+oI1hMrlph04ezEyEGoK/4saFIm79D3f+jdPHQlsPDBzOuYd753gJZ1JZ1rdRWFpeWV0rrpc2Nre2d8zdvZaMU0GhSWMei45HJHAWQVMxxaGTCCChx6HtDa/HfvsBhGRxdKdGCTgh6UcsYJQoLbnmAbgZO8E9riM+ubLz++w0d82yVbUmwIvEnpEymqHhml89P6ZpCJGinEjZta1EORkRilEOeamXSkgIHZI+dDWNSAjSySbX5/hYKz4OYqFfpPBE/Z3ISCjlKPT0ZEjUQM57Y/E/r5uq4NLJWJSkCiI6XRSkHKsYj6vAPhNAFR9pQqhg+lZMB0QQqnRhJV2CPf/lRdKqVe2zau32vFyvzOoookN0hCrIRheojm5QAzURRY/oGb2iN+PJeDHejY/paMGYZfbRHxifP29nlHw=</latexit>

e�

i,�=2

<latexit sha1_base64="LlKe4eCpL8RCNAJaefRMcVCiSSA=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1iELrQkVdCNUHDjsoJ9QBvDZDJph04ezEyEGoK/4saFIm79D3f+jdPHQlsPDBzOuYd753gJZ1JZ1rdRWFpeWV0rrpc2Nre2d8zdvZaMU0Fok8Q8Fh0PS8pZRJuKKU47iaA49Dhte8Prsd9+oEKyOLpTo4Q6Ie5HLGAEKy255gF1M3aCelxHfHxVy++z09w1y1bVmgAtEntGyjBDwzW/en5M0pBGinAsZde2EuVkWChGOM1LvVTSBJMh7tOuphEOqXSyyfU5OtaKj4JY6BcpNFF/JzIcSjkKPT0ZYjWQ895Y/M/rpiq4dDIWJamiEZkuClKOVIzGVSCfCUoUH2mCiWD6VkQGWGCidGElXYI9/+VF0qpV7bNq7fa8XK/M6ijCIRxBBWy4gDrcQAOaQOARnuEV3own48V4Nz6mowVjltmHPzA+fwBw8JR9</latexit>

e�

i,�=3

<latexit sha1_base64="e+GK3ZuXsRkDrTmS3796RW6bE94=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1iELrQkraAboeDGZQX7gDaGyWTSDp08mJkINQR/xY0LRdz6H+78G6ePhVYPDBzOuYd753gJZ1JZ1pdRWFpeWV0rrpc2Nre2d8zdvbaMU0Foi8Q8Fl0PS8pZRFuKKU67iaA49DjteKOrid+5p0KyOLpV44Q6IR5ELGAEKy255gF1M3aC+lxHfHxZz++y09w1y1bVmgL9JfaclGGOpmt+9v2YpCGNFOFYyp5tJcrJsFCMcJqX+qmkCSYjPKA9TSMcUulk0+tzdKwVHwWx0C9SaKr+TGQ4lHIcenoyxGooF72J+J/XS1Vw4WQsSlJFIzJbFKQcqRhNqkA+E5QoPtYEE8H0rYgMscBE6cJKugR78ct/SbtWtevV2s1ZuVGZ11GEQziCCthwDg24hia0gMADPMELvBqPxrPxZrzPRgvGPLMPv2B8fANyeZR+</latexit>

e+i,�=1,!=1

<latexit sha1_base64="x2DIFg7HU8a6eatXE5f1UBZ2s18=">AAACBnicbVDLSgMxFM34rPVVdSlCsAgFS5mpgm6EghuXFewD2nHIZG7b0MxkSDJCGWblxl9x40IRt36DO//G9LHQ1gOBwzn3cHOPH3OmtG1/W0vLK6tr67mN/ObW9s5uYW+/qUQiKTSo4EK2faKAswgammkO7VgCCX0OLX94PfZbDyAVE9GdHsXghqQfsR6jRBvJKxyBl7Jyl5tEQK6cMu6KEPqGZffpaeYVinbFngAvEmdGimiGulf46gaCJiFEmnKiVMexY+2mRGpGOWT5bqIgJnRI+tAxNCIhKDednJHhE6MEuCekeZHGE/V3IiWhUqPQN5Mh0QM1743F/7xOonuXbsqiONEQ0emiXsKxFnjcCQ6YBKr5yBBCJTN/xXRAJKHaNJc3JTjzJy+SZrXinFWqt+fFWmlWRw4domNUQg66QDV0g+qogSh6RM/oFb1ZT9aL9W59TEeXrFnmAP2B9fkDkXyX0w==</latexit>

e+i,�=2,!=1

<latexit sha1_base64="J1LIc/TqGqaU2xphP67glTB2294=">AAACBnicbVDLSgMxFM3UV62vUZciBItQsJSZKuimUHDjsoJ9QGccMpnbNjTzIMkIZejKjb/ixoUibv0Gd/6N6WOhrQcCh3Pu4eYeP+FMKsv6NnIrq2vrG/nNwtb2zu6euX/QknEqKDRpzGPR8YkEziJoKqY4dBIBJPQ5tP3h9cRvP4CQLI7u1CgBNyT9iPUYJUpLnnkMXsbKDteJgNSqZezEIfRJzR7fZ2djzyxaFWsKvEzsOSmiORqe+eUEMU1DiBTlRMqubSXKzYhQjHIYF5xUQkLokPShq2lEQpBuNj1jjE+1EuBeLPSLFJ6qvxMZCaUchb6eDIkayEVvIv7ndVPVu3IzFiWpgojOFvVSjlWMJ53ggAmgio80IVQw/VdMB0QQqnRzBV2CvXjyMmlVK/Z5pXp7UayX5nXk0RE6QSVko0tURzeogZqIokf0jF7Rm/FkvBjvxsdsNGfMM4foD4zPH5MPl9Q=</latexit>

e+i,�=3,!=1

<latexit sha1_base64="GSetEzUONP8qBBDjvs354P9/Xtw=">AAACBnicbVDLSgMxFM34rPU16lKEYBEKljLTCropFNy4rGAf0BmHTHqnDc08SDJCGbpy46+4caGIW7/BnX9j+lho64HA4Zx7uLnHTziTyrK+jZXVtfWNzdxWfntnd2/fPDhsyTgVFJo05rHo+EQCZxE0FVMcOokAEvoc2v7weuK3H0BIFkd3apSAG5J+xAJGidKSZ56Al7GSw3WiR2rVEnbiEPqkZo/vs/OxZxassjUFXib2nBTQHA3P/HJ6MU1DiBTlRMqubSXKzYhQjHIY551UQkLokPShq2lEQpBuNj1jjM+00sNBLPSLFJ6qvxMZCaUchb6eDIkayEVvIv7ndVMVXLkZi5JUQURni4KUYxXjSSe4xwRQxUeaECqY/iumAyIIVbq5vC7BXjx5mbQqZbtartxeFOrFeR05dIxOURHZ6BLV0Q1qoCai6BE9o1f0ZjwZL8a78TEbXTHmmSP0B8bnD5Sil9U=</latexit>

e+i,�=`2,!=1

<latexit sha1_base64="VOXN1RLKcjGF5vjCNc4gWBvCz/I=">AAACDXicbVC7SgNBFJ2NrxhfUUubwSgEDGE3CtoEAjaWEcwDsusyO3uTDJl9MDMrhGV/wMZfsbFQxNbezr9x8ig08cDA4Zx7uHOPF3MmlWl+G7mV1bX1jfxmYWt7Z3evuH/QllEiKLRoxCPR9YgEzkJoKaY4dGMBJPA4dLzR9cTvPICQLArv1DgGJyCDkPUZJUpLbvEE3JRVbK4TPqnbwLmb1rIKtqMABqRuZffpWeYWS2bVnAIvE2tOSmiOplv8sv2IJgGEinIiZc8yY+WkRChGOWQFO5EQEzoiA+hpGpIApJNOr8nwqVZ83I+EfqHCU/V3IiWBlOPA05MBUUO56E3E/7xeovpXTsrCOFEQ0tmifsKxivCkGuwzAVTxsSaECqb/iumQCEKVLrCgS7AWT14m7VrVOq/Wbi9KjfK8jjw6QseojCx0iRroBjVRC1H0iJ7RK3oznowX4934mI3mjHnmEP2B8fkDZJWbCg==</latexit>

e+i,�=`2,!=2

<latexit sha1_base64="xLzwwPqUwUC9v8JygrImgE+zTIo=">AAACDXicbVC7SgNBFJ2NrxhfUUubwSgEDGE3CtoEAjaWEcwDsusyO7lJhsw+mJkVwrI/YOOv2FgoYmtv5984SbbQxAMDh3Pu4c49XsSZVKb5beRWVtfWN/Kbha3tnd294v5BW4axoNCiIQ9F1yMSOAugpZji0I0EEN/j0PHG11O/8wBCsjC4U5MIHJ8MAzZglCgtucUTcBNWsblO9EndBs7dpJZWsB36MCT1WnqfnKVusWRWzRnwMrEyUkIZmm7xy+6HNPYhUJQTKXuWGSknIUIxyiEt2LGEiNAxGUJP04D4IJ1kdk2KT7XSx4NQ6BcoPFN/JxLiSznxPT3pEzWSi95U/M/rxWpw5SQsiGIFAZ0vGsQcqxBPq8F9JoAqPtGEUMH0XzEdEUGo0gUWdAnW4snLpF2rWufV2u1FqVHO6sijI3SMyshCl6iBblATtRBFj+gZvaI348l4Md6Nj/lozsgyh+gPjM8fZh6bCw==</latexit>

e+i,�=1,!=2

<latexit sha1_base64="Qs1sFrchtQO2mvG2N0eNQQAYEB0=">AAACBnicbVDLSgMxFM3UV62vUZciBItQsJSZKuimUHDjsoJ9QGccMpnbNjTzIMkIZejKjb/ixoUibv0Gd/6N6WOhrQcCh3Pu4eYeP+FMKsv6NnIrq2vrG/nNwtb2zu6euX/QknEqKDRpzGPR8YkEziJoKqY4dBIBJPQ5tP3h9cRvP4CQLI7u1CgBNyT9iPUYJUpLnnkMXsbKDteJgNTsMnbiEPqkVh3fZ2djzyxaFWsKvEzsOSmiORqe+eUEMU1DiBTlRMqubSXKzYhQjHIYF5xUQkLokPShq2lEQpBuNj1jjE+1EuBeLPSLFJ6qvxMZCaUchb6eDIkayEVvIv7ndVPVu3IzFiWpgojOFvVSjlWMJ53ggAmgio80IVQw/VdMB0QQqnRzBV2CvXjyMmlVK/Z5pXp7UayX5nXk0RE6QSVko0tURzeogZqIokf0jF7Rm/FkvBjvxsdsNGfMM4foD4zPH5MFl9Q=</latexit>

e+i,�=2,!=2

<latexit sha1_base64="gdCUkaSwJSXjTurIHjDytbzg/RY=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBEKlpJUQTdCwY3LCvYBTS2TyU07dPJgZiKUkJUbf8WNC0Xc+g3u/BunbRbaemDgcM493LnHjTmTyrK+jaXlldW19cJGcXNre2fX3NtvySgRFJo04pHouEQCZyE0FVMcOrEAErgc2u7oeuK3H0BIFoV3ahxDLyCDkPmMEqWlvnkE/ZRVHK4THrmqVbATBTDQLLtPT7O+WbKq1hR4kdg5KaEcjb755XgRTQIIFeVEyq5txaqXEqEY5ZAVnURCTOiIDKCraUgCkL10ekaGT7TiYT8S+oUKT9XfiZQEUo4DV08GRA3lvDcR//O6ifIveykL40RBSGeL/IRjFeFJJ9hjAqjiY00IFUz/FdMhEYQq3VxRl2DPn7xIWrWqfVat3Z6X6uW8jgI6RMeojGx0geroBjVQE1H0iJ7RK3oznowX4934mI0uGXnmAP2B8fkDlJiX1Q==</latexit>

e+i,�=3,!=2

<latexit sha1_base64="6l7mFwWbPJCx86HnnOO9FiqeJP4=">AAACBnicbVDLSgMxFM34rPU16lKEYBEKljLTCropFNy4rGAf0I5DJr1tQzOTIckIZejKjb/ixoUibv0Gd/6N6WOhrQcCh3Pu4eaeIOZMacf5tlZW19Y3NjNb2e2d3b19++CwoUQiKdSp4EK2AqKAswjqmmkOrVgCCQMOzWB4PfGbDyAVE9GdHsXghaQfsR6jRBvJt0/AT1mhw02iSyrlAu6IEPqkUhrfp+dj3845RWcKvEzcOcmhOWq+/dXpCpqEEGnKiVJt14m1lxKpGeUwznYSBTGhQ9KHtqERCUF56fSMMT4zShf3hDQv0niq/k6kJFRqFAZmMiR6oBa9ifif105078pLWRQnGiI6W9RLONYCTzrBXSaBaj4yhFDJzF8xHRBJqDbNZU0J7uLJy6RRKrrlYun2IlfNz+vIoGN0ivLIRZeoim5QDdURRY/oGb2iN+vJerHerY/Z6Io1zxyhP7A+fwCWK5fW</latexit>

e+i,�=3,!=`1

<latexit sha1_base64="fCwSb+GHAQpQE2W4ml48e+uZxWU=">AAACDXicbVC7SgNBFJ31GeMramkzGIWAIewmgjaBgI1lBPOA7LrMTm6SIbMPZmaFsOwP2PgrNhaK2Nrb+TdOki008cDA4Zx7uHOPF3EmlWl+Gyura+sbm7mt/PbO7t5+4eCwLcNYUGjRkIei6xEJnAXQUkxx6EYCiO9x6Hjj66nfeQAhWRjcqUkEjk+GARswSpSW3MIpuAkr21wn+qReK2M79GFI6jZw7iZWmt4n56lbKJoVcwa8TKyMFFGGplv4svshjX0IFOVEyp5lRspJiFCMckjzdiwhInRMhtDTNCA+SCeZXZPiM6308SAU+gUKz9TfiYT4Uk58T0/6RI3kojcV//N6sRpcOQkLolhBQOeLBjHHKsTTanCfCaCKTzQhVDD9V0xHRBCqdIF5XYK1ePIyaVcrVq1Svb0oNkpZHTl0jE5QCVnoEjXQDWqiFqLoET2jV/RmPBkvxrvxMR9dMbLMEfoD4/MHXq6bCw==</latexit>

e+i,�=`2,!=`1

<latexit sha1_base64="W0Nz56DLkpaSrXTeHU32CXwPYRo=">AAACFHicbVDLSgMxFM3UV62vqks3wSIUWspMFXQjFNy4rGAf0BmHTOa2Dc08SDJCGeYj3Pgrblwo4taFO//G9CFo64HA4Zx7uLnHizmTyjS/jNzK6tr6Rn6zsLW9s7tX3D9oyygRFFo04pHoekQCZyG0FFMcurEAEngcOt7oauJ37kFIFoW3ahyDE5BByPqMEqUlt1gBN2VVm+uETy5t4NxN61kV21EAgx/ByrK7tJK5xZJZM6fAy8SakxKao+kWP20/okkAoaKcSNmzzFg5KRGKUQ5ZwU4kxISOyAB6moYkAOmk06MyfKIVH/cjoV+o8FT9nUhJIOU48PRkQNRQLnoT8T+vl6j+hZOyME4UhHS2qJ9wrCI8aQj7TABVfKwJoYLpv2I6JIJQpXss6BKsxZOXSbtes05r9ZuzUqM8ryOPjtAxKiMLnaMGukZN1EIUPaAn9IJejUfj2Xgz3mejOWOeOUR/YHx8A0OMnkA=</latexit>

e+i,�=2,!=`1

<latexit sha1_base64="dW64ZMdI+2scxryX+RX9dOuhOKU=">AAACDXicbVC7SgNBFJ2NrxhfUUubwSgEDGE3CtoEAjaWEcwDsusyO3uTDJl9MDMrhGV/wMZfsbFQxNbezr9x8ig08cDA4Zx7uHOPF3MmlWl+G7mV1bX1jfxmYWt7Z3evuH/QllEiKLRoxCPR9YgEzkJoKaY4dGMBJPA4dLzR9cTvPICQLArv1DgGJyCDkPUZJUpLbvEE3JRVbK4TPqnXKtiOAhiQug2cu6mVZffpWeYWS2bVnAIvE2tOSmiOplv8sv2IJgGEinIiZc8yY+WkRChGOWQFO5EQEzoiA+hpGpIApJNOr8nwqVZ83I+EfqHCU/V3IiWBlOPA05MBUUO56E3E/7xeovpXTsrCOFEQ0tmifsKxivCkGuwzAVTxsSaECqb/iumQCEKVLrCgS7AWT14m7VrVOq/Wbi9KjfK8jjw6QseojCx0iRroBjVRC1H0iJ7RK3oznowX4934mI3mjHnmEP2B8fkDXRSbCg==</latexit>

e+i,�=1,!=`1

<latexit sha1_base64="V4cumJtgeWCsQ7emFkzOtJZ8Pg8=">AAACDXicbVC7SgNBFJ2NrxhfUUubwSgEDGE3CtoEAjaWEcwDsusyO3uTDJl9MDMrhGV/wMZfsbFQxNbezr9x8ig08cDA4Zx7uHOPF3MmlWl+G7mV1bX1jfxmYWt7Z3evuH/QllEiKLRoxCPR9YgEzkJoKaY4dGMBJPA4dLzR9cTvPICQLArv1DgGJyCDkPUZJUpLbvEE3JRVbK4TPqlbFWxHAQxI3QbO3dTKsvv0LHOLJbNqToGXiTUnJTRH0y1+2X5EkwBCRTmRsmeZsXJSIhSjHLKCnUiICR2RAfQ0DUkA0kmn12T4VCs+7kdCv1Dhqfo7kZJAynHg6cmAqKFc9Cbif14vUf0rJ2VhnCgI6WxRP+FYRXhSDfaZAKr4WBNCBdN/xXRIBKFKF1jQJViLJy+Tdq1qnVdrtxelRnleRx4doWNURha6RA10g5qohSh6RM/oFb0ZT8aL8W58zEZzxjxziP7A+PwBW3qbCQ==</latexit>

…

...

...

...

Augmentation Set

...

.12 .91 .14 … .09

.13 .94 .17 … .06

.11 .13 .14 … .93

...

.92 .08 .11 … .12

.12 .91 .14 … .09

.13 .94 .17 … .06

.11 .13 .14 … .93

...

.92 .08 .11 … .12

0 1 0 … 0

0 1 0 … 0

0 0 0 … 1

...

1 0 0 … 0

.10 .89 .04 … .11

.03 .94 .17 … .16

.01 .21 .04 … .93

...

.99 .18 .04 … .05 ... .03 .92 .04 … .19

.11 .96 .13 … .16

.01 .16 .11 … .92

...

.94 .18 .13 … .02

.02 .96 .16 … .19

.12 .93 .17 … .07

.12 .26 .01 … .87

...

.92 .11 .11 … .18

0 1 0 … 0

0 1 0 … 0

0 0 0 … 1

...

1 0 0 … 0

.05 .97 .11 … .03

.12 .90 .05 … .04

.15 .03 .02 … .89

...

.93 .18 .14 … .12 ... .01 .91 .02 … .19

.11 .94 .07 … .16

.10 .07 .18 … .93

...

.92 .01 .10 … .02

.11 .98 .04 … .11

.08 .98 .15 … .03

.12 .10 .01 … .87

...

.90 .08 .01 … .19

...

...

Augmentation Set

…

…...

......

…

…

…...

......

…

e�

i,�=

2̀


e�

i,�=1


e�

i,�=2


e�

i,�=3


…

e�

i,�=

2̀


e�

i,�=1


e�

i,�=2


e�

i,�=3


…



1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...





1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...





1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

... 1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

... 1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

...

Negative (inter-attribute) data augmentations

Positive (intra-attribute) data augmentations

...

. . .

. . .

#!=1<latexit sha1_base64="qN+Yu6KUqwp3/2BJw8iy/VteKv4=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5CRbBVUlqQTdCwY3LCvYBTQiT6W07dCYTZiaFGoq/4saFIm79D3f+jdM2C209cOFwzr0z954oYVRp1/22CmvrG5tbxe3Szu7e/oF9eNRSIpUEmkQwITsRVsBoDE1NNYNOIgHziEE7Gt3O/PYYpKIiftCTBAKOBzHtU4K1kUL7xB9jqYegcZj5gsMA33jT0C67FXcOZ5V4OSmjHI3Q/vJ7gqQcYk0YVqrruYkOMvMyJQymJT9VkGAywgPoGhpjDirI5ttPnXOj9Jy+kKZi7czV3xMZ5kpNeGQ6OdZDtezNxP+8bqr710FG4yTVEJPFR/2UOVo4syicHpVANJsYgomkZleHDLHERJvASiYEb/nkVdKqVrzLSvW+Vq7X8jiK6BSdoQvkoStUR3eogZqIoEf0jF7Rm/VkvVjv1seitWDlM8foD6zPH6nElU4=</latexit>

#!=2<latexit sha1_base64="pELdhV9NNX0XkRkDl4H3Pf/PDX0=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5CRbBVUlqQTdCwY3LCvYBTQiT6W07dDITZiaFGoq/4saFIm79D3f+jdM2C209cOFwzr0z954oYVRp1/22CmvrG5tbxe3Szu7e/oF9eNRSIpUEmkQwITsRVsAoh6ammkEnkYDjiEE7Gt3O/PYYpKKCP+hJAkGMB5z2KcHaSKF94o+x1EPQOMx8EcMA31SnoV12K+4czirxclJGORqh/eX3BElj4JowrFTXcxMdZOZlShhMS36qIMFkhAfQNZTjGFSQzbefOudG6Tl9IU1x7czV3xMZjpWaxJHpjLEeqmVvJv7ndVPdvw4yypNUAyeLj/opc7RwZlE4PSqBaDYxBBNJza4OGWKJiTaBlUwI3vLJq6RVrXiXlep9rVyv5XEU0Sk6QxfIQ1eoju5QAzURQY/oGb2iN+vJerHerY9Fa8HKZ47RH1ifP6tJlU8=</latexit>

#!=`1<latexit sha1_base64="Dhwh8qhcXRz/bx9oMW8TPd/MXbc=">AAACBHicbVDLSsNAFJ3UV62vqMtuBovgqiS1oBuh4MZlBfuAJoTJ9KYdOnkwMymUkIUbf8WNC0Xc+hHu/BunbRbaeuDC4Zx7ufceP+FMKsv6Nkobm1vbO+Xdyt7+weGReXzSlXEqKHRozGPR94kEziLoKKY49BMBJPQ59PzJ7dzvTUFIFkcPapaAG5JRxAJGidKSZ1adKRFqDIp4mROHMCI3DnDuZXaee2bNqlsL4HViF6SGCrQ988sZxjQNIVKUEykHtpUoN9MLGOWQV5xUQkLohIxgoGlEQpButngix+daGeIgFroihRfq74mMhFLOQl93hkSN5ao3F//zBqkKrt2MRUmqIKLLRUHKsYrxPBE8ZAKo4jNNCBVM34rpmAhClc6tokOwV19eJ91G3b6sN+6btVaziKOMqugMXSAbXaEWukNt1EEUPaJn9IrejCfjxXg3PpatJaOYOUV/YHz+AGKKmIQ=</latexit>

L!=1NCE

<latexit sha1_base64="kPmZNCfcWgkJJZo60Y3fDk3jJCU=">AAACBXicbVDLSsNAFJ3UV62vqEtdDBbBVUlqQTdCoQguRCrYBzQxTKaTduhMEmYmQgnZuPFX3LhQxK3/4M6/cdJ2oa0HLhzOuZd77/FjRqWyrG+jsLS8srpWXC9tbG5t75i7e20ZJQKTFo5YJLo+koTRkLQUVYx0Y0EQ9xnp+KNG7nceiJA0Cu/UOCYuR4OQBhQjpSXPPHQ4UkOMWHqd3adOxMkAXdiZl940LjPPLFsVawK4SOwZKYMZmp755fQjnHASKsyQlD3bipWbIqEoZiQrOYkkMcIjNCA9TUPEiXTTyRcZPNZKHwaR0BUqOFF/T6SISznmvu7Mb5bzXi7+5/USFZy7KQ3jRJEQTxcFCYMqgnkksE8FwYqNNUFYUH0rxEMkEFY6uJIOwZ5/eZG0qxX7tFK9rZXrtVkcRXAAjsAJsMEZqIMr0AQtgMEjeAav4M14Ml6Md+Nj2lowZjP74A+Mzx+IbZiJ</latexit>

L!=2NCE

<latexit sha1_base64="KEpnNU8JieQPgny5CZNyPGMOEhA=">AAACBXicbVDLSsNAFJ3UV62vqEtdBIvgqiS1oBuhUAQXIhXsA5oYJtNJO3RmEmYmQgnZuPFX3LhQxK3/4M6/cdJ2oa0HLhzOuZd77wliSqSy7W+jsLS8srpWXC9tbG5t75i7e20ZJQLhFopoJLoBlJgSjluKKIq7scCQBRR3glEj9zsPWEgS8Ts1jrHH4ICTkCCotOSbhy6DaoggTa+z+9SNGB7Ai2rmpzeNy8w3y3bFnsBaJM6MlMEMTd/8cvsRShjmClEoZc+xY+WlUCiCKM5KbiJxDNEIDnBPUw4Zll46+SKzjrXSt8JI6OLKmqi/J1LIpByzQHfmN8t5Lxf/83qJCs+9lPA4UZij6aIwoZaKrDwSq08ERoqONYFIEH2rhYZQQKR0cCUdgjP/8iJpVyvOaaV6WyvXa7M4iuAAHIET4IAzUAdXoAlaAIFH8AxewZvxZLwY78bHtLVgzGb2wR8Ynz+J+JiK</latexit>

L!=`1NCE

<latexit sha1_base64="eoN86n3ZPPTlQI9AR38NhAVSIyQ=">AAACDHicbVDLSsNAFJ34rPVVdekmWARXJakF3QiFIrgQqWAf0MQwmd62QycPZiZCGfIBbvwVNy4UcesHuPNvnLRZaOuBgcM55zL3Hj9mVEjL+jaWlldW19YLG8XNre2d3dLefltECSfQIhGLeNfHAhgNoSWpZNCNOeDAZ9Dxx43M7zwAFzQK7+QkBjfAw5AOKMFSS16p7ARYjghm6jq9V04UwBBfOMCYp+w09dRN4zLVKatiTWEuEjsnZZSj6ZW+nH5EkgBCSRgWomdbsXQV5pISBmnRSQTEmIzxEHqahjgA4arpMal5rJW+OYi4fqE0p+rvCYUDISaBr5PZ6mLey8T/vF4iB+euomGcSAjJ7KNBwkwZmVkzZp9yIJJNNMGEU72rSUaYYyJ1f0Vdgj1/8iJpVyv2aaV6WyvXa3kdBXSIjtAJstEZqqMr1EQtRNAjekav6M14Ml6Md+NjFl0y8pkD9AfG5w9YdZu/</latexit>

Positive Pair (Attract) Negative Pair (Repel)

...

.01 0.96 .21 … .04

1.0 0.0 0.0 … .06

0.0 0.0 0.0 … .03

0.0 0.0 0.0 … .12

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

.01 .98 .12 … .14

.94 .14 .07 … .03

.03 .15 .06 … .89

.11 .23 .02 … .87

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

...

.01 0.96 .21 … .04

1.0 0.0 0.0 … .06

0.0 0.0 0.0 … .03

0.0 0.0 0.0 … .12

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

.01 .98 .12 … .14

.94 .14 .07 … .03

.03 .15 .06 … .89

.11 .23 .02 … .87

...

0.0 1.0 0.0 … 0.0

1.0 0.0 0.0 … 0.0

0.0 0.0 0.0 … 1.0

0.0 0.0 0.0 … 1.0

...

InfoNCE Sim. Matrix

InfoNCE Sim. Matrix

InfoNCE Sim. Matrix

Figure 3: Overview of the proposed journal entry augmentation technique. Given an encoded journal entry 𝑥𝑖 , negative andpositive augmentation policies are sequentially applied to yield distinct augmentation sets 𝜗𝜔 . For each set, the InfoNCE lossL𝜔NCE [45] is derived over the pair-wise embeddings. The accumulation of the set losses increases the task invariance of thelearned journal entry representations.

Table 1: Number of neurons per layer [ of the encoder 𝑓\and projection-head 𝑔𝜙 network that comprise the SimCLRarchitecture [10] used in our experiments.

Net Dataset [ = 1 2 3 4 ... 11 12𝑓\ (𝑧 |𝑥) A 4,096 2,048 1,024 512 ... 4 2𝑔𝜙 (𝑒 |𝑧) A 2 2 - - ... - -

𝑓\ (𝑧 |𝑥) B 2,048 1,024 512 256 ... 4 2𝑔𝜙 (𝑒 |𝑧) B 2 2 - - ... - -

We set _ = 20 in the negative augmentation policy T−. Thepositive augmentation policy T + uses the following parameters:`𝑅 = 0.0 and 𝜎𝑅 = 0.05 (Random-Noise), `𝐶 = 0.2 (Random-Cut), aswell as `𝐺 = 0.0 and 𝜎𝐺 = 0.8 (Gaussian-Kernel). The parameters \and 𝜙 of the encoder and projection-head are randomly initializedas described in [16]. We train all models for a max. of 1,000 trainingepochs, a mini-batch size of 𝑚 = 128 journal entries, and earlystopping once the loss converges. We use Adam optimization [37]with 𝛽1 = 0.9, 𝛽2 = 0.999 and a cosine learning rate schedule inthe optimization of the network parameters. Upon completion ofthe pre-training, we discard the projection head and transfer thelearned encoder parameters \∗ to a given downstream audit tasks.

4.3 Downstream Audit Task EvaluationWe evaluate the generalization performance of the learned jour-nal entry representations based in three downstream audit tasks,namely (i) anomaly detection, (ii) audit sampling, and (iii) auditdocumentation.

Anomaly Detection: When assessing the risk of material mis-statement due to fraud, auditors are required to detect ‘unusual’or anomalous journal entries (ISA 240, SAS 99) [4, 24]. Thereby,two classes of anomalies can be distinguished (i) global anomaliesthat exhibit unusual attribute values and (ii) local anomalies thatexhibit unusual attribute value combinations [48]. We inject a small

fraction of 200 randomly sampled anomalies (60 global and 140 lo-cal) into each city payment dataset.4 To evaluate the quality of thelearned representations for the purpose of anomaly detection, weutilize Autoencoder Neural Networks (AEN) [21]. The AENs encoder𝑓\ and decoder 𝑞𝜓 are of symmetrical architecture. We initializethe encoder parameters with the pre-trained \∗ parameters, whilethe decoder parameters 𝜓 are randomly initialized as describedin [16]. Throughout the AEN fine-tuning, the encoder parameters\∗ are not updated to retain the pre-trained representations. Thedecoder parameters are fine-tuned for a max. of 100 training epochsby minimizing a reconstruction loss, formally defined by:

argmin𝜓

= | |𝑥𝑖 − 𝑞𝜓 (𝑓\ ∗ (𝑥𝑖 )) | |2, (4)

where 𝑥𝑖 ∈ 𝑋 denotes an encoded payment. We use a combinedloss [50] that defines the reconstruction error of a given encodedjournal entry 𝑥𝑖 , defined as:

L𝜓 (𝑥𝑖 , 𝑥𝑖 ) = a L𝐵𝐶𝐸𝜓,𝑐𝑎𝑡

+ (1 − a) L𝑀𝑆𝐸𝜓,𝑛𝑢𝑚

, (5)

where 𝑥𝑖 = 𝑞𝜓 (𝑓\ ∗ (𝑥𝑖 )) denotes the i-th journal entry reconstruc-tion, L𝐵𝐶𝐸

𝜓,𝑐𝑎𝑡= 1𝑁

∑𝑁𝑖=1 𝑥

𝑐𝑎𝑡𝑖

· log(𝑥𝑐𝑎𝑡𝑖

) + (1 − 𝑥𝑐𝑎𝑡𝑖

) · log(1 − 𝑥𝑐𝑎𝑡𝑖

) defines thebinary-cross-entropy error over the categorical attributes, whileL𝑀𝑆𝐸𝜓,𝑛𝑢𝑚

= 1𝑁

∑𝑁𝑖=1 (𝑥𝑛𝑢𝑚𝑖

− 𝑥𝑛𝑢𝑚𝑖

)2 denotes the mean-squared error overthe numerical attributes. To account for the higher number of cat-egorical attributes in both datasets we set a = 2

3 . To quantitativelyassess the anomaly detection capability of the fine-tuned modelswe derive the average precision 𝐴𝑃 over the sorted payments re-construction errors L𝜓 . The 𝐴𝑃 summarizes the model’s precision-recall curve as weighted mean of precisions at each reconstructionerror threshold, formally defined by:

4To create both classes of anomalies, we built upon the ‘Faker’ project, which is publiclyavailable via the following URL: https://github.com/joke2k/faker

https://github.com/joke2k/faker


𝐴𝑃 (L𝜓 ) =∑︁𝑁

𝑖=1(𝑅𝑖 − 𝑅𝑖−1)𝑃𝑖 , (6)

where 𝑃𝑖 (L𝜓 ) = 𝑇𝑃/(𝑇𝑃 + 𝐹𝑃) denotes the detection precision,and 𝑅𝑖 (L𝜓 ) = 𝑇𝑃/(𝑇𝑃 + 𝐹𝑁 ) denotes the detection recall of thei-th reconstruction error threshold.

Audit Sampling: When performing substantive audit procedures,auditors are required to audit a representative sample of an orga-nization’s accounting records to derive a ‘reasonable basis for anopinion’ and mitigate sampling risks (ISA 530, SAS 39) [3, 25]. Toevaluate the quality of the learned representations for the purpose ofaudit sampling we utilize Vector Quantized Variational Autoenoders(VQ-VAE) [58]. The VQ-VAE defines a discrete latent space 𝑍 ′ anda set of 𝐾 latent quantization vectors 𝑧′

𝑗∈ 𝑍 ′, where 𝑗 = 1, 2, ..., 𝐾 .

For a given journal entry 𝑥𝑖 the VQ-VAE’s encoder network 𝑓\produces a latent representation 𝑧𝑖 . To obtain a quantization 𝑧𝑖of 𝑧𝑖 , a nearest neighbour lookup is performed 𝑧𝑖 = 𝑧′

𝑘, where

𝑘 = argmin𝑗 | |𝑧𝑖 − 𝑧′𝑗 | |2. Afterwards, the quantised representation𝑧𝑖 is passed to the decoder 𝑞𝜓 . Both, encoder 𝑓\ and decoder 𝑞𝜓 , areof symmetrical architecture. We initialize the encoder parameterswith the pre-trained \∗ parameters, while the decoder parameters𝜓are randomly initialized as described in [16]. The decoder parame-ters are fine-tuned for a max. of 100 training epochs by minimizinga reconstruction loss, formally defined by:

argmin𝜓

= | |𝑥𝑖 − 𝑞𝜓 (𝑧𝑖 ) | |2 + 𝛼 | |𝑠𝑔[𝑧𝑖 ] − 𝑧′ | |22+ 𝛽 | |𝑧𝑖 − 𝑠𝑔[𝑧′] | |22 + 𝛾 | |𝑥𝑖 − 𝑞𝜓 (𝑧𝑖 ) | |2,

(7)

where 𝑧𝑖 denotes the encoder output, 𝑧𝑖 the quantised encoder out-put, 𝑧′ the set of quantization vectors, and 𝑠𝑔[·] the stop gradientoperator. Throughout the VQ-VAE fine-tuning, the encoder param-eters \∗ are not updated to retain the pre-trained representations.We use the combined reconstruction loss [50] defined in Eq. 5 tooptimize the first and fourth optimization term of Eq. 7. We quan-titatively assess the model’s fine-tuned audit sampling capabilityby measuring (i) quantization perplexity and (ii) quantization pu-rity. The average quantization perplexity defines the likelihoodof a quantised representation 𝑧𝑖 of being assigned to a particularquantization 𝑧′

𝑗, formally expressed by:

P𝑒𝑟𝑝 (𝜋) =1𝐾

∑︁𝐾

𝑗=12−

∑𝐾𝑗=1 𝑝 (𝜋 𝑗 ) log2 𝑝 (𝜋 𝑗 ) , (8)

where 𝜋 𝑗 =∑𝑁𝑖=1 1[𝑧𝑖 = 𝑧′𝑗 ] denotes the number of payments quan-

tized by a particular quantization and 1 the indicator function. Wedetermine the average purity of quantizations, as defined by:

P𝑢𝑟𝑖𝑡𝑦 (𝜋) =1𝐾

∑︁𝐾

𝑗=11 −

|𝜋 ′𝑗|

𝜋 𝑗, (9)

where denotes |𝜋 𝑗 | the cardinality of de-duplicated payments 𝜋 ′𝑗

given a particular quantization 𝜋 𝑗 . The payment de-duplicationis conducted over the categorical attributes of each 𝑥𝑖 ∈ 𝜋 𝑗 . Inaddition, we obtain the weighted purity P ′𝑢𝑟𝑖𝑡𝑦 by normalizing theaverage purity of each quantization by the fraction of quantizedpayments.

Audit Documentation: Auditors are obliged to prepare audit doc-umentation that enables an experienced external auditor to ‘un-derstand the results of the audit procedures performed and the

audit evidence obtained’ (ISA 230, SAS 103) [5, 22]. As shown inTab. 1 we restrict the encoder 𝑓\ bottleneck to two dimensions forall trained models. The limitation of 𝑍 ∈ R2 allows for a human(visual) interpretation of the learned representations and improveddocumentation of associated audit findings.

5 EXPERIMENTAL RESULTSIn this section, we quantitatively and qualitatively assess the trans-ferablity of the learned representations towards the downstreamaudit tasks.

Anomaly Detection: We evaluate the fine-tuned autoencodermodel (AENSSL) and compare its anomaly detection capability todifferent autoencoders (AEN) proposed in [48] which we use as abaseline. The baseline models are specifically designed for the pur-pose of journal entry anomaly detection. To conduct a fair compar-ison, we trained distinct baseline model comprised of [ ∈ [1, 6, 12]encoder and decoder layers for 1,000 training epochs following thetraining objective defined in Eq. 5.

Table 2: Average precision scores obtained for all journal en-try anomalies 𝐴𝑃𝑎𝑙𝑙 , global anomalies 𝐴𝑃𝑔𝑙𝑜𝑏𝑎𝑙 , local anom-alies 𝐴𝑃𝑙𝑜𝑐𝑎𝑙 and distinct temperature parameters 𝜏 on bothcity payment datasets.

Model Data 𝜏 𝐴𝑃𝑎𝑙𝑙 𝐴𝑃𝑔𝑙𝑜𝑏𝑎𝑙 𝐴𝑃𝑙𝑜𝑐𝑎𝑙

AEN1 A - 0.452 ± 0.05 0.404 ± 0.01 0.186 ± 0.01AEN6 A - 0.587 ± 0.03 0.211 ± 0.08 0.439 ± 0.13AEN12 A - 0.565 ± 0.02 0.082 ± 0.02 0.591 ± 0.14AENSSL A 0.1 0.586 ± 0.04 0.371 ± 0.05 0.312 ± 0.03AENSSL A 0.5 0.680 ± 0.02 0.412 ± 0.03 0.383 ± 0.02AENSSL A 0.8 0.882 ± 0.02 0.673 ± 0.06 0.483 ± 0.03

AEN1 B - 0.430 ± 0.04 0.051 ± 0.01 0.480 ± 0.04AEN6 B - 0.435 ± 0.10 0.098 ± 0.01 0.394 ± 0.48AEN12 B - 0.4993 ± 0.06 0.113 ± 0.02 0.459 ± 0.08AENSSL B 0.1 0.472 ± 0.10 0.087 ± 0.01 0.426 ± 0.16AENSSL B 0.5 0.334 ± 0.05 0.089 ± 0.11 0.294 ± 0.08AENSSL B 0.8 0.416 ± 0.13 0.256 ± 0.08 0.318 ± 0.17

Variances originate from parameter initialization using five distinct random seeds.

Table 2 shows the average precision scores obtained for bothanomaly classes 𝐴𝑃𝑎𝑙𝑙 , the global anomalies 𝐴𝑃𝑔𝑙𝑜𝑏𝑎𝑙 , and the localanomalies 𝐴𝑃𝑙𝑜𝑐𝑎𝑙 over distinct temperature parameters 𝜏 . Com-paring the obtained results shows that the fine-tuned contrastiveSSL models yield a high overall detection precision for the distinctanomaly classes. Furthermore, the task-invariant representationsdemonstrate the ability to significantly outperform the baselines inboth city payment datasets for the global anomalies. For the localanomalies, the task-invariant representations underperform thebaselines of highly specialized AENs. In summary, the obtainedresults provide initial evidence of the generalization capabilitiesof contrastive SSL, closing the gap to learning highly specializedaccounting data representations.

Audit Sampling: We evaluate the fine-tuned VQ-VAE models(VAESSL) and compare its audit sampling capability to differentvector-quantized VAE models (VAE) proposed in [49] which we useas a baseline. The baseline models are specifically designed for thepurpose audit sampling. To conduct a fair comparison, we trained


(a) Attribute: Payment Type (b) Attribute: Fiscal Month (c) Anomaly Detection (d) Audit Sampling

Figure 4: Learned task invariant accounting data representations 𝑧𝑖 ∈ R2 with 𝜏 = 0.5 of the 238,894 ‘City of Philadelphia’ ven-dor payments (dataset A). The visualisations on the left show the representations coloured according to selected payment char-acteristics: payment type (a) and posting month (b). The visualisations on the right show the same representations colouredaccording to the downstream audit task: anomaly detection (c) and audit sampling (d).

distinct baseline models comprised of different audit sample sizes𝐾 ∈ [23, 25, 26] respectively for 1,000 training epochs following thetraining objective defined in Eq. 7.

Table 3: Quantization purity P𝑢𝑟𝑖𝑡𝑦, weighted quantizationpurity P ′𝑢𝑟𝑖𝑡𝑦, and codebook perplexity P𝑒𝑟𝑝 obtained fordifferent codebook size 𝐾 on both city payment datasets.

Model Data 𝐾 P𝑢𝑟𝑖𝑡𝑦 P′𝑢𝑟𝑖𝑡𝑦 P𝑒𝑟𝑝VAE A 8 0.174 ± 0.02 0.897 ± 0.01 6.856 ± 0.52VAE A 64 0.195 ± 0.03 0.974 ± 0.01 37.618 ± 3.88VAE A 128 0.226 ± 0.05 0.988 ± 0.02 41.251 ± 4.32VAESSL A 8 0.174 ± 0.01 0.897 ± 0.01 7.369 ± 0.45VAESSL A 64 0.313 ± 0.02 0.985 ± 0.01 26.211 ± 1.04VAESSL A 128 0.372 ± 0.02 0.992 ± 0.01 37.931 ± 2.21

VAE B 8 0.672 ± 0.01 0.956 ± 0.01 5.950 ± 0.63VAE B 64 0.756 ± 0.08 0.989 ± 0.01 21.636 ± 9.89VAE B 128 0.914 ± 0.04 0.997 ± 0.01 17.965 ± 8.01VAESSL B 8 0.955 ± 0.01 0.994 ± 0.01 5.940 ± 0.19VAESSL B 64 0.995 ± 0.01 0.999 ± 0.01 12.983 ± 3.47VAESSL B 128 0.998 ± 0.01 0.999 ± 0.01 12.904 ± 0.25

Variances originate from parameter initialization using five distinct random seeds.

Table 3 shows the average codebook perplexity P𝑒𝑟𝑝, quantiza-tion purity P𝑢𝑟𝑖𝑡𝑦, and weighted quantization purity P ′𝑢𝑟𝑖𝑡𝑦 ob-tained for distinct audit sample sizes 𝐾 . The obtained results showthat the average codebook usage and quantization purity increaseswith increased codebook sizes. Hence, a more fine-grained quan-tization of the latent generative factors is learned. For both citypayment datasets, the fine-tuned contrastive SSL models achievesignificantly higher purity scores when compared to the baselinesof specialized VQ-VAEs. This observation confirms the general-ization capabilities of the pre-trained representations to yield arepresentative audit sample.

Audit Documentation: We qualitatively assess the accounting-specific semantics captured by the task invariant accounting repre-sentations in different regions of the latent space 𝑍 ∈ R2. Figure4 shows the learned representations of the City of Philadelphia

Payments (Dataset A). The visualizations illustrate that the learnedrepresentations correspond to latent manifolds that ‘disentangle’the underlying data semantics. In the context of audit documen-tation the characteristics and underlying semantics of the latentspace could serve as reference point for explaining deep learninginspired audit procedures and corresponding findings.

6 SUMMARYIn this work, we proposed a self-supervised learning framework tolearn task invariant accounting data representations. We demon-strated that the SimCLR architecture, enhanced by deliberatelydesigned intra-attribute and inter-attribute data augmentation poli-cies, yield excellent performance on the evaluated downstreamaudit tasks, namely (i) anomaly detection, (ii) audit sampling, and(iii) audit documentation. In summary, the obtained experimentalresults based on real-world datasets of city payments provide ini-tial evidence that contrastive SSL could provide a starting pointfor a variety of further downstream audit procedures. Given thetremendous amount of journal entries recorded by organisationsnowadays, such techniques offer the ability to increase the effi-ciency of audits by pre-training a single multi-purpose model andtherefore mitigate the time pressure faced by auditors nowadays.

REFERENCES[1] Practice Aid for Testing Journal Entries and Other Adjustments Pursuant to AU

Section 316. Center for Audit Quality (CAQ), 2008.[2] P. Agrawal, J. Carreira, and J. Malik. Learning to see by moving. In Proceedings of

the IEEE International Conference on Computer Vision (ICCV), pages 37–45, 2015.[3] AICPA. Statement on Auditing Standards No. 39.: Audit Sampling. 1981.[4] AICPA. Statement on Auditing Standards No. 99.: Consideration of Fraud in a

Financial Statement Audit. 2002.[5] AICPA. Statement on Auditing Standards No. 103.: Audit Documentation. 2015.[6] A. Argyrou. Auditing Journal Entries Using Self-Organizing Map. In Proceedings

of the Eighteenth Americas Conference on Information Systems, 2012.[7] A. Argyrou. Auditing Journal Entries Using Extreme Vale Theory. In Proceedings

of the 21st European Conference on Information Systems, 2013.[8] S. Bay, K. Kumaraswamy, M. G. Anderle, R. Kumar, D. M. Steier, A. Blvd, and

S. Jose. Large Scale Detection of Irregularities in Accounting Data. In SixthInternational Conference on Data Mining, pages 75–86. IEEE, 2006.


[9] A. R. Bowrin and J. King. Time pressure, task complexity, and audit effectiveness.Managerial auditing journal, 2010.

[10] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework forcontrastive learning of visual representations. In International Conference onMachine Learning (ICML), pages 1597–1607. PMLR, 2020.

[11] S. Cho, M. A. Vasarhelyi, T. Sun, and C. Zhang. Learning from machine learningin accounting and assurance. Journal of Emerging Technologies in Accounting,17(1):1–10, 2020.

[12] R. S. Debreceny and G. L. Gray. Data mining journal entries for fraud detection:An exploratory study. International Journal of Accounting Information Systems,11(3):157–181, 2010.

[13] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scalehierarchical image database. In 2009 IEEE Conference on Computer Vision andPattern Recognition (ICPR), pages 248–255, 2009.

[14] C. Doersch, A. Gupta, and A. A. Efros. Unsupervised visual representationlearning by context prediction. In Proceedings of the IEEE International Conferenceon Computer Vision (ICCV), pages 1422–1430, 2015.

[15] A. Dosovitskiy, P. Fischer, J. T. Springenberg, M. Riedmiller, and T. Brox. Dis-criminative unsupervised feature learning with exemplar convolutional neu-ral networks. IEEE Transactions on Pattern Analysis and Machine Intelligence,38(9):1734–1747, 2015.

[16] X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforwardneural networks. In Proceedings of the thirteenth international conference onartificial intelligence and statistics, pages 249–256, 2010.

[17] P. Goyal, D. Mahajan, A. Gupta, and I. Misra. Scaling and benchmarking self-supervised visual representation learning. In Proceedings of the IEEE/CVF Inter-national Conference on Computer Vision, pages 6391–6400, 2019.

[18] S. V. Grabski, S. A. Leech, and P. J. Schmidt. A review of erp research: A futureagenda for accounting information systems. Journal of information systems,25(1):37–78, 2011.

[19] R. Hadsell, S. Chopra, and Y. LeCun. Dimensionality reduction by learning aninvariant mapping. In 2006 IEEE Computer Society Conference on Computer Visionand Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE, 2006.

[20] K. He, H. Fan, Y.Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervisedvisual representation learning. In Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (CVPR, pages 9729–9738, 2020.

[21] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data withneural networks. science, 313(5786):504–507, 2006.

[22] IFAC. International Standards on Auditing 230, Audit Documentation. 2006.[23] IFAC. International Accounting Standard (ISA) 1 - Presentation of Financial State-

ments. 2007.[24] IFAC. International Standard on Auditing (ISA) 240, The Auditor’s Responsibility

to Consider Fraud in an Audit of Financial Statements. 2009.[25] IFAC. International Standard on Auditing (ISA) 530, Audit sampling and Other

Means of Testing. 2009.[26] IFAC. International Standards on Auditing 240, The Auditor’s Responsibilities

Relating to Fraud in an Audit of Financial Statements. 2009.[27] A. K. Islam, M. Corney, G. Mohay, A. Clark, S. Bracher, T. Raub, and U. Flegel.

Fraud detection in ERP systems using Scenario matching. IFIP Advances inInformation and Communication Technology, 330:112–123, 2010.

[28] A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon. A survey oncontrastive self-supervised learning. Technologies, 9(1):2, 2021.

[29] M. Jans, M. G. Alles, and M. A. Vasarhelyi. A Field Study on the Use of ProcessMining of Event Logs as an Analytical Procedure in Auditing. The AccountingReview, 89(5):1751–1773, 2014.

[30] M. Jans, N. Lybaert, and K. Vanhoof. Internal Fraud Risk Reduction: Results of aData Mining Case Study. International Journal of Accounting Information Systems,11(1):17–41, 2010.

[31] M. Jans, J. M. Van Der Werf, N. Lybaert, and K. Vanhoof. A Business ProcessMining Application for Internal Transaction Fraud Mitigation. Expert Systemswith Applications, 38(10):13351–13359, 2011.

[32] L. Jing and Y. Tian. Self-supervised visual feature learning with deep neuralnetworks: A survey. IEEE Pattern Analysis and Machine Intelligence, 2020.

[33] R. Khan, A. Clark, G. Mohay, and S. Suriadi. Detecting Fraud Using TransactionFrequency Data. Information Technology in Industry, 2(3):79–88, 2014.

[34] R. Khan, M. Corney, A. Clark, and G. Mohay. A role mining inspired approachto representing user behaviour in erp systems. In Proceedings of Asia PacificIndustrial Engineering & Management Systems Conference 2009, pages 2541–2552.APIEMS Society, 2009.

[35] R. Khan, M. Corney, A. Clark, and G. Mohay. Transaction Mining for FraudDetection in ERP Systems. Industrial Engineering and Management Systems,9(2):141–156, 2010.

[36] E. Kharitonov, M. Rivière, G. Synnaeve, L. Wolf, P.-E. Mazaré, M. Douze, andE. Dupoux. Data augmenting contrastive learning of speech representationsin the time domain. In 2021 IEEE Spoken Language Technology Workshop (SLT),pages 215–222. IEEE, 2021.

[37] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXivpreprint arXiv:1412.6980, 2014.

[38] J. Kokina and T. H. Davenport. The emergence of artificial intelligence: Howautomation is changing auditing. Journal of emerging technologies in accounting,14(1):115–122, 2017.

[39] Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. nature, 521(7553):436, 2015.[40] M. McGlohon, S. Bay, M. G. M. Anderle, D. M. Steier, and C. Faloutsos. SNARE:

A Link Analytic System for Graph Labeling and Risk Detection. In KDD’09:15th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages1265–1273, 2009.

[41] I. Misra and L. v. d. Maaten. Self-supervised learning of pretext-invariant repre-sentations. In Proceedings of the IEEE/CVF Conference on Computer Vision andPattern Recognition, pages 6707–6717, 2020.

[42] J. Nonnenmacher and J. M. Gómez. Unsupervised anomaly detection for internalauditing: Literature review and research agenda. The International Journal ofDigital Accounting Research, 21(27):1–22, 2021.

[43] J. Nonnenmacher, F. Kruse, G. Schumann, and J. Marx Gómez. Using autoencodersfor data-driven analysis in internal auditing. In Proceedings of the 54th HawaiiInternational Conference on System Sciences, page 5748, 2021.

[44] M. Noroozi and P. Favaro. Unsupervised learning of visual representations bysolving jigsaw puzzles. In European Conference on Computer Vision (ECCV), pages69–84. Springer, 2016.

[45] A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastivepredictive coding. arXiv preprint arXiv:1807.03748, 2018.

[46] D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V.Le. Specaugment: A simple data augmentation method for automatic speechrecognition. arXiv preprint arXiv:1904.08779, 2019.

[47] J. Robinson, C.-Y. Chuang, S. Sra, and S. Jegelka. Contrastive learning with hardnegative samples. arXiv preprint arXiv:2010.04592, 2020.

[48] M. Schreyer, T. Sattarov, D. Borth, A. Dengel, and B. Reimer. Detection of anom-alies in large scale accounting data using deep autoencoder networks. arXivpreprint arXiv:1709.05254, 2017.

[49] M. Schreyer, T. Sattarov, A. S. Gierbl, B. Reimer, and D. Borth. Learning samplingin financial statement audits using vector quantised variational autoencoderneural networks. In Proceedings of the International Conference on ArtificialIntelligence (ICAIF)’20. Association of Computing Machinery (ACM), 2020.

[50] M. Schreyer, T. Sattarov, C. Schulze, B. Reimer, and D. Borth. Detection ofaccounting anomalies in the latent space using adversarial autoencoder neuralnetworks. 2nd KDD Workshop on Anomaly Detection in Finance, Anchorage,Alaska, USA, 2019.

[51] F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for facerecognition and clustering. In Proceedings of the IEEE conference on computervision and pattern recognition, pages 815–823, 2015.

[52] M. Schultz andM. Tropmann-Frick. Autoencoder neural networks versus externalauditors: Detecting unusual journal entries in financial statement audits. InProceedings of the 53rd Hawaii International Conference on System Sciences, 2020.

[53] P. S. SEOW, G. Pan, and T. Suwardy. Data Mining Journal Entries for FraudDetection: A Replication of Debreceny and Gray’s (2010) Techniques. Journal ofForensic and Investigative Accounting, 8(3):501, 2016.

[54] T. Sun. Applying deep learning to audit procedures: An illustrative framework.Accounting Horizons, 33(3):89–109, 2019.

[55] S. Thiprungsri and M. A. Vasarhelyi. Cluster Analysis for Anomaly Detection inAccounting Data: An Audit Approach. International Journal of Digital AccountingResearch, 11, 2011.

[56] Y. Tian, D. Krishnan, and P. Isola. Contrastive multiview coding. In ComputerVision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020,Proceedings, Part XI 16, pages 776–794. Springer, 2020.

[57] Y. Tian, C. Sun, B. Poole, D. Krishnan, C. Schmid, and P. Isola. What makes forgood views for contrastive learning? arXiv preprint arXiv:2005.10243, 2020.

[58] A. van den Oord, O. Vinyals, et al. Neural discrete representation learning. InAdvances in Neural Information Processing Systems, pages 6306–6315, 2017.

[59] S. Wang and D. Borth. Noisevc: Towards high quality zero-shot voice conversion.arXiv preprint arXiv:2104.06074, 2021.

[60] X. Wang and A. Gupta. Unsupervised learning of visual representations usingvideos. In Proceedings of the IEEE International Conference on Computer Vision(ICCV), pages 2794–2802, 2015.

[61] K. Q. Weinberger and L. K. Saul. Distance metric learning for large marginnearest neighbor classification. Journal of machine learning research, 10(2), 2009.

[62] M. Werner and N. Gehrke. Multilevel Process Mining for Financial Audits. IEEETransactions on Services Computing, 8(6):820–832, 2015.

[63] Z. Wu, Y. Xiong, S. X. Yu, and D. Lin. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition, pages 3733–3742, 2018.

[64] B. Xu, N. Wang, T. Chen, and M. Li. Empirical evaluation of rectified activationsin convolutional network. arXiv preprint arXiv:1505.00853, 2015.

[65] M. Ye, X. Zhang, P. C. Yuen, and S.-F. Chang. Unsupervised embedding learningvia invariant and spreading instance feature. In Proceedings of the IEEE/CVFConference on Computer Vision and Pattern Recognition, pages 6210–6219, 2019.

[66] M. Zupan, V. Budimir, S. Letinic, et al. Journal entry anomaly detection model.Intelligent Systems in Accounting, Finance and Management, 27(4):197–209, 2020.

multi-view contrastive self-supervised learning of

Documents