fingers to frets - a mathematical approachvibeserver.net/scripties/2017/fingers to frets - master...

FACULTY OF ECONOMICS AND BUSINESS

Fingers to frets - A Mathematical ApproachOptimizing Finger Assignment for Guitar Tablature

Bernd Tahonr0377485

Thesis submitted to obtainthe degree of

Master in de toegepaste economische wetenschappen:Handelsingenieur

Major Data science en business analytics

Promotor: Prof. Dr. Frits SpieksmaAssistant: Bart Vangerven

Academic year: 2016-2017

Contents

Preface v

1 Introduction 1

2 Literature review 5

3 Problem description 93.1 Model 1: Scale-based guitar music . . . . . . . . . . . . . . . . . . . . . . 10

3.1.1 Mathematical Model . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2 Model 2: Chord-based guitar music . . . . . . . . . . . . . . . . . . . . . . 17

3.2.1 Mathematical Model . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4 Implementation and Results 23

5 Conclusion 29

Appendix 33A.1 Python Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

iii

Preface

To hell with the rules. If it sounds right, then it is. - Eddie Van Halen

As Eddie Van Halen, considered one of the world’s best guitarists, points out: rules,especially in the art of music, have their limitations. In this thesis, we try to model theleft-hand finger assignment of a right-handed guitarist through the use of certain rules.Yet, just as Van Halen points out, sometimes the rules don’t apply. Sometimes, for somenotes, an experienced guitar player would opt for another finger assignment than theone the model returns. Sometimes, what may look wrong can sound right.

This does not mean that the models are not useful - it rather reinforces what this thesisis meant to be: first off, a tool for beginners, who are about to embark on the wonderfuljourney of becoming a guitar player; secondly, a building block, a piece of work thatcan be built on, that can be improved and that can be incorporated in more complexmodels with different aims; lastly, and maybe most importantly, a thought process, thatgives insight into decisions that experienced guitarists make unconsciously and why theymake them. This insight can make you a better guitarist.

Personally, writing this thesis has taught me many things. Not only have I honed myresearch, mathematical and programming skills, and have I gained the insight describedabove, most of all I was amazed by the power of the human mind. I was surprisedto learn that something that is relatively easy and happens unconsciously for a semi-experienced guitarist like myself, can be so challenging to write down and ‘explain’ to acomputer. What sometimes seems trivial for the trained human mind is often a rathercomplex process.

Throughout my writing of this thesis, many people have supported me. First of all,I would like to thank my promotor, prof. dr. Frits Spieksma, and his assistant, BartVangerven, who were of great help in showing me the world of linear optimization andin decomposing the thought process of guitar finger assignment into mathematical equa-tions. Secondly, I would like to thank my parents and friends for their unconditionalsupport and help over the years that brought me to this point in life, finishing the chap-ter of my studies. Thirdly, I would like to thank you, the reader, for taking the time forgoing through this work. I wish you an interesting journey, hopefully not to hell, with

v

the rules for optimal finger assignment that are developed throughout this thesis. Let’shope it sounds right.

Leuven, 19/05/2017.

Chapter 1

Introduction

Guitar tablature1 allows musicians to play musical pieces without having to read stan-dard musical (“staff”) notation. Tablature consists of six parallel lines representing aguitar’s six strings2 with the bottom string (i.e. the thinnest one which produces a highe) at the top. Numbers on these lines indicate the frets that should be pressed downon the respective strings, chronologically ordered from left to right. As a consequence,notes that need to be played simultaneously are situated on one vertical line. In theremainder of this thesis, we will call each moment in time at which a note or multiplenotes should be played a ‘timestamp’.

Figure 1.1 shows an example: the first two bars of the intro of Eric Clapton’s acousticversion of ‘Layla’. During the first timestamp, the A-string should be plucked withoutpressing down a fret (this is called an ‘open string’), followed by the third fret on thesame A-string. Subsequently, during the third timestamp, the second fret on the G-string is played simultaneously with an open D-string, and so on.

Figure 1.1: The tablature for the first two bars of the intro of the acoustic version ofLayla by Eric Clapton [1]

1We use ‘tablature’ to indicate guitar tablature in the remainder of this thesis2A standard guitar has six strings, that produce, in standard tuning, the following notes from top to

bottom: low E, A, D, G, B, high e

1

2 CHAPTER 1. INTRODUCTION

Consequently, tablature tells a guitarist which strings and frets to play at each point intime. It does not, however, give the finger assignment, i.e. which left-hand3 finger shouldbe used to press down a string and fret. For more advanced guitarists, this finger assign-ment is an acquired skill and happens unconsciously for the majority of guitar pieces.On the other hand, beginning guitarists often struggle with this, adding another layer ofcomplexity to their learning. To cope with this, a guitar teacher would typically writethe fingers to be used at each point in time under the tablature. Nowadays, however,more and more people try to teach themselves how to play the guitar using the internet.One can find tablature for virtually every song on the world wide web, but they seldominclude the finger assignment.

Therefore, a tool that calculates and displays the optimal finger assignment for a givenpiece of tablature, could offer significant gains in guitarists’ learning curves and takeaway motivational hurdles that beginners often face. A good tool would also take intoaccount the personal preferences and the physical characteristics of the player. More-over, a functioning model could be implemented in models for automatic transcriptionof audio signals or for translating staff notation into tablature4.

In this thesis, we make use of the software package Guitar Pro 7 to generate tabla-tures. For the majority of popular songs, free Guitar Pro tablature files can be foundon the internet. In addition, Guitar Pro is able to parse the common ASCII-format fortablature. Guitar Pro enables the user to manually write the left-hand finger assignmentbelow the tablature with a number: number 1 refers to the index finger, 2 to the middlefinger, 3 to the ring finger and 4 to the pinky finger. In the remainder of this thesis, wewill use that convention. Two examples of different finger assignments are depicted inFigures 1.2 and 1.3 .

Figure 1.2: A good finger assignment for the first two bars of the intro of the acousticversion of Layla by Eric Clapton

In order to illustrate the importance of a good finger assignment, consider those two

3We assume, without loss of generality, a right-handed player4These concepts are further explained in Chapter 2

3

Figure 1.3: A bad finger assignment for the first two bars of the intro of the acousticversion of Layla by Eric Clapton

examples. In the first case (Figure 1.2), the ring finger plays the third fret during thesecond timestamp, whereas the middle finger plays the following note on the second fret.Until the last depicted timestamp, the left-hand wrist does not move and all fingers arekept in position on the same fret. At the seventh timestamp, the index finger plays thefirst fret and the ring finger plays the third fret simultaneously, keeping the distancebetween fingers comfortable.

Consider now the bottom case (Figure 1.3), where the wrist is required to move around:the index finger jumps around from the third fret to the second, back to the third andfinally, at the seventh timestamp, to the first fret. In addition, between the second andthird timestamp, it needs to move down two strings, complicating the ease as well as thefluency of playing. Lastly, playing the frets at the seventh timestamp with the index andmiddle finger requires the player to stretch his fingers and makes it hard to play this.In conclusion, the upper finger assignment would be preferred by most guitarists as itimplies no wrist movement, limited movement across strings and comfortable inter-fingerdistances.

While there exist good and bad finger assignments, as demonstrated in the exampleabove, there is no such thing as THE optimal finger assignment for a given song. Con-sider the example in Figure 1.4 on page 4. A guitarist with large fingers could opt for thefirst finger assignment, where the middle finger stays on the A-string, the pinky fingeron the B-string and the index finger on the e-string. He can play all the notes by justsliding over these strings, minimizing the effort required. Someone with small hands andfingers, however, will have trouble stretching the middle finger and the pinky finger farenough from each other to cover the distance required between the third and the fifthfret. Consequently, this guitarist could choose for the second finger assignment, whichrequires the index finger to jump around from the A-string to the e-string and back, butallows the relatively wide spread between the third and the fifth fret to be played with

4 CHAPTER 1. INTRODUCTION

the index and the pinky finger. This example illustrates the need for personalization,allowing user-specific inputs.

In summary, the aim of this thesis is to develop a user-specific model that gives goodfinger assignments for any given guitar tablature. The remainder of this thesis is struc-tured as follows. In chapter 2, we discuss the existing literature on similar problemsfor both guitar and other musical instruments. In chapter 3 we give a precise problemdescription. Chapter 4 discusses the implementation and its results. Chapter 5 gives themain conclusions and possible steps for subsequent research.

Finger assignment 1: for a guitarist with large fingers1

Finger assignment 2: for a guitarist with small fingers2

Figure 1.4: The importance of user-specific inputs: different finger assignments for dif-ferent people. (From “The Beatles - Blackbird”)

Chapter 2

Literature review

Concerning guitar pieces, little has been written on finger assignment. Most work focuseson automatic transcription of audio signals or, more specifically, the determination ofoptimal fret-string combinations for a given piece of music. This is due to the fact thatseveral fret-string combinations (so called ‘fingering’1), are possible for a given guitarpiece since the same note can be played on multiple combinations of frets and strings: agiven note can have up to six different positions on the fretboard [2]. Despite the differ-ent and wider focus, this work is still useful for this paper, as they attempt to model andminimize the difficulty of certain finger movements and make an implicit assumption offinger assignment in their model. We will now discuss the most relevant literature onthis fingering decision.

Radisavljevic and Driesen [3] make use of dynamic programming with cost functionsto map music score into playable fret-string combinations. Each set of simultaneouslyplayed notes is broken down into a set of corresponding fingering alternatives, whichthey refer to as different ‘states’. These states are represented by matrices with one row(string, fret, finger) for each note. Subsequently, they split their cost function in a staticcost and a transition cost. The static cost function models the cost of one single state, orthus the cost of playing one chord. The transition cost function models the cost of tran-sitioning from one state to the following state. To avoid assigning a cost to each singlestate or transition and in this way reduce computational memory requirements, they usefeatures to uncover desired states. Static cost is based on features such as “number offrets between consecutive fingers”. Transition cost, in turn, is based on features such as“numbers of frets traversed by a specific finger” or “finger changes from used to unused”.

Additionally, they developed a technique called ‘path difference learning’ to automati-cally select cost function weights by using a training set of published tablatures. In thisway, they avoid lengthy interviews with guitarists and manual adjustment of cost func-

1‘Fingering’ or ‘fingering decision’ needs to be discerned from ‘finger assignment’. In this thesis, theformer indicates the choice of fret and string to play a note, while the latter refers to the choice ofleft-hand finger to play this note with.

5

6 CHAPTER 2. LITERATURE REVIEW

tions leading to rather subjective weightings. In their experiments, they shorten songsby removing repeated sections such as a returning chorus. Radisavljevic and Driesenremark that a complete convergence between the model and the published tablaturescan never be reached, due to personal preferences of guitarists, the features of the costfunction not being exhaustive and the limited flexibility of the linear cost functions.

Yazawa, Itoyama and Okuno [4] developed a dynamic programming method for au-tomatic transcription of tablature from audio signals, taking into account the player’sskill. In this method, acoustical reproducibility (i.e. how similar are the pitches andtimbres to the original piece) and fingering easiness are balanced and can be attributeddifferent weights. For example, beginners will generally favor fingering easiness andthus allocate it a higher weight. This fingering easiness is broken down into the cost ofplaying a fingering configuration and the cost of changing fingering configuration. Theformer is determined by finger spread, number of fingers used, the fret position of theindex finger and whether or not it is a barre chord2. The cost of changing fingeringconfiguration is, in turn, broken down into the distance the player’s wrist travels duringa transition (i.e. the distance the index finger travels) and the Manhattan distances ofthe fingers thereafter. Each of these cost drivers can again be assigned different weightsbased on a player’s preferences. The model is solved by finding the longest path in aweighted acyclic graph where the vertexes represent potential fingering configurations.These configurations are enumerated through a standard guitar chord book.

Miura and Yanagida [5] created S2T (Score to tablature) software to automaticallytranslate a musical score to a playable tablature. Again, the cost of finger position pat-terns is split up in the weighted sum of left hand movements and finger movements. Inthis system, they automatically assign a placing finger to each note as follows: if thefret span is lower than four, each of the fingers is assigned to the corresponding fret,index to little finger from left to right. If the fret span is higher (i.e. five or six), theindex or little finger should be stretched to reach one fret further. After allocating thefingers, the appropriate position is obtained as the one which minimizes the total sumof left hand movements. Their software outperforms current commercial software but is,in turn, still outperformed by a trained guitarist. The S2T software is the only tool inthis literature review that explicitly outputs left-handed finger assignment. It does thiswith a number below the notes on the tablature: 1 for the index finger, 2 for the middlefinger, 3 for the ring finger and 4 for the little finger.

Hori, Kameoka and Sagayama [6] make use of an Input-Output Hidden Markov Model tofind the most probable sequence of forms (i.e. string-fret combinations) given a certainmelody. They assume that beginner guitarists want to minimize left hand movementalong the neck of the guitar while avoiding difficult left hand forms. The probabilityof a form being selected as the next form in a sequence depends on the previous form,

2When a guitarist uses one finger to simultaneously press down multiple strings in one fret, it is calleda barre chord.

7

characterized by the distance the index finger travels, and on the time interval betweenthe two. Moreover, the difficulty of the form is characterized by the index finger po-sition, the width (i.e. the maximum distance between two fingers) and the number ofactive fingers (i.e. fingers that are being pressed down on a string). ‘Idling fingers’ (i.e.non-active fingers) are assumed to be on consecutive frets with active fingers. Theycompare their model to existing commercial tools Power Tab Editor and Guitar Pro,and conclude with a clearly better performance, but they do note that future studiesshould include the possibility to tailor the model to personal preferences.

Tuohy and Potter [2] developed a genetic algorithm (“GA”) to map sequences of musicnotes in staff notation to positions on the fretboard for tablature. Their fitness functionconsists out of two main components: hand movement and hand manipulation. Handmovement means that the more the hand and fingers move across the strings and fret-board, the lower the fitness of this fingering will be. Hand manipulation, on the otherhand, assesses whether a chord is an open chord (i.e. each finger presses down maximallyone string) or a barre chord, but also how long a position can be held without movingthe hand.Tuohy and Potter compare their GA with commercial software such as Guitar Pro 4,which tends to underperform their GA since it creates tablature note by note withoutlooking ahead, while a GA looks at the piece as a whole and tries to improve the wholepiece simultaneously.

Lastly, we consider the finger assignment for another instrument: Balliauw [7] presentsthe problem of determining the optimal finger assignment for a piano piece as a com-binatorial optimization problem, picking a finger for every note that has to be played.The objective is to minimize a cost function based on rules that penalize certain fingercombinations and sequences. Rules include, for example, “add 2 if the piano key distancebetween two consecutive notes exceeds the ‘comfortable distance’ (i.e. a parameter) be-tween the finger pair concerned” or “add 1 if the pinky finger is used”. To model thepenalties for distances between fingers, Balliauw makes use of three types of parametermatrices that represent minimum and maximum distances between finger pairs: firstly,relaxed distances which can be easily played; secondly, practical distances which are theabsolute minimum and maximum inter-finger distances that are possible to play; andthirdly, comfortable distances which can be played without hamper. These distance ma-trices are user-specific and can thus be tailored to a single user. As Balliauw points out,the use of rules allows to identify multiple origins of difficulties resulting from a specificfinger assignment, assign weights to them and allow trade-offs. He does not, however,give an explicit integer programming formulation in the paper.

Balliauw introduces a variable neighborhood search (“VNS”) algorithm to solve theproblem in a reasonable amount of time. While the performance of this VNS algorithmis satisfying, Balliauw suggests two potential improvements with regards to the algo-rithm and the objective function. The algorithm could, for example, be improved by

8 CHAPTER 2. LITERATURE REVIEW

reducing the calculation time through splitting the music piece in smaller parts (e.g. ata rest). Improving the objective function, on the other hand, could be done throughparameter training.

Compared to Balliauw’s model for piano, certain complicating factors arise in the caseof the guitar. First, in contrast with one-dimensional piano keys, guitars have bothmultiple strings and multiple frets, making it a two-dimensional exercise. Second, fretsget smaller when one moves up the neck of the guitar, making it easier to stretch fingersacross multiple frets the more you move up the fretboard. Finally, while typically onefinger can play at most one key on the piano, the use of barre chords allows guitar playersto press down multiple strings across one fret.

Chapter 3

Problem description

The aim of this thesis is to develop a model which outputs ‘good’ finger assignments.A good finger assignment is one that would be picked by trained guitarists, and can bepersonalized for different users.

We introduce two concepts to differentiate between two types of guitar pieces: scale-based and chord-based guitar music. Scale-based1 music includes mainly lead gui-tar pieces such as solos and electric guitar intros, that consist of mostly single notes(monophony) or notes on adjoining strings that can be strummed with a pick ratherthan plucked with one’s right-hand fingers. Chord-based music, on the other hand, in-cludes mainly guitar pieces that contain various notes that are played simultaneously onnon-adjoining strings and are often plucked with one’s right-hand fingers (‘fingerpick-ing’). In this kind of guitar pieces, certain finger positions (‘chords’) are held for a longerperiod of time while the different strings are being plucked. Note that the distinctionbetween scale-based and chord-based pieces is not straightforward, and one guitar piececan include both scale-based and chord-based parts.

We will build two slightly different integer programming models that accommodate forthe different needs of these types of guitar music. Both models consist of minimizing acost function that penalizes different components of difficult finger assignments. Theircomponents are the same, but they differ in the way some of these components are buildup.

For both models, two simplifying assumptions are used:

• One finger plays at most one note (i.e. one string and one fret): no (partial) barrechords.

• The thumb cannot be used to press down a string

1Guitar scales are standard patterns of notes that sound well together and that can be used to writemusic or to improvise

9

10 CHAPTER 3. PROBLEM DESCRIPTION

3.1 Model 1: Scale-based guitar music

The scoring of a specific finger assignment is based on both static and transitionalcomponents [3, 4]. The static component measures the difficulty of a finger assignmentat one timestamp and consists of inter-finger distances [3, 4, 5, 6, 7] and the use ofthe pinky finger [7]. The transitional component, on the other hand, measures thedifficulty of transitioning from one timestamp to the next, given the finger assignment,and consists of the wrist movement (i.e. a horizontal component) [2, 4, 5, 6] and themovement across the strings (i.e. a vertical component) [2, 4]. Figure 3.1 summarizesthe different components of the objective function. Each of these components is discussedin detail below.

Figure 3.1: The different components of the objective function.

Inter-Finger Distances This component of the objective function is used to avoidimpossible finger combinations and to penalize uncomfortable ones. This includes havingto stretch fingers too far, or squeezing them together too much. Similar to Balliauw [7],we make use of parameter sets of comfortable and absolute distances between eachfinger pair, uniquely defined for each user. Distances that fall outside of the minimaland maximal comfortable distances are penalized while distances that fall outside theabsolute bounds are excluded through a high cost.

Use of the pinky finger Beginning guitarists often struggle with using their pinkyfinger. The user can personalize the weight of this component in the objective functionto obtain a finger assignment of their liking. Some beginning guitarists prefer limitingthe pinky finger’s use to be able to play a certain guitar piece more fluently, while othersmight want to train their pinky finger to become a better guitarist and thus not penalizeit (heavily).

Wrist movement Since fingers are interconnected and moving one finger over thefretboard implies moving the others too, the movement of the wrist is the most impor-tant horizontal determinant of the ease of playing. In order to play a guitar piece fluentlyand easily, one should aim to keep their hand as much as possible in the same position.

3.1. MODEL 1: SCALE-BASED GUITAR MUSIC 11

The wrist position is usually determined through the fret position of the index finger.When, however, the index finger is not used at a certain point in time (i.e. not pressingdown a string), an assumption needs to be made. In this model for scale-based guitarpieces, we will use the hypothesis that non-active fingers are on consecutive frets, called“in-place performance” [5, 6]. For example, if the middle finger on the sixth fret is theonly finger being used at one point in time, the index finger and thus the wrist positionis assumed to be one fret lower, i.e. on the fifth fret.

Movement across strings A good finger assignment is one that limits the move-ment of fingers. While the most important horizontal movements are covered in thewrist movement component, this second transitional component penalizes (large) verti-cal movements across strings. It is easier to slide one’s finger across the fretboard than tolift it to move across strings. Moreover, this component can be tuned to reward stayingon the same string with one finger: evidently, when the same string and fret needs tobe played two times in a row, this should be done with the same finger - but also whenthe fret changes, it is still easier to slide one’s finger to this fret.


3.1.1 Mathematical Model

A piece of music consists of a sequence of notes to be played at a specific moment intime. To that end, we define T as the set of timestamps. It follows that we have to makedecisions at every moment t ∈ T : what finger(s) do we use to play the required notes?Note that there is one exception: no finger is required to play open strings. A guitarusually has six strings, which are contained in the set S = {E,A,D,G,B, e}2. The setof frets is denoted by G. The number of frets is fixed at n and equal for every strings ∈ S. In order to identify each fret-string combination (i.e. location on the fretboard)uniquely, we give all frets a unique number, as depicted in Figure 3.2 for a guitar with12 frets. In this example for a guitar with n = 12 frets, the first fret on the A-string

61

49

37

25

1

13

2 3 4 5 6 7 8 9 10 11 12

14 15 16 17 18 19 20 21 22 23 24

26 27 28 29 30 31 32 33 34 35 36

38 39 40 41 42 43 44 45 46 47 48

14 15 16 17 18 19 20 21 22 23 2450 51 52 53 54 55 56 57 58 59 60

62 63 64 65 66 67 68 69 70 71 72

1 2 3 4 5 6 7 8 9 10 11 12Frets:

e

B

G

D

A

E

Strings:

Figure 3.2: From strings and frets to identifiers: position numbering starts on the thickE string and continues on to the A, D, G, B, and e strings respectively.

would have number, or “identifier”, 13. We denote the set of these position identifiersby P = S × G. Note that we do not include open strings in this set. The reason forthis is simple: no finger is needed to play an open string. The set of fingers, availableto play the guitar, is F = {1, 2, 3, 4}3, where the first finger corresponds to the indexfinger, the second finger to the middle finger, the third finger to the ring finger, and thefourth finger to the pinky finger. The different sets are summarized below:

• T is the set of timestamps, i.e. the set of all points in time during which a note isplayed.

2We assume, without loss of generality, the guitar to be in the standard tuning. Moreover, this setcan be customized for, for example, 12-string guitars or 4-string bass guitars.

3If necessary, the set F can be adapted for people with injured fingers


• S = {E,A,D,G,B, e} is the set of strings.

• F = {1, 2, 3, 4} is the set of fingers available to play the guitar.

• G = {1, .., n} is the set of frets for a guitar with n frets.

• G′ = {−2, .., n} is the set of ‘extended frets’, as the wrist position can be locatedleft of the first fret based on the consecutive fret assumption.

• P = S × G is the set of unique fretboard position identifiers.

• Pt denotes the set of fretboard positions to be played at timestamp t, based on thetablature of the concerned song.

• Th denotes the set of (t, p)-vectors that give the timestamp t ∈ T when, and posi-tion p ∈ Pt where, a hammer-on4 starts, based on the tablature of the concernedsong.

• Tp denotes the set of (t, p)-vectors that give the timestamp t ∈ T when, andposition p ∈ Pt where, a pull-off5 starts, based on the tablature of the concernedsong.

• Ts denotes the set of (t, s)-vectors that give the timestamp t ∈ T when, and strings ∈ S on which, a slide6 starts, based on the tablature of the concerned song.

Decision variables

• Ft, p, f is a binary variable which is one iff at timestamp t ∈ T , position p ∈ Pt ispressed by finger f ∈ F .

• St, s, f is a binary variable which is one iff at timestamp t ∈ T , string s ∈ S ispressed by finger f ∈ F .

• Gt, g, f is a binary variable which is one iff at timestamp t ∈ T , fret g ∈ G is pressedby finger f ∈ F .

• ∆g, g′, f, f ′, t is a binary variable which is one iff distinct fingers f and f ′ ∈ F areboth used on respectively frets g and g′ ∈ G at time t ∈ T .

4A hammer-on is a special guitar playing technique where a left-hand finger is ‘hammered’ on thefretboard, causing the note of that string and fret to sound without the string being plucked with theright hand. A hammer-on is denoted on tablature by an arc with the letter ‘H’ above

5A pull-off is a special guitar playing technique where a left-hand finger is pulled off a position onthe fretboard, causing a lower fret that is being pressed down with another finger to sound without thestring being plucked with the right hand. A pull-off is denoted on tablature by an arc with the letter‘P’ above

6A slide is a special guitar playing technique where a string is plucked by a right-handed finger andsubsequently the left-hand finger on this string slides (up- or downwards) to a different fret. A slide istypically denoted on tablature by a slash (‘/‘) symbol for upward slides and a backslash (‘\‘) symbol fordownward slides


• µs, s′, f, t is a binary variable which is one iff finger f ∈ F moves from string s tostring s′ ∈ S from timestamp t− 1 to t ∈ T \ {1}.

• wg, t is a binary variable which is one iff at timestamp t ∈ T the wrist position isat fret g ∈ G′.

• Wg, g′, t is a binary variable which is one iff the wrist moves from fret g to g′ ∈ G′from timestamp t− 1 to t ∈ T \ {1}.

Constants

• Ip, s is a binary constant which is one iff fretboard position p ∈ P is located onstring s ∈ S.

• Ip, g is a binary constant which is one iff fretboard position p ∈ P is located on fretg ∈ G.

Parameters

• ag, g′, f, f ′ is the cost of using both fingers f and f ′ ∈ F simultaneously on respec-tively frets g and g′ ∈ G. Three instances exist: (i) the distance between fingersf and f ′ is comfortable: no cost is imposed; (ii) the distance between fingers fand f ′ is uncomfortable but playable: a cost is imposed; (iii) the distance betweenfingers f and f ′ is impossible to play: a high cost is imposed to prevent the modelfrom selecting this finger assignment.

• b is the cost of using the pinky finger. This cost is independent of time, fret orstring.

• cg, g′ is the cost of moving the wrist from position g to position g′ ∈ G. Shortermovements should have a lower cost, longer movements should have a higher cost.

• ds, s′ is the cost of moving a finger from string s to s′ ∈ S.


Integer programming model

min∑

g, g′∈Gf,f ′∈F :f 6=f ′

t∈T

ag, g′, f, f ′∆g, g′, f, f ′, t + b∑t∈Tp∈Pt

Ft,p,4 +∑

g, g′∈G′

t∈T \{1}

cg, g′Wg, g′, t +∑

s, s′∈Sf∈F

t∈T \{1}

ds, s′µs, s′, f, t

(3.1)∑f∈F

Ft,p,f = 1 ∀t ∈ T , p ∈ Pt (3.2)

∑p∈Pt

Ft,p,f ≤ 1 ∀f ∈ F , t ∈ T (3.3)

St,s,f =∑p∈Pt

Ip, sFt,p,f ∀f ∈ F , t ∈ T , s ∈ S (3.4)

Gt,g,f =∑p∈Pt

Ip, gFt,p,f ∀f ∈ F , t ∈ T , g ∈ G (3.5)

Gt,g,f +Gt,g′,f ′ − 1 ≤ ∆g, g′, f, f ′, t ∀t ∈ T , g, g′ ∈ G, f, f ′ ∈ F : f 6= f ′ (3.6)

St−1,s,f + St,s′,f − 1 ≤ µs, s′, f, t ∀s, s′ ∈ S, f ∈ F , t ∈ T \ {1} (3.7)

µs, s′, f, t ≤ St−1,s,f ∀s, s′ ∈ S, f ∈ F , t ∈ T \ {1} (3.8)

µs, s′, f, t ≤ St,s′,f ∀s, s′ ∈ S, f ∈ F , t ∈ T \ {1} (3.9)∑g∈G′

wg,t = 1 ∀t ∈ T (3.10)

wg,t ≥ Gt,g,1 ∀t ∈ T , g ∈ G (3.11)

wg−1,t +∑g′∈G

Gt,g′,1 ≥ Gt,g,2 ∀t ∈ T , g ∈ G (3.12)

wg−2,t +∑

g′∈G,f∈{1,2}

Gt,g′,f ≥ Gt,g,3 ∀t ∈ T , g ∈ G (3.13)

wg−3,t +∑

g′∈G,f∈{1,2,3}

Gt,g′,f ≥ Gt,g,4 ∀t ∈ T , g ∈ G (3.14)

wg,t−1 + wg′,t − 1 ≤Wg,g′,t ∀g, g′ ∈ G′,∀t ∈ T \ {1} (3.15)∑f∈F

f × Ft,p,f ≤∑f∈F

f × Ft,p′,f ∀t ∈ T , p, p′ ∈ P, k ∈ {1, .., 5} : p′ = p+ nk (3.16)

Ft,p,f + Ft+1,q,f ≤ 1 ∀(t, p) ∈ Th ∪ Tp, q ∈ Pt+1, f ∈ F (3.17)∑f∈F


f × Ft+1,q,f ∀(t, p) ∈ Th, q ∈ Pt+1 (3.18)

∑f∈F

f × Ft,p,f ≥∑f∈F

f × Ft+1,q,f ∀(t, p) ∈ Tp, q ∈ Pt+1 (3.19)

St,s,f = St+1,s,f ∀(t, s) ∈ Ts, f ∈ F (3.20)

Ft,p,f , St,s,f , Gt,g,f ∈ {0, 1} ∀t ∈ T , p ∈ Pt, f ∈ F , s ∈ S, g ∈ G (3.21)

wg,t ∈ {0, 1} ∀g, g′ ∈ G′, t ∈ T (3.22)

Wg,g′,t, µs, s′, f, t ∈ {0, 1} ∀g, g′ ∈ G′, s, s′ ∈ S, f ∈ F , t ∈ T \ {1} (3.23)

∆g, g′, f, f ′, t ∈ R≥0 ∀g, g′ ∈ G′, f, f ′ ∈ F : f 6= f ′, t ∈ T (3.24)


The objective function, Equation 3.1, minimizes the cost due to difficult inter-fingerdistances, the use of the pinky finger, wrist movement and movement across strings.Constraints 3.2 make sure that all positions p ∈ Pt are actually played, i.e. are assigneda finger, while constraints 3.3 ensure that at all timestamps, one finger can only playone position. Constraints 3.4 and 3.5 translate the position identifiers to respectively thestrings and the frets that are being played by a specific finger. Constraints 3.6 activatethe inter-finger distance variables when appropriate: if two fingers are used in the sametimestamp. Note that this is done for all possible pairs of fingers

(42

)= 6. Also note that

the ∆ variables, while not restricted to be binary, will only take values 0 or 1 due to theminimization of the objective function and the nature of these constraints. Similarly,constraints 3.7 activate the inter-string distance variables µ when a finger moves fromone string to the other. Constraints 3.8 and 3.9 are necessary in case the parametersinclude a profit (i.e. a negative cost) - for example to reward fingers for staying on thesame string - since these constraints prevent the µ variables from taking value 1 unlessthe concerned finger effectively travels from string s to string s′.

Constraints 3.10 ensure that a wrist position is determined at all times. Constraints 3.11-3.14 take care of setting this wrist position as follows: if the index finger is active , thenconstraints 3.11 set the wrist position to the index position. In this case, the summationof the G variables in constraints 3.12-3.14 makes these constraints non-binding for the wvariables. If, however, the index finger is not active at timestamp t, constraints 3.12-3.14implement the assumption that non-active fingers are on consecutive frets with activefingers and thus determine the position of the index finger and by extension the wristposition: if the middle finger is active, constraints 3.12 set the wrist position one fretlower than the fret the middle finger is on. If the middle finger is not active, we look atthe ring finger using constraints 3.13 and finally at the pinky finger with constraints 3.14.Constraints 3.15 then take care of the wrist movement.

Constraints 3.16 eliminate awkward hand positions by forbidding a finger to be on aphysically lower (thinner) string in the same fret as a higher numbered finger. Con-straints 3.17-3.20, in turn, take care of the special techniques as follows: first, con-straints 3.17 prevent the finger that starts a hammer-on or pull-off to be used during thefollowing timestamp. Second, constraints 3.18 state that a hammer-on should start witha lower numbered finger than the one it ends with (e.g. starting with your index fingerand ending with your ring finger is allowed, but the opposite is not), and constraints 3.19state the opposite for pull-offs. Lastly, constraints 3.20 make sure the same finger is usedfor the starting and the ending position of a slide.

Constraints 3.21-3.23 restrict the F , S, G, w, W and µ variables to be binary, andconstraints 3.24 restrict the ∆ variables to positive real numbers.

3.2. MODEL 2: CHORD-BASED GUITAR MUSIC 17

3.2 Model 2: Chord-based guitar music

In this model for chord-based guitar pieces, the composition of the objective function re-mains the same, as again depicted in Figure 3.3. The static components remain identical,

Figure 3.3: The different components of the objective function.

but the transition components are determined differently, since in chord-based music onetypically keeps the same finger position throughout multiple timestamps, thus affectingtransitions.

Wrist movement The consecutive fret assumption is typically too restrictive forchord-based pieces. As an example, consider the small part of Eric Clapton’s Tearsin Heaven depicted in Figure 3.4.

Figure 3.4: Why the consecutive fret assumption is too restrictive for chord-based guitarpieces: an excerpt from Tears in Heaven.

In this case, the guitarist would keep on pressing down the second fret on the E-stringwith his index finger, the second fret on the G-string with his middle finger and the thirdfret of the B-string with his ring finger during the first four timestamps, even when thestring does not need to be played. Consider now using the consecutive fret assumptionhere: for the first two timestamps, the index finger is used and thus determines the wrist


position to be on fret 2. When we move to the third timestamp, however, the index fin-ger is not used and, in that case, the middle finger determines the wrist position. Whenassuming fingers to be on consecutive frets, the index finger is assumed to be on fret1 and thus the theoretical wrist position changes, even though the guitarist keeps theexact same form as before. This illustrates the need for a less restrictive determinationof the wrist position.

In this second model, we let the model determine the wrist position endogenously byrequiring the index finger to be assigned to a fret at all times, even when it is not ac-tually playing a position on the fretboard. The wrist position is then set equal to theposition of the index finger. The determination of this ‘floating’ index finger is subjectto the inter-finger distances, just like the determination of the index finger is subject tothem for positions p ∈ Pt that actually need to be played.

Movement across strings Due to the fact that a player would keep its fingers inposition for a longer time (playing a ‘chord’ with repeated notes), the horizon for themovement across strings is extended from t+1 to t+2.

3.2.1 Mathematical Model

The whole chord-based model is given below, and the differences with the scale-basedmodel are explicitly highlighted at the end of this chapter.

Sets

• T is the set of timestamps, i.e. the set of all points in time during which a note isplayed.

• S = {E,A,D,G,B, e} is the set of strings.

• F = {1, 2, 3, 4} is the set of fingers available to play the guitar.

• G = {1, .., n} for a guitar with n frets is the set of frets.

• P = S × G is the set of unique fretboard position identifiers.

• Pt denotes the set of fretboard positions to be played at timestamp t, based on thetablature of the concerned song.

• Th denotes the set of (t, p)-vectors that give the timestamp t ∈ T when, andposition p ∈ Pt where, a hammer-on starts, based on the tablature of the concernedsong.

• Tp denotes the set of (t, p)-vectors that give the timestamp t ∈ T when, andposition p ∈ Pt where, a pull-off starts, based on the tablature of the concernedsong.


• Ts denotes the set of (t, s)-vectors that give the timestamp t ∈ T when, and strings ∈ S on which, a slide starts, based on the tablature of the concerned song.

Decision variables

• Ft, p, f is a binary variable which is one iff at timestamp t ∈ T , position p ∈ P ispressed by finger f ∈ F .

• St, s, f is a binary variable which is one iff at timestamp t ∈ T , string s ∈ S ispressed by finger f ∈ F .

• Gt, g, f is a binary variable which is one iff at timestamp t ∈ T , fret g ∈ G is pressedby finger f ∈ F .

• ∆g, g′, f, f ′, t is a binary variable which is one iff distinct fingers f and f ′ ∈ F areboth used on respectively frets g and g′ ∈ G at time t ∈ T .

• µhs, s′, f, t is a binary variable which is one iff finger f ∈ F moves from string s tostring s′ ∈ S from timestamp t− h to t ∈ T \ {1, .., h} with h ∈ {1, 2}.

• Wg, g′, t is a binary variable which is one iff the wrist moves from fret g to g′ ∈ Gfrom timestamp t− 1 to t ∈ T \ {1}.

Constants

• Ip, s is a binary constant which is one iff fretboard position p ∈ P is located onstring s ∈ S.

• Ip, g is a binary constant which is one iff fretboard position p ∈ P is located on fretg ∈ G.

Parameters

• ag, g′, f, f ′ is the cost of using both fingers f and f ′ ∈ F simultaneously on fretsg and g′ ∈ G respectively. Three instances exist: (i) the distance between fingersf and f ′ is comfortable: no cost is imposed; (ii) the distance between fingers fand f ′ is uncomfortable but playable: a cost is imposed; (iii) the distance betweenfingers f and f ′ is impossible to play: a high cost is imposed to prevent the modelfrom selecting this finger assignment.

• b is the cost of using the pinky finger. This cost is independent of time, fret orstring.

• cg, g′ is the cost of moving the wrist from position g to position g′ ∈ G. Shortermovements should have a lower cost, longer movements should have a higher cost.

• dhs, s′ is the cost of moving a finger from string s to s′ ∈ S for a horizon of h ∈ {1, 2}.


Integer programming model

min∑

g, g′∈Gf,f ′∈F :f 6=f ′

t∈T

ag, g′, f, f ′∆g, g′, f, f ′, t+b∑t∈Tp∈Pt

Ft,p,4+∑

g, g′∈Gt∈T \{1}

cg, g′Wg, g′, t+∑

s, s′∈Sf∈F

h∈{1,2}t∈T \{1,..,h}

dhs, s′µhs, s′, f, t

(3.25)∑f∈F

Ft,p,f = 1 ∀t ∈ T , p ∈ Pt (3.26)

∑p∈Pt

Ft,p,f ≤ 1 ∀f ∈ F , t ∈ T (3.27)

St,s,f =∑p∈Pt

Ip, sFt,p,f ∀f ∈ F , t ∈ T , s ∈ S (3.28)

Gt,g,f =∑p∈P

Ip, gFt,p,f ∀f ∈ F , t ∈ T , g ∈ G (3.29)

Gt,g,f +Gt,g′,f ′ − 1 ≤ ∆g, g′, f, f ′, t ∀t ∈ T , g, g′ ∈ G, f, f ′ ∈ F : f 6= f ′ (3.30)

St−h,s,f + St,s′,f − 1 ≤ µhs, s′, f, t ∀s, s′ ∈ S, f ∈ F , h ∈ {1, 2}, t ∈ T \ {1, .., h} (3.31)

µhs, s′, f, t ≤ St−h,s,f ∀s, s′ ∈ S, f ∈ F , h ∈ {1, 2}, t ∈ T \ {1, .., h} (3.32)

µhs, s′, f, t ≤ St,s′,f ∀s, s′ ∈ S, f ∈ F , h ∈ {1, 2}, t ∈ T \ {1, .., h} (3.33)∑

g∈GGt,g,1 = 1 ∀t ∈ T (3.34)

Gt−1,g,1 +Gt,g′,1 − 1 ≤Wg,g′,t ∀g, g′ ∈ G′,∀t ∈ T \ {1} (3.35)∑f∈F


f × Ft,p′,f ∀t ∈ T , p, p′ ∈ P, k ∈ {1, .., 5} : p′ = p+ nk (3.36)

Ft,p,f + Ft+1,q,f ≤ 1 ∀(t, p) ∈ Th ∪ Tp, q ∈ Pt+1, f ∈ F (3.37)∑f∈F


f × Ft+1,q,f ∀(t, p) ∈ Th, q ∈ Pt+1 (3.38)

∑f∈F

f × Ft,p,f ≥∑f∈F

f × Ft+1,q,f ∀(t, p) ∈ Tp, q ∈ Pt+1 (3.39)

St,s,f = St+1,s,f ∀(t, s) ∈ Ts, f ∈ F (3.40)

Ft,p,f , St,s,f , Gt,g,f , Wg,g′,t ∈ {0, 1} ∀t ∈ T , p ∈ P, f ∈ F , s ∈ S, g, g′ ∈ G (3.41)

µhs, s′, f, t ∈ {0, 1} ∀h ∈ {1, 2}, s, s′ ∈ S, f ∈ F , t ∈ T \ {1, .., h} (3.42)

∆g, g′, f, f ′, t ∈ R≥0 ∀g, g′ ∈ G, f, f ′ ∈ F : f 6= f ′, t ∈ T (3.43)

Compared to the scale-based model, the wg,t variables disappear, together with theiraccompanying constraints 3.11-3.14. Constraints 3.10 are replaced by constraints 3.34,requiring the index finger to be assigned to a fret at all times. To make this possibe, theFt,p,1 variables now need to be defined for all p ∈ P and not just p ∈ Pt (3.41), whichis also reflected in the determination of the G variables in constraints 3.29. The wristmovement is now directly determined through the Gt,g,1 variables in constraints 3.35.

In addition, the µs,s′,f,t variables are replaced by µhs,s′,f,t variables, indicating the horizon


of the movement across strings, just like the dhs, s′ parameters that determine their cost.The relevant constraints 3.31-3.33 are likewise updated.

Chapter 4

Implementation and Results

We implement the model in IBM ILOG CPLEX Optimization Studio 12.7.1.0. Theparameters were determined as follows:

• The ag,g′,f,f ′ parameters are determined based on matrices that indicate the min-imum and maximum comfortable and absolute distances between fingers. Thesedistances are expressed in centimeters, to account for the different widths fretshave. Note that these comfortable and absolute distances are personal and shouldbe tailored to the guitarist. Inter-finger distances that fall below or exceed thecomfortable distances are penalized with a cost of 5 while those that fall belowor exceed the absolute distances are penalized with a cost of 100. We use a semi-acoustic guitar (LAG Tramontane T222DCE) with 20 frets to map the distancesto frets.

• The b parameter for penalizing the use of the pinky is set to 0.25

• The cg,g′ parameters are set to 1 for a wrist movement of 1 fret, 2 for a movementof 2 frets and 3 for all larger movements. Again, a guitar of 20 frets is assumed.

• The d1s,s′ parameters are set to the distance between the two concerned strings.For example, a movement from the first to the third string equals a cost of 2.When, however, a finger stays on the same string, a profit of 1 (i.e. a cost of -1) isassigned.

• The d2s,s′ parameters are set to 0.5 for a movement of one string, and 1 for all largermovements (2 to 5 strings). When, however, a finger stays on the same string, aprofit of 0.5 (i.e. a cost of -0.5) is assigned.

In order to avoid the long manual work of converting the tablature to useable CPLEXinputs, i.e. an array of sets of position identifiers and frets that are used at each times-tamp, we created a small Python preprocessing program that parses Guitar Pro tabsand produces the required format for CPLEX. In addition, we developed another Pythonpostprocessing program to process the output of CPLEX back into a Guitar Pro file andautomatically print the left-hand finger assignment on the tablature. These programs

23

24 CHAPTER 4. IMPLEMENTATION AND RESULTS

were written using PyGuitarPro [8], a freeware python package, and can be found in theAppendix. This way, one can download any Guitar Pro Tab from the internet, quicklyparse it to useable CPLEX inputs, solve the model, and parse it again to a Guitar Profile with the finger assignment below the tablature.

We selected a sample of nine songs in different genres, to test the performance of themodel. Four songs were directly ruled out as the Python preprocessing program flaggedthey included barre chords. Table 4.1 gives the different songs, the number of times-tamps the guitar track includes and whether or not they include barre chords.

BarreArtist Song # Timestamps Chords?

Kansas Dust In The Wind 1079 NEric Clapton Tears in Heaven 675 NEric Clapton Layla 575 YThe Beatles Blackbird 489 NLed Zeppelin Stairway To Heaven 1140 YSantana Samba Pa Ti (Intro) 230 NChuck Berry Johnny B Goode (Intro) 215 NJimi Hendrix Purple Haze 726 YJohn Williams Cavatina (Theme 399 Y

from The Deer Hunter)

Table 4.1: The list of songs that were parsed through the Python implementation

The instances that do not include barre chords were run in CPLEX on a laptop with anIntel(R) Core(TM) i5-2430M CPU @ 2.40Ghz (4 CPUs) processor with a RAM memoryof 6144MB, running on Windows 10.

Total Root + Branch & Cut Time (Seconds)Song Type Model 1: Scale-Based Model 2: Chord-Based

Dust In The Wind Chord-based 711 125Tears in Heaven Chord-based 50 40Blackbird Chord-based 229 124Samba Pa Ti Scale-based 39 333Johnny B Goode Scale-based 166 201*

Table 4.2: The results of the experiments: for each instance, the type of the song isgiven and the solving time for both models. An asterisk (*) indicates that the instancewas stopped at a 2% optimality gap because it could not be solved to optimality within30 minutes.

The results are given in table 4.2. For each song, the assumed ‘type’ is given, togetherwith the solving time (Total Root + Branch & Cut Time) in both models. As can be

25

seen from the table, the chord-based songs are solved significantly faster in the chord-based model, and vice versa for the scale-based songs. Chuck Berry’s Johnny B Goodecan even not be solved to optimality in the chord-based model within 30 minutes, butsolves relatively fast in the scale-based model.

Concerning the quality of the outputs of the model, it is not possible to give one ob-jective metric. This is due to the fact that, as expressed in the introduction, there isno such thing as THE optimal finger assignment. For example, how Paul McCartneyplays Blackbird might not be the easiest or preferred way for a beginning guitarist andeven an experienced guitarist might opt for another finger assignment, due to differentpreferences. To illustrate, however, the difference in quality of both models for differ-ent pieces, we will discuss some extracts of the output of the songs in the table above.

Firstly, consider the finger assignment for the first two bars of “Dust in the wind” forboth models in Figure 4.1 on page 26. Note that there are a lot of repeated notes (i.e.the same fret and string) in this song, which is typical for a chord-based guitar piece.Consequently, in the chord-based model at the bottom, the first three fingers can stayon their position during the whole part: the index finger on the first fret of the B-string,the middle finger on the second fret of the D-string and the ring finger on the third fretof the A-string. For the first half of the second bar, the guitarist can simply add itspinky finger on the third fret of the B-string and remove it again for the last part. Thisfinger assignment requires very little effort. In the scale-based model on top, however, atthe 13th timestamp, the index finger moves to the D-string and all fingers assume a newposition, only to return to their original position at the second half of the last bar. Ev-idently, this finger assignment requires more effort and the chord-based one is preferred.

Secondly, consider the finger assignment for the first bar of “Blackbird” for both modelsin Figure 4.2 on page 26. In the chord-based model at the bottom, the same fingersstay on the same frets and can effortlessly be slid over the fretboard. In the scale-basedmodel, the finger used for these two strings constantly switch, increasing the necessaryeffort and decreasing the comfort of the guitarist. Again, the chord-based finger assign-ment is preferred over the scale-based finger assignment.

Lastly, consider the finger assignment for the first bar of “Samba Pa Ti” for both modelsin Figure 4.3 on page 27. Notice the lack of repeated notes, typical for scale-based guitarpieces. In the scale-based model on top, the fingers can stay in the same fret during thewhole bar, and the wrist does not need to move. In the chord-based model, however,the pinky finger jumps around from fret to fret, requiring the wrist to constantly movearound. Evidently, the scale-based finger assignment is preferred for this song.


Model 1: Scale-based1

2 Model 2: Chord-based

Figure 4.1: The finger assignment for the first two bars of Kansas’ “Dust in the wind”for both models.



Figure 4.2: The finger assignment for the first two bars of The Beatles’ “Blackbird” forboth models.

27



Figure 4.3: The finger assignment for the first two bars of Santana’s “Samba Pa Ti” forboth models.

Based on the descriptive examples above, it can be seen that the scale-based modelis preferred for scale-based songs and the chord-based model for chord-based songs.

In the existing literature on automatic transcription and fingering decisions discussed inChapter 2, Hori et al. [6] and Miura and Yanagida [5] use the consecutive fret assumptionwhile Yazawa et al. [4] enumerate fingering configurations through a guitar chord book,thus focusing on chords. The results in this thesis point out that the area of applicabil-ity of their work potentially could be expanded or the quality of their results improvedthrough explicitly modeling the distinction between chord-based and scale-based songs.

The remaining question is: how to discern scale-based from chord-based pieces? Whilethis is relatively easy for the human eye or ear, it is less evident to explain this to acomputer program. Typically, scale-based pieces have mostly single notes per timestamp(monophony), have limited repeated notes, and the movements across the fretboard tendto be larger than in chord-based pieces. In addition, these songs typically contain more


special techniques such as hammer-ons, slides, pull-offs and bends. Future work couldfocus on finding objective metrics to discern the two different types, even for differentparts within songs.

In addition, another improvement point is a current lack of consistency in repeatedparts: identical parts are sometimes assigned different fingers. This can be explainedthrough a multitude of equivalent optimal solutions. Table 4.3 gives the size of the so-lution pool for the instances above.

Solution PoolSong Model 1: Scale-Based Model 2: Chord-Based

Dust In The Wind 22 19Tears in Heaven 20 14Blackbird 20 24Samba Pa Ti 19 41Johnny B Goode 25 24*

Table 4.3: The solution pool of the instances solved. An asterisk (*) indicates that theinstance was stopped at a 2% optimality gap because it could not be solved to optimalitywithin 30 minutes.

This solution pool could be reduced by introducing additional rules for ordering so-lutions in terms of preference. For example, a rule could be “in case of equivalence, alower numbered finger should be preferred over a higher numbered finger”. Moreover,adding these rules could potentially reduce processing time.

Chapter 5

Conclusion

A tool for automatic finger assignment can bring significant advantages for beginningguitarists. In this thesis, two user-specific models to assign fingers for tablature weredeveloped and implemented in CPLEX. Moreover, a Python program was developed toparse Guitar Pro tablatures (which can be found on the world wide web for the majorityof existing songs), produce the necessary inputs for CPLEX and, after solving, outputit back into Guitar Pro tablature, this time including the finger assignment.

In this thesis, we discovered an important distinction in model composition for twodifferent types of guitar pieces, with an impact on both processing time and the qualityof the resulting finger assignment. The existing literature on automatic transcriptionand translation of staff notation into tablature implicitly focuses on either one of thesetypes but not on both. Therefore, including this distinction could potentially improvethe quality of the models suggested as well as expand their applicability.

Potential future work includes three key focal areas: first, refining the distinction be-tween the two different types of songs and finding objective metrics to differentiatebetween them, even for different parts within songs; second, expanding the model toinclude barre chords; and lastly, reducing the solution pool through additional rules toimprove the quality and potentially decrease processing time for finding solutions.

29

30 CHAPTER 5. CONCLUSION

Bibliography

[1] Mikkel Winther Jensen. Layla acoustic (ver 2) guitar pro. https://tabs.ultimate-guitar.com/e/eric clapton/layla acoustic ver2 guitar pro.htm, 2006. Accessed: 2017-05-03.

[2] D.R. Tuohy and W.D. Potter. A genetic algorithm for the automatic generation ofplayable guitar tablature. Proceedings of the International Computer Music Confer-ence, 2005.

[3] Aleksander Radisavljevic and Peter Driesen. Path difference learning for guitar fin-gering problem. Proceedings of the International Conference on Music Perceptionand Cognition, 2004.

[4] Kazuki Yazawa, Katsutoshi Itoyama, and Hiroshi G. Okuno. Automatic transcrip-tion of guitar tablature from audio signals in accordance with player’s proficiency.Proceedings of the IEEE International Conference on Acoustic, Speech and SignalProcessing (ICASSP), 2014.

[5] Masanobu Miura and Masuzo Yanagida. Finger-position determination and tablaturegeneration for novice guitar players. Proceedings of the 7th International Conferenceon Music Perception and Cognition, 2002.

[6] Gen Hori, Hirokazu Kameoka, and Shigeki Sagayama. Input-output HMM appliedto automatic arrengement for guitars. Journal of Information Processing Vol. 21 No.2, 2013.

[7] Matteo Balliauw, Dorien Herremans, Daniel Palhazi Cuervo, and Kenneth Sorensen.A variable neighborhood search algorithm to generate piano fingerings for polyphonicsheet music. International Transactions in Operational Research, 2015.

[8] Sviatoslav Abakumov. Pyguitarpro. https://pypi.python.org/pypi/PyGuitarPro,2014.

31

32 BIBLIOGRAPHY

Appendix A

A.1 Python Programs

On the next pages, the Python programs that were written for this thesis can be found:first, the preprocessing program that parses Guitar Pro tabs and produces the requiredformat for the CPLEX inputs; second, the postprocessing program that processes theoutput of CPLEX back into a Guitar Pro file and automatically prints the left-handfinger assignment on the tablature.

33

1 import sys2 import guitarpro3 from os import path4 5 gp5_version=(5,1,0)6 7 MAPPING = {8 1: 100,9 2: 80,

10 3: 60,11 4: 40,12 5: 20,13 6: 014 }15 16 def main(tab_file):17 18 song = guitarpro.parse(tab_file)19 20 if song.versionTuple[0]==3: # fingering not possible on gp3-files with this

version of pyguitarpro; therefore first converting to gp521 dest = '%s-conv.gp5' % path.splitext(tab_file)[0]22 guitarpro.write(song, dest,gp5_version)23 print("GP3 input file converted to GP5-format : " + dest)24 print("")25 song = guitarpro.parse(dest)26 27 print(song.title + " - " + song.artist)28 29 for track_nr, track in enumerate(song.tracks, start=1):30 print("track " + str(track_nr) + " = " + track.name)31 if track.isPercussionTrack:32 print("Percussion ==> skipping")33 break34 if track.is12StringedGuitarTrack:35 print("12 string ==> skipping")36 break37 if track.isBanjoTrack:38 print("Banjo ==> skipping")39 break40 time_cnt=041 notes_not_0=042 outstr=""43 outstr_real=""44 barrestr=""45 for measure_nr, measure in enumerate(track.measures, start=1):46 for voice_nr, voice in enumerate(measure.voices, start=1):47 for beat_nr, beat in enumerate(voice.beats, start=1):48 tmpstr=""49 tmpstr_real=""50 isfirst=151 note_cnt=052 for note_nr, note in enumerate(beat.notes, start=1):53 if note.value > 0 and note.type.name != "tie": # open

string and tie notes will not be outputted54 if isfirst:55 isfirst=056 tmpstr=tmpstr+str(MAPPING[note.string] + note.value)57 tmpstr_real=tmpstr_real+str(note.value)58 else:59 tmpstr=tmpstr+","+str(MAPPING[note.string] +

note.value)60 tmpstr_real=tmpstr_real+","+str(note.value)61 notes_not_0=162 note_cnt=note_cnt+163 if outstr=="":64 outstr="{"+tmpstr+"}"65 outstr_real="{"+tmpstr_real+"}"66 else:67 outstr=outstr+",{"+tmpstr+"}"68 outstr_real=outstr_real+",{"+tmpstr_real+"}"69 time_cnt=time_cnt+1

70 if note_cnt > 4: # more than 4 notes in a one timestamp, must be a barre

71 if barrestr=="":72 barrestr="{"+tmpstr+"}"73 else:74 barrestr=barrestr+",{"+tmpstr+"}"75 if notes_not_0:76 print(outstr)77 print("times = " + str(time_cnt))78 if barrestr != "":79 print("ATTENTION - This track contains barre chords: " + barrestr)80 print("\nOutput with just fret values:")81 print(outstr_real)82 else:83 print("No playable notes in this track")84 print("")85 86 if __name__ == '__main__':87 import argparse88 89 parser = argparse.ArgumentParser(description="Preprocessor for adding left-hand

fingering positions into a Guitar Pro tablature. Output is written to standard output")

90 parser.add_argument("tab_file",help="path to the Guitar Pro tablature file")91 args = parser.parse_args()92 kwargs = dict(args._get_kwargs())93 main(**kwargs)

1 import sys2 import guitarpro3 from os import path4 import re5 6 gp5_version=(5,1,0)7 8 MAPPING = {9 1: 100,

10 2: 80,11 3: 60,12 4: 40,13 5: 20,14 6: 015 }16 17 FINGERMAPPING = {18 1: guitarpro.models.Fingering.index,19 2: guitarpro.models.Fingering.middle,20 3: guitarpro.models.Fingering.annular,21 4: guitarpro.models.Fingering.little,22 }23 24 def main(tab_file, track_num, finger_file):25 26 song = guitarpro.parse(tab_file)27 print(song.title + " - " + song.artist)28 29 if song.versionTuple[0]==3:30 print("GP3 tab-file not allowing finger positions! Quiting...")31 exit()32 33 dest_file = '%s-LHF.gp5' % path.splitext(tab_file)[0] # we will always write in

GP5-format34 35 list_of_fingerings = list()36 ff=open(finger_file)37 line=ff.readline()38 cline=re.sub("\}\,","}|",line).strip() # this can be removed if the |-delimiter

is used in the inputfile39 list_of_fingerings=cline.split("|")40 41 track = song.tracks[track_num - 1]42 print("track " + str(track_num) + " = " + track.name)43 if track.isPercussionTrack or track.is12StringedGuitarTrack or track.isBanjoTrack:44 print("Invalid track")45 exit()46 47 time_cnt=048 notes_not_0=049 outstr=""50 51 for measure_nr, measure in enumerate(track.measures, start=1):52 for voice_nr, voice in enumerate(measure.voices, start=1):53 for beat_nr, beat in enumerate(voice.beats, start=1):54 tmpstr=""55 isfirst=156 fingerpos=list_of_fingerings[time_cnt]57 if fingerpos=="{}": # no finger positions on this timestamp58 time_cnt=time_cnt+159 else:60 fingerpos=fingerpos.strip("{}")61 list_of_pos=[int(x) for x in fingerpos.split(",")]62 fnr=063 for note_nr, note in enumerate(beat.notes, start=1):64 if note.value > 0 and note.type.name != "tie": # open

string and tie notes have been ignored65 notes_not_0=166 note.effect.leftHandFinger=FINGERMAPPING[list_of_pos[fnr]]67 fnr=fnr+168 time_cnt=time_cnt+169 if notes_not_0:

70 guitarpro.write(song, dest_file ,gp5_version)71 print("%s has been created with left-hand fingering added" % dest_file)72 73 if __name__ == '__main__':74 import argparse75 76 parser = argparse.ArgumentParser(description="Merge left-hand fingering

positions into a Guitar Pro tablature")77 parser.add_argument("tab_file",78 help="path to the Guitar Pro tablature file")79 parser.add_argument("track_num",80 type=int,81 help="number of the track that will get the left-hand

fingering")82 parser.add_argument("finger_file",83 help="path to the file that has the fingering")84 args = parser.parse_args()85 kwargs = dict(args._get_kwargs())86 main(**kwargs)

fingers to frets - a mathematical approachvibeserver.net/scripties/2017/fingers to frets - master...

Documents