![Page 1: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/1.jpg)
![Page 2: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/2.jpg)
Just Dance 4
![Page 3: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/3.jpg)
Ok, once more
![Page 4: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/4.jpg)
Variation from (Victoria Zhou)
1. Step forward towards corner 2, into croisé derrière à terre, arms demi-seconde
2. Plié in 5th change direction to corner 13. Relevé derrière, arms in 4th en avant
palms down4. Passé and change direction to corner 2
place foot on pointe in 5th, arms move to a low kissing gesture
5. Retiré and back to 5th on pointe with the front foot, back foot and front foot arms move to demi-seconde
6. Repeat to the right, and repeat all.7. Posé coupé effacé towards corner 1,
arms in 4th en avant8. Posé coupé arms in 4th palms down9. Step and posé retiré croisé, place back
foot in 5th on pointe, port de bras with the left arm from 5th en haut to 5th enavant, end with the arms forward in a low line, cross the wrists.
![Page 5: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/5.jpg)
One more time
![Page 6: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/6.jpg)
![Page 7: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/7.jpg)
What representation for dance?for humans
and machines Sci-Fi Dan
cing G
IF by C
olin
Raff
![Page 8: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/8.jpg)
While for music we have good (machine readable) representations, we lack these for dance.
![Page 9: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/9.jpg)
Why do we need knowledge representation for dance?
Three reasons
![Page 10: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/10.jpg)
Archival and retrieval
![Page 11: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/11.jpg)
Analysis: Digital humanities
![Page 12: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/12.jpg)
Supporting creativity
![Page 13: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/13.jpg)
Towards a choreography assistant tool
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
![Page 14: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/14.jpg)
Existing representations and tools
![Page 15: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/15.jpg)
Labanotation
![Page 16: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/16.jpg)
LabanXML and Laban Editor
LED Labanotation editor http://donhe.topcities.com/pubs/led.htmlNakamura & Hachimura (2006)
![Page 17: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/17.jpg)
Benesh
![Page 18: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/18.jpg)
Dance Forms
![Page 19: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/19.jpg)
Cecchetti system7 elementary movements: [plie (bend), etandre(stretch), releve (rise), sauter (jump), glisse(glide), tourne (turn), elancer (dart)]Positions: 1st, 2nd, 3rd,.. (left right croise)Facing position (1…8)Position in spaceDirection of movement (de cote, dessous, dessus, en avant, en arriere, devant, derriere)Combinations (100+) pas-de-chat, pas-de-bourre, piroutte
Ballet languages/systems
Based on interview with Marije Koning
![Page 20: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/20.jpg)
XML Dance Grammar
Balakrishnan Ramadoss and Kannan Rajkumar. Modeling the Dance Video Semantics using Regular Tree Automata Fundamenta Informaticae 86 (2008) 175–189 175 IOS Press
![Page 21: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/21.jpg)
Interactive Dance Choreography Assistance
Victor de BoerJosien Jansen Ana-Liza Tjon-A-Pauw
Frank Nack
![Page 22: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/22.jpg)
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
![Page 23: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/23.jpg)
To what extent can choreographers be supported by
semi-automatic dance analysis and the generation of new
creative elements in choreographies?
![Page 24: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/24.jpg)
Method
Questionnaire: How do choreographers work (withtechnologies)
Tool: Proof of concept digital choreo assistant
Evaluation: Test application toand different strategies
![Page 25: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/25.jpg)
Questionnaire
54 Dutch choreographers
Online questionnaire
![Page 26: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/26.jpg)
Personal choreography archiving
0
2
4
6
8
10
12
14
16
18
writtendance
notation
digital dancenotation
videotaping other memory,without
problems
memory,forget things
![Page 27: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/27.jpg)
Preferred Notations
0
5
10
15
20
25
30
35
![Page 28: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/28.jpg)
Notations Laban & Benesh
0
5
10
15
20
25
30
Never heard ofit
Cannot workwith it
Other Know Laban Can write both
![Page 29: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/29.jpg)
Interest in support in the creative process
Originality, Creativity and Emotion are most important aspects
One very negative sub-group> Afraid to lose humanity
One positive towardscreative assistance
Two sub-groups:
![Page 30: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/30.jpg)
Tool Requirementsbased on MoSCoW method
• A dancer must be able to add their dance style to the tool• A dancer must be able to add their existing choreography to the tool• The tool must be able to give new suggestions for variations of the
choreography• The suggestions must be based on different strategies • The dancer must be able to see the whole choreography at any
moment in time (written)• The communication of the tool are written dance terms• The tool must be “easy to use”, which means getting suggestions may
take no longer than 2 minutes• The tool does have simplified body movements (legs, feet, arms, hands
and head)
![Page 31: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/31.jpg)
Proof-of-concept mobile app
3 different dance styles
Ballet (including 78 steps)
Modern dance (including 57 steps)
Street dance (including 31 steps)
Dancepiration – a tool for choreography assistance
![Page 32: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/32.jpg)
4 rule-based strategies for creating variations on existing choreographies
1. Random step replaced by random other step
2. Random step replaced by ontology-based other step
3. Random steps replaced by multiple strategies
4. Specific step replaced by ontology-based steps
![Page 33: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/33.jpg)
Ontology-based variation for the 3 dance styles.
El Raheb, et al. BalOnSe: Ballet Ontology for Annotating and Searching Video performances. In Proceedings of the 3rd International Symposium on Movement and Computing (p. 5). ACM, 2016
![Page 34: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/34.jpg)
Evaluation
Evaluation
![Page 35: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/35.jpg)
6 choreography students
Random-based versus Ontology-based
Each dance style is tested 3 times with both strategies per person
Rate original choreography and each variation (10pt scale)
Rate on 5pt Likert scale: Correctness, Creativity, Helpfulness, Meaningfulness
![Page 36: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/36.jpg)
Results
Respondents are positive about the tool
…prefer to choose a specific step to change themselves
… consider creativity in this tool very high (avg 4.2/5)
Correctness is important to improve, it influences other factors the most
![Page 37: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/37.jpg)
Ontology-based variant outperforms random variations
Score OriginalRandomOntology-Based Difference SigAverage grade 6.17 5.50 6.35 +0.85 **Correctness 2.89 3.37 +0.48 *Creativity 3.19 3.37 +0.18
Helpfulness 2.59 3.00 +0.41
Meaningfulness 2.70 2.96 +0.26
** = statistically sign
ificant at α
=0.0
5 (t-test/an
ova)
![Page 38: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/38.jpg)
Style matters* = statistically sign
ificant at α
=0.1
0 (t-test/an
ova)
** = statistically sign
ificant at α
=0.0
5 (t-test/an
ova)
Element Style Random Ontology-Based DifferenceCorrectness Ballet 2.89 2.56 -0.33
Streetdance 2.78 3.56 +0.78 *Modern 3.00 4.00 +1.00 **
Creativity Ballet 3.44 3.56 +0.12Streetdance 2.78 3.11 +0.33Modern 3.11 3.44 +0.33
Helpfulness Ballet 2.67 2.67 0.00Streetdance 2.44 2.89 +0.45Modern 2.89 3.44 +0.55
Meaningfulness Ballet 2.89 2.78 -0.11Streetdance 2.33 2.67 +0.34Modern 2.89 3.44 +0.55
![Page 39: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/39.jpg)
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
![Page 40: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/40.jpg)
Which presentation methods are considered most effective for an interactive dance choreography assistant tool?
![Page 41: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/41.jpg)
Experiment: Comparing 4 choreography presentation methods
![Page 42: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/42.jpg)
1: Textual descriptions
![Page 44: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/44.jpg)
3: 3D animations
DanceForms 2 (http://charactermotion.com/products/danceforms/
![Page 45: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/45.jpg)
4: auditory instructions
![Page 46: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/46.jpg)
Experiment• 7 choreographers• 2 new(!) styles
– Hip-hop and dancehall
• Simple choreography + pre-generated variations
• Large projection screen in practice space• 4 presentation variations (random)
![Page 47: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/47.jpg)
![Page 48: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/48.jpg)
Overall assessment
Dancehall vs Hip-hop
0123456789
10
Textual 2Danimations
3Danimations
Auditory
Sco
re
0123456789
10
Textual 2Danimations
3Danimations
Auditory
Sco
re
![Page 49: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/49.jpg)
0
1
2
3
4
5
6
7
8
9
10
Overall assessment Stimulation ofcreativity
Understandability (Un-)disruptiveness
Textual 2D an. 3D an. Auditory
3D animations are the best
![Page 50: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/50.jpg)
A significant sub-group of choreographers is interested in and enthusiastic about automatic choreography support
Needs to be able to understand ‘dance language’
Knowledge representation matters
Style matters
Presentation styles matter -> 3D + dance language
![Page 51: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/51.jpg)
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
Next level: Representation and Reasoning
Multi-tiered semantic model
Low-level image features
Atomic movements (Labanotation?)
Compound movements (100+ movements)
Emotional content, Socio-cultural layers etc.
Machine Learning for classification and pattern detection
Generative module (automatic choreographer)
![Page 53: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/53.jpg)
![Page 54: Interactive Dance Choreography Assistance presentation for ACE entertainment 2017](https://reader031.vdocuments.net/reader031/viewer/2022022415/5a64794a7f8b9a27568b47e9/html5/thumbnails/54.jpg)
Sensing
• Motion capture– Marker-based
– Marker-less
• Joint rotations, limb positions etc.– unintuitive
• Backup: Video annotation
img: news.stanford.edu