Story Analogy
character A character B
character
typestudent teacher
situationnot knowing where
to go for the future
willing to give
guidance to character
A
emotion /
attitude
anxious, lost, hoping
for help, pessimistic
caring, insightful,
appreciatory, advisory
character A character B
character
typeStudent teacher
situation
Thought that she was
clear where to go for
the future
willing to give
suggestion to
character A
emotion /
attitudeConfused, hesitated
encouraging, insightful,
advisory
Idea
Build system that allows users to tell different stories from sequences of video clips with meaningful relationships between clips (using analogies to other stories)
Original System DesignWhat’s the story about? Presidential election
1. Who were the candidates?
2. When was the election?
3. What was the outcome of the election?
4. What were the issues?
Video Corpus
1. How do you feel about him?
2. What has he done in the past?
Challenges for Q/A scenario
How to derive context-relative questions based on provided information
How to derive plausible annotations for a video sequence based on the context of using it
How to design representation for video sequence such that the above can be conducted more easily
Generating Questions
Assume video sequences annotated with Roger Schank’s form of indexing stories (theme,goal,plan,result,lesson)
Questions come from:Generic question templatesCommon sense reasoningStory analogies
Curiosity
Ask questions people tend to be more curious about
Introduce new pieces of information that produce effect of new direction of navigation
Curiosity
Relationship between people Why do people do what they do?
External forces Values/Beliefs Internal desire
New System Design
Different usersUsers that know video corpus very wellUsers that are seeing videos for first time
Help new users create stories by using information first set of users have already entered (system developers)
New system design
1Q1Q2Q3
System Developer
End User
23Q4
Q5
Q5 Q6
44
5Q7Q8Q9
2Why is Mary sad?
Using semantic similarity and common sense44
3
View suggested questions
Q7Q8Q9
555
System Developer1. System Developer types in story
Gustave is one of the dancers. He likes to suggest ideas for the dance. He gets very
frustrated that he can only do what he is told.
2. Sentence/Sentences attached to clips
1 2 3
3. Question set developed for each sequence
1
2
3
Q1Q2Q3
Q1Q2Q3
Q1Q2Q3
4. Generate Q/A sets between video sequences
12Q1
Q2
Q3 Q4
4
Steps 1 – Segment StoriesGustave is one of the dancers.
He gets very frustrated that he is only allowed to do what he is told.
At one point he was crying. This happened during the first week.
The second week was different.
Dancers were encouraged to design their own dance.
Gustave had a wonderful time.
He likes to suggest ideas for the dance.
1. Always include topic sentence2. Try topic sentence with new sentence, if sentence does not return 4/5 same clips as previous sentence/segment, start new segment. Otherwise, add to previous sentence/segment
3/5
4/5
4/5
3/5
4/5
Step 2 – Assign clips
He gets very frustrated that he is only allowed to do what he is told.
At one point he was crying. This happened during the first week.The second week was different.
Dancers were encouraged to design their own dance.
Gustave had a wonderful time.
He likes to suggest ideas for the dance.
1
2
3
Gustave is one of the dancers.
Gustave is one of the dancers.
Gustave is one of the dancers.
Step 3 – Question GenerationGustave is one of the dancers.
He gets very frustrated that he is only allowed to do what he is told.At one point he was crying. This happened during the first week.The second week was different.
Dancers were encouraged to design their own dance.
He likes to suggest ideas for the dance.1
2
3
Gustave is one of the dancers.
Gustave is one of the dancers.
Gustave had a wonderful time.
1. General templates What,when,how,why
How does Gustave dance? What are Gustave’s ideas?
Why did Gustave have a wonderful time?
2. Emotions – why…? Ever feel the opposite?
Why was Gustave frustrated?
Is Gustave ever happy?
3. Conflict – ask for resolution
How was the conflict resolved?
Common Sense