unique two-handed interaction techniques
TRANSCRIPT
Unique Two-Handed Interaction Techniques
Manipulating graphic objects on large displays
By:
Martin A. N. Khamala
Student
Matrikelnummer: 952366
Interactive Media Systems (MA)
Faculty of Art and Design
Hochschule Augsburg, Augsburg University of Applied Sciences (AUAS)
Friedberger Str. 2, D - 86161 Augsburg
Email: [email protected]
Mobile: +49 1520 536 9830
Antony Sande
Student
Matrikelnummer: 952374
Interactive Media Systems (MA)
Faculty of Art and Design
Hochschule Augsburg, Augsburg University of Applied Sciences (AUAS)
Friedberger Str. 2, D - 86161 Augsburg
Email: [email protected]
Mobile: +49 1520 536 9176
This report is submitted in partial fulfillment of the requirement of the Interactive Media Systems
Masters course in Design.
Date: 5-February, 2016
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 2 | 31
Table of Contents
Acknowledgements 3
Abstract 4
1.0 Introduction 5
2.0 A use case in Education 6
3.0 Methodology 7
4.0 Findings 15
5.0 Conclusion and Extended Work 16
6.0 Appendices: Demo Video 17
7.0 Appendices: Research Instruments 18
8.0 Bibliography 31
List of Figures
Figure 1: Cue the system from inside the near zone: 12
Figure 2: Highlight and select from the near zone: 12
Figure 3: Move Transformation from the near zone: 12
Figure 4: Highlight and select: 13
Figure 5: Scale Transformation: Figure 13
Figure 6: Cue the pointing zone: 14
Figure 7: Highlight and Select in the pointing zone: 14
Figure 8: Transform the object using two-handed gestures. 14
List of Tables
Table 1: Sampling Table of success 11
Table 2: Sampling Table of success 15
List of Charts
Chart 1: Comparison sheet of success rates between activity 1 and 2 16
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 3 | 31
ACKNOWLEDGEMENTS
We acknowledge with appreciation the contribution of all the people who played a part in the success of
this study. Special mention goes to our lecturer and project supervisor, for his assistance with the
proposal ideation, conceptualization, planning, development and evaluation over the semester.
1. Prof. Dr. Michael Kipp Lecturer and course supervisor, Interaction Engineering
Also, deserving of mention are the following officers,
2. Herald Lesti The Faculty of Informatik
3. Prof. Dr. Phil. Doris A. Binger: Text, Culture and Media Studies, International Faculty Coordinator
University of Applied Sciences Augsburg and Faculty of Design
4. Finally, we wish to acknowledge the following students who accepted to participate in evaluating our
prototype:
Christian Spiegel, WirschaftInformatik student
Stefan Berner, Interactive Media Systems (MA) student
Stefan Reichinger, Interactive Media Systems (MA) student
Patrick Schroeter, Interactive Media Systems (MA) student
Sebastian Harter, Interactive Media Systems (MA) student
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 4 | 31
ABSTRACT
Gesture interactions are being deployed increasingly, in Human Computer Interaction, in an attempt to
create more natural interfacing with the plethora of new digital environments. If these new applications
are to successfully enter into use, in our day to day lives, their development must be underpinned in
equal measure, by robust and intuitive interfacing methods. This research implementation looks at two-
handed spatial gestures and moots the question that seeks to inherit the merits of isotonic gesture
recognition on touch screens, in physical space.
The advantage of evolving unique gesture sets, from 2D isotonic interactions is that there exists the real
possibility, that the new styles will be easy to learn and intuitive to use. This research explores this
possibility by building a prototype that responds to two-handed gestures sets. The underlying purpose is
to construct manipulation styles to transform computer graphics. These styles are comprised of unique
combinations of two-handed gesture inputs, location data, and proximity information. To keep the
system as intuitive as possible, this implementation draws from the extensive library of information
available in two finger isotonic gesture actions.
The prototype is evaluated in terms of gesture recognition and orientation, while its use is demonstrated
with the execution of two object transformations: specifically, move and scale. Users will be tested on
how successfully they manipulate the graphics using the prototype’s predefined styles.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 5 | 31
1.0 INTRODUCTION
Background
This study proposes to examine natural and guided human interaction with 2D objects on a projected
display. Emphasis will be laid on using proxemic interaction and two hands to simulate two finger
isotonic gestures in physical space. Input data will be captured using a motion sensing device. The data
throughput will be processed using a gesture recognition method and output to manipulate graphic
objects using “move” and “scale” transformations.
1.1 The Opportunity for Research
During presentations, there is often a need to hold viewer’s attention while navigating on-screen
information by repositioning, enlarging, rotating or revealing an area of focus. Any break from the
presentation to access a laptop to interface with the information comes at a cost of attention by placing
demands on viewer's physical and cognitive resources.
This research study’s innovation is the evolution and implementation of unique manipulation styles that
provide natural two-handed interfacing while transforming graphics on a large projected display area.
1.2 The Research Question
The contention is, whether the physical manipulation of computer graphics using two-handed
manipulation styles on large displays can benefit from leveraging the merits of isotonic gestures already
in widespread use.
The hypothesis asserts that Human Computer Interaction techniques are translatable from a 2D realm,
as with multi-touch screens to a physical 3-dimensional space without the loss of the natural know-how
in intuitively manipulating graphic objects. The researchers proffer the view that, to meaningfully
emulate 2D gesture recognition, manipulation styles in physical space must take into consideration
location, distance, and orientation input data to cue and execute a unique combination of gestures.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 6 | 31
1.3 The Objective
The objective of this research implementation is to investigate the suitability of using natural, physical
two-handed gestures that mimic 2D isotonic gestures, for use in transforming a rectangular object on
large display screens.
1.4 The Scope
The scope of the project is limited to building a prototype to demonstrate Human Computer Interaction
techniques and testing the validity of the assertion made in the hypothesis about two-handed gesture
movements.
The project’s success will be determined by the complete implementation of the prototype, the
demonstration its function and the completion of the documentation of the project findings from
usability tests.
1.5 Assumptions
The main assumption that presentation methods do not go far enough in offering presenter’s ways to
hold viewer attention on specific areas as they navigate areas on-screen will be tested. Often time’s
presenter’s retreat to their laptops to manage viewer prompted manipulation interactions involving
repositioning, zooming, rotating and revealing of information.
2.0 Use case for Education
In blended teaching environments, teachers using presentation technology in the classroom experience
few navigational problems when the presentation is linear. But what happens during a question and
answer session when to aid in expounding a concept, the teacher needs to zoom into and reposition a
particular part of a diagram on-screen. User prompted explanations such as these, are usually preceded
by a trip to and from a laptop. Software on laptops allows the presenter to set the navigation style
accordingly. Should the explanation require a reference from another part of the diagram, another trip to
and from will be prompted. Herein lies a situation in which learner attention is broken each time.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 7 | 31
3.0 METHODOLOGY
The development of the interface technique
i) Through a series of brainstorming sessions, it was determined that the simplest hand gestures to effect
two-handed interactions are those that mimic isotonic gestures, which through widespread usage over
the passage of time, have become intuitive to most users.
ii) The next step determined the implementation concept for the project. The idea was to project a graphic
onto a large display area. An individual designated to a focus group test would move the object using his
hands.
iii) The next step was to determine a set of manipulation styles correlating to paired gesture actions that
allow for selection in both the near and pointing zone in the experiment. From the sensor hardware
limitations, the near zone was set to 30 cm from the display area and the pointing zone was set from
30cm to 3m the display area.
The manipulation styles that were used were segmented according to proximity zones:
In the near zone (set at 0-30 cm)
a) Orientation: use facial recognition data to determine intent. The idea being to establish that the user is, in
fact, facing the presentation and, therefore, intends to interact with it.
Proxemics: use distance input data to determine that the user is in the near zone.
Highlight: use the location data of the first hand, as input data to determine whether there is a hit
between the hand and the object.
Select: use a closing fist gesture, made with the other hand-form below the waist line, as a grabbing
action to select.
Move: use location input data, to detect the motion of the first hand, to update the position of the
projected graphic object.
Scale: use location input data, to detect the distance between both hands, to update the distance between
hands along a connection line between them.
In the pointing zone (set at 30-300 cm)
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 8 | 31
iv) Orientation: use facial recognition data to determine intent. The idea being to establish that the user is, in
fact, facing the presentation and, therefore, intends to interact with it.
Proximity: use distance input data to determine that the user is in the pointing zone.
Highlights: using the location data of the first elbow and hand as input data to determine whether there is
a hit between the arms projection and the object.
Select: using a closing fist gesture, made with the other hand-form below the waist line, as a grabbing
action to select.
Move: using location input data, to detect the motion of the first arm, to update the position of the
projected graphic object.
Scale: using location input data, to detect the distance between both hands, to update the distance
between hands along a connection line between them.
v) Next, a developer must determine a methodology for calibrating the screen to let the correlate the
projected display to the virtual space.
The scheme of work.
To program a pair of hand gestures, a developer must track each hand's movement change. This proved
to be tricky when you consider that the Kinect sensors tracking system at times failed to track the hand
movement. This irregular sensor detection resulted in inconsistent event handling on the code. For
example, the coordinates for any hand movements close to the limit of the kinetics’ field of view would
be dropped from the motion tracking feedback.
To decide on when to draw on the screen, a developer must track the hand action. To confirm that a
selection between the real hand the virtual graphic object has been registered,
The developer must analyze the hand's real position and relate its position in virtual space. This is done
using a linear interpolation formula, that relates at any one time, to one of the four corners of the display
box.
This step is repeated rapidly to update the graphic position as often as possible for the action of:
a) Moving the box
A developer must calculate the new position of the hand in relation to the last position. This describes a
vector in the line of motion of the hand and can be used to reposition the graphic object.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 9 | 31
b) Scaling the box
A developer must capture the coordinates of both hands and track the change in distance between them.
vi) UX testing
The size of our sample group was 5 respondents. All are male and are attending university.
The user context for the test was a medium sized room of approximately 12 x 12m.
The ambiance was quiet.
Each test comprised two activities, and each activity comprised of 9 tasks. (Ref. Appendix for details).
The results for each respondent was documented after the test.
3.2 Interaction hardware and software
We expect to make use of:
- Microsoft Kinect v2 Sensor device, for live motion data capture.
- A projector to build a prototype demonstrating the projects outcomes.
- A laptop to create a desktop application to analyze and implement the new gesture data.
Prototype
In order to track the presenter’s body, the Microsoft Kinect v2 Sensor, an integrated set of sensors
designed to detect human movement, voice and gesture, was used. Its depth sensing technology, a high-
definition colour camera, and the infra-red emitter were the features used in the prototype.
The prototype is a Windows 8.1 App, with the interface developed using XAML (Extensible Application
Markup Language) and the logic developed in C# programming language using Microsoft Visual Studio
2013, and it was built on the Kinect for Windows Software Development Kit (SDK) 2.0.
The KinectSensor exposes a set of Sources that provide streams of a specific data type. Typically a
Reader is attached to a Source and the FrameArrived event of the Reader is subscribed to. Using this, we
were able to subscribe to the BodyFrameSource which allowed us to track all the body joints at an
output range of 30 Frames per Second (FPS). Since the Frame objects are being created and destroyed at
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 10 | 31
a rapid rate, we incorporated Reactive Extensions, which are a great way to handle the torrent of event
data that the Kinect generated. Adding a reactive extension layer above the raw Kinect event handlers
allowed us to abstract away the messy details and simplify the application significantly.
The Kinect Sensor tracks up to 6 people, but we only needed one person to control the presentation. The
BodyFrame returns the set of bodies that the Kinect is currently tracking. From the array of bodies, we
then calculated the closest body to the presentation (by comparing distance data). We then passed the
reference onto a second Source; the HighDefinitionFaceFrameSource, which allowed us to access the
face alignment of the tracked user. With this were able to know when they were facing the presentation.
We then tracked the user's hands, spine, wrists and elbows coordinates, which we used in manipulating
the object.
The setup involved a Kinect Sensor mounted on the ceiling beside a projector that was projecting
presentation on a wall. The Kinect sensor was placed approximately 3.5 Metres from the wall.
In order for the Kinect to detect the presentation area, the user had to perform calibration first by
touching each of the four corners of the presentation area with their right hand. Then the system will
map the presentation area with the Kinect’s Coordinates. After that, the user can start interacting with
the presentation by using his hands as a pointer. In order to grab the object, we monitored the
HandStates from the Kinect and detected whether it was Open or Closed (object grabbed).
3.3 Mapping Methods
We suggest the following interface interactions that combine distance and location proximity input data
with two unique spatial gestures, that will allow presenters to intuitively navigate on-screen information.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 11 | 31
Table 1 - Sense, select, and transform gestures
Event
Application
function
Manipulation Style
Interaction
outcome Cue Technique
Proximity input Gesture
combination
Cue the interaction
from the near zone
Orientation is facing the display at a
distance of 0 – 30 cm from the display
screen
Hold hand over the
selectable object
Selectable object
highlights
Cue the interaction
from the pointing
zone
Orientation is facing the display at a
distance of 0.3 – 3m from the display
screen
Point to the
selectable object
Selectable object
highlights
Move object with
selecting hand Distance of hand from display
Touch or hover
with hand and
move
Move the display
screen
Enlarge object with
selecting hand Distance of hand from display
Touch or hover
with one hand and
move the other arm
up and down
Scale the display
screen with selecting
hand as anchor and
the other hand as
scale slider
Move object with
selecting hand
Determines the (x, y, z) coordinates of
the shoulder and hand
Point to select, and
then concurrently
move the left hand
up/down
Calculates the target
position of the
pointing hand
Enlarge object with
selecting hand
Determines the (x, y, z) coordinates of
the shoulder and hand
Point to select, and
then concurrently
tilt the hand
up/down
Calculates the
amount of zoom and
is applied
progressively.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 12 | 31
3.4 Manipulation style list
Figure 1: Cue the system from inside the near zone: Use the first hand using an open hand gesture.
Figure 2: Highlight and select from the near zone: Select the object with the first hand (left) and use the
second (right) hand to make a grabbing fist gesture.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 13 | 31
Figure 3: Move Transformation from the near zone: Move the object by repositioning it using the first
hand. Note the second hand is still in a fist action to keep the selection valid.
Figure 4: Highlight and select: Use the first hand (left) to hover over the graphic and the second (right)
hand to make a selection by using a grabbing action.
Figure 5: Scale Transformation: Open arms apart to enlarge and bring them together to reduce.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 14 | 31
Figure 6: Cue the pointing zone: Step away from the display to stand between 30 to 300cm.
Figure 7: Highlight and Select in the pointing zone: The first-hand highlights the second-hand selects.
Figure 8: Transform the object using two-handed gestures. Move the first hand while the other is in a fist
to reposition the graphic. And move both hands simultaneously to reduce or apart to enlarge.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 15 | 31
4.0 FINDINGS
The respondents were guided on how to complete an activity and were then timed to see how long it
took to complete a task. An early finishing time (under 2 minutes) done correctly translated to a perfect
score of 2, whereas a late finishing time completing the tasks after 2 minutes translated to a score of 1.
Completing the task after 2 minutes in more than 1 attempt or not completing the task translated to a
score of 0.
Activity 1 Highlight using first hand, select using second hand, reposition by moving first hand
Activity 2 Highlight using first hand, select using second hand, scale by moving both hands
For both activities, the active Orientation is facing the display
Table 2: Sampling Table of success
Activity 1 Activity 2
Name Score Attempts
Success Score Attempts
Success
rate rate
1 Sebastian Spiegel 14 8 1.75 14 13 1.08
2 Stefan Berner 14 8 1.75 11 13 0.85
3 Sebastian Harter 16 8 2.00 19 11 1.73
4 Patrick Schroeter 11 8 1.38 15 8 1.88
5 Stefan Reichinger 11 15 0.73 12 17 0.71
Score guide Success rate scale 2.00 perfect
1.5 to 2 good
1.0 to 2 above average
0.5 to 1 average
0 to 0.5 below average
All respondents are male, between the age of 24 to 30 years and university students.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 16 | 31
Activity 2
Activity 1
i. Most respondents achieved a higher success rate with Activity 1 which was done in close proximity to
the display area, far easier than activity 2.
ii. The comparison of both activity results appeared to be consistent as they are outside the margin of error.
iii. More than half the respondents scored an above average score in activity one, whereas more than half
scored a below average performance in activity 2.
3 CONCLUSION AND EXTENDED WORK
From the analysis of the results, it seems that the manipulation of the graphics was achieved from close
proximity with less effort that at a distance using the pointing action. From respondents comments on the
questionnaire, it would seem the difficulty arose from the unsteadiness of the pointer action.
Respondents, in general, attributed the failure to complete the tasks quickly was attributed to a difficult
selection process. This could be attributed to the sensor when provided inconsistent tracking
functionality.
From the completion of the exercise to the point of data gathering means the objective was achieved and
the outcome, to implement and test the hypothesis was successful.
In closing, the manipulation of graphic objects was not as natural compared to isotonic gesture action.
The selection and movement action were made more difficult by hardware limitations.
On the matter of the two handed selection approach, respondents, in general, felt it was not natural, and a
one handed approach would be more intuitive.
-
0.50
1.00
1.50
2.00
2.50
SebastianSpiegel
Stefan Berner SebastianHarter
PatrickSchroeter
StefanReichinger
Suce
ss r
ate
sco
res
Names of respondents
Chart 1: Comparison sheet of sucess rates between activity 1 and 2
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 17 | 31
4 APPENDIX
4.1 Link to video demonstration: https://vimeo.com/154373014
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 18 | 31
4.2 Questionnaire template
Research Instrument:
Data capture sheet No:
Project Title: Date:
Two-Handed Interaction
Techniques 29 Jan 16
Supervisor: Researchers:
Test: iteration No. A. Sande
Prof. Michael Kipp M. Khamala
Most Viable Product 1 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interactive
Engineering
Att
emp
ts
Activity 1 Activity 2
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
1
2
3
Activity 1
Task 1: Orientation: Face away from the display in the neutral zone, then turn to the display
Task 2: Highlight object - single-handed gesture, standing less than 30cm from the display zone
Task 3: Select object - 2 hand gesture, make a fist with the other hand while highlighting with the first hand.
Task 4: Move the object to a new location on the display using the highlighting hand.
Task 5: While still in the display zone, highlight the object again using a single-handed gesture.
Task 6: Select the object by making a fist using the other hand while highlighting with the first hand.
Task 7: Enlarge the object by parting hands in a 2 handed gesture.
Task 8: Reduce the object size by bringing hands together in a 2 handed gesture.
Activity 2
Task 1: Orientation: Face away from the display in the neutral zone, then turn to the display
Task 2: Highlight object - single-handed gesture, standing in the pointing zone: 30cm to 3m from the display.
Task 3: Object selection - 2 handed gesture, make a fist with the other hand while pointing with the first hand.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 19 | 31
Task 4: Move the object to a new location on the display
Task 5: While in the pointing zone, highlight the object by pointing again using a single-handed gesture.
Task 6: Select the object by making a fist using the other hand while pointing with the first hand.
Task 7: Enlarge the object by parting hands in a 2 handed gesture.
Task 8: Reduce the object size by bringing hands together in a 2 handed gesture.
Performance Index:
2: complete success
1: moderate success
0: failure
Recorded by: Name Signature:
Research Instrument:
Questionnaire No:
Project Title: Date:
Towards natural,
2-handed gestures for 2D
interactions
29 Jan 16
Supervisor: Researchers:
Test: iteration No. A. Sande
Prof. Michael Kipp M. Khamala
Most Viable Product 2 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interactive
Engineering
The Interaction prototype is a 2D interaction system that uses 2 hand interfacing to move and scale screen objects.
The user objective during the experiment was to move the object around and as quickly and efficiently as possible.
Having completed the activities, please answer the questions below:
1 Have you interacted with sensor technology before, to manipulate or navigate content?
Yes No A little
2 To what extent do you feel that the prototype was helpful in moving the screen object.
Very helpful Helpful Not helpful impossible
3 How natural where the 2 hand gestures in manipulating the object.
Very natural natural Not natural unnatural
4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of
efficiency.
5 Little effort needed Some effort needed A lot of effort needed Too much effort needed
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 20 | 31
6 Which activity was easier for you to execute?
Activity 1 while in the display zone Activity 2 while in the pointing zone
Explain:
7 Which task was most difficult for you to execute?
Ref. the back of the sheet for the task list.
Explain.
8 What would you suggest as improvements to the whole system?
9 Is there a user case in which you see yourself using the prototype system?
Thank you.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 21 | 31
5.2.1 User 1
Research Instrument:
Data capture sheet No: 001
Project
Title: Date:
Two-Handed Interaction
Techniques 29 Jan 16
Supervis
or:
Resear
chers:
Test: iteration No. A.
Sande
Prof.
Michael
Kipp
M.
Kham
ala Most Viable Product 2 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interac
tive
Sebastian Spiegel 30 Male Right Engin
eering
Att
emp
ts
Activity 1 Activity 2
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
1 2 2 2 1 2 2 1 2 2 2 2 1 1 1 0 2
2 0 1 1 1
3 0
Having completed the activities, please answer the questions below:
1 Have you interacted with sensor technology before, to manipulate or navigate content?
Yes No x A little
2 To what extent do you feel that the prototype was helpful in moving the screen object.
Very helpful x Helpful Not helpful impossible
3 How natural where the 2 hand gestures in manipulating the object.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 22 | 31
Very natural x natural Not natural unnatural
4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of efficiency.
5 Little effort needed
x Some effort needed
A lot of effort needed
Too much effort
needed
6 Which activity was easier for you to execute?
x Activity 1 while in the display zone Activity 2 while in the pointing zone
Explain:
At the Activity second I can't control the point. It flickered too much
7 Which task was most difficult for you to execute?
4, act. 2
Ref. the back of the sheet for the task list.
Explain.
Because I can't move it I can't grab it good.
8 What would you suggest as improvements to the whole system?
Not answered
9 Is there a user case in which you see yourself using the prototype system?
Not answered
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 23 | 31
Research Instrument:
Data capture sheet No: 002
Project Title: Date:
Two-Handed Interaction
Techniques 29 Jan 16
Supervisor: Researchers:
Test: iteration No. A. Sande
Prof. Michael Kipp M. Khamala
Most Viable Product 2 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interactive
Engineering
Att
emp
ts
Activity 1 Activity 2
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
1 2 2 2 2 2 2 1 1 2 2 0 1 0 0 0 0
2 0 1
3 1 2 2
Having completed the activities, please answer the questions below:
1 Have you interacted with sensor technology before, to manipulate or navigate content?
Yes No x A little
2 To what extent do you feel that the prototype was helpful in moving the screen object.
Very helpful x Helpful x Not helpful impossible
3 How natural where the 2 hand gestures in manipulating the object.
Very natural x natural Not natural unnatural
4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of
efficiency.
5 Little effort needed Some effort needed x A lot of effort needed Too much effort needed
6 Which activity was easier for you to execute?
x Activity 1 while in the display zone Activity 2 while in the pointing zone
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 24 | 31
Explain:
Tracking worked on the first try
7 Which task was most difficult for you to execute?
Ref. the back of the sheet for the task list.
Explain.
Tracking of arm didn't work well
8 What would you suggest as improvements to the whole system?
Additional Hardware may be needed on your wrist for better tracking
9 Is there a user case in which you see yourself using the prototype system?
Maybe at exhibitions and conventions
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 25 | 31
Research Instrument:
Data capture sheet No: 003
Project Title: Date:
Two-Handed Interaction
Techniques 29 Jan 16
Supervisor: Researchers:
Test: iteration No. A. Sande
Prof. Michael Kipp M. Khamala
Most Viable Product 2 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interactive
Sebastian Harder 25 Male Right Engineering
Att
emp
ts
Activity 1 Activity 2
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1
2 2 2 2
3
Having completed the activities, please answer the questions below:
1 Have you interacted with sensor technology before, to manipulate or navigate content?
x Yes No A little
2 To what extent do you feel that the prototype was helpful in moving the screen object.
Very helpful x Helpful Not helpful impossible
3 How natural where the 2 hand gestures in manipulating the object.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 26 | 31
Very natural x natural Not natural unnatural
4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of
efficiency.
5 Little effort needed
Some effort needed
x A lot of effort needed
Too much effort needed
6 Which activity was easier for you to execute?
x Activity 1 while in the display zone Activity 2 while in the pointing zone
Explain:
The selection in Activity 2 was more difficult
7 Which task was most difficult for you to execute?
3, act 2
Ref. the back of the sheet for the task list.
Explain.
Selection didn't work properly (maybe I wasn't in the correct pointing zone)
8 What would you suggest as improvements to the whole system?
I think it would be more natural to use 1 hand while moving, pointing, fist, then move
9 Is there a user case in which you see yourself using the prototype system?
Maybe for interactive presentations
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 27 | 31
Research Instrument:
Data capture sheet No: 004
Project Title: Date:
Two-Handed Interaction
Techniques 29 Jan 16
Supervisor: Researchers:
Test: iteration No. A. Sande
Prof. Michael Kipp M. Khamala
Most Viable Product 2 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interactive
Patrick Schroeter 24 Male Right Engineering
Att
emp
ts
Activity 1 Activity 2
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
1 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 1
2
3
Having completed the activities, please answer the questions below:
1 Have you interacted with sensor technology before, to manipulate or navigate content?
x Yes No A little
2 To what extent do you feel that the prototype was helpful in moving the screen object.
Very helpful x Helpful Not helpful impossible
3 How natural where the 2 hand gestures in manipulating the object.
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 28 | 31
Very natural x natural Not natural unnatural
4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of
efficiency.
5 Little effort needed
x Some effort needed
A lot of effort needed
Too much effort needed
6 Which activity was easier for you to execute?
Activity 1 while in the display zone x Activity 2 while in the pointing zone
Explain:
You have a better overview because of the distance from the screen,
7 Which task was most difficult for you to execute?
3, act. 2
Ref. the back of the sheet for task list.
Explain.
It did not work the way I expected
8 What would you suggest as improvements to the whole system?
More precise hardware
9 Is there a user case in which you see yourself using the prototype system?
Dashboard to move and resize information
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 29 | 31
Research Instrument:
Data capture sheet No: 005
Project Title: Date:
Two-Handed Interaction
Techniques 29 Jan 16
Supervisor: Researchers:
Test: iteration No. A. Sande
Prof. Michael Kipp M. Khamala
Most Viable Product 2 Course: Unit:
Respondent Name: Age Gender Handedness
Interactive Media
Systems (MA)
Interactive
Stefan Reichinger 25 Right Engineering
Att
emp
ts
Activity 1 Activity 2
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
Tas
k 1
Tas
k 2
Tas
k 3
Tas
k 4
Tas
k 5
Tas
k 6
Tas
k 7
Tas
k 8
Act
ivit
y
Co
mp
leti
on
tim
e
(min
:sec
)
1 2 1 1 0 0 0 0 0 2 2 0 0 0 0 2 0
2 1 1 1 1 1 2 0 0 0 0 2 0
3 2 0 1 1
Having completed the activities, please answer the questions below:
1 Have you interacted with sensor technology before, to manipulate or navigate content?
x Yes No A little
2 To what extent do you feel that the prototype was helpful in moving the screen object.
Very helpful Helpful x Not helpful impossible
3 How natural where the 2 hand gestures in manipulating the object.
Very natural x natural Not natural unnatural
4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of
efficiency.
5 Little effort needed Some effort needed x A lot of effort needed Too much effort needed
6 Which activity was easier for you to execute?
Activity 1 while in the display zone x Activity 2 while in the pointing zone
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 30 | 31
Explain:
7 Which task was most difficult for you to execute?
Ref. the back of the sheet for the task list.
Explain.
Selecting the object
8 What would you suggest as improvements to the whole system?
Just better selection and hand detection
9 Is there a user case in which you see yourself using the prototype system?
Presentations
Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 31 | 31
5.0 BIBLIOGRAPHY
1. Towards user-defined multi-touch gestures for 3D objects; Authors: Sarah Buchanan, Bourke Floyd,
Will Holderness, Joseph J. LaViola; Publisher: ACM New York, NY, USA ©2013
2. Toward localizing audiences' gaze using a multi-touch electronic whiteboard with PieMenu; Authors:
Kazutaka Kurihara, Naoshi Nagano, Yuta Watanabe, Yuichi Fujimura, Akinori Minaduki, Kushiro,
Hidehiko Hayashi, Yohei Tutiya; Publisher: (13–16, 2011), Palo Alto, California, USA. Copyright 2011
ACM
3. Adaptive gesture recognition with variation estimation for interactive systems; Authors: Baptiste
Caramiaux, Nicola Montecchio, Atau Tanaka, and Fred´ eric Bevilacqua. 2014. Publisher: ACM Trans.
Interact. Intell. Syst. 4, 4, Article 18 (December 2014),
4. The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies; Authors:
Nicolai Marquardt1, Robert Diaz-Marino2, Sebastian Boring1, Saul Greenberg
Publisher: UIST’11, October 16–19, 2011, Santa Barbara, CA, USA. Copyright © 2011
5. The use and abuse of PowerPoint in Teaching and Learning in the Life Sciences: A Personal
Overview; Author: ALLAN M JONES; Publisher: Life Sciences Teaching Unit, Old Medical School,
University of Dundee, Dundee, DD1 4HN, UK
6. Cognitive Supports for Analogies in the Mathematics Classroom
Authors: Lindsey E. Richland,1* Osnat Zur,2 Keith J. Holyoak2; Publisher: Department of Education,
University of California, Irvine, CA 92697, USA.
7. Virtual sensors: rapid prototyping of ubiquitous interaction with a mobile phone and a Kinect; Lauren
Norrie, Roderick Murray-Smith; Publisher: ACM, August 2011
8. Kinect analysis: a system for recording, analyzing and sharing multimodal interaction elicitation
studies; Authors: Michael Nebeling, Carnegie Mellon University; David Ott, ETH Zurich; Moira C.
Norrie, ETH Zurich; Publisher: ACM New York, NY, USA ©2015
9. Teaching natural user interaction using OpenNI and the Microsoft Kinect sensor
Authors: Norman Villaroman, Dale Rowe, Bret Swan; Publisher: ACM, October 2011