make my desktop app great with touch, mouse, and pen
Post on 07-Feb-2016
106 Views
Preview:
DESCRIPTION
TRANSCRIPT
Make my desktop app great with touch, mouse, and pen
Nick WaggonerSenior Program Manager4-022
Demo: Photo annotation
Input and hit-testingWindows touch languagePanning and zoomingPen and ink
Agenda
Where we are todayDeep and mature
Diverse ecosystem
Primarily mouse and keyboard
PC Form Factorshave evolved.
Key platform APIs
Direct2D, DirectWrite
Direct3D
Dyna
mic
DPI
Desktop Applications
DirectComposition
Rem
aind
er o
f Win
32
Real
time
Styl
us
Ink
Reco
gniti
on
InteractionContext
WM_POINTER Family
DirectManipulation
Win32 apps
Windows kernel services
Input and Interaction Rendering and composition
Real
time
styl
us
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
Input and interaction
Input and hit-testing
WM_POINTER
Unified messages for touch, mouse, and pen.EnableMouseInPointer(…)
GetPointerInfo(…) / GetPointerInfoHistory(…)
GetPointerTouchInfo(…) / GetPointerPenInfo(…)
Real
time
styl
usIn
k re
cogn
ition
InteractionContext
WM_Pointer family
DirectManipulation
Touch hit testing
RegisterTouchHitTestingWindow(…)WM_TOUCHHITTESTINGEvaluateProximityToRect(…) or EvaluateProximityToPolygon(…)PackTouchHitTestingProximityEvaluation(…)
Control C Button 1
Button 2Real
time
styl
usIn
k re
cogn
ition
InteractionContext
WM_Pointer family
DirectManipulation
Demo: Moving to WM_POINTER & WM_TOUCHHITTESTING
Windows touch language
Windows touch languageRe
altim
e st
ylus
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
InteractionContextInitialization
CreateInteractionContext(…)SetInteractionConfigurationInteractionContext(…)RegisterOutputCallbackInteractionContext(…)
Feeding inputAddPointerInteractionContext(…) ProcessPointerFramesInteractionContext(…)
Handling output
Real
time
styl
usIn
k re
cogn
ition
InteractionContext
WM_Pointer family
DirectManipulation
void INTERACTION_CONTEXT_OUTPUT_CALLBACK( _In_opt_ void *clientData, _In_ const INTERACTION_CONTEXT_OUTPUT *output );
typedef struct INTERACTION_CONTEXT_OUTPUT { INTERACTION_ID interactionId; INTERACTION_FLAGS interactionFlags; POINTER_INPUT_TYPE inputType; float x; float y; union { INTERACTION_ARGUMENTS_MANIPULATION manipulation; INTERACTION_ARGUMENTS_TAP tap; INTERACTION_ARGUMENTS_CROSS_SLIDE crossSlide; } arguments; } INTERACTION_CONTEXT_OUTPUT;
InteractionContext – Output structures
typedef struct INTERACTION_ARGUMENTS_MANIPULATION { MANIPULATION_TRANSFORM delta; MANIPULATION_TRANSFORM cumulative; MANIPULATION_VELOCITY velocity; MANIPULATION_RAILS_STATE railsState; } INTERACTION_ARGUMENTS_MANIPULATION;
InteractionContext – Output structures
Demo: Integrating the touch language
Panning and zooming
Drives panning and zooming in the systemSlide and pinch
Encapsulates the personalityInertiaBounce effectsSnap-points
DirectManipulationRe
altim
e st
ylus
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
Pre-declare your behaviors/interactions
DirectManipulationRe
altim
e st
ylus
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
Harness the power of Windows with DirectComposition
InitializationIDirectManipulationManagerIDirectManipulationViewport IDirectManipulationContent
Hit-testingSetContact()WM_POINTERDOWN / DM_POINTERHITTEST
Manipulation beginsWM_POINTERCAPTURECHANGED
DirectManipulation – Key interfaces/messages
Real
time
styl
usIn
k re
cogn
ition
InteractionContext
WM_Pointer family
DirectManipulation
Adding panning and zooming
Best performance by integrating DirectManipulation with a 3 thread model:
UI threadDelegate threadIndependent hit-test thread
IDirectManipulationManager::RegisterHitTestTarget(…)
DirectManipulation - threadingRe
altim
e st
ylus
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
Pen and ink
Independent of the UI threadHandwriting recognition
RealTimeStylusRe
altim
e st
ylus
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
Initialization IRealTimeStylus
AddStylusSyncPlugin(…)Processing Input
IStylusSyncPlugin StylusDown(…), Packets(…), StylusUp(…)
IStrokeBuilderBeginStroke(…), AppendPackets(…),
EndStroke(…)Handwriting Recognition
InkRecognizerContextRecognize(…), BackgroundRecognize(…), BackgroundRecognizeWithAlternates(…)
RealTimeStylus - Key interfaces/methods
Real
time
styl
usIn
k re
cogn
ition
InteractionContext
WM_Pointer family
DirectManipulation
RealTimeStylus - Key interfaces/methods
Real
time
styl
usIn
k re
cogn
ition
InteractionContext
WM_Pointer family
DirectManipulation
IRealTimeStylus
IStylusSyncPlugin
IStrokeBuilder
InkRecognizerContext
Adding ink and recognition
Performance is critical for both inking as well as panning and zooming experiences.
Recap
Use WM_POINTER for touch, mouse, and pen.
Use the InteractionContext to incorporate the Windows touch language into your app.
Use DirectManipulation for great panning and zooming.
Differentiate your app by making ink pop with the RealTimeStylus APIs.
TakeawaysRe
altim
e st
ylus
Ink
reco
gniti
on
InteractionContext
WM_Pointer family
DirectManipulation
ResourcesRelated talks
3-017 - Power up your desktop app with Windows 4-021 - DirectComposition: Smooth composition and animation for desktop apps4-9147 – Exploring customized touch interactions for your Windows store app
Reference documentationWM_POINTER - http://msdn.microsoft.com/en-us/library/windows/desktop/hh454916(v=vs.85).aspx InteractionContext - http://msdn.microsoft.com/en-us/library/windows/desktop/hh437192(v=vs.85).
aspx DirectManipulation - http://msdn.microsoft.com/en-us/library/windows/desktop/hh446972(v=vs.85).
aspx RealTimeStylus - http://msdn.microsoft.com/en-us/library/windows/desktop/ms701683(v=vs.85).aspx InkRecognizerContext -
http://msdn.microsoft.com/en-us/library/windows/desktop/ms696371(v=vs.85).aspxProcess touch input with DirectManipulation sample
http://go.microsoft.com/fwlink/?LinkID=309852&clcid=0x409Design guidelines
Touch target sizes - http://msdn.microsoft.com/en-us/library/windows/apps/hh465415.aspx#touch_targets
Device distribution starts after sessions conclude today (approximately 6:00pm) in the Big Room, Hall D. If you choose not to pick up your devices tonight, distribution will continue for the duration of the conference at Registration in the North Lobby.
Get your goodies
Acer Iconia W3, Surface Pro, and Surface Type Cover
Evaluate this session
Scan this QR code to evaluate this session and be automatically entered in a drawing to win a prize!
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
top related