![Page 1: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/1.jpg)
Build Your Own Multi-Touch Interface with Java and JavaFX Technology
Simon Ritter, Sun MicrosystemsAngela Caicedo, Sun Microsystems
TS-6127
![Page 2: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/2.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 2
Learn how to build a JavaTM technology-powered touch screen that recognises multiple touch points. Also see how easy it is to use JavaFXTM technology to build user interfaces that work with multi-touch input.
![Page 3: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/3.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 3
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 4: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/4.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 4
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 5: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/5.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 5
Touch Screen Basics
Several different approaches• Special surface coating - capacitive, resistive• Surface acoustic wave, acoustic pulse• Optical – Frustrated total internal reflection
Most screens only respond to single touch point• Touch replaces use of mouse
Multi-touch is becoming more popular• Apple iPhone• Samsung multi-touch display• Microsoft table
![Page 6: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/6.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 6
Frustrated Total Internal Reflection
Similar concept to fibre optic cableTouching screen causes IR to appear where touched
Web-cam
Infra-RedSource (LED)
Finger
Acrylic Sheet
ScreenProjector
![Page 7: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/7.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 7
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 8: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/8.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 8
Touch Surface
Thick acrylic sheet• We used 6mm thick, 50x40cm
Edges need to be polishedWe used a three stage process• Back of a hacksaw blade• Very fine sandpaper• Car polish
Screen for display needs to be separate• Coating the underside of the touch surface was a complete disaster• Used a separate 3mm thick acrylic sheet covered in tracing paper
![Page 9: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/9.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 9
Infra-Red Source/Detection
Infra-red LEDs mounted along edge of screen• Initially mounted one LED every 7mm• This was reduced to one LED every 3cm (lower power consumption)
Webcam mounted under screen to observe touch points• Webcams can see infra-red, unlike humans• Most webcams have an infra-red filter which must be removed
• This can be difficult (we destroyed several webcams!)
• Replace this with a filter that only lets IR through• Use the filter from a remote control
• Tricky bit is getting the webcam far enough away to see whole screen• Wide angle lens makes this easier
![Page 10: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/10.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 10
Infra-Red LED Wiring
LEDs wired in parallel100 Ohm resistor for each LED • Good, bright LED• Not too much power consumption• Not too hot
![Page 11: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/11.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 11
Infra-Red LED Mounting
Each LED machined flatBetter optical connectivity to touch surface
![Page 12: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/12.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 12
Touch Surface
IR LEDs
![Page 13: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/13.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 13
Touch Surface with Screen
Display screen
Touch surface
IR LEDs
![Page 14: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/14.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 14
Webcam
Without filter With IR bandpass filter
![Page 15: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/15.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 15
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 16: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/16.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 16
Getting an Image From a Webcam
Java Media Framework (JMF) API• The forgotten Java API, last update 2003
Version 1 only for playback, version 2 introduced captureStill works well for SolarisTM (SPARC®) technology, Linux and Windows• Pure Java technology version available• Performance packs for specific OS• Decision was to use Ubuntu Linux for driver support
Performance was not an issue• Able to grab and process images at suitably high frame rate
![Page 17: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/17.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 17
JMF API Frame Grabbing
Locate device with CaptureDataManagerGet DataSource through MediaLocatorSet FormatCreate and Realize ProcessorCreate a PushBufferDataSource• We really want a PullBufferDataSource, but can't have one
Get BufferStream through DataSourceRead BufferConvert Buffer to BufferedImage with BufferToImageConvert BufferedImage to RGB array with PixelGrabber
![Page 18: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/18.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 18
Image Processing: Stage 1
Image from camera was not good for processing• Camera had automatic adjustment of white balance, contrast, etc.
Use webcam device driver ioctls• Turn off automatic adjustment• Retrieve current settings• Manually change settings
Driver needed modification• Yay open source!
![Page 19: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/19.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 19
Image Processing: Stage 2
Java 2DTM and Java Advanced Imaging APIConvert to 8-bit greyscale for simplified processing• ColorConvolveOp
BufferedImage is really useful for this kind of processing
BufferedImage
ColorModel Raster
SampleModel
DataBufferColorSpace
![Page 20: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/20.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 20
Brightness and Contrast Adjustment
Create a ByteLookupTableApply LookupOp filter
Image Pixel Values
Filte
red
Pix
el V
alue
s
Window
Window / 2
Level
LookupOp lop = new LookupOp(byteTable, null);filteredImage = lop.filter(oldImage, null);
![Page 21: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/21.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 21
Image Processing: Stage 3
Identify where bright spots are in imageAlgorithm is basically simple• Although certain cases make life a bit harder
Ensure image is mono-chrome, each pixel is white or black• 8-bit greyscale -> 1 bit mono-chrome• Tunable level for change from black to white (0-255)
Sum pixels in each row and column (white = 1, black = 0)Rows or columns with high values indicate touch pointGenerate a set of x, y co-ordinates for all touch points
![Page 22: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/22.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 22
2 4 2 3 3 3
233
333
Image Processing: Stage 3
Points can be reversedTest possible locations to get accurate result
x point 1 x point 2
y point 1
y point 2
![Page 23: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/23.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 23
Generating Touch Events
We now have a set of touch point co-ordinatesSimple events are somewhat like mouse events• Change of position• New touch point• Touch point disappeared
Touch point code is more complex than mouse• Must track points and figure out when a new one appears• Or an existing one disappears
Gesture recognition is even more complex• Swipe movement• expand/shrink/rotate using two points• Requires time-based analysis
We defined an extendable touch event interface
![Page 24: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/24.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 24
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 25: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/25.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 25
JavaFX Technology Basics
Programming Language for the Java PlatformObject-orientedDeclarative SyntaxStatically-typed with type-inferenceAutomatic data bindingExtensive Widget library encompassing Swing and Java 2D APIDevelopment tools including NetBeansTM and Eclipse IDE plugins
![Page 26: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/26.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 26
Java/JavaFX Technology and Multi-Touch Software Integration
Why JavaFX technology:• Great for image manipulation• Data binding simplifies the software implementation• Simple and easy to design and implement the user interface
Why Java technology:• Heavy multi-threading implemented in Java technology• Java based model bound to the interface with JavaFX technology
capabilities• Model change <-> view change
Why Multi-touch:• Intuitive user interaction• Multiple point of interactions• Allows multiple users interacting with the software• Cool and interesting technology
![Page 27: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/27.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 27
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 28: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/28.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 28
Multi-Touch Events
They capture:• Initial position (finger down)• Direction (finger trajectory)• Speed (finger speed)• Final position (finger up)
Dynamically associated with SmartObjectDynamically released from SmartObjectsUniverse vs. individual interactionDirection = 0 and speed = 0 then locking eventsSpeed > threshold then animation triggered
![Page 29: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/29.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 29
Smart Objects
Register multi-touch eventsObject's behavior based on multi-touch events and areas of interactions
rotating &stretching
rotating &stretching
rotating &stretching
rotating &stretching
rotating,stretching &
flipping
rotating,stretching &
flipping
rotating,stretching &
flipping
rotating,stretching &
flipping rotating,
![Page 30: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/30.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 30
Stretching with Locking Action
![Page 31: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/31.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 31
Both Sides Stretching
![Page 32: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/32.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 32
Flipping
![Page 33: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/33.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 33
Locking and Rotating
![Page 34: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/34.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 34
Free Rotation
![Page 35: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/35.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 35
Putting Things Together
![Page 36: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/36.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 36
Universe Interaction (1 of 2)
![Page 37: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/37.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 37
Universe Interaction (2 of 2)
![Page 38: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/38.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 38
Smart Objects Still Active (1 of 2)
![Page 39: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/39.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 39
Smart Objects Still Active (2 of 2)
![Page 40: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/40.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 40
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 41: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/41.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 41
Where Next?
Infra-red is very useful as we can't see it• No interference with rest of display
Wiimote from Nintendo has IR detector• Can track up to four points simultaneously• Reports positions via bluetooth interface• Used to figure out where the Wiimote is relative to the display
Why not keep the Wiimote stationary, move the IR?Great examples of this• Johnny Chung Lee of CMU• 3D position interpolation
Working on combining this with touch screen• Using wiiremotej open source project
![Page 42: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/42.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 42
Agenda
Basic ideas of the multi-touch displayConstruction of the displayUsing Java technology to recognise touch pointsJavaFX technology integration with multi-touch eventsJavaFX technology multi-touch user interfacesFuture directionsSummary and further information
![Page 43: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/43.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 43
Summary
Multi-touch screens are simple and cheap to build• Most expensive bit is the projector
Java technology makes the software easyJavaFX technology is easy to integrate with new types of eventJavaFX technology makes building multi-touch user interfaces simpleThis is just the beginning...
![Page 44: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/44.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 44
For More Information
Java Media Framework• java.sun.com/products/java-media/jmf/
Java Advanced Imaging• java.sun.com/javase/technologies/desktop/media/jai/
JavaFX technology• openjfx.org
Jeff Hahn (FTIR multi-touch screen)• cs.nyu.edu/~jhan/ftirtouch/• www.perceptivepixel.com/
Building Imaging Applications With Java Technology• Lawrence H. Rodrigues
![Page 45: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/45.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 45
Multi-touch screen in action with JavaFX technology
![Page 46: Build Your Own Multitouch Interface with Java and JavaFX Technology… · Build Your Own Multitouch Interface with Java and JavaFX Technology, TS-6127, JavaOne 2008 Subject: 2008](https://reader030.vdocuments.net/reader030/viewer/2022041005/5eaa9b38a568e12f674e19b1/html5/thumbnails/46.jpg)
2008 JavaOneSM Conference | java.sun.com/javaone | 46
Simon Ritter, Sun MicrosystemsAngela Caicedo, Sun Microsystems
TS-6127