ts-5722 - interactive application development for iptv · ts-5722 - interactive application...

TS-5722 - INTERACTIVE APPLICATION DEVELOPMENT FOR IPTV

Ronan McBrien, Senior Staff Engineer, Sun MicrosystemsSourath Roy, Staff Engineer, Sun MicrosystemsJohn Allen, CEO, Digisoft

2008 JavaOneSM Conference | java.sun.com/javaone | 2

Show how simple compelling Java™ technology-based applications can be written for IPTV set top boxes, and learn about challenges, tips and tricks along the way


Agenda

IPTV EnvironmentDeveloper APIsIPTV Application development• Device Features and Application support• Challenges – Tips 'n' Tricks• Debugging and development environment

DemosQ&A


IPTV Environment

Device Anatomy• RISC Processor (150MHz-300MHz)

• Predominantly MIPS, some ARM, ST40, others• Memory: System, Graphics, Video, HDD

• HDD is optional• IO & Peripherals: Ethernet port, IR, USB port• Limited Graphics acceleration (Blit, Alpha blending, Image

decoding)• Hardware Codecs (MPEG2, MPEG4-2, H.264)• Video output: SD (S-Video, composite), HD (HDMI)

IPTV Use cases• Games• Streaming • PVR


IPTV Environment – End to end Architecture


Agenda


DemosQ&A


Developer APIsCore, I/O, Networking and Graphics

Connected Device Configuration & Foundation Profile 1.1• Core Java Programming Language support• java.net, java.io• Java Archive (JAR) file / Zip utils• Security: Fine grained support plus Security Optional Package 1.0

(Java Authentication and Authorization Service, Java Cryptography Extension, Java Secure Socket Extension)

Personal Basis Profile 1.1• Xlets & Inter Xlet Communication (application life-cycle

management)• Basic AWT Graphics

• Frame• Drawing Primitives• Color Models (Indexed, Direct), Alpha blending, XOR drawing• Fonts & Font Metrics, Image support (decoding, loading and blitting)• Input Events


Developer APIsJava Media Framework API

Component-based APIs for encoding/decoding, rendering and manipulating mediaUse native codecs & hardware acceleration when availableExtensible - 3rd Party components can be deployed in existing systemComponents are generally transparent to the application



Player(s) and DatasourceStreaming extensible content types• ... from files or Network (UDP/RTSP/RTP)• javax.media.rtp, javax.media.rtp.event, javax.media.rtp.rtcp

Presentation: Controls, Latency, UI Component, QualityProcessing: Demultiplexers & Multiplexers, Codecs, Effect filters and RenderersCapture: Devices & Controls



Codec Chain

Manager PlayerCreates Codec

Renderer

MediaLocator

DataSource

Describes

Feeds data to

Codec

ControllerListener

Events

Events


Developer APIsSun™ Java Technology-Based Media Client Architecture


Agenda


DemosQ&A


Display• User Interface

Network• Video Data Streaming

• Unicast & multicast UDP sockets• Live TV and 'As Live'• Virtual Channel select• Remote VOD (RTSP)

Storage – HDD, Peripheral (USB stick/drive)• File Playback• PVR – Recording, seeking, trick-play

Decoders• Rendering Video/Audio codecs• Protocols and BitRate capabilities• Positional support, resizing

IPTV ApplicationsDevice Features


Personal Basis Profile has no widgetsExtend component (add KeyListeners)Group components in containersLeverage LayoutManagersCustomize paint method• Use Images or primitives and Fonts • Customized UI and Look and feel

IPTV ApplicationsApplication support - Lightweight widgets

public class MyTVButton extends Component implements KeyListener {paint() {} // create customized UI component

}


MediaLocator ml = new MediaLocator("rtp://239.255.1.1/video/1");

final Player p = Manager.createPlayer(ml);p.addControllerListener((new ControllerListener(){ public void controllerUpdate(ControllerEvent event){ if (event instanceof RealizeCompleteEvent){ Frame f = new Frame(); f.setLayout(new BorderLayout()); Component c; if ((c = p.getVisualComponent()) != null){ f.add(c, BorderLayout.CENTER); f.pack(); f.setVisible(true); } } }}));p.start();

IPTV ApplicationsApplication support – Playing & Rendering Stream


MediaLocator ml = new MediaLocator("file://home/user/media/movie.ts");final Player p = Manager.createPlayer(ml);...p.start(); //Render at standard rate (1.0)p.setRate(-2.0); //Render at 2x rewindp.setRate(0.0); //Pause player/* getRate also supported */p.setRate(1.0f); // Render at standard ratep.stop();

p.setMediaTime(Time offset); //Set Player Clock locationp.start();/* getMediaTime also supported */

IPTV ApplicationsApplication support – Trickplay


MediaLocator source = new MediaLocator(“rtp://239.255.1.1/video/1”);

Processor p = Manager.createProcessor(source);MediaLocator dest =

new MediaLocator("file://stream.mpeg");DataSink sink =

Manager.createDataSink(p.getDataOutput(), dest);sink.open();sink.start();p.start();

IPTV ApplicationsApplication support – Recording


Avoid making strict thread priority assumptions• Might not be properly acknowledged on some platforms

Use Xlet model• Helps managing application life cycle

Graphics• Use BufferedImage – fast blit for complex components

• e.g. 'MyTVButton' = RoundRect + Fonts + Gradient + Icon• Consider display capabilities

• Not entire screen is necessarily visible - Stick to TV safe area• Draw thick lines (fillRect?)• Use large fonts• Alpha transparency might vary

IPTV ApplicationsChallenges – Tips 'n' Tricks


IPTV ApplicationsTips 'n' Tricks – User Interface & Games

Load all images at startup• Faster image processing at runtime• Host all images on device

Use suitable image formats & color depth• Consider platform capabilities• Limit saturation of colors on NTSC systems

Emulate double buffering• Eliminates work-in-progress impression• Use off-screen image (BufferedImage) for all drawings• Draw off-screen on to primary graphics at paint

Minimize updates on screen• Enhances responsiveness• Track screen area updated by user response• Split screen into active area and static area


IPTV ApplicationsTips 'n' Tricks – User Interface & Games

Alpha blending – not all values may be supportedUsing AlphaComposite.SRC• Ensures Fast drawings (lines, rectangles, images)• Can leverage hardware acceleration in all cases• SRC_OVER rule not required for major parts• JPEG Image format do not have alpha channel• Off-screen in double buffer emulation needs just a copy to screen

Input Events• Add KeyListener at right time

• To ensure consistency addKeyListener after the game is ready• Extra Key Events required for TV (trick play)


Avoid use of processor intensive code• Graphics : Animations (GIF), Alpha blending (SRC_OVER)

Be aware of memory intensive operations• Large Zip files, Image files• Stream data in chunks where possible

Release resources when not used (or no longer required)• Native player resources• Buffered Images

IPTV ApplicationsChallenges – Tips 'n' Tricks – Resource Management


Media support• Quick channel switching

• Avoid Player destruction (heavyweight)• Multiplex DataSource to different streaming URLs

• Caveat: Must be identical format (size, aspect ratio, encoding etc...)Match Media playing to device capabilities• Trick play support• Video positioning and re-sizing

Use Alpha (SRC) blending over VideoBeware picture-in-graphics to full screen switching• Full screen repaint

IPTV ApplicationsChallenges – Tips 'n' Tricks


Netbeans™ 5.5+ softwareNetbeans Mobility Pack 5.0 for CDCSun CDC Java Media API Client Emulator

IPTV ApplicationsDebugging & development Environment


Offering Use Case• Streaming from existing DVB

network• DVB server and client deployed

outside Java environment• Interactive Java TV™ extension • ITV is implemented over IP• VOD content to be followed

Key Benefits• Potential mass market• Compliant with existing standards

Digivision Introduction 2008 JMC Use Cases

Oriental Cable Network


Traditional IPTV• IPG, VOD and PVR

Movie download to CPE devices• Eliminate Broadband requirement

Java Technology Games• From our large Java Technology Games

CommunityLearning• Extensive learning Catalog

Information Services• Tickers, Weather, Yellow Pages

Karaoke

Digisoft Introduction 2008 JMC Use Cases


STB Middleware is expensive (~$25 per STB)STB Middleware is inflexible• Most have no developer partners – because they can't• Most stuck with a Fat “Frankenstein” Client• All are failing to scale in functionality – up to 18 month lead times

Browser based solutions have come up very short• www.digisoft.tv/whitepapers

Telcos etc. have very limited control over their own UI

IPTV Application DevelopmentChallenges in legacy market


Client Side - The Java platform Set Top Box (HD < $100)• Eliminating expensive and complex client middleware

Developers – Digisoft SDKs• NetBeans software plug-ins – Client, Server and Integration SDKs

Client Side - The Digisoft Java technology-based IPG• Bringing Java platform flexibility to IPG

Client Side – Revenue-generating Java platform applications• Leveraging the vast Java platform developer community

Server Side – Digihost IPTV Application Server• The brains of the applications

Digisoft Key Components


IPTV Application DevelopmentDevelopment Environment'IPTVBox' for Developers• DigiHost Application Server• DigiHost NetBeans software

integrated SDKs • Video Server• Digisoft IPG / Application

Launcher / Demo Apps• Training Course

Digisoft Developers Group, Sun Java CommunityDigisoft Deployment Group

Off you go ... Make Money ... Your way!


Sun Media Receiver

Digisoft & Sun Java based media client

Digivision & Sun Java based media client


Summary

IPTV – new media consumption paradigmApplication complexity relates to device/Java Media Framework API capabilitiesDevices need hardware acceleration (graphics & codecs)


Agenda - Q&A


For More Information

Java Community ProcessSM service resources: jcp.org• CDC 1.1 (JSR218), FndP 1.1 (JSR219), PBP 1.1 (JSR 217)• Java Media Framework API 2.1.1 (JSR 2.1.1)

Java Media Framework API (Spec and Programmers guide)

• http://java.sun.com/products/java-media/jmf/2.1.1/documentation.htmlJava Media Framework API FAQ

• http://java.sun.com/products/java-media/jmf/reference/faqs/index.htmlExhibition booth• Sun Media Receiver and Sun Java Media Client

http://java.sun.com/products/java-media/jmf/2.1.1/documentation.html

http://java.sun.com/products/java-media/jmf/reference/faqs/index.html


Ronan McBrien, Senior Staff Engineer, Sun MicrosystemsSourath Roy, Staff Engineer, Sun MicrosystemsJohn Allen, CEO, Digisoft

ts-5722 - interactive application development for iptv · ts-5722 - interactive application...

Documents