timeshift everything, miss nothing - mashup your pvr with kamaelia
TRANSCRIPT
Timeshift Everything, Miss Nothing
Mash up your PVR with Kamaelia
Euro OSCON 2006, Brussels
Michael Sparks,
Senior Research Engineer, BBC Research
Prologue
Time shifting...
Enshrined as a right in UK Law
17A. - (1) The making in domestic premises for private and domestic use of a recording of a broadcast solely for the purpose of enabling it to be viewed or listened to at a more convenient time does not infringe any right conferred by Part 2 in relation to a performance or recording included in the broadcast.
Time shifting...
Enshrined as a right in UK Law
17A. - (1) The making in domestic premises for private and domestic use of a recording of a broadcast solely for the purpose of enabling it to be viewed or listened to at a more convenient time does not infringe any right conferred by Part 2 in relation to a performance or recording included in the broadcast.
TiVO*, Sky+, Goodmans, Grundig, Humax, Sagem, Topfield, Sharp, Sony, Thomson, Technosonic, etc...
* not sold in UK anymore
Kamaelia Macro
Kamaelia Macro
It records and transcodes what is broadcast over DTT for future viewing.
(video)
How do you build this?
DVB MultiplexKamaelia Macro: Target
outbox Channel Transcoderinbox DVB Demuxer
BBC_ONEinbox Channel Transcoderinbox Channel Transcoderinbox Channel Transcoder Channel Transcoder Channel TranscoderinboxinboxinboxBBC_TWOBBC_THREEBBC_FOURCBEEBIESCBBC
DVB MultiplexKamaelia Macro: Status
outbox Channel Transcoderinbox DVB Demuxer
BBC_ONEinbox Channel Transcoderinbox Channel Transcoderinbox Channel Transcoder Channel Transcoder Channel TranscoderinboxinboxinboxBBC_TWOBBC_THREEBBC_FOURCBEEBIESCBBC
DVB MultiplexKamaelia Macro: Status
outbox Channel Transcoderinbox DVB Demuxer
BBC_ONEinbox Channel Transcoderinbox Channel Transcoderinbox Channel Transcoder Channel Transcoder Channel TranscoderinboxinboxinboxBBC_TWOBBC_THREEBBC_FOURCBEEBIESCBBC
Scaling up simplyawaiting hardware
DVB MultiplexTop Level Schematic
outbox Channel Transcoderinbox DVB Demuxer
BBC_ONEinbox
Channel Transcoder EITDemuxChannel Transcoder Schematic
inbox_eit_inbox EITParsingoutboxinbox Carousel
transcoder_factoryinboxnextoutbox
EIT ParsingPSI PacketReconstructorProgramme Identification Schematic
inboxinbox EIT PacketParseroutboxinbox
Now Nextservice filter Now NextChangesoutboxinboxoutboxinboxoutboxoutbox
Programme TranscoderProgramme Transcoding Schematic
(a pipeline)
Pipethrough(mencoder -o file options...)
inboxcontrolsignalinboxcontrol_transcodingcomplete_stopsignal
But code isn't pretty web 2.0 esque boxes gradiented , it's never like that...
End of Prologue
Aims of Kamaelia
Aims of this Talk
To stave off doom from buggy software on parallel machines
Aims of this Talk
ahem
Aims of this Talk
(nb, that's something that concerns me,Kamaelia is partly aimed at staving that off,I'm not claiming it does that but it is oneapproach we're finding useful)
Aims of this Talk
Understand what Kamaelia is
Aims of this Talk
Understand what Kamaelia is
Take away our research results, and assimilate our code or approach in your systems
Aims of this Talk
Understand what Kamaelia is
Take away our research results, and assimilate our code or approach in your systems
To make it clear concurrency can be much simpler if you have the right tools
Aims of this Talk
Understand what Kamaelia is
Take away our research results, and assimilate our code or approach in your systems
To make it clear concurrency can be much simpler if you have the right tools
Can be simpler than single threaded code
Aims of this Talk
Understand what Kamaelia is
Take away our research results, and assimilate our code or approach in your systems
To make it clear concurrency can be much simpler if you have the right tools
To encourage you to think about using Kamaelia, and collaborating with us
Learning More: Kamaelia Open Space
What: kinda like a project's Foo/Bar Camp
Why: To help you get started, share knowledge etc,
When: This Friday 22nd Sept, 11am-5pm
Where: Here, Brussels, The FoAM Offices
Koolmijnenkaai 30-34, 1080 Brussels
Details: http://tinyurl.com/f8szz
Map: http://tinyurl.com/hsarl
Aims of this Talk
To share an approach we find fun and useful
What ?
What ?
(demo)
What you've just seen.
ComponentWhat you've just seen
outboxinbox
ComponentWhat you've just seen
outboxinbox Componentoutboxinbox
ComponentWhat you've just seen
outboxinbox Componentoutboxinbox
ComponentWhat you've just seen
outboxinbox Componentoutboxinbox
Componentoutboxinbox
ComponentWhat you've just seen
outboxinbox Componentoutboxinbox
Componentoutboxinbox
What you've just seen
Componentoutboxinbox Componentoutboxinbox
Componentoutboxinbox
What you've just seen
Componentoutboxinbox Componentoutboxinbox
Componentoutboxinbox
What you've just seen
Componentoutboxinbox Componentoutboxinbox
Componentoutboxinbox
What you've just seen
Componentoutboxinbox Componentoutboxinbox
Componentoutboxinbox
What you've just seen
Componentinbox Component Componentoutboxoutbox | inboxoutbox | inbox
What you've just seen
Componentinbox Component Componentoutboxoutbox | inboxoutbox | inbox
What you've just seen
Componentinbox Component Componentoutboxoutbox | inboxoutbox | inboxWhat's inside the box?
Python Generators
Python Generators
Fibonacci Demo
Component Basics
Component Basics
Rotating Angle Demo
turning a generator into a component
Component Basics
What use is this?
class RotatingAngle(Axon.Component.component): def main(self): while 1: self.send(x, "outbox") x = x + 1 if x > 359: x = 0 yield 1
Component Basics
Consider these components:
class loopingCounter(Axon.Component.component) class cartesianPingPong(Axon.Component.component) class bouncingfloat(Axon.Component.component) class continuousIdentity(Axon.Component.component) class continuousZero(Axon.Component.component) class continuousOne(Axon.Component.component) (from Kamaelia.Automata.Behaviours...)
Component Basics
Bouncing Cats
Component Basics
Graphline( CAT = BasicSprite(image=cat), ROTATOR = loopingCounter(rotation_speed), MOVER = cartesianPingPong(position, 800, 600, 40), SCALER = bouncingFloat(scale_speed), IMAGER = continuousIdentity(cat), linkages = { ("ROTATOR","outbox" ) : ("CAT", "rotator"), ("MOVER","outbox" ) : ("CAT", "translation"), ("SCALER","outbox" ) : ("CAT", "scaler"), ("IMAGER","outbox" ) : ("CAT", "imaging"), } ).activate()
From Examples/SimpleGraphicalApps/BouncingCatGame/Simplegame.py(shutdown fanout removed for space)
What's it good for?
Some stuff we've done
Networked Audio Mixer MatrixDirac/Vorbis distribution/decode/playersSubtitling distribution server (mobiles, PCs)Presentation ToolsCollaborative Whiteboards * with audioSimple gamesVisual programming toolsVisual Introspection toolsKamaelia Macro...
Stuff trainees have done
pre-university trainee:
Previewing PVR content on a mobile phone Shakespeare script parsing and doing something fun with events & data
Stuff trainees have done
pre-university trainee:
Previewing PVR content on a mobile phone Shakespeare script parsing and doing something fun with events & data
Later adapted simply to have the characters talk
the play on an IRC channel
Stuff trainees have done
Summer trainee: (2nd year student)
Tools for simple reliable multicastMulticast island joining
Google Summer of Code Students
Ryan Lothian
Thomas Flanitzer
Devendra Laulkar
Anagha Mudigonda
Google Summer of Code
For us SoC was an opportunity to have components implemented that we've wanted for some time
Students implementing them appear to have had fun.
Also they've been guinea pigs to test our ease of use hypothesis
Large code chunks in our 0.5.0 release
Google Summer of Code
For us SoC was an opportunity to have components implemented that we've wanted for some time
Students implementing them appear to have had fun.
Also they've been guinea pigs to test our ease of use hypothesis
Large code chunks in our 0.5.0 release
Hopefully uploaded to sourceforge today :-)
Google Summer of Code
Components for:
Bit Torrent integration, a web server, web clients, IRC clients, icecast clients tone generation
3D systems widgets, pygame component wrapping, 3D scribbling, simple image viewer
secure communications including a nascent (choppy) secure phone.
prototype trusted communications tools
eg reporter trusting his comms to the BBC won't be changed without detection
Ease of Use
Now have 6 data points for ease of use
The youngest, most nave students have produced the most complex and interesting systems
The older ones have generally had more baggage, and unless used to thinking about X talking to Y, to Z, have had more problems.
Even our weakest SoC student has produced interesting data
Stuff other parts of the BBC have done
Radio & Music Interactive:
Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data.
Stuff other parts of the BBC have done
Radio & Music Interactive:
Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data.
Bootstrapped the final system rapidly
Stuff other parts of the BBC have done
Radio & Music Interactive:
Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data.
Bootstrapped the final system rapidly
Created the system without any involvement from us
Stuff other parts of the BBC have done
Radio & Music Interactive:
Prototyped a system for recording all BBC radio output, transcoding it and making podcasts from the resulting data.
This directly led on to the Macro workas a result, including optimisations
Kamaelia Macro: Revisited
What Macro looks like inside
(yes, it really is boxes)
DVB MultiplexTop Level Schematic
outbox Channel Transcoderinbox DVB Demuxer
BBC_ONEinbox
Top Level Code
Graphline( SOURCE=DVB_Multiplex(freq, pids["BBC ONE"]+pids["EIT"], feparams), DEMUX=DVB_Demuxer({ 610: ["BBCONE"], 611: ["BBCONE"], 18: ["BBCONE"], }), BBCONE_HI = ChannelTranscoder(service_ids["BBC ONE"], **params["HI"]), linkages={ ("SOURCE", "outbox"):("DEMUX","inbox"), ("DEMUX", "BBCONE"): ("BBCONE_HI", "inbox"), }).run()
Channel Transcoder EITDemuxChannel Transcoder Schematic
inbox_eit_inbox EITParsingoutboxinbox Carousel
transcoder_factoryinboxnextoutbox
Channel Transcoder Code
def ChannelTranscoder(service_id, mencoder_options, dir_prefix):
def transcoder_factory(eit): return ProgrammeTranscoder(eit, mencoder_options, dir_prefix)
return Graphline( PROG_CODER = Carousel( transcoder_factory ), EIT_PARSE = EITParsing(service_id), DEMUX = EITDemux(), linkages = { ("self","inbox") : ("DEMUX","inbox"), ("DEMUX","outbox") : ("PROG_CODER","inbox"), ("DEMUX","_eit_") : ("EIT_PARSE", "inbox"), ("EIT_PARSE", "outbox") : ("PROG_CODER", "next"), } )
EIT ParsingPSI PacketReconstructorProgramme Identification Schematic
inboxinbox EIT PacketParseroutboxinbox
Now Nextservice filter Now NextChangesoutboxinboxoutboxinboxoutboxoutbox
Programme Identification Code
def EITParsing(*service_ids): return pipeline( PSIPacketReconstructor(), EITPacketParser(), NowNextServiceFilter(*service_ids), NowNextChanges(), )
Programme TranscoderProgramme Transcoding Schematic
(a pipeline)
Pipethrough(mencoder -o file options...)
inboxcontrolsignalinboxcontrol_transcodingcomplete_stopsignal
Programme Transcoding Code
class ProgrammeTranscoder(Axon.Component.component): Inboxes = { "inbox" : "TS packets containing audio and video", "control" : "the usual", "_transcodingcomplete" : "signal from the transcoder", } Outboxes = { "outbox" : "", "_stop" : "stop the transcoder", "signal" : "the usual", } def __init__(self, eitdata, mencoder_options, dir_prefix): super(ProgrammeTranscoder,self).__init__() self.eitdata = eitdata self.mencoder_options =mencoder_options self.dir_prefix = dir_prefix def main(self): # first, generate unique filename uid = str(time.time()) encodingfile = "/data/encoding"+self.dir_prefix+"/"+uid+".avi" waitingEIT = "/data/encoding"+self.dir_prefix+"/"+uid+".eit" finishedfile = "/data/finished"+self.dir_prefix+"/"+uid+".avi" finishedEIT = "/data/finished"+self.dir_prefix+"/"+uid+".eit" print uid,"Starting transcoding into: "+encodingfile transcoder = Pipethrough("mencoder -o "+encodingfile+" "+self.mencoder_options) print uid,"Transcoder pipethough =",transcoder.name data_linkage = self.link( (self,"inbox"), (transcoder,"inbox"), passthrough=1 ) ctrl_linkage = self.link( (self,"_stop"), (transcoder,"control")) done_linkage = self.link( (transcoder,"signal"), (self,"_transcodingcomplete") )
Kamaelia Macro
Code is in the distribution is you want to play
(only works under Linux due to Linux having a uniform DVB-T API)
But Why Mashup?
A web mashup often takes data from multiple sources (websites) transforms them and republishes them somehow, allowing recomposition.
Kamaelia Components are essentially the same, except you're not mashupping websites but systems, including potentially your PVR.
It's just data.
Kamaelia: Ideas
Things you might want to do (to Macro and beyond)
3D user interface to the PVR
IRC based control (your PVR sitting on
an IRC channel)
Web integration (beyond blogs/rss)
Integrate common sense engine
An open alternative to Second Life
Video Conferencing
Build an open technology for BBC iPlayer?
Kamaelia Futures
We want to replace the need to write code when writing components.
Further optimisations
More concurrency mechanisms
More tutorials
(AKA Specfic stuff we want to do)
Kamaelia Futures
Collaborative Storyboarding/programme production tools
Open source iPlayer
Integrate PGU
Better Tk Components
PyQT, wxWindows integration
(AKA Specfic stuff we want to do)
Thank you!
Thank you!
Kamaelia Open Space
This Friday 22nd Sept, 11am-5pm
FoAM's Offices:
Koolmijnenkaai 30-34, 1080 Brussels
Map: http://tinyurl.com/hsarl
Possible discussions: Get started; integrate your favourite libraries; apply to teaching (gesture recognition); flesh out our system to be more relevant to your problem domain (as we do for the BBC)? Re-implement in C++, Ruby?
http://tinyurl.com/f8szz
Click to edit the title text format
Click to edit the outline text format
Second Outline Level
Third Outline Level
Fourth Outline Level
Fifth Outline Level
Sixth Outline Level
Seventh Outline Level
Eighth Outline Level
Ninth Outline Level
Michael Sparks, [email protected], http://kamaelia.sf.net/
Kamaelia