implementation work on open source web of things€¦ · • bad data • violation of design...
TRANSCRIPT
-
Implementationworkonopensourcewebofthings
serversandgateways
DaveRaggett,W3C
Monday,11April2016
-
ReferenceArchitecture
GatewayCloudbasedServices
FirewallAmbientorbatteryoperated IoT devices
Powered,multi-protocol
BrowserforHMI
2/27
-
Introduction
• IamworkingontwoopensourceWebofThingsserver&gatewayprojects• NodeJS
• Forrelativelypowerdevices,e.g.runningLinux• C++formicrocontrollers
• ForArduinoUnoandWiznet basedEthernetShield• Highlyconstrainedlowcostmicrocontroller• FutureplansforextensiontomorepowerfulMCUs• Andotherkindsofcommunications technologies
• Why?• Nothinglikeworkingcodeforbetterunderstanding• Hugepotentialmarketforlow-endIoT devices
3/27
-
• ArduinoEthernetShield• 16KBRAM• MicroSD cardslot• ControlledthroughSPIbus• PollingorH/Winterrupt• Cost:4.75GBPoneBay
• ArduinoUnowithATmega328PMCU• 2KBRAM• 1KBEEPROM• 32KBFLASH• LotsofI/Opins• Cost:2.33GBPoneBay
RequiredHardware
4/27EEPROMusefulforstoringsettingsanderror logs,etc.
-
RequiredSoftware
• ArduinoIDE• Freefromhttps://www.arduino.cc/en/Main/Software
• SerialdriverforCH340USBtoserialchip• RequiredforcheapArduinoclones
• AbitofapaintofindandsetupL
5/27
-
Components
• ArduinoSketchwithapplicationcode• ThisiswhereyouwriteyourcodetointerfacetosensorsandactuatorsusingtheArduinolibraries• GPIOpins• A2DandPWM• SPIandI2Cbuses• Timers
• WebofThingsLibraryfortheArduinoIDE• InstallablefromGitHub• Smallfootprintwithcustomnetworkingmodule
• Wiznet W5100withnativeIPsupport• Multicastfordiscovery• UDPandTCP• ARP,ICMP,IGMPetc.
6/27
-
Configuration
• UsestaticIPaddressorDHCPfordynamicaddress• UsesmDNS todiscoverWebofThingsgateway
• Servicetype:_wot._tcp.local (unregistered)• MyMacBook’sfirewallblocksallothermulticastsockets
• RegistersremoteproxyongatewayforlocalthingontheArduino• UsesTCPformessagingwithgateway
• LesscodeduetonativeIPsupportonW5100chip• VerysimilartomyWebSocketimplementationforNodeJS webofthingsserver
• CoAPwouldtakemorespaceandbelesscapable• Notethechoiceofprotocoldependsonthechoiceofhardware,whichinthiscaseisanEthernetcontroller
7/27
-
Softwarearchitecture• Targetis2KBforapplicationcode• 2KBforRAM• Avoidmalloc andfreeforrobustoperation
• Fragmentationandlongterminstability• Staticnodepool
• JSONfornumbers,booleans,objects,arraysetc.• AverageLengthBinarytrees(objectsandarrays)• Mark/Sweepgarbagecollector• 6bytespernodeontheATmega328P
• JSONobjectpropertynamesreplacedbysymbols• ToreduceRAMfootprintto2KBisverychallenging
• Efficientbinarymessageencoding/decoding• Reliesondeterministic assignmentofsymbols• Designedwithshortpackettechnologies inmind
e.g.NordicnRF24L01+with32bytepayloads
8/27
-
RAMvsFLASHmemory
• SomeMCUsliketheATmega328PusetheHarvardmemoryarchitecture• Thishasseparatedataandcodeaddressspaces
• StringliteralsarestoredinFLASHandcopiedtoRAMjustbeforeenteringafunction• Thisisobviouslybadifyou’reshortofRAM
• Firststep:tellcompilertokeepliteralsinFLASH• F(“stringliteral”)vs“stringliteral”• Youthenneedtousesystemcallstoaccessthem
• Secondstep:useabstractionlayertohidethedifferencebetweenRAMandFLASH• Signatures:const __FlashStringHelper *vsconst char*• Mergeaddressspacesthroughadding0x8000toFLASHpointersandusestringfunctionsthatknowhowtodealwiththem
9/27
-
InterruptsandEvents
• Hardwareinterruptsarevaluableforhandlingsensorsandactuatorsetc.• Buttheinterruptserviceroutine(ISR)mustbequicktoavoidblockingotherinterrupts• Eventdrivenbehaviourisveryconvenient,buteventhandlerscantaketimetoexecute• Solution:ISRpusheseventontoqueue,whichisdispatchedfromwithinArduinoloop()method• SamesinglethreadingexecutionmodelasforWebbrowsers– eventhandlersareexecutedsequentially• Avoidsappdevelopersfromhavingtodealwithcomplexsynchronisationissues
10/27
-
LifeafterDeath
• Eventuallysomethingbadwill happen…• Memoryissues
• RunningoutofRAMfortheprogramstack• RunningoutoffreenodesforusewithJSON• Overflowofcriticalqueues
• Programbugs• Youcanneverbe100%confidentthattherearenobugs
• Baddata• Violationofdesignassumptions
• Softwarerestartondetectionoffatalerrors• Hardwarewatchdogtimerforcesrestartifnotresetintime
• Thisallowsthedevicetorecoverafterbeingstuckinaloop• Gatewaysandotherdevicesmustberesilienttosuchrestarts
11/27
-
ThingsandProxychains
• Thingsaresoftwareobjectswithproperties,actions,eventsandmetadata• IfyouwanttointeractwithaThingonanotherdeviceyouwillneedaProxyobjectforit• ThingsLayerkeepProxiesinsyncwiththeirThings• ProxiescanbechainedtoformProxytrees rootedintheThingtheystandinfor• Updatingapropertyonaproxyresultsinmessagesthattraveltowardstherootandleavesofthetree
12/27
-
LocallyScopedIdentifiers
• EachdeviceintheproxytreegenerateslocallyscopedIDsfortheproxies/thingsithosts• Theseareusedtomatchmessagestotheobjectstheyaretargetedat• Messagesaredistinguishedbywhethertheyaresenttoaparentorchildofanodeintheproxytree• ProxyobjectsholdtheirIDforthisdeviceaswellastheirparent’sIDforthecorrespondingThing/Proxyontheparentdevice• Messagehandlersortsthisallouttoroutemessagesbetweenthedesiredsoftwareobjects
13/27
-
CleanSeparationofLayers
Application Scripts that define thing behaviour in terms of their properties, actions and events, using APIs for control of sensor and actuator hardware
Things Software objects that hold their stateAbstract thing to thing messagesSemantics and Metadata, Data models and Data
Transfer Bindings of abstract messages to mechanisms provided by each protocol, including choice of communication pattern, e.g. pull, push, pub-sub, peer to peer, etc.
Transport REST based protocols, e.g. HTTP, CoAPPub-Sub protocols, e.g. MQTT, XMPPOthers, including non IP transports, e.g. Bluetooth
Network Underlying communication technology with support for exchange of simple messages (packets)Many technologies designed for different requirements
ApplicationDeveloper(WoT focus)
PlatformDeveloper(IoT focus)
Focus on data types, APIs and error handling
14/27
-
ImplementingtheCommunicationsStack• Designthesoftwarearchitecturetoallowforuseofmultiplecommunicationstechnologies• ThingLayerthathooksintothesoftwareobjectsforthingsandhandlescorrespondingabstractmessages• Transferlayermapsthesetocommunicationpatternssubjecttocommunicationsmetadata• TransportLayerforspecificprotocolsimplementedaspluggablemodulesforthetransferlayer• Thechoiceoftransportisindicatedbytheconnectioninformationforeachparentandchild• Lowenddevicelikelytosupportonlyoneprotocol
15/27
-
ThingDataModels
• Thingsmayhaveproperties,actions,eventsandmetadata• Eventshaveatypeandavalue• Propertieshavevalues• Actionsmaypassavaluewhentheyareinvoked• Actionsmayreturnzeroormorevaluesasresponses• Valueshavetypes*
• null,true,false,integers,floats,strings• Arraysandobjects(setsofname/valuepairs)• Valuescanbecompound,e.g.nestedarraysandobjects• ThingsandStreamsasfirstclasstypes
*Should enumerations beallowedascoretypes?16/27
-
LifecycleConsiderations
• Thingscanhaveotherthingsastheirproperties*• Thingsreferencedaspropertiescanbelocalorremote• Forlocalthings,thereferencedthingmaynothavebeencreatedyet• Forremotethings,serverneedstosetupproxies• ProxiestaketimetosetupduetotheneedtogettheThingdatamodeldescription
• Solution:pushthepropertytoapendingqueueandsetthepropertyvaluewhenthereferencedthing/proxyisready
17/27*Exceptforveryconstrainededgedevices
-
CyclicDependencies
• Withthingsaspropertiesthereisthepotentialforcyclicdependenciesbetweenthings• ThingAhasThingBasaproperty• ThingBhasThingCasaproperty• ThingChasThingAasaproperty
• Thiscouldinvolveamixoflocalandremotethings• IfwestartsuchThingswhenalloftheirpropertieshavebeenstartedthenwewillgetadeadlock!• Solution:startaThing/Proxywhenallofitspropertiesareresolved,andforeachThing/Proxyholdmessagestothatobjectinaqueueuntilithasbeenstarted
18/27
-
LateBinding
• Valueswhosefulltypeisonlygivenatrun-time• Thetypemaybepartiallyspecifiedatcompiletime
• Canapplytoproperties,actionsandevents• Variantdatatypeswithtagthatdetermineswhichvariantappliestothisvalue(e.g.C++unions)• Thingsaslateboundvalues• Youneedtoretrievethedatamodeltoinstantiatethesoftwareobjectforthethingyou’vebeenpassed• Thevalueincludesareferencetothedatamodel
• ThingLayermanagesthisautomatically
19/27
-
Pointers
• SayaThinghasapropertywhichisanarray• Ascriptupdatesthe3rd iteminanarray• Thisrequiresameansforupdatemessagestoidentifyareferencetothe3rd iteminthearray• InotherwordssomekindofpointerthatoperatesontheThing’sdatamodel• SimpleexpressionwithinJSONmessages,e.g.
“path”:foo[3]and“path”:foo[3].bar• ForCoAP aURIbasedsyntaxwouldbeappropriate
20/27
-
RefinementofJSON-LD
• UsedforexpressingThingdatamodels• @contextforbindingshortnamestoRDFURIs
• Implicitcontextforthemostcommonnames• Linkedcontextfordomainspecificnames
• Usedforsemanticsearchandservicecompositions• Canbesafelyignoredbyedgedevices
• Toplevelobjectwith“properties”,“actions”&“events”• Valueseitherdeclaredasname:type
• e.g. “on”:“boolean”,or“level”:“float”• Ordeclaredasname:{annotations}
• e.g.“temperature”:{“type”:“integer”,“min”:-20,“max”:100,“units”:“celcius”}
• Neededfornestedproperties• Actionsdeclaredwith“in”and“out”forthevaluesthatarepassedtothem,andfortheresponsestheactiongenerates
21/27
-
IntegrityConstraints*
• Enabledetectionofbaddataforincreasedresilienceinthepresenceoffaultsandattacks
• Goodprogrammingpractice:assert(expression)• Basicintegrityconstraintsonasinglevalue
• Upperandlowerboundsfornumbers• Closedsetofvaluesforstrings
• Integrityconstraintsacrossmultiplevalues• Relationships betweenpropertiesofasingleThing• Relationships acrossmultiple Things
• Cardinalityconstraints• e.g.ForeveryXyoumusthaveatleasttwoY
• Metadataconstraints• e.g.forapplicationversioning
• Needforanexpressionlanguagefordefiningconstraints
22/27*Temporalconstraintscanbeexpressedindomainmodels
-
IntegrityConstraintsinJSON-LD
• Simplevalueconstraints• Asannotationsonthetype,e.g.minandmax
• Complexconstraintsinvolvingexpressions*• OperatorsandFunctions
• Numericcomparisons,Stringcomparisons,Booleanoperators• Extensible setoffunctions
• Complexconstraintsinvolvingpointers• PointersasreferencestoothervaluesinthisThing
• Otherproperties,whichcanincludeotherthings• Otherpartsofthedatainanevent,orpassedto,orreturnedbyanaction
• TooexpensiveforlowenddeviceswithlimitedRAM• Iplantoimplementthisonthegatewayserver
*CanwegeneraliseJSONtoalloweasytounderstandexpressionswithoperatorprecedence?23/27
-
DomainModels
• Richontologybaseddescriptions• Canexpresssemanticconstraints• Ifxisatemperaturesensorthenxmustdefineitsphysicalunitsfromtheset{Kelvin,Celsius,Fahrenheit}• Temporalconstraints,e.g.onsequencesofactions
• Usefulforsemanticsearchandservicecompositions• Verifycompatibilityofservices
• RequiressupportforOWL,SPARQL,etc.• Notneededforresourceconstrainededgedevices
24/27
-
DemoProjects
• Aimtoprovidesomeexamplesforhowtousethelibrarywithcommonlyavailablesensorsandactuators,e.g.• Temperature&humidity• Ultrasonicrangesensor• MulticolourLEDs• Servomotor
• Needtoprovidesketch,wiringdiagramandnotes• Betteryettoalsoprovideavideo• Lookingforvolunteerstohelpwiththiseffort!• StartinginJune2016
25/27
-
Projectsite
• TheArduinoprojectisonGitHub• https://github.com/w3c/wot-arduino
• Stillaworkinprogress• HavingtodothisasabackgroundactivityduetolotsofcompetingtasksL
• CurrentlyworkingonThingLayerfollowingrecentworkongarbagecollectionanddiscovery
• Re-usingvirtuallyallthecodeforgateway• Willenable lowcostmicrocontrollerbasedgateways
• PlantoupdateNodeJS project*tomatch• GoaltohavefullydocumentedprojectwithexamplesintimeforBeijingmeetinginJuly2016
*https://github.com/w3c/web-of-things-framework 26/27
-
WhereNext?• Morepowerfulprocessors
• ArduinoMega• 8KBRAM,4KBEEPROM,256KBFLASH
• ARMbasedMCUs• MoreRAM,butnoEEPROM
• WorkonStreams,e.g.forelectrocardiograms• Othercommunication technologies
• SensornetworkswithNordicnRF24L01+• BluetoothSmartwithXBee Arduinoshield
• Sleepydevices• Usingtimer towakeupeveryfewminutes• Mainlyrelevanttowireless technologies
• NotethatArduinoboardsarepowerhungryduetoLEDsandlinearvoltageregulators
• Synchronisation acrossclustersofdevices• Synchronisedcontrolofrobotjoints
• Stronghardwarebasedsecurity
27/27