Immaterials: Light Painting Wi-fi explored the invisible terrain of WiFi networks in urban spaces by light painting signal strength in long-exposure photographs.

physical web exploring the

a workshop for UX Lisbon 2016

• A brief history • Understanding Physical Web and Web Bluetooth. • Three key use cases. • Design considerationsCoffee Break :-)• Group exercise (~35 minutes) • Group discussion (~45 minutes)

Workshop agenda

In this workshop, we will discuss the Physical Web and Web Bluetooth: two unrelated but highly complimentary technologies that enable us to create signposts and pathways between the web and physical world.

Image source: Wired

The number of smart devices is going to explode, and the assumption that each new device will require its own application just isn't realistic.

We need a system that lets anyone interact with any device at any time…[this] isn’t about replacing native apps, it’s about enabling interaction when native apps just aren't practical.

— Scott Jenson, Physical Web Lead, Google

“

1. A brief history

5

The Physical Web isn’t our first attempt to place digital markers within the physical world. Most older approaches in fact are still in use as they serve slightly different use cases.

QR codes (Japan, 1994) Invented for use in retail but rapidly adopted across industries.

Advantages • Cheap to implement and distribute. • Can hold up to 1000 bytes of data. • Easily discoverable.

Disadvantages • Hard to scan from a distance. • Many people think they’re ugly. • Most people (outside Asia) don’t

have a QR code reader.

QR codes (Japan, 1994) • Invented for use in retail but rapidly

adopted across industries. • Advantages

• Cheap. Easy to implement. • Disadvantages (practical)

• Hard to scan from a distance. • One to one: If you’re close enough

to scan the code then (probably) no-one else can scan it.

• Disadvantages (sociocultural) • Most people (outside Asia) don’t

have a QR code reader. • Many people think they’re ugly.

RFID, NFC (1983, 2002) • Both employ radio signals to

exchange data between nearby objects.

Advantage • Proximity often requires explicit

user action, which improves trust, and makes these technologies ideal for payment and identification.

Disadvantage • Poor discoverability. Must be

signposted or rely on learned behaviour.

British Airways NFC + e-paper baggage tag prototype

Proximity marketing 1.0 (2004) • A small server broadcasts short

messages, images, mp3s etc. into a space using Bluetooth.

Disadvantage • Anyone nearby with Bluetooth

turned on received the broadcast. • Was mostly used for advertising

so became a form of spam (which forced many people to shut Bluetooth off).

https://www.flickr.com/photos/voyages-provence/13541325644

Apple iBeacon (2013) • Small battery powered Beacons

broadcast messages to iOS + Android apps using Bluetooth Low Energy (BLE).

• Only the app that matches the beacon ID recognizes the broadcast.

Advantage • Users aren’t bombarded with messages

from brands they don’t already have a relationship with.

Disadvantage • Users without the specific app that matches

the beacon can’t discover/take advantage of the data or service it provides.

*kontakt.io beacon shown

iBeacon On iOS, a user’s app will automatically receive a broadcast each time it passes by a beacon—even if the app is in background mode or switched off. Advantage

• If a user walks by several beacons, it’s therefore possible track their path and use it to trigger location-relevant offers or notifications (i.e. a user’s gate number when they enter the airport).

Disadvantage • Potential privacy concerns. • Even if the user likes a brand,

automatically triggered notifications can rapidly become annoying.

The goal of the Physical Web isn’t to replace these technologies, but to address some of their challenges by leveraging the ‘super-powers’ of URLs and the open, flexible, decentralised and universally supported web.

2. How does the Physical Web work?

15

The Physical Web (PW) is an open-source* project. Google initiated the technology, and has the most widespread implementation (~800m devices) through Chrome on Android and iOS—but there’s already a large ecosystem of other contributors.

It’s all open source!

*Visit GitHub to download the source and/or contribute to these conversations.

1. The beacon (a thing that

broadcasts a URL)

The Physical Web is pretty simple—it has three parts

2. The scanner/browser (a thing that scans for, retrieves

and displays a list of URLs)

3. Proxy (user advocate)

optional cloud service

The specifications for all three are open source.

What is a beacon?

BLE radio transmitter (intermittently on, sips power

compared to regular BT)

low-cost, long-life (~2yrs) battery or

other power source

many beacons also include some form of remote management appThis beacon is by Estimote (but there are many others)

many sizes and form factors

A beacon’s (only) job is to repeatedly* transmit a signal that other devices can see.

The message itself is transmitted using a BLE ‘advertising packet’, a standard broadcast format limited to 17 bytes.

Once in range, Bluetooth-equipped devices like a smartphone can “see” the beacon and receive its broadcast (if BT is switched on).

*recommended every 700 milliseconds

broadcast range

What it does

Unidirectional broadcastThe Bluetooth ‘advertising’ function is by-design unidirectional. • A single beacon can advertise to

multiple nearby smartphones but these devices cannot send data back to the beacon.

• Beacons also cannot physically detect when clients scans them so cannot track passers-by.

to broadcast URLs could also be attached or embedded into all sorts of

smart (and otherwise dumb) things around us

https://www.flickr.com/photos/naan/2398024748

…in the near future, the ability

Permanently broadcasts “I love you”

Turn me off from a distance

Change my colour!

Check what materials i’m made of when Craigslisting me.

Understand how I work and where to recycle me

2. The scanner/browserAs the beacons broadcast URLs, the most common scanners are (and will probably remain?) web browsers.

ChromeAndroid + iOS

~800M devices

Opera(Labs browsers)

Mozilla(Beta)

Microsoft Edge (coming soon)

Safari Mobile ????

Unlike iBeacon, there is however no background scanning. All scanning for URLs must be user-initiated.

As the Physical Web is new, users will probably not scan unless they’re aware of nearby content, or notice the logo.

(This is how the ‘normal’ web began as well :-)

How do you ‘scan’ (in Chrome)

A user opens their Android Notification screen or Today Widget on iOS.

If there are beacons nearby, they will see a low priority (no vibration) notification. Tapping the notification displays a list of

all beacons within range, showing their URL, page title, description and favicon*.

…dis

play

ed in

ord

er o

f pro

xim

ity

*see this article for details of metadata retrieval in each browser

(Behaviour varies by browser)

And then…?

https://webPage.io

That’s (technically) it. The beacon’s job is done. All subsequent interactions between the user and brand take place ‘on the web’.

https://shortURL.io

user taps to load the page (on the internet—the beacon does not serve the page)

The proxy sits in between the beacon and the user. A proxy is optional but serves two important purposes: improve performance and protect the user.

The Chrome proxy is called the Physical Web Service (PWS). Other browsers have a similar (but probably not identical) service.

3. The proxy

*more details on the PWS and its role

What it does

https://shortURL.io

2

1

(based on Google’s PWS…each proxy is a bit different)

parses the document to extract metadata• final URL

• page title • description • favicon

What it does

https://shortURL.io

BEACONS BOOKMARKTOP SITES

Search or enter address

Haro Sushi and Izakayaharo-sushi.com

Hons on Robsonhons-noodles.com

Beacons are small devices that broadcast links to web sites. Tap here to learn more

2

analysis + optimisation

4

1one-way broadcast

(based on Google’s PWS…each proxy is a bit different)

parses the document to extract metadata3

• final URL • page title • description • favicon

Security + privacy Masks the user’s device information from a web site until the user has chosen to interact with it. Ensures site doesn’t contain SPAM or malicious content.

De-duplication*If several beacons with the same URL are used within a space the user's will only see one URL.

Optimisation Augments and enriches the basic results to improve usefulness and usability. e.g. sorting, ranking and filtering for relevance

User benefits

*Chrome only (for now)

Miscellaneous FAQ

HTTPS only (on Chrome*) All communications between your browser and the website are encrypted.

All interactions are ‘on the web’ Once a user selects a site, all subsequent interactions take place on the web so automatically conform to privacy-preserving behaviours such as opting-in to enable geolocation or web notifications.

Q: Are any other user safeguards built in?

*For now. Other scanners will hopefully emulate this.

No. There are many ways to broadcast a URL. Today, the Physical Web uses BLE because of its ubiquity on mobile devices today. Additionally, it is highly energy efficient—many bluetooth beacons today have multi-year battery lives.

The hope however, is that other useful formats will be supported.

Two strong candidates are mDNS and uPnP—transport protocols that enables users who are logged into wi-fi to discover beacons broadcasting on that same wi-fi network.

Q: Will/does the Physical Web only support BLE?

See Github for the latest discussions on mDNS

An outstanding issue with wi-fi based protocols is that the devices that are most likely to broadcast this way (e.g. TVs, printers, smart home appliances) will most likely broadcast a local IP address rather than a public URL.

The (cloud-based) proxy will therefore not be able to follow this link to retrieve the page title and description, perform customary security checks, or further optimise the discovery experience.

Challenges with wi-fi based discovery

See Github for the latest discussions on local IP-based discovery.

Eddystone is a new open source protocol specification from Google that defines a BLE message format for proximity interactions.

Eddystone broadens what can be done with beacons by broadcasting up to four formats (or ‘frame’ types): • Eddystone UID, EID and TLM—Which all work with apps and

enable interactions similar to iBeacon. • Eddystone URL—Which the Physical Web is now based on and works

with a browser.

Q: What is Eddystone?

Thought experiment: Does the scanner have to be a browser?

No. The browser is merely an app that incorporates the Physical Web specification. This specification is open source, so companies could build the ability to “see” Physical Web URLs into other apps.

“…the watches glow and vibrate when you walk somewhere in the real world

that corresponds with somewhere inPokémon Go's virtual world”

Source: The Verge - Pokemon Go Plus hands on photos

https://www.flickr.com/photos/25958224@N02/8122856863

…an antenna implanted in his skull allows him to ‘hear’ the racks upon racks of brightly coloured packaging in the aisles as a sensor converts colour frequencies into sound.

- Vice: The Creator’s Project

“artist Neil Harbisson

Martin Luther King Jr.'s "I Have A Dream" painted by Neil Harbisson.

which means Neil can listen to paintings…

TED: I listen to color

and also paint everything he hears.

Each colour is assigned an audible frequency,

3. Three key use cases

40

Use case 1: Pure discovery (“A much smarter QR code”)

Use case 2: Interact with moderately “smart” things

Use case 3: Directly control an object

tadaslab on Instagram “Call a taxi” button attached to a tree #iot

“…the significance of technologies such as RFID and 2D barcoding is that they offer a low impact way to import physical objects into the datasphere—to endow them with an informational shadow.

- Adam Greenfield

Seriously? Is that it? That’s only marginally

more useful than typing a URL yourself, or just

googling it…

https://www.flickr.com/photos/neilghamilton/10389735244

url/greatKurdishFoodNearby

…unless you’re waaaaay at the backtop range for beacons is currently ~100m (300ft) and while a wider

smart poster

range will drain more power, this poster is powered, so that’s ok

https://www.flickr.com/photos/alsaarom/8258444009

Tube, DLR, London Overground

Part closureLondon Overgroud

Part closureNorthern

Service closedWaterloo & City

Good service on all other lines

Average wait at North Entrance is 12 minutes

https://www.flickr.com/photos/oatsy40/24775669489/

you can also dynamically deep-diveto the exact content that suit a user’scontext and location

https://www.flickr.com/photos/marketingfacts/6323249188/

...in Korea, grocery stores are embeddedon Subway platforms where users scan QR codes to

buy items that are delivered just-in-time for dinner

Snapchat Geofilters are small content bundles that become available once a user reaches a given (geofenced) place.

instead of merely delivering information,the URL could provide a location-relevant

utility or experiential application

…or attach URLs to a “thing”whose identity is more important

https://www.flickr.com/photos/morebyless/14246207164

Hi, i’m Narelle! Ask me anything about vikings (or join my class on Thursdays and Sundays at 14:00).

REGISTER

than its context

Miscellaneous FAQ

By design, the Physical Web does not push messages (and it will hopefully remain this way).

You can however expand on its base behaviours by combining it with other web technologies.

For example…

Q: Is it possible to push messages or notifications from a Physical Web beacon?

https://www.flickr.com/photos/sfj/288526372

Yikes, as you can see, we’re *really* busy!

There’s about a 20 minute wait, but our sister location Oishii still has three tables.

JOIN WAITLIST

SHOW ME OISHII

BOOK ANOTHER DAY

once the table is ready

customers can add their names toScenario:

a wait-list and receive a notification

1. Once a user has joined the queue, ask her to opt-in* to Web Push Notifications. These will be managed by a Service Worker—a new web standard that enables the browser to run scripts in the background—even when the page is not open.

2. Once the customer’s table is ready, a web service will message the service worker, that will then trigger the push notification.

User-centered push notification (…an example)

*opting-in is a default requirement for privacy sensitive web standards such as Geolocation and Push Notifications

Use case 1: Pure discovery (“A much smarter QR code”)

Use case 2: Interact with moderately “smart” things

Use case 3: Directly control an object

50

Most “smart” things we use these days are not that smart. While they can often be controlled using an app, very rarely does the app “speak” directly to the thing.

A brief “smart thing” primer…

Instead, the app often communicates with the cloud, or a local hub (or ‘bridge’) which then relays the command to the device.

1

Issue a command: ”Lamp on!”

For example…

1

cloud service API

The bridge in your home receives the command via wi-fi.

2

For example…

Issue a command: ”Lamp on!”

1

cloud service API

The bridge in your home receives the command via wi-fi.

23

It transmits the command (P2P) to nearby bulbs

For example…

Issue a command: ”Lamp on!”

1

cloud service API

The bridge in your home receives the command via wi-fi.

23

It transmits the command (P2P) to nearby bulbs

If other bulbs are too far from the bridge, the closest bulb uses a mesh network to pass the message along

4

For example…

Issue a command: ”Lamp on!”

1

cloud service API

The bridge in your home receives the command via wi-fi.

23

It transmits the command (P2P) to nearby bulbs

If other bulbs are too far from the bridge, the closest bulb uses a mesh network to pass the message along

4

For example…

smart

pretty smart

less smart

less smart

less smart

less smart

smart

Issue a command: ”Lamp on!”

…by extending this pattern to the web we can create all

sorts of rich and yet casual interactions while completely bypassing the friction of first

downloading an app

https://www.flickr.com/photos/charlottemorrall/3778508426

GUMBOT Bet you don’t have a quarter? Am I right or am I right?

GUMBOT No sweat. How about one of these? That’ll be $0.25 please.

And once you reconsider how “smart” a device needs to be to create a useful (and convincing) experience—you may also want to reconsider the term ‘device’.

https://www.flickr.com/photos/neo_ii/7483010074

Now playing

LOGIN with SPOTIFY

VOTE FOR THE NEXT SONG

I Didn’t see it comingBelle and Sebastian

Monthly special for Spotify members.Log in to redeem your complementary virtual jukebox credit and choose a song we will play in the next 18 minutes.

the music system…or the café itself? in this scenario, is ‘the device‘

https://www.flickr.com/photos/charlottemorrall/3778508426

PS - Android users can even download the native app directly from the plane :-)

Air Canada Rouge replaced their seat-back entertainment system with a web app whose content is streamed from within the plane to a passenger’s personal device (or a rented iPad). Anyone with a browser can access the service—but passengers are incentivised to download the app to access premium content.

Use case 1: “A much smarter QR code”

Use case 2: Interact with moderately “smart” things

Use case 3: Directly control an object

60

Web Bluetooth is an open web standard that enables users—in a secure and privacy-preserving way—to discover smart devices, communicate with them, and use a web page to directly control them.

Support levelsThe technology is completely separate from the Physical Web but highly complimentary. It is nearing launch on Chrome and is currently testable behind a Chrome ‘flag’.

ChromeAndroid + iOS

(behind Dev flag)

Opera(Labs browser)

Mozilla(Experimental)

Microsoft Edge (coming soon?)

Safari Mobile ????

Built-in security features HTTPS Only All communications between your browser and the website (and in this case, the object) are encrypted.

User Gesture Required As a security feature, discovering nearby Bluetooth devices must be called via a user gesture like a touch or mouse click.

FitBit

Heart Rate Monitor GO9

Visit the thing’s URL e.g. shown on the thing’s package, accessible via QR code, Physical Web beacon Tap to connect. You can now

interact with the device!

4

How it works: Device discovery and pairing

1

Choose device and grant permission to pair with the device.

2

Parrot Drone

3

CONNECTED success!

https://www.youtube.com/watch?v=-FO9thLaiug

PAIR

Playbill candle

…one more thing

One neat thing about BLE—is that you can also use it to create new (personalized) web UIs for known devices*.

*whose services are exposed with the necessary read/write permissions.

• A standardised way for BLE devices to advertise their services to the outside world.

• Each device has services (e.g. battery service) which has characteristics (e.g. battery level: 0-100).

• BLE has a list of generic services for common devices such as, but products can also create their own.

• The value of a characteristics can be read, you can also write to it and request notifications* when the value changes.

*the web app (not the user) receives the notifications and uses them to update the UI as needed

Bluetooth (Generic Attributes) GATT 101

Each of these examples, taken by itself, is modestly useful. Taken as a whole, however, they imply a vast "long tail" where anything can offer information and utility.

— Scott Jenson, Google“

https://www.flickr.com/photos/jsome1/1243493095

4. Design considerations

70

It used to be fairly straightforward for users to understand what a thing was, what it could do, and how to make it do those things.

Photo of Sniff by Timo Arnall on Flickr, used with permission

Sniff - the RFID enabled toy dog

With IoT (and connected things in general) this becomes far more challenging.

Photo of Sniff by Timo Arnall on Flickr, used with permission

speaker

RFID reader

vibration motor

battery

vibration motor

The conceptual model is the understanding and expectations you want the user to have of the system.

What components does it have, how does it work, and how can they interact with it? It’s the mental scaffolding that enables users to figure out how to interact with your service.”

— Claire Rowland

“

1 cloud service API

23

4

The system might in reality do this…

Issue a command: ”Lamp on!”

The bridge in your home receives the command via wi-fi.

It transmits the command (P2P) to nearby bulbs

If other bulbs are too far from the bridge, the closest bulb uses a mesh network to pass the message along

But (if you’re lucky) the user sees (something like) this…

It’s OK for a user’s conceptual model to be incorrect, so long as this doesn’t prevent them from easily and safely operating the product.

But (if you’re lucky) the user sees (something like) this…

The whole point of using the web to interact with things and spaced is it to enable random, often one-time interactions that occur ‘in the wild’.

In this context (whether we like it or not) there will often be little time or incentive for a user to ‘properly’ learn how things work.

https://twitter.com/collision/status/729166303253041152

The guidelines in this section tend to work best the more of them you implement. The ultimate goal is to enable users to quickly and easily develop a useful and plausible conceptual model.

i. Tune the brain’s broadcast range Tune the range to aid discoverability and clarify the relationship between the object and the space it inhabits.

*The advertising range can drop significantly if adjacent to walls which are made of metal or brick.

BLE has a typical unobstructed broadcast range of 70-100m*. Most beacons allow you to adjust their beacon’s transmission power—which will in effect—adjust this range.

An ideal setting prioritises user context while being mindful of beacon battery life (>power = >battery consumption).

Sample beacon range for a Kontakt.io beacon.

Unnecessarily wide or greatly overlapping ranges can result in premature discovery, introduce ambiguity and increase cognitive load.Is there actual value in

being able to “see” this parking meter’s URL from 100m away? Are you merely creating noise?

Tuning the range to minimise or avoid overlap can reduce friction, minimise errors, and increase trust in the overall service.

This is particularly important if the service has a cost (…which in the case of a vending machine might prove a minor inconvenience, while in the case of the parking meter could result in a substantial fine or a towed vehicle!)

How costly is human error in this particular context?

https://www.flickr.com/photos/crondeau/14314596362/

url/specialExhibitDouglasCoupland

url/thisDonutThingHereThatDougWantsHelpIdentifying

Douglas Coupland, “The Brick Wall”, 2005/2014 assemblage with pieces from the following toys and various untraceable construction sets.

this technique isn’t limited toobjects you interact with…

ii. Provide clear (and ideally) real-time feedback Strong feedback improves learnability, promotes trust and helps users feel in-control.

iii. Use consistent signposting Content in the physical environment should—where possible—match that used on the web interface.

Space #2304

Meter #2304

Meter #2304Main St, 10 m from Fir St.

$2.50FOR 1 HOUR

PARK HERE

ends at 13:47pm

SPOT is brilliantly simple way to improve usability and increase trust.

“Tap a colored button and look for the Uber with the matching colored SPOT light. Passengers can even hold down on one of the buttons to turn their phone that colour and wave it in the air to help their driver find them.”

iv. Provide just enough content Consider how much content you really need.

there’s nothing wrong with simply mirroringa beacon architecture to its physical

https://www.flickr.com/photos/morebyless/14246207164

url/grandGallery

url/artDesign

url/artDesign

url/artDesign

url/naturalWorldurl/naturalWorld

url/naturalWorld

and virtual equivalent

Does my interaction with this flower pot require much more than a disembodied record-set?

[yes] [no] [maybe]

https://www.flickr.com/photos/badlydrawn/15972048661

SODERHAMNFind it in store105.523.23

Aisle: Location: 0227

Download instructions

If however, the information’s role is to help complete a specific task—then the content should probably be more specific to that task.

Does my interaction with this flower pot require much more than a disembodied record-set?

[yes] [no] [maybe]

Source: Estimote Nearables

https://www.flickr.com/photos/crondeau/14314596362/

Douglas Coupland, “The Brick Wall”, 2005/2014 assemblage with pieces from the following toys and various untraceable construction sets.

What is this object? Where is it from? Share your ideas at #dougsDonutThing

Doug Coupland @douglascoupland

63%

37%

@douglascoupland

511 votes

A toilet float

A toy

url/thisDonutThingHereThatDougWantsHelpIdentifying

bundles of just-in-time content and micro-interaction…

as we exchange smaller and smaller

…it’s not clear we’ll evenneed to open an app* at all*native or otherwise—remember, notifications (with actions)now exist in the browser as well

url/starbucksBranch_0123

https://www.flickr.com/photos/130000572@N03/16285653016/

Receipt AvailableTipping available until 12:09

$0.50 $1.50 $2.00

11:20 AM

v. Anticipate novel uses Anticipate URL re-use in novel contexts and, where needed provide clear guidance to avoid unexpected errors.

A few thoughts… • Remember the old debate about all the things “mobile users

won’t want to do”? (Hint: that list is super tiny)

• Once a user discovers a URL in the real world there’s no reason they can’t bookmark it for later use.

• So…what (if anything) will users “NOT want to do” with that URL when they are no longer nearby?

https://www.flickr.com/photos/sfj/288526372

Yikes, as you can see, we’re *really* busy!

There’s about a 20 minute wait, but our sister location Oishii still has three tables.

JOIN WAITLIST

SHOW ME OISHII

BOOK ANOTHER DAY it’s not hard to imagineremote URL use for almost

any physical web context

What happens if someone joins remotely. Is the outcome positive or negative? And for whom is it so? [the user] [the restauraunt] [the other people in the queue]

Is the physical touchpoint merely an entry point to future (virtual) interaction?

When officials in Melbourne assigned email addresses to trees (to enable people to report dangerous branches)…some people sent in letters and poems for the trees.

https://www.flickr.com/photos/rbh/14431865903

APOLOGIES To: London Plane, Tree ID 1032398, St Kilda Rd

My dog pee’d on you the other day.

Sincerest apologies.

A STRANGE QUESTION To: Western Red Cedar,Tree ID 1058295 Hi Tree, Are you worried about being affected by the Greek debt crisis? Should Greece be allowed to stay in the European Union?

HELLO TREE To: Green Leaf Elm, Tree ID 1022165

Dear Green Leaf Elm, I hope you like living at St. Mary’s. Most of the time I like it too. I have exams coming up and I should be busy studying. You do not have exams because you are a tree. I don’t think that there is much more to talk

3. Group exercise

11:00-11:50 • Group brainstorming, discussion and design.

11:50-12:30 • Group presentations • Limited to ~5 minutes each so we can fit as many as

possible.

Group exercise (11:00-12:30)

http://www.flickr.com/photos/tinou/453593446

thank you

many thanks to the amazing photographers on

http://creativecommons.org/licenses/by/2.5

@yiibu

hello@yiibu.comcontact us at

Presentation deck available @ http://www.slideshare.net/yiibu