pervasive web content delivery with efficient data reuse chi-hung chi and cao yang school of...
TRANSCRIPT
![Page 1: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/1.jpg)
Pervasive Web Content Delivery with Efficient Data Reuse
Chi-Hung Chi and Cao Yang
School of ComputingNational University of SingaporeEmail: [email protected]
![Page 2: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/2.jpg)
Outline Introduction
Scenarios Observation Potentials and Challenges
System Model and Architecture Requirements and Design Consideration Scalable Data Model System Architecture and Proxy Example
Related Work
Conclusion
![Page 3: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/3.jpg)
Part 1
Introduction
![Page 4: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/4.jpg)
Scenarios Digital Images on the Internet – accounts for 40% of the data bytes accessed across the
WWW
Internet Client Variation – in terms of hardware capabilities, user tolerance level,
personal preference, etc.
Network Variation– in terms of network bandwidth, timeout latency, etc.
![Page 5: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/5.jpg)
Problem
Web-based digital image libraries
Single copy approach Problem: not able to accommodate the wide
spectrum of Internet clients
Multiple copy approach Problem: hard to maintain
– What versions to keep? – Users’ requirements keep changing.
![Page 6: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/6.jpg)
Observation HTTP/1.1 protocol – Allows a client to request
portions of an object via Range requests.Example:Range Request
HTTP/1.1 206 Partial ContentDate: Thu, 03 May 2001 09:22:12 GMTLast-Modified: Thu, 03 May 2001 07:12:27 GMTContent-Range: bytes 2500-4000/7614Content-Length: 1501
Get http://www.nus.edu.sg Range: bytes=2500-4000
Server’s reply
![Page 7: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/7.jpg)
Potentials Single-client scenario
ClientServer
Request: 1 - 100K byte
Image X 400KB
Reply: 1 – 100K byte
1 – 100KB
Effective network speed: 1KB/s
Timeout latency: 100s
![Page 8: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/8.jpg)
Potentials Single-client scenario
ClientServer
Request: 100K - 200K byte
Reply: 100K – 200K byte
1 – 200KB
Effective network speed: 1KB/s
Timeout latency: 100s
1 – 100KB
Image X 400KB
![Page 9: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/9.jpg)
Potentials Multiple-clients scenario
Client 1
Server
Image X 400KB
ProxyClient 2
Client 3
Effective network speed: 1KB/s
Timeout latency: 100s
Request XRequest X
1 – 100KB
1-100KB
1 – 100KB
1 – 100KB
![Page 10: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/10.jpg)
1 – 200KB
Potentials Multiple-clients scenario
Client 1
Server
Image X 400KB
ProxyClient 2
Client 3
Effective network speed: 1KB/s
Timeout latency: 100s
1 – 100KB
Request XRequest X
100KB – 200KB
1 – 100KB
1 – 200KB1 – 200KB
![Page 11: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/11.jpg)
Big Idea! The potential benefits of JPEG 2000 and
HTTP/1.1 protocol can be maximized by storing and reusing partial objects.
Over slow network link, as more clients requests for the target image, the visual effect improves until the original quality is obtained.
![Page 12: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/12.jpg)
Our Solution Exploits the potentials of JPEG 2000 and
HTTP/1.1 protocol.
A proxy-based transcoding system that stores and reuses partial objects.
Allows scalable image web delivery.
Minimizes network bandwidth consumption and client latency.
![Page 13: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/13.jpg)
Part 2
System Model and
Architecture
![Page 14: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/14.jpg)
System Requirements
Content adaptation
Minimal bandwidth consumption
Real-time delivery
![Page 15: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/15.jpg)
Design Considerations
Proxy-based
On-demand transcoding
Streamed transcoding
Small transcoding overhead
Bandwidth optimization (Along entire path)
Data reuse
![Page 16: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/16.jpg)
Scalable Data Model Basic Terminology
B: Set of all data bits BLK: Set of all blocksO: Set of all data objects P: Set of all presentations
Definition 1: Bit StreamOn = Union(b1b2b3 … bm-1bm) where OnO, and biB with i[0,m].
Definition 2: Blockblk(i, j) = Union( bibi+1 … bj-1bj ) where blk(i,j)BLK and bnB with 0<i<j and n[i, j].
Definition 3: PresentationBit expression: Pi = f (b1b2b3 … bi+1bi )where bi B, i size of original object, and f represents the encoding function.
Block Expression:
Pi = f (blk(1,I1) blk(I1+1, I2) blk(I2+1, I3) … blk(Ik+1, i))where 0< I1< I2< I3… < Ik <i-1, and f represents the encoding function.
![Page 17: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/17.jpg)
Scalable Data Model
Basic Properties
Property 1: Inclusive Property
Pj = F (Pi , blk(i+1, j)) where i<j, blk(i+1, j) BLK, and F denotes the encoding function that generate Pi by using Pi and blk(i+1, j)
Property 2: Union Function F
The encoding function F should be union function.
![Page 18: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/18.jpg)
Scalable Data Model
Property 3: Single Pass Property
According to inclusive property, we have:Pi = F (Pi-1 , blk(i, i) ) or Pi = F (Pi-1 , bi )
Similarly,Pi-1 = F (Pi-2 , bi-1 )Pi-2 = F (Pi-3 , bi-2 )
… … P2 = F (P1 , b2 )P1 = F (b1 )
![Page 19: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/19.jpg)
Scalable Data Model Transcoding OperationsSuppose we want to transform between a lower quality presentation Pi and a higher presentation Pj. Based on our data model, the following equation holds: Pj = Union (Pi , blk(i+1, j))
Conversion from higher quality to lower qualityTo transform Pj into Pi, simply discard the data bits ranging from bit (i+1)
to bit j. Conversion from lower quality to higher qualityTo transform Pi into Pj, we only need to retrieve the data bits ranging from
i+1 to j, and then append them to presentation Pi.
![Page 20: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/20.jpg)
Scalable Data Model
Impact on Image Transcoding
Allows reuse of partial objects
Allows streamed transcoding
Incurs minimal transcoding overhead
Transcoding Computation Instead Block Synthesis
![Page 21: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/21.jpg)
System Architecture
Assembling
Transcoding HTTP Proxy
Client
Server
ProxyCach
eFiltering
Request Customizatio
n
Client’s Request
Server’s Reply
![Page 22: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/22.jpg)
Proxy Cache Caching partial objects
Introducing the notion of “partial hit” Cache hit Cache miss Partial hit
Enforcing single range Multiple ranges are not allowed. The cached content must start with the first data bit.
![Page 23: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/23.jpg)
Example: Cache Miss
ProxyClient 2
Client 3
Client 1
Client 4
Content Provide
rImage X 400KB
Req: 1-100KBReq: 1-100KB
1-100KB
1-100KB
1-100KB
1-100KB
1-50KB
1-200KB
![Page 24: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/24.jpg)
1-150KB
1-100KB1-150KB
Example: Partial Hit 1
ProxyClient 2
Client 3
Client 1
Client 4
Content Provide
rImage X 400KB
1-100KB
1-50KB
Req: 50KB-150KB
100KB-150KB
Req: 100KB-150KB
50KB-150KB
1-200KB
![Page 25: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/25.jpg)
1-150KB
1-150KB
Example: Partial Hit 2
ProxyClient 2
Client 3
Client 1
Client 4
Content Provide
rImage X 400KB
1-100KB
Req:
1-100KB
1-200KB
1-100KB1-100KB
![Page 26: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/26.jpg)
1-300KB
1-300KB
1-150KB
Example: Partial Hit 3
ProxyClient 2
Client 3
Client 1
Client 4
Content Provide
rImage X 400KB
1-100KB
Req:
200KB-300KB
1-150KB
1-200KB
1-100KB
Req: 150KB-300KB
150KB-300KB
200KB-300KB
![Page 27: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/27.jpg)
Streaming and PipeliningTranscodin
g HTTP Proxy
Client
Server
Client Proxy Server
Cached content
Unavailable content
Pipelining of data bytes transferred over proxy-client link and server-proxy link
![Page 28: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/28.jpg)
Example JPEG 2000 – Scalability in image quality
2KB 4KB 8KB
16KB24KB32KB (original)
![Page 29: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/29.jpg)
Model Analysis
Proxy-based
On-demand transcoding
Streamed transcoding
Small transcoding overhead
Bandwidth optimization (both client & server)
Data reuse
![Page 30: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/30.jpg)
Part 3
Related Work
![Page 31: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/31.jpg)
Related Work Progressive Data Formats
Interlaced GIF
Progressive JPEG
JPEG 2000
MPEG-4
![Page 32: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/32.jpg)
Related Work Transcoding Systems
Mowser
GloMop
InfoPyramid
Hybrid Transcoding System
![Page 33: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/33.jpg)
Current Transcoding Systems Assumption
The transcoding process cannot commence until the object has been downloaded in its entirety.
Limitations Does not support scalable image web delivery. Can not cache or reuse partial objects. Store-and-forward transcoding – break the
pipeline of data bytes transferred across the network
![Page 34: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/34.jpg)
Part 6
Conclusion
![Page 35: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/35.jpg)
Contributions Scalable data model
Proposed a general data model that allows reuse of partial objects.
Identified inclusive property that enables scalable web delivery.
The data model enables streamed transcoding and minimizes transcoding overhead.
The data model applies to any multimedia type such as image, audio and video.
![Page 36: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/36.jpg)
Contributions System Model
Exploited the potential benefits of scalable data format and HTTP/1.1 protocol.
Supports scalable web delivery with cumulative effect.
Proposed the idea of caching and reusing partial objects.
Introduced the concept of partial hit. Benefits: content adaptation, minimal
network bandwidth consumption and real-time delivery.
![Page 37: Pervasive Web Content Delivery with Efficient Data Reuse Chi-Hung Chi and Cao Yang School of Computing National University of Singapore Email: chich@comp.nus.edu.sg](https://reader036.vdocuments.net/reader036/viewer/2022062713/56649ccb5503460f949945ba/html5/thumbnails/37.jpg)
Q & A