exact and distributed algorithms for collaborative camera control dezhen song * a. frank van der...
TRANSCRIPT
![Page 1: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/1.jpg)
Exact and Distributed Algorithms for
Collaborative Camera Control
Dezhen Song*
A. Frank van der Stappen†
Ken Goldberg*
* UC Berkeley, USA
† Utrecht University, Netherlands
![Page 2: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/2.jpg)
Dezhen Song
![Page 3: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/3.jpg)
Webcams
![Page 4: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/4.jpg)
![Page 5: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/5.jpg)
n users 1 pan, tilt, zoom robotic camera
![Page 6: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/6.jpg)
“ShareCam”
![Page 7: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/7.jpg)
Example input: 7 requested frames:
![Page 8: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/8.jpg)
One Optimal Frame
ShareCam Problem: Given n requests, find optimal frame
![Page 9: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/9.jpg)
Taxonomy (Tanie, Matsuhira, Chong 00)
Multiple Operator, Single Robot (MOSR):
Single Operator, Single Robot (SOSR):
Single Operator, Multiple Robot (MOSR):
![Page 10: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/10.jpg)
Related Work• MOSR systems
– Cinematrix (91)– Cannon, McDonald, et al. (97) – Goldberg, Chen, et al. (00, 01) – Goldberg, song, et al. (02)
• Internet robots– Tanie, K., Chong, N. Et al(01)– Jia, S. And K. Takase (01)– Hu, H., Yu, L., Tsui, P., Zhou, Q (01)– Safaric, R. Et al. (01)– Goldberg and Siegwart (02)
![Page 11: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/11.jpg)
Related Work• Facilities Location Problems
– Megiddo and Supowit [84]– Eppstein [97]– Halperin et al. [02]
• Rectangle Fitting – Grossi and Italiano [99,00]– Agarwal and Erickson [99]– Mount et al [96]– Kapelio et al [95]
![Page 12: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/12.jpg)
Related Work• Similarity Measures
– Kavraki [98]– Broder et al [98, 00]– Veltkamp and Hagedoorn [00]
• Distributed robot algorithms – Sagawa et al [01], Safaric[01]– Parker[02], Bulter et al. [01]– Mumolo et al [00], Hayes et al [01]– Agassounon et al [01], Chen [99]
![Page 13: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/13.jpg)
Problem Definition
Requested frames: i=[xi, yi, zi], i=1,…,n
![Page 14: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/14.jpg)
Problem Definition• Assumptions
– Camera has fixed aspect ratio: 4 x 3– Candidate frame = [x, y, z] t
– (x, y) R2 (continuous set)– z Z (discrete set)
(x, y)3z
4z
![Page 15: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/15.jpg)
Problem Definition• “Satisfaction” for user i: 0 Si 1
Si = 0 Si = 1
= i = i
![Page 16: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/16.jpg)
•Symmetric Difference
•Intersection-Over-Union
SDArea
AreaIOU
i
i
1)(
)(
)(
)()(
i
ii
Area
AreaAreaSD
Satisfaction Metrics
Nonlinear functions of (x,y)
![Page 17: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/17.jpg)
• Intersection over Maximum:
),(
)(
),max(
)1,)/min(()/(),(
i
i
i
i
biiii
Max
Area
aa
p
zzaps
Requested frame i , Area= ai
Candidate frame
Area = a
Satisfaction Metrics
pi
![Page 18: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/18.jpg)
Intersection over Maximum: si( ,i)
si = 0.20 0.21 0.53
Requested frame i
Candidate frame
![Page 19: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/19.jpg)
),(),( yxpyxs iii
),( yxpi
Requested frame i Candidate
frame (x,y)
)1,)/min(()/(),( biiii zzaps
(for fixed z)
![Page 20: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/20.jpg)
Satisfaction Function
– si(x,y) is a plateau •One top plane•Four side planes•Quadratic surfaces at corners•Critical boundaries: 4 horizontal, 4
vertical
![Page 21: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/21.jpg)
Objective Function• Global Satisfaction:
n
iii
n
i
biii
yxpyxS
zzapS
1
1
),(),(
)1,)/min(()/()(
for fixed z
ShareCam problem: Find * = arg max S()
![Page 22: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/22.jpg)
S(x,y) is non-differentiable, non-convex, but
piecewise linear along axis-parallel lines.
Properties of Global Satisfaction
![Page 23: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/23.jpg)
ShareCam Algorithms
Bruteforce Algorithm– Compute S at each pixel (x,y)– O(whmn):
•w, h: width and height of panoramic image
•m: number of zoom levels•n: # users
![Page 24: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/24.jpg)
Approximation Algorithm
spacing zoom :
spacing lattice :
zd
dx
y
d
Compute S(x,y) at lattice of sample points:
![Page 25: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/25.jpg)
Approximation Algorithm
– Run Time: – O(w h m n / d2)
* : Optimal frame
: Optimal at lattice ~
: Smallest frame at lattice that encloses *
)ˆ()~
()( * sss
)(
)ˆ(
)(
)~
(1
**
s
s
s
s
ddz
z
z
min
min...
![Page 26: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/26.jpg)
Exact Algorithm
• Define “Virtual corners” – Consider a pair of requested frames
and – Their critical boundariesy
x
![Page 27: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/27.jpg)
Exact Algorithm• Virtual corner: Intersection between
boundaries– Self intersection:– Frame intersection:
y
x
![Page 28: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/28.jpg)
Exact Algorithm
• Claim: An optimal point occurs at a virtual corner. Proof:– Along vertical boundary, S(y) is a 1D
piecewise linear function: extrema must occur at x boundaries
![Page 29: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/29.jpg)
Exact Algorithm
Exact Algorithm:Check all virtual corners
(mn2) virtual corners(n) time to evaluate S for each
(mn3) total runtime
![Page 30: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/30.jpg)
Improved Exact Algorithm• Sweep horizontally: solve at each
vertical – Sort critical points along y axis: O(n
log n)– 1D problem at each vertical boundary
O(nm) – O(n) 1D problems– O(mn2) total runtime
O(n) 1D problems
![Page 31: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/31.jpg)
Distributed Algorithm
More users More computers available
![Page 32: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/32.jpg)
Distributed Algorithm• At the Server
– Sort horiz. boundaries– O(n log n)
• At the Client– Solve 1D problem
for own vertical boundaries.
– O(nm)
• O(n(m+ log n)) Total Four 1D problems
![Page 33: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/33.jpg)
Examples
![Page 34: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/34.jpg)
Examples
![Page 36: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/36.jpg)
• New Approx Algorithms:– With Har-Peled, Koltun– Stair-like
approximation– Dynamic segment tree– O(n log n)
• Weighted Requests
Current Work
![Page 37: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/37.jpg)
Future Work
• Continuous zoom (m=)• Multiple outputs:
– p cameras – p views from one camera
• “Temporal” version: fairness– Integrate si over time: minimize accumulated
dissatisfaction for any user
• Network / Client Variability: load balancing
• Obstacle Avoidance
![Page 38: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/38.jpg)
The Tele-Actor
Operators
Server
![Page 39: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/39.jpg)
Summary• Satisfaction Metric: • Intersection over Maximum• ShareCam Problem : find * = arg max
S()• Critical Points at Virtual Corners• Exact Algorithms: • Distributed Algorithm:• tele-actor.net/sharecam
O(mn2)
O(mn)
![Page 40: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/40.jpg)
![Page 41: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/41.jpg)
Summary
• A collaborative camera control system
• Satisfaction metric• Virtual corner based
algorithms• Distributed algorithm• www.tele-actor.net/
sharecam
![Page 42: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/42.jpg)
Introduction
• Regular web-cam • Collaborative camera control
Queue
![Page 43: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/43.jpg)
![Page 44: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/44.jpg)
Internet Interface:
![Page 45: Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht](https://reader030.vdocuments.net/reader030/viewer/2022012910/5697bf831a28abf838c86463/html5/thumbnails/45.jpg)
Results & Discussion
• Speed of naive (B) and fast (V): – AMD K7 950Mhz– 1.2 GB memory– JDK 1.3.1– For a fixed z