15-440 distributed systems kmeansmhhammou/15440-f19/... · p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14...
TRANSCRIPT
![Page 1: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/1.jpg)
15-440 Distributed Systems
Kmeans
November 14, 2019
Zeinab Khalifa
![Page 2: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/2.jpg)
Agenda
• Sequential Kmeans
• Parallelizing
• DNA clustering
![Page 3: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/3.jpg)
Sequential Kmeans
![Page 4: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/4.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
The blue and red starts are called unlucky centroids (*)A poor choice of the initial centroids will take longer to converge or may result in bad clustering. You can handle this in: 1. Your data generators (generate first k points to be far
apart and pick them in your implementation)2. Try different sets of random centroids, and choose the
best set.
Initial centroids/means
![Page 5: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/5.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
![Page 6: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/6.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
4
10
2
9C0
C1
C2
C3
![Page 7: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/7.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
4
10
2
9C0
C1
C2 P1
C3
![Page 8: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/8.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
C0
C1 P2
C2 P1
C3
![Page 9: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/9.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
C0
C1 P2 + P3
C2 P1
C3
* P1 + P2 = (x1,y1) + (x2,y2) = (x1+x2, y1+y2)
![Page 10: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/10.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
C0
C1 P2 + P3
C2 P1 + P4
C3
![Page 11: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/11.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
C0
C1 P2 + P3
C2 P1 + P4
C3 P5
![Page 12: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/12.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Initial centroids/means
C0 P6 + P8 + P10 + P13
C1 P2 + P3 + P7 + P11
C2 P1 + P4 + P12 + P15 + P16
C3 P5 + P9 + P14
![Page 13: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/13.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
Centroids after iteration 1
C0 (P6 + P8 + P10 + P13)/4
C1 (P2 + P3 + P7 + P11)/4
C2 (P1 + P4 + P12 + P15 + P16)/5
C3 (P5 + P9 + P14)/3
* P/N = (x/N,y/N)
![Page 14: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/14.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 Pc0
C1 Pc1
C2 Pc2
C3 Pc3
When to stop?
![Page 15: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/15.jpg)
Parallel K-Means
![Page 16: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/16.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
How can we parallelize?C0 P2 + P3
C1 0
C2 P1
C3 P4
C0 P5
C1 P7
C2 P8
C3 P6
C0 0
C1 P12
C2 P10 + P11
C3 P9
C0 P13 + P14 + P16
C1 0
C2 0
C3 P15
![Page 17: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/17.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
How can we parallelize?C0 P2 + P3
C1 0
C2 P1
C3 P4
C0 P5
C1 P7
C2 P8
C3 P6
C0 0
C1 P12
C2 P10 + P11
C3 P9
C0 P13 + P14 + P16
C1 0
C2 0
C3 P15
![Page 18: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/18.jpg)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
C0 P1
C1 P6
C2 P3
C3 P9
How can we parallelize?C0 P2 + P3 + P5 + P13 + P14 + P16
C1 P7 + P12
C2 P8 + P10 + P11
C3 P6 + P9 + P15
/6
/2
/3
/3
![Page 19: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/19.jpg)
DNA stranding
![Page 20: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/20.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
![Page 21: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/21.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
How to get the centroid of these DNA strands?
![Page 22: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/22.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
A
C
G
T
Output strand
How many repetitions of A in index 0 of all
strands
![Page 23: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/23.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
A 2
C
G
T
Output strand
![Page 24: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/24.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
A 2 1 2 1
C 0 1 1 0
G 1 1 1 1
T 1 2 1 2
Output strand
![Page 25: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/25.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
A 2 1 2 1
C 0 1 1 0
G 1 1 1 1
T 1 2 1 2
Output strand
Get the mean or the median (sort the values and select the
middle one)
![Page 26: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/26.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
A 2 1 2 1
C 0 1 1 0
G 1 1 1 1
T 1 2 1 2
Output strand
T G C A
![Page 27: 15-440 Distributed Systems Kmeansmhhammou/15440-f19/... · P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 C0 P1 C1 P6 C2 P3 C3 P9 The blue and red starts are called unlucky centroids](https://reader030.vdocuments.net/reader030/viewer/2022040413/5f09b3b57e708231d4281acc/html5/thumbnails/27.jpg)
ACTG
GTCA
SGGT
TAAA
ATAT
A 2 1 2 1
C 0 1 1 0
G 1 1 1 1
T 1 2 1 2
Output strand
T G C A