performance of simulated annealing, simulated quantum ...performance of simulated annealing,...
TRANSCRIPT
||Matthias Troyer
Performance of simulated annealing, simulated quantum annealing and D-Wave on hard spin glass instances
1
||Matthias Troyer
▪ Troels Rønnow (ETH) ▪ Sergei Isakov (ETH → Google) ▪ Sergio Boixo (USC → Google) ▪ Joshua Job (USC) ▪ Zhihui Wang (USC) ▪ Bettina Heim (ETH, BSc student) ▪ Damian Steiger (ETH) ▪ Ilia Zintchenko (ETH) ▪ Dave Wecker (Microsoft Research) ▪ John Martinis (UCSB) ▪ Daniel Lidar (USC)
2
Collaborators
||Matthias Troyer 3
!Spin glasses on the D-Wave chimera graph
||Matthias Troyer
▪ NP-hard for non-planar lattices (Barahona 1982) ▪ No spin glass phase in finite-D lattices in a magnetic field (Young et al, 2004)
4
Worst-case complexity and spin glass physics
no magnetic field with magnetic field
2D planar polynomial spin glass phase at T=0
NP-hard no spin glass phase
2D non-planar NP-hard spin glass phase at T=0
NP-hard no spin glass phase
3D or higher dimensions
NP-hard spin glass phase with
NP-hard no spin glass phase
Infinite dimensions NP-hard spin glass phase with
NP-hard spin glass phase with
H = Jijij∑ sis j + hi
i∑ si + const. with si = ±1
||Matthias Troyer
▪ In the absence of a spin glass phase correlations are short-ranged. Can we thus solve typical spin glass problems locally?
▪ Spin glass Tc=0 in 2D. Is a 2D lattice the wrong system for realizing hard problems? see following talk by Helmut Katzgraber Katzgraber, Hamze, Andrist, Phys. Rev. X (2014)
5
Average case complexity
We observe average case polynomial scaling for a new algorithm see tomorrow’s talk by Ilia Zintchenko
||Matthias Troyer 6
!Annealing Simulated annealing !Adiabatic quantum optimization Quantum annealing Simulated quantum annealing
||Matthias Troyer
Image credit ANFF NSW node, University of New South Wales
Annealing A neolithic technology
Slowly cool metal or glass to improve its properties and get closer to the ground state
7
Long history of annealing
Simulated annealingKirkpatrick, Gelatt and Vecchi, Science (1983)
A classical optimization algorithm
Slowly cool a model in a Monte Carlo simulation to find the solution to an optimization problem
||Matthias Troyer
Quantum annealing for a transverse field Ising modelKadowaki and Nishimori (1998) Farhi, Goldstone, Gutmann and Sipser (2000)
8
Add a transverse magnetic field to induce quantum fluctuations
Initial time t=0: all spins aligned with the transverse field
Final time t=tf: ground state of the Ising spin glass
8
H (t) = B(t) Jijσ izσ j
z
i< j∑ − A(t) σ i
x
i∑
� ��� ��� ��� ��� ��
�
�
�
�
��
��
��
��
�� ���������
�����
�����
||Matthias Troyer
▪ Quantum annealing not necessarily stays adiabatic Kadowaki and Nishimori (1998)
▪ Adiabatic quantum optimization is the special case of perfectly coherent adiabatic evolution in the ground state Farhi, Goldstone, Gutmann and Sipser (2000)
▪ Experimental quantum annealing (QA) Quantum mechanical evolution in a material or device, potentially at finite temperatures Brooke, Bitko, Rosenbaum, Aeppli, Science (1999)
9
Quantum annealing
||Matthias Troyer
▪ “Schrödinger” dynamics (unitary) ▪ Exponential complexity on classical hardware
▪ Simulates the time evolution of a quantum system
▪ Unitary evolution in the ground state: U-QAKadowaki and Nishimori (1998)
▪ Open systems dynamics using master equations: OS-QA
▪ Quantum Monte Carlo dynamics (stochastic) ▪ Classical algorithms with polynomial complexity
▪ QMC samples the equilibrium thermal state of a quantum system
▪ Typically based on path integral Monte Carlo simulations: QMC-QAApolloni et al (1988), Santoro at al (2002)
▪ Mean-field MC version using coherence but no entanglement: MC-QAShin, Smolin, Smith, Vazirani, arXiv:1401.7087
10
Simulated quantum annealing
||Matthias Troyer
▪ Many quantum systems have effective semi-classical (mean-field) descriptions.
▪ “Quantum annealing” describes a process performed on a quantum system.
▪ “Quantum annealing” is not necessarily a statement about large-scale entanglement being important for the performance or it outperforming classical approximations.
11
Is quantum annealing quantum or classical?
||Matthias Troyer 12
Are the D-Wave devices quantum annealers?
||Matthias Troyer
Find hard test problems for the machine to solve
random couplings on all bonds of the chimera graph
13
Our experiments
hundred million experiments on D-Wave One
billions of simulations classical and quantum Monte Carlo
1000s of choices of couplings 1000-10000 repetitions of the annealing
10s of problem sizes vary the annealing time and schedule
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
1
S. Boixo et al, Nature Physics (2014)
||Matthias Troyer
1. Pick a specific instance of the couplings Jij and fields hi
2. Perform N = 1000 or more annealing runs and measure the final energy ▪ count the number of times S that we find a ground state ▪ calculate the success probability s = S/N of finding a ground state in
one run
3. Repeat for many instances of the couplings Jij and fields hi
4. Make a histogram of the success probabilities s
14
Success probability histograms
Num
ber o
f ins
tanc
es
S. Boixo et al, Nature Physics (2014)
||Matthias Troyer
Num
ber o
f ins
tanc
es
▪ Bimodal histogram for D-Wave One and the simulated quantum annealer ▪ D-Wave One is inconsistent with a classical annealer ▪ D-Wave One is consistent with a simulated quantum annealer ▪ D-Wave One does not look too similar to mean field spin dynamics
15
Comparing the histograms
Simulatedclassical annealer
Simulated quantum annealerD-Wave One
Mean field spin dynamics
Num
ber o
f ins
tanc
es
S. Boixo et al, Nature Physics (2014)
||Matthias Troyer
D-Wave One, gauge transformed
D-W
ave
One
16
Correlations
The correlation between a simulated quantum annealer and D-Wave is as good as the correlation of D-Wave with itself
16
Simulated quantum annealer
but 5% outliers:
calibration problems?
excellent correlations
H = Jiji, j∑ σ iσ j
σ i ← aiσ i with ai = ±1Jij ← aiajJij
Investigate calibration issues by using a gauge transformation
si = +1 si = -1
si = +1si = -1
or
D-W
ave
One
S. Boixo et al, Nature Physics (2014)
||Matthias Troyer
The same instances are hard and easy on D-Wave and the simulated quantum annealer
but not on D-Wave and mean-field spin dynamics or classical annealing17
D-Wave One
classical annealerclassical
spin dynamics
QMC-QA
Comparing the performance characteristicsby correlating success probabilities for N=108
||Matthias Troyer
Shin et al’s model
Sim
ulat
ed q
uant
um a
nnea
ler
▪ …QMC-QA correlates well with a finely tuned mean field version Shin et al., arXiv:1401.7087 !!!!!!!!!!!!
▪ My conclusion from this work: a quantum annealer at the temperatures where D-Wave operates might not profit much from quantum effects
18
D-Wave performs like a quantum annealer, but …
||Matthias Troyer
1. Does adiabatic quantum computing for the Ising spin glass have any speedup over classical algorithms?
2. Does finite-temperature quantum annealing have any speedup over classical algorithms?
3. Does the implementation in the D-Wave devices have any speedup over classical algorithms?
19
Three open questions about quantum annealing for typical (not worst case) problems
||Matthias Troyer 20
! the ultimate and only important test is quantum speedup
Scaling to larger problem sizes
||Matthias Troyer
▪ The hope that a quantum annealer outperforms a classical one is based on experiences with QMC-QA
▪ Simulated quantum annealing by QMC gives lower residual energies than simulated thermal annealing, but not for all models
21
Simulated classical versus quantum annealing
Santoro et al (2002), similar results by
Matsuda, Nishimori, Katzgraber (2009)
Spin glass Topical Review R419
10
100
100 1000 10000 100000 1e+06
ε res
(τ)
τ (inverse annealing rate)
CAQA (P=50)
QA+G (P=50)Field Cycling
Gardner Energy
35
40
45
50
10 100 1000P
Figure 9. Comparison between optimal linear-schedule classical (CA) and path-integral MonteCarlo quantum annealing (PIMC-QA) for a 3-SAT problem with N = 104 and α = M/N = 4.24.CA always performs better than PIMC-QA simulated with P = 50 Trotter replicas. The averageperformance of linear PIMC-QA is worse than that of CA, even if an improvement in the resultscan be obtained by introducing global moves (G) and by increasing P (in the inset the final averageenergy found by PIMC-QA after 2000 iterations for increasing P is plotted and compared with theaverage result of a CA of the same length, dashed line). The solid triangles are the data obtainedby the field cycling PIMC-QA hybrid strategy described in [39].
same experiment; in the case of QA, a second average was performed among the energiesof the P replicas, which are in general different. It can be seen that the linear-schedule CAalways performs better than the linear-schedule QA. No further improvement can be obtainedfor P ! 100, see inset of figure 9—a much larger value than in the case of the Ising spinglass and the TSP instance—but we chose P = 50 in order to extend the simulation time asmuch as possible. The asymptotic slope of the linear-schedule QA curves seems indeed to bedefinitely less steep than that of CA, independently of the number of replicas involved in thesimulation and of the use of global moves.
The sobering message converged by this failure is that superiority of QA over CA isnot universal, and is only achieved when we can use some understanding of the problem,especially when building the kinetic energy operator.
4.3. PIMC-QA of a double-well: lessons from a simple case
We would like to finish our discussion about path-integral Monte Carlo -based QA bymentioning recent results on a very simple case from which one can learn much about thelimitations of the method [42]. Suppose we want to perform a QA optimization of the simpledouble-well potential which was investigated in section 2.2 using PIMC. One is then lead tosimulate the behaviour of a closed polymer made up of P Trotter replicas {xk}(k = 1, . . . , P )
of the original particle, held at temperature β/P and moving in the potential Vasym with anearest-neighbour spring coupling, as shown in equation (37). One can actually be moresophisticated than that, and perform a higher order Trotter break-up, correct to O(β/P )4
instead of O(β/P )2, using, for instance, the Takahashi–Imada approximation [75]. Moreover,instead of performing single-bead moves, i.e., moves involving a single xk at a time, one canreconstruct, during the move, entire sections of the polymer, using the bisection method [29].We have applied this rather sophisticated PIMC to our textbook double-well problem, working
3SAT
Battaglia, Santoro, Tossati (2005)
||Matthias Troyer
▪ Split the total annealing time t into R faster repetitions with time ta = t / R ▪ escape a local minimum by a fresh attempt ▪ optimize ta for a class of problems to get the best algorithm
▪ Simulated annealing now shows better scaling with problem size
22
A different approach: many fast annealing attempts
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
Median SAMedian MFMedian SQA
32 72 128 200 288 392 512101
102
103
104
105
106
107
108
spins
meannumberofsweeps We need to carefully revisit the
evidence showing advantage of QA
||Matthias Troyer
We see different results if we extrapolate in the Trotter number
23
Preliminary new results: Trotter error dependenceB. Heim , T.F. Rønnow et al, unpublished
||Matthias Troyer
Multiply the effort with the Trotter number to compare computational cost
24
Performance as a classical optimization algorithmB. Heim , T.F. Rønnow et al, unpublished
||Matthias Troyer
▪ Quantum annealing has a bias towards a subset of the ground states ▪ Classical annealing better for exploring states uniformly
25
Differences in the ground states found
Matsuda, Nishimori, Katzgraber (2009)
Simulated thermal annealing nearly equal distribution
QMC-quantum annealing very uneven distribution
||Matthias Troyer
▪ Time to solution varies hugely between instances ▪ Simulated quantum annealing seems to have problems for high
quantiles (hard instances)
26
Look at more than the median (typical) scaling
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
32 72 128 200 288 392 512101
102
103
104
105
106
107
108
spins
meannumberofsweeps
quantiles
0.010.050.100.250.500.750.900.950.99
Simulated thermal annealing
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!!
!
!!
!!
!
!
!
!!
!!
!!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
32 72 128 200 288 392 512101
102
103
104
105
106
107
108
spins
meannumberofsweeps
quantiles
0.010.050.100.250.500.750.900.950.99
QMC-quantum annealing
||Matthias Troyer
▪ Distribution of time to find a ground state has fat (power law) tails ▪ Tails are fatter for QMC-QA and D-Wave than for thermal annealing
!▪ see tomorrow’s talk by Damian Steiger
27
Fat tails
100 101 102 103 104 105 106 107 108 109
100
101
102
103
104
mean number of repetitions t
instances
SQA 288 spins
100 101 102 103 104 105 106 107 108 109
100
101
102
103
104
mean number of repetitions t
instances
SA 288 spins
D. Steiger, T.F. Rønnow et al, unpublished
||Matthias Troyer
▪ D-Wave shows the same very fat tails as QMC-QA
28
Compare to D-Wave
Thermal annealing
QMC-QA (SQA)
D-Wave Two
||Matthias Troyer 29
!!! …. a non-trivial endeavor
Detecting and defining quantum speedup
||Matthias Troyer
Quantum speedup exists if grows asymptotically with the problem size N !!
30
Defining quantum speedup
S(N ) = TC (N )TQ (N )
Seems easy and trivial to define, but … !
one can easily get fooled into believing there is speedup
T.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ Provable quantum speedup ▪ when we can prove a separation between TQ and TC ▪ example: Grover search
▪ Strong quantum speedup (Traub et al, 2013) ▪ speedup compared to bound for best classical algorithm,
whether that algorithm is known or not
▪ Quantum speedup ▪ speedup compared to best known classical algorithm ▪ example: Shor’s algorithm
▪ Potential (quantum) speedup ▪ speedup compared to a (selection of) classical algorithms
▪ Limited quantum speedup ▪ speedup compared to a “classical version” of the quantum algorithm
31
Five types of quantum speedupT.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ Initially too flat slope when running at a fixed annealing time ▪ To determine asymptotic scaling we have to find the optimal annealing
time for each problem size
32
Performance at fixed annealing timeT.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ Compare simulated quantum annealing at fixed (suboptimal) annealing time to classical annealing at optimal annealing time.
▪ What is a slowdown suddenly looks like speedup
33
“Fake” speedup due to suboptimal performanceT.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ On DW2 the optimal annealing time is much shorter than 20 µs. ▪ The annealing times are far longer than is needed ▪ The machine could be much faster ▪ We cannot demonstrate quantum speedup without doubt
34
Optimizing the total effortT.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ D-Wave Two is a parallel machine acting on all spins simultaneously
▪ We need to compare to a (hypothetical) classical machine with same hardware layout performing simulated annealing in parallel
▪ One could build such a classical machine in an FPGA or ASIC if there is interest
35
Parallel speedup
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
1
TC(N )∝1NTSA(N )
S(N ) = TC(N )TDW(N )
∝ TSA(N )TDW
1N
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
1
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
1
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
1
T.F. Rønnow et al, Science (2014)
||Matthias Troyer 36
Two types of speedup benchmarks
||Matthias Troyer
▪ Since we don’t know a-priori whether an instance is hard or easy we have to assume that it may be hard and look at the high quantiles
▪ Focus just on annealing time to get the intrinsic scaling
37
The device as an optimizer: solve almost all instancesT.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ Focus on high quantiles
38
Benchmark 1: ratio of quantilesT.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ Scientifically is the question whether for a subset of problems we there is at least limited quantum speedup compared to SA
▪ Look at the instance-by-instance comparison and search for problems that are faster on D-Wave
39
Is there quantum speedup for a subset of problems?T.F. Rønnow et al, Science (2014)
||Matthias Troyer
▪ Compare quantiles of individual instance ratios
40
Benchmark 2: quantiles of ratios
potentially there is speedup here for a subblass of problems with ±1 couplings
but we cannot know since annealing times are not optimal
No speedup for problems with couplings ±1/7, ±2/7, …. ±6/7, ±1
is this due to calibration errors or intrinsic?
T.F. Rønnow et al, Science (2014)
||Matthias Troyer 41
Instance by instance comparisons: wallclock time
Faster on D-Wave
Faster classically
N=503
T.F. Rønnow et al, Science (2014)
||Matthias Troyer 42
Wallclock time for harder problemsN=503
Couplings are ±1/7, ±2/7, …. ±6/7, ±1
T.F. Rønnow et al, Science (2014)
||Matthias Troyer ||Matthias Troyer
Classical algorithms: QMC-QA versus classical annealing ▪ Evidence for advantage of QMC-QA needs to be revisited and checked ▪ QMC-QA shows much fatter tails in the distribution of time to solution ▪ QMC-QA visits state space selectively and can be trapped in the wrong
neighborhood ▪ Classical annealing has much better “hard case” behavior
Performance of the D-Wave device ▪ Behavior is consistent with quantum annealing: D-Wave built what they claim ▪ The Shin et al paper shows that entanglement may not play a big role in the
performance of the device for spin glass instances ▪ Ultimate test for quantum-supremacy is scaling, but we have not found evidence for
quantum speedup in our tests so far ▪ The device has the same very fat tails and bad hard-case performance
as QMC-QA (and the Shin et al model) 43
ConclusionsS. Boixo et al, Nature Physics (2014)
T.F. Rønnow et al, Science (in press, 2014)