a framework for selecting hyper-parameters · a framework for selecting hyper-parameters jim o’...
TRANSCRIPT
![Page 1: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/1.jpg)
A Framework for Selecting Hyper-Parameters
Jim O’ Donoghue
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979
British International Conference on Databases
7th July 2015
![Page 2: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/2.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979
Background + Motivation
Algorithms + The CDN
Experiments + Results
Future Work
Conclusions
NEED TO FIX NUMBERS
![Page 3: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/3.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 3
1
![Page 4: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/4.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 4
1
![Page 5: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/5.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 5
1
Dementia Awareness + Prevention
![Page 6: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/6.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 6
1
Dementia Awareness + Prevention
Online Environment
![Page 7: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/7.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 7
1
Dementia Awareness + Prevention Online Environment
Risk Prediction Algorithm
![Page 8: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/8.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 8
1
Dementia Awareness + Prevention Online Environment
Risk Prediction Algorithm - Validation
![Page 9: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/9.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 9
3
![Page 10: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/10.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 10
3
![Page 11: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/11.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 11
3
High-Dimensional
![Page 12: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/12.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 12
3
High-Dimensional
Variable Interactions
![Page 13: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/13.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 13
3
High-Dimensional
Variable Interactions
![Page 14: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/14.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 14
3
High-Dimensional
Variable Interactions
Hyper-Parameter Selection
![Page 15: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/15.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 15
3
High-Dimensional
Variable Interactions
Hyper-Parameter Selection
![Page 16: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/16.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 16
4
![Page 17: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/17.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 17
4
Class Class
Input Features
![Page 18: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/18.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 18
4
Class Class
Input Features
Learned Features
![Page 19: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/19.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 19
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 20: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/20.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 20
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 21: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/21.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 21
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 22: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/22.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 22
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 23: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/23.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 23
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 24: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/24.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 24
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 25: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/25.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 25
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 26: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/26.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 26
4
Class
Connection Weights
Class
Input Features
Learned Features
![Page 27: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/27.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 27
4
Connection Weights
Class
Input Features
Learned Features
![Page 28: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/28.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 28
5
Class
Connection Weights
Class
Input Features
![Page 29: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/29.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 29
6
Connection Weights
Class
Input Features
![Page 30: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/30.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 30
7
Connection Weights
Input Features
Learned Features
![Page 31: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/31.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 31
8
Class
Connection Weights
Input Features
Learned Features
![Page 32: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/32.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 32
9
![Page 33: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/33.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 33
9
MySql
File System
Configurable Deep Network Framework
![Page 34: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/34.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 34
9
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
![Page 35: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/35.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 35
9
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
e1, e2, … , en
![Page 36: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/36.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 36
9
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
Query
e1, e2, … , en
![Page 37: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/37.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 37
9
MySql
File System
Grid
Algorithm
Configurable Deep Network Framework
Final Model
Query
e1, e2, … , en
![Page 38: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/38.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 38
10
![Page 39: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/39.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 39
11
CDN
![Page 40: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/40.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 40
12
Subset of the Data – dimensions What the variables are What the predictor is Purpose
![Page 41: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/41.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 41
13
To Choose:
![Page 42: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/42.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 42
13
To Choose:
learning rate α
![Page 43: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/43.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 43
13
To Choose:
learning rate α weight decay term λ
![Page 44: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/44.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 44
13
To Choose:
learning rate α weight decay term λ training iterations t
![Page 45: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/45.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 45
13
The Grid:
![Page 46: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/46.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 46
13
The Grid:
α, λ: [0.001, 0.003, 0.009, … , 0.1, 0.3, 0.9]
![Page 47: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/47.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 47
13
The Grid:
α, λ: [0.001, 0.003, 0.009, … , 0.1, 0.3, 0.9]
t: [100, 1000, 10000]
![Page 48: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/48.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 48
13
Categorical Continuous
![Page 49: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/49.jpg)
Alpha 0.9 0.3 0.09 0.003
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 49
13
![Page 50: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/50.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 50
13 Alpha 0.9 0.3 0.09 0.003
Lambda 0.009 0.003 0.001
![Page 51: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/51.jpg)
Lambda 0.009 0.003 0.001
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 51
13
100 1,000
10,000
Training Iterations
Alpha 0.9 0.3 0.09 0.003
![Page 52: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/52.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
Categorical Continuous Lambda 0.009 0.003 0.001
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 52
13
100 1,000
10,000
Training Iterations
Alpha 0.9 0.3 0.09 0.003
![Page 53: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/53.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 53
13
100 1,000
10,000
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 54: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/54.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 54
13
100 1,000
10,000
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 55: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/55.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 55
13
100 1,000
10,000
0.3046
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 56: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/56.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 56
13
100 1,000
10,000
0.3046
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 57: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/57.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 57
13
100 1,000
10,000
0.3046
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 58: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/58.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 58
13
100 1,000
10,000
0.3046 0.2815
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 59: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/59.jpg)
0
5
10
15
20
25
30
35
40
45
50
Valid. Cost
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 59
13
100 1,000
10,000
0.3046 0.2815
Training Iterations
Categorical Continuous Lambda 0.009 0.003 0.001
Alpha 0.9 0.3 0.09 0.003
![Page 60: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/60.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 60
15
To Choose:
![Page 61: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/61.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 61
15
To Choose:
layer 1 nodes h(1)n
![Page 62: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/62.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 62
15
To Choose:
layer 1 nodes h(1)n
pre-training epochs e
![Page 63: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/63.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 63
15
The Grid:
h(1)n:
[10, 30, 337, 900, 1300, 2000]
![Page 64: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/64.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 64
15
The Grid:
h(1)n:
[10, 30, 337, 900, 1300, 2000]
e [1, 5, 10, 15, 20]
![Page 65: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/65.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 65
15
Parameter Initialisation:
− 4 6
𝑓𝑎𝑛_𝑖𝑛 + 𝑓𝑎𝑛_𝑜𝑢𝑡, + 4
6
𝑓𝑎𝑛_𝑖𝑛 + 𝑓𝑎𝑛_𝑜𝑢𝑡
![Page 66: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/66.jpg)
Valid. Cost
-25
-20
-15
-10
-5
0
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 66
15
Epochs 1 5
![Page 67: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/67.jpg)
Valid. Cost
-25
-20
-15
-10
-5
0
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 67
15
Nodes 10 30 100 337 900 1 300 2000
![Page 68: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/68.jpg)
Valid. Cost
-25
-20
-15
-10
-5
0
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 68
15
Epochs 1 5
Nodes 10 30 100 337 900 1300 2000
![Page 69: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/69.jpg)
Valid. Cost
-25
-20
-15
-10
-5
0
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 69
15
Epochs 1 5
Nodes 10 30 100 337 900 1300 2000
![Page 70: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/70.jpg)
Valid. Cost
-25
-20
-15
-10
-5
0
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 70
15
Epochs 1 5
Nodes 10 30 100 337 900 1300 2000
![Page 71: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/71.jpg)
Valid. Cost
-25
-20
-15
-10
-5
0
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 71
15
Epochs 1 5
Nodes 10 30 100 337 900 1300 2000
-19.580
![Page 72: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/72.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 72
14
To Choose:
Last layer nodes h(1)n
![Page 73: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/73.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 73
14
To Choose:
Last layer nodes h(1)n
The Grid:
[10, 30, 337, 900, 1300, 2000]
![Page 74: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/74.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 74
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
![Page 75: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/75.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 75
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Nodes 10 30 100 337 900 1300 2000
![Page 76: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/76.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 76
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
![Page 77: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/77.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 77
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
![Page 78: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/78.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 78
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
![Page 79: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/79.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 79
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
![Page 80: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/80.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 80
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
0.232
![Page 81: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/81.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 81
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1300 2000
0.232
![Page 82: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/82.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 82
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1 1300 2000
0.232
![Page 83: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/83.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 83
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1 1300 2000
0.232
![Page 84: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/84.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 84
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1 1300 2000
0.232 0.291
![Page 85: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/85.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 85
14
0
10
20
30
40
50
60
70
80
90
Valid. Cost
Categorical Continuous
Nodes 10 30 100 337 900 1 1300 2000
0.232 0.291
![Page 86: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/86.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 86
16
Lambda @ 0.03
10 2000 200 337
10 200 3567
10 200 3567
10 200 3567
10 200 3567
10 30
10 337
10 100 337
0
1
2
3
4
5
6
7
Alpha 0.001 0.01 0.9
![Page 87: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/87.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 87
16
Lambda @ 0.03
10 2000 200 337
10 200 3567
10 200 3567
10 200 3567
10 200 3567
10 30
10 337
10 100 337
0
1
2
3
4
5
6
7
Step 3000 1000 100
![Page 88: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/88.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 88
16
Lambda @ 0.03
10 2000 200 337
10 200 3567
10 200 3567
10 200 3567
10 200 3567
10 30
10 337
10 100 337
0
1
2
3
4
5
6
7
0.265 0.245
Alpha 0.001 0.01 0.9 Steps 3000 1000 100
0.272
![Page 89: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/89.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 89
16
Lambda @ 0.03
10 2000 200 337
10 200 3567
10 200 3567
10 200 3567
10 200 3567
10 30
10 337
10 100 337
Alpha 0.001 0.01 0.9 Steps 3000 1000 100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
![Page 90: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/90.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 90
18
Activation functions Algorithms Inference Framework – to Mongo and input from Visualising learning Implementing Early Stopping Mini-batch Stochastic Gradient Descent
![Page 91: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/91.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 91
19
Much easier to model when you have one extensible network that can handle many type of data Constituent models can be used to select a starting point for deep learning configurations
![Page 92: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/92.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 92
![Page 93: A Framework for Selecting Hyper-Parameters · A Framework for Selecting Hyper-Parameters Jim O’ Donoghue ... Configurable Deep Network Framework Final Model Query e 1, e 2, …](https://reader034.vdocuments.net/reader034/viewer/2022050503/5f94dfd73ccbc60f026c4a75/html5/thumbnails/93.jpg)
In-MINDD is funded under the European Union Seventh Framework Programme, Grant Agreement Number 304979 93
16
Lambda @ 0.03
10 2000 200 337
10 200 3567
10 200 3567
10 200 3567
10 200 3567
10 30
10 337
10 100 337
0
1
2
3
4
5
6
7
0.265 0.245
Alpha 0.001 0.01 0.9 Steps 3000 1000 100
0.272