messagepassing two steps least square algorithms for...
TRANSCRIPT
![Page 1: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/1.jpg)
1
Messagepassing Two Steps Least Square Algorithms for Simultaneous Equations Models
José Juan López EspínUniversidad Miguel Hernández (Elche, Spain)Domingo Giménez CánovasUniversidad de Murcia (Murcia, Spain)
![Page 2: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/2.jpg)
2
Contents Introduction Simultaneous equations models OLS and 2SLS techniques Three different versions of 2SLS algorithm
General Inverse decomposition QR decomposition
Experimental results Conclusions and future works
![Page 3: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/3.jpg)
3
Introduction The solution of a S.E.M. in high performance
parallel systems is studied using 2SLS. Three different versions of 2SLS are studied. Parallel algorithms for distributed memory have
been developed for the three versions. The methods have been analyzed in different
parallel systems.
![Page 4: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/4.jpg)
4
Simultaneous Equations ModelsThe scheme of a system with M equations, M endogenous variables and k predetermined variables is (structural form)
…
These equations can be represented in matrix form
1 12 2 13 3 1 11 1 1 1... ...t t t M Mt t k kt tY Y Y Y X X uβ β β γ γ= + + + + + + +2 21 1 23 3 2 21 1 2 2... ...t t t M Mt t k kt tY Y Y Y X X uβ β β γ γ= + + + + + + +
1 1 2 2 3 3 1 1 1 1... ...Mt M t M t M t MM M t M t Mk kt MtY Y Y Y Y X X uβ β β β γ γ− −= + + + + + + + +
0t t tBY X u+G + =
![Page 5: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/5.jpg)
5
Simultaneous Equations ModelsThe structural form can be expressed in reduced form
with and
…
t t tY X v= P +
1B-P = - G 1t tv B u-= -
1 11 1 1 1...t t k kt tY X X vp p= + + +
1 1 ...Mt M t Mk kt MtY X X vp p= + + +
![Page 6: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/6.jpg)
6
OLS (Method)OLS (Ordinary Least Square) can be used to solve a
regression model
In matrix form
The expression of the estimator is
1 1 ...t t n nt tY X X ua a= + + +
Y X ub= +
1ˆ ( )X X X Yb -=
![Page 7: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/7.jpg)
7
2SLS (Two Step Least Squares) OLS can not be used in
structural form because random variable and endogenous variables are correlated
Endogenous variables are replaced for approximations (proxys variables)
The proxy of Y is calculated using OLS with Y and the exogenous in the system.
When the endogenous have been replaced, OLS is used again in the equation
![Page 8: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/8.jpg)
8
Parallel Algorithm for distributed memory
Try to parallelize at the upest level Share the maximum of information. Each call to 2SLS must share more information
to reduce the number of operations. Perform the maximum number of operations
between all the processors at the beginning of the algorithm to be used for any processor in other parts of the algorithm.
ScaLAPACK and PBLAS libraries are used to make a portable program
![Page 9: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/9.jpg)
9
OLSp (Parallel OLS)
In the experiments pdgemm has been used to perform the multiplications, and pdgesv to compute the inverse. The use of ScaLAPACK allows us to obtaina portable routine.
![Page 10: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/10.jpg)
10
2SLS for a system (Parallel 2SLS) Three different versions of the 2SLS
algorithm are presented. The first is a basic algorithm which
will be improved in the second and the third versions.
In the first version, the structure of the parallel 2SLS algorithm is stated. In the others versions, the same structure is followed but matrix decompositions are used to obtain lower costs.
![Page 11: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/11.jpg)
11
The first version of 2SLS All the proxys are
calculated at the beginning of the algorithm
All the proxys are distributed in all the processors
Each processor solves an equation using OLS sequentially
![Page 12: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/12.jpg)
12
The 2nd v. of 2SLS (inverse decomposition)
Solve an equation where the proxy variables have been substituted before (they are calculated at the beginning)
The set of endogenous variables of the equation is and X1 is the set of predetermined, and then the variables of the equation are the matrix [ X1]
And ([ X1]t [ X1])1[ X1]t yj must be solved1̂Y
1̂Y
1 10 1 1ˆ ˆ... ...m kj j m j j k jy y y x xa a a g g e= + + + + + + +
1̂Y
1̂Y1̂Y1̂Y
![Page 13: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/13.jpg)
13
The 2nd v. of 2SLS (inverse decomposition)
The inverse:
Using
11 1 1 1 1
1 11 1 1 1 1
ˆ' ' 'ˆˆ ˆ ˆ ˆ' ' '
X X X X YX Y
Y Y X Y Y
-
= =
1 11 1 11 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1
ˆ( ' ) 0 ( ' ) ' ˆ ˆ ˆ ˆ ˆ( ' ' ( ' ) ' ) ( ' ( ' ) , )0 0
X X X X X Y Y Y Y X X X X Y Y X X X IdId
- -- - --
+ - -
1 1 11 1 10 ( ' ) ( , )
' 0 0A B A A B D B A B A B IdB D Id
- - -- - --
= + - -
![Page 14: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/14.jpg)
14
The 2nd v. of 2SLS (inverse decomposition)
(X1’X1) is taken from X’X
(X1’X1)1 is calculated
X1’ is taken from X’ Y
(X1’X1)1 X1’ is calculated (cost 2k2m+ 2/3k3)
’X1(X1’X1)1 X1’ is calculated (cost 2m2k)
’ is taken from ( ‘ ‘X1(X1’X1)1 X1’ )1 is calculated (cost 2/3m3 )
1̂Y1̂Y
1̂Y
1̂Y 1̂YYY 'ˆ
1̂Y1̂Y 1̂Y
1̂Y 1̂Y
![Page 15: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/15.jpg)
15
The 2nd v. of 2SLS (inverse decomposition)
To calculate [X1 ]’yj X’1yj can be taken from XtY which was
calculated to obtain Pi ( ’yj ) can be taken from 1̂Y YY 'ˆ
1̂Y
![Page 16: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/16.jpg)
16
The 2nd v. of 2SLS (inverse decomposition)
Finally, the algorithm is
![Page 17: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/17.jpg)
17
The 3rd v. of 2SLS (QR decomposition)
X is decomposed as QR using Householder method, where Q is orthogonal and R upper triangular.
![Page 18: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/18.jpg)
18
The 3rd v. of 2SLS (QR decomposition)
The algorithm is
![Page 19: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/19.jpg)
19
![Page 20: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/20.jpg)
20
Computer System Kefren: A cluster of 20 biprocessors Pentium Xeon 2
Ghz interconnected by a SCI net with a Bull 2D topology in a mesh of 4 £ 5. Each node has 1 Gigabyte RAM.
Marenostrum: A supercomputer based on PowerPC processors, BladeCenter architecture, a Linux system and a Myrinet interconnection. The main characteristics are: 10240 IBM Power PC 970MP processors at 2.3 GHz (2560 JS21 blades), 20 TB of main memory, 280 + 90 TB of disk storage and a peak Performance of 94,21 Teraflops. Marenostrum is the most powerful supercomputer in Europe and the fifth in the world, according to the last TOP500 list.
![Page 21: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/21.jpg)
21
The first version of 2SLS
![Page 22: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/22.jpg)
22
The first version of 2SLS
![Page 23: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/23.jpg)
23
The 2nd v. of 2SLS (inverse decomposition)
![Page 24: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/24.jpg)
24
The 2nd v. of 2SLS (inverse decomposition)
![Page 25: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/25.jpg)
25
The 3rd v. of 2SLS (QR decomposition)
![Page 26: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/26.jpg)
26
Comparison between the three techniques
![Page 27: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/27.jpg)
27
Comparison of the precisions between the three techniques
2,79031E122,7896E1220008002000
7,78951E127,81023E1215008002000
2,49918E092,63E0915006001500
2,18451E082,13886E0810006001500
4,64279E134,65709E1310004001000
3,00927E123,00996E125004001000
9,08442E129,13657E12500200500
dif. InvNormaldif InvQrSampleExogenousEndogenous
![Page 28: Messagepassing Two Steps Least Square Algorithms for ...dis.um.es/~domingo/07/PPAMJJ/presentation.pdf · Domingo Giménez Cánovas Universidad de Murcia (Murcia, Spain) 2 Contents](https://reader035.vdocuments.net/reader035/viewer/2022063014/5fd1584aac40d073be426422/html5/thumbnails/28.jpg)
28
Conclusions and Future works Sometimes a
Simultaneous Equations Model needs special software and be solved in High Performance Systems
Tools will be made freely available to the scientific community
Application to real problems
Develop an algorithm to find the best model