Financial Filtered NetworksUntangling financial data complexity

Tomaso Aste & Tiziana Di Matteo

Zurich 9 Jan 2014

UCL, London KCL, London

Aplollonian Networks

JS. Andrade, Jr., HJ Herrmann, R FS Andrade and LR. da Silva, PRL 94 (2005) 018702

M. Tumminello, TA, T. Di Matteo, and R. N. Mantegna, PNAS 102 (2005) 10421–10426

TA, R Gramatica, T Di Matteo PRE 86 (2012) 036109

T1

T2J. W. Alexander, “The combinatorial theory of complexes” Ann. Math. 31 (1930) 292.

Planar triangulationsPMFG

Filtered networks

Information and Big DataWe are witnessing interesting times rich of information, readily available for us all. Using, understanding and filtering such information has become one of the major tasks and a crucial bottleneck for scientific and industrial endeavors

Information content and flow are often associated with large degrees of interdependency that can be used to reduce data complexity

To filter information we must first understand the data inetr-dependency structure

This large amount of information must be filtered and meaning extracted

Linear measures• Correlations• Partial Correlations• Granger causality • Transfer EntropyNon-linear and kernel measures• Kernelized Granger/Geweke’s causality • Hilbert-Schmidt Normalised Conditional Independence Criterion (HSNCIC) • Transfer Entropy

X Y

Z

Interdependency

A Zaremba

Is the Consumer Price Index (US) causing interest rates (LIBOR)?

1 month lag 7 months lag

CPI->LIBOR

LIBOR->CPI

CPI->LIBOR

LIBOR->CPI

Kernelized Granger/Geweke’s causality Kernelized Granger/Geweke’s causality

Transfer Entropy

A Zaremba & TA, Measures of Causality in Complex Datasets with application to financial data http://arxiv.org/abs/1401.1457

Dependency and Causality:

Measuring and validating

The surface constraints the complexity of the network (the degree of interwoveness)

We can achieve this by embedding interdependency nets on surfaces

Information Filtering

Simplifying Complex Big Datasets

Planar surfaces are the simplest

To reduce complexity we must find ways to decrease the number of interrelations and gather data into clusters and hierarchical structures

Big data demand new algorithms

Sort similarities form the largest to the smallest

Connect the first two nodes on the top line of the list

Is the resulting graph planar?

Delete the top line from the list

Discard the edgeKeep the edge

Have we reached the maximum number of edges?

yes no

yes

no

http://www.mathworks.com/matlabcentral/fileexchange/27360

M. Tumminello, TA, T. Di Matteo, R.N. Mantegna, “A tool for filtering information in complex systems”, PNAS 102 (2005) 10421-10426.

Planar Maximally Filtered Graph

T2 (Apollonian) construction

G Previde Massara

The novelty of the method is that we do not longer rely on any particular ordering but at every stage we calculate the gain that would be obtained by adding any of the remaining vertices inside any triangle, complexity is O(n2) and results improve PMFG

Numerically efficient algorithm for big dataT1

T2

Non planar graph can be generated with this methodTA, R Gramatica and T. Di Matteo, Phys. Rev. E., 86 (2012) 036109.

Some 1000s times faster than PMFG and scalable to millions of vertices

Clique Tree construction for Markov Random Field inference modeling

Network Interdependence Modelling

Local Sparse Inverse Covariance (LoGo)

G Previde Massara

Wolfram Barfuß

The Join probability distribution factorizes over the cliques (for exponential classes) The T2 (Apollonian) construction generates a 4-clique tree!

In linear models (and kernel-linearized as well) interactions are associated with the inverse of the covariance, but noise makes the inverse meaningless. Local inversion on the clique-tree produce meaningful interactions.

Risk modelling

Efficient diversification / risk hedging

F Pozzi

Correlation networks can be used for efficient portfolio differentiation by selecting stocks from the periphery of the PMFG

Portfolio performance

Probability of negative returns

F. Pozzi, T. Di Matteo, and TA , “Spread of risk across financial markets: better to invest in the peripheries”, Scientific Reports 3 (2013) 1665.

We extract clusters and hierarchies form maximal planar graphs by using the fact that 3-cliques on Maximal Planar Graphs contain other cliques inside or/and they are contained inside the other cliques providing a natural hierarchy

Complexity reduction: DBHT

v1 v2 v3 v5 v4 v6 v8 v7b

α

b

β

b

3

b

2

v

4

v

8

v

3

v

6v

2

v

7

v

5

W.M. Song, T. Di Matteo and T. Aste, “Hierarchical information clustering by means of topologically embedded graphs”, PLoS ONE, 7 (2012) e31929 Won-Min Song, T. Di Matteo, TA, Nested hierarchies in planar graphs, Discrete Applied Mathematics 159 (2011) 2135-2146.

v

4

v

8

v

3

v

6v

2

v

7

v

5v4

v3v2

v1

b1

b2

k1

v2

v3

v4

v6

v5

v2 v4

v5 v7

v4

v3

v6

v8

b3

b4

k3

k2

v9

α β

bα= b1

k

3

k

2

k

1 bβ = b4

b2

b3

A deterministic method to capture both local clustering and global hierarchical organization without introducing any characteristic scale

WM Song

Directed Bubble Hierarchical Tree

100%

56%

14%

29%

W.M. Song, T. Di Matteo and T. Aste, “Hierarchical information clustering by means of topologically embedded graphs”, PLoS ONE, 7 (2012) e31929

PMFGDBHT

Conclusions and PerspectivesBig data demand complexity reduction: information filtering

Networks are especially suited for this purpose because one can reduce size by clustering without loosing the information about the whole hierarchy

PMFG generated via T2 (Apollonian nets) are especially suited for this purpose