french banks sequester your data - schedule2013.rmll.info · intro weboob banks applications...

34
Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon [email protected] LSM, July 2013

Upload: others

Post on 13-Jun-2020

25 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

French banks sequester your data

Romain Bignon

[email protected]

LSM, July 2013

Page 2: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Overview

• What are bank data

• No standard way to access to your own data

• How banks prevent scrapping

• How weboob scaps them anyway

• What is possible with standard API

Page 3: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Paper statement

Page 4: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Online statement

Page 5: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Downloadable files

QIF

CSV

OFX

Page 6: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Transpac & SEPA

Page 7: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

WEB OUTSIDE OF BROWSERS

Page 8: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Capabilities

• Common API

• Aggregation

• Examples: CapVideo, CapSubtitle, CapMessages, etc.

Page 9: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Diagram

Page 10: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

CapBank

+---------------------------+

| Account |

+---------------------------+

| Name: Check account |

| Balance: 5000 |

| Currency: EUR |

| Coming: 290 |

| Type: CHECKING |

| | +--------------------------+

| Transactions[] |------| Transaction |

+---------------------------+ +--------------------------+

| Date: 2013-07-01 |

| Raw: FACTURE CARTE DU 130613 SNCF

| Amount: -118 |

| Rdate: 2013-06-13 |

| Type: CARD |

| Label: SNCF INTERNET |

+--------------------------+

Page 11: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Difficultés du scrapping

Page 12: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Pierre Tramo

Page 13: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Bad HTML structure

• No IDs neither classes

• Inlined styles

Concerned banks:

About all of them.

Page 14: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Bad date format

Examples:

• 18 sep.

• 13/02

• hier

Concerned banks:

1. Crédit Agricole

2. ING

Page 15: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Limited historic

• Last 20 transactions

• Last 30 days

Concerned banks:

1. Crédit Agricole

2. Banque Postale

Page 16: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

No ID for transactions

• Difficult to recognize them

Not concerned banks:

1. Paypal

2. HSBC

3. Crédit du Nord

4. Bred

Page 17: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Non integrity of data

• Shown transactions not considered in account balance

• Transactions details may change

Concerned banks:

About all of them.

Page 18: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Sequential navigation

• Java applications

• Sateful navigation

Concerned banks:

1. BNP

2. Banque Populaire

3. AXA Banque

4. Crédit Agricole

Page 19: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Javascript needed

• Set cookies

• Fill pages from arrays in js

Concerned banks:

1. Banque Populaire

2. Barclays

3. BNP

4. Crédit Agricole

5. Crédit du Nord

6. ...

Page 20: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Headers checking

Concerned banks:

1. Banque Populaire

2. BNP Paribas

3. AXA Banque

Page 21: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Variants between regions

• Various version of the same software

Concerned banks:

1. Crédit Agricole

2. Banque Populaire

Page 22: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Numeric keypad

Concerned banks:

1. Crédit Agricole

2. BNP Paribas

3. AXA Banque

4. Banque Postale

5. ...

Page 23: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Authentication token

Concerned banks:

1. HSBC

2. Other banks soon

Page 24: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Ranking

1. Crédit Agricole

2. Banque Populaire

3. BNP Paribas

4. AXA Banque

5. HSBC

Page 25: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

KMyMoney

Page 26: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Page 27: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Page 28: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Page 29: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

OFX

• Microsoft protocol

• Mostly in USA and Germany

Page 30: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Open Bank Project

• Standard API

• Nobody uses it

Page 31: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Crédit Agricole Store

• Crappy.

Page 32: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

AXA Banque

• Open API

• Application register

Page 33: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

French banks are outdated

• Open to use aggregation

• Closed to opening their data

Page 34: French banks sequester your data - schedule2013.rmll.info · Intro Weboob Banks Applications Alternatives Conclusion French banks sequester your data Romain Bignon romain@weboob.org

Intro Weboob Banks Applications Alternatives Conclusion

Questions?

http://weboob.org

#weboob @ irc.freenode.net

https://www.budgea.com