QFDs and BLACHOLE Technology behind GCHQ/INOC

Slide 1

© This information is exempt under the Freedom of Information Act 2000 (FOIA) and may be exempt under other UK information legislation. Refer any FOIA queries to GCHQ on

UK SECRET STRAP2 COMINT ORCON

Probes Flat Store Analytic engines

Multi

<10G

Question specific DBs (INOC today)

Data-mart extraction

MESME/Netezza etc.

Clouds

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on

Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

Slide 2 UK SECRET STRAP2 COMINT ORCON

UK SECRET STRAP2 COMINT ORCON

• Flat data store - not a database

• — 1,100,000,000,000 events since 6th August 2007 12%

- 53.3TB compressed ^ ^ ^ ^ ^ ^ - 217TB uncompressed \ - 47% of data is from last 3 months

19%

-10 Billion events per day

Cheap system: - £1 k per TB of storage V - £20k per 10Gbps probe 1 - ~£400k for 10x1 OGbps prototype

I R HTTP

B Websearch

[ • S M T P

I R Webmai l

CNE Server Log

I R Social networking

Instant Messenger

I R Routing Protocols

r Anonymiser

• Other

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of I n l T l ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ f f ^ s u b j e c t to exemption under other UK information legislation. Refer disclosure requests to GCHQ on

Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

Slide 2 UK SECRET STRAP2 COMINT ORCON

; " ri • i I ^ I.

UK SECRET STRAP2 COMINT ORCON

Who, Where, When, What? - Online presence (TDI)

MUTANT BROTH

Who, which website(s)? - Online browsing activity

KARMA POLICE

Who, which fora?- Bulletin board usage

INFINITE MONKEYS

Where? - Google maps/earth usage

© Crown Copyrigh reserved.This inrormaïTon is exempffrom disclosure under the Freedom of Information Act 2000 and may be subject to exemption under Snrrei-orrTnformation legislation. Refer disclosure requests to GCHQ on

Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

Slide 2 UK SECRET STRAP2 COMINT ORCON

J I Id {

UK SECRET STRAP2 COMINT ORCON

Who, Where, When, What? - Online presence (TDI)

MUTANT BROTH

Who, which website(s)? - Online browsing activity

KARMA POLICE

Who, which fora?- Bulletin board usage

INFINITE MONKEYS

Where? - Google maps/earth usage

© Crown Copyriglj Reserved! "Thîs intormâûonTs*ëxernpTfrom clîsclosiîre under the Freedom of Information Act 2000 and may be subject to exemption under Snrrei-orrTnformation legislation. Refer disclosure requests to G C H ^ B

Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

Slide 2 UK SECRET STRAP2 COMINT ORCON

... And many more

UK SECRET STRAP2 COMINT ORCON

Too many specific questions... Too much for analysts to learn.

Need analyst systems to be simple and intuitive.

Let analysts find & simply understand their targets easily...

drill in to systems only when they need to © Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under

other UK information legislation. Refer disclosure requests to GCHQ on Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must

be obtained for dissemination outside the organisation.

, fi UK SECRET STRAP2 COMINT ORCON

S S I IES EAI tC H m

UK SECRET STRAP2 COMINT ORCON

AR Prototypes - EPR Stats - Feb 2008 - Jan 2009

n J* y # J' f

© Crown Copyright. All qpms reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to GCHQ on

Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

, 8 UK SECRET STRAP2 COMINT ORCON

y J"

• Grade C • Grade B • Grade A

•r é f y /

UK SECRET STRAP2 COMINT ORCON

Number of users who executed >10 queries, by area

140

120

100

80

60

4 0

Area

other UK information legislation. Refer disclosure requests to GCHQ on Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must

be obtained for dissemination outside the organisation.

Slide 2 UK SECRET STRAP2 COMINT ORCON