digital intelligence gathering · digital intelligence gathering using the powers of osint for both...
TRANSCRIPT
Digital Intelligence GatheringUsing The Powers Of OSINT For Both Blue And Red Teams
BSidesSF February 2016
1 / 71
Ethan Dodge DFIR @ Nuna Health.
DFIR professional and perpetual learner.
@__eth0
dodgesec.com
2 / 71
Nuna Health We work with the government and self-insured employers tounderstand and improve how people use healthcare.
3 / 71
Nuna Health We work with the government and self-insured employers tounderstand and improve how people use healthcare.
Security is the foundation of our culture and products.
4 / 71
Nuna Health We work with the government and self-insured employers tounderstand and improve how people use healthcare.
Security is the foundation of our culture and products.
We're accepting resumes!
5 / 71
OSINT
6 / 71
What is OSINT?
Using information available to everyone to gatherintelligence
7 / 71
What is OSINT?
Using information available to everyone to gatherintelligence
Social Networks
8 / 71
What is OSINT?
Using information available to everyone to gatherintelligence
Social Networks
Public Data Records
9 / 71
What is OSINT?
Using information available to everyone to gatherintelligence
Social Networks
Public Data Records
Leaked Customer Data
10 / 71
11 / 71
12 / 71
Why OSINT?
Private Investigators/Detectives
Investigative Journalism
Criminal Activity/Law Enforcement
Threat Intelligence
13 / 71
14 / 71
15 / 71
DISCLAIMER
16 / 71
Identify Source
Identify possible sources of intel
Validate
Automate
Basic Work�ow
17 / 71
Identify Source
Analyze Does it apply to our target?
Determine probability
Apply con�dence
Generate new potential sources
Basic Work�ow
18 / 71
Identify Source
Analyze
Enrich
Add context to target
Add probability, con�dence level to details
Develop narrative
Basic Work�ow
19 / 71
Maltego
20 / 71
Mal...what?
Link Analysis Visualization Tool
Enrich entity with other sources of information automatically
Identify relationships between entities
Visualize relationships
21 / 71
Common Terms
Entities
22 / 71
Common Terms
Entities
Transforms
23 / 71
Common Terms
Entities
Transforms
Machine
24 / 71
Transform Example
25 / 71
Transform Development Primerfrom MaltegoTransform import *
26 / 71
Transform Development Primerme = MaltegoTransform()me.parseArguments(sys.argv)location = sys.argv[1]
27 / 71
Transform Development Primerent = me.addEntity("maltego.Location","DNA Lounge")me.returnOutput()
28 / 71
Gavel Custom maltego transform we developed.
29 / 71
Gavel Custom maltego transform we developed.
Digs up court case records from individual states.
30 / 71
Gavel Custom maltego transform we developed.
Digs up court case records from individual states.
Tons of sensitive information.
31 / 71
Gavel Custom maltego transform we developed.
Digs up court case records from individual states.
Tons of sensitive information.
https://github.com/brianwarehime/gavel
32 / 71
Gavel Example
33 / 71
Story Time
34 / 71
Ever seen this?
35 / 71
Or this?
36 / 71
37 / 71
Twitter Data
Start with best source of data - Twitter
38 / 71
Twitter Data
Start with best source of data - Twitter
We needed a way to parse through all the data
39 / 71
Twitter Data
Start with best source of data - Twitter
We needed a way to parse through all the data
We identi�ed it, validated it, now we analyze...
40 / 71
Get the tweetsdef download_tweets(screen_name,number_of_tweets,max_id=None):
api_url = "%s/statuses/user_timeline.json?" % base_twitter_url api_url += "screen_name=%s&" % screen_name api_url += "count=%d" % number_of_tweets
if max_id is not None: api_url += "&max_id=%d" % max_id
# send request to Twitter response = requests.get(api_url,auth=oauth)
if response.status_code == 200:
tweets = json.loads(response.content)
return tweets
41 / 71
All the tweets!def download_all_tweets(username): full_tweet_list = [] max_id = 0
tweet_list = download_tweets(username,200)
oldest_tweet = tweet_list[::-1][0]
while max_id != oldest_tweet['id']:
full_tweet_list.extend(tweet_list)
max_id = oldest_tweet['id']
time.sleep(3)
tweet_list = download_tweets(username,200,max_id-1)
if len(tweet_list): oldest_tweet = tweet_list[-1]
full_tweet_list.extend(tweet_list)
return full_tweet_list
42 / 71
43 / 71
Where the tweets at?for tweet in tweets: templist = [] if tweet.has_key("geo") and tweet['geo']: latitude,longitude = tweet['geo'].get("coordinates") r = requests.get("http://maps.googleapis.com/maps/api/geocode/json? latlng="+str(latitude)+","+str(longitude)+"&sensor=true") res = json.loads(r.text) try: for i in res['results'][0]['address_components']: if "neighborhood" in i['types'] or "administrative_area_level_2" in i['types'] or "postal_code_suffix" in i['types'] or "country" in i['types'] or "postal_code" in i['types']: pass else: templist.append(i['long_name']) except: pass
44 / 71
Enriching Maltegolast = Counter(newlist).most_common(5)x = 5for address in last: ent = me.addEntity("maltego.Location",address[0]) ent.addAdditionalFields('link#maltego.link.thickness','','',x) x = x - 1
45 / 71
Transform in Action
46 / 71
Then we found this...
47 / 71
28702eb195e8b292ceb8c5eb4d8b94d05b6c2a62 SHA1 : 3nd3rwiggin
48 / 71
49 / 71
Unique Username = Win
50 / 71
51 / 71
52 / 71
Personal Home address (Twitter & Etsy)
Class locations (Twitter)
Password (Have I Been Pwnd?)
Close Friends (Twitter & Instagram)
Job History (LinkedIn & Facebook)
Home IP Address (Reddit Login History)
Birthdate (Etsy)
Barber (Twitter)
What we found:
53 / 71
Personal
Family
Addresses (Whitepages & Property Records)
Members (Google+)
Names (Maltego)
What we found:
54 / 71
Use CasesRed Team
55 / 71
56 / 71
57 / 71
Use CasesBlue Team
58 / 71
WALK THE LINE
59 / 71
Twitter See if public activity is malicious
Following with competitors?
Talking with competitors?
Talking about your brand?
Blue Team
60 / 71
Work badges
Passwords
Network Diagrams
Blue Team
61 / 71
Github
Committed sensitive �les
Committed proprietary code
Committed company info
Blue Team
62 / 71
Github
See if public activity is malicious
Friends with competitors?
Talking about your brand?
Blue Team
63 / 71
Github
Brand Monitoring
Scumblr by Net�ix
Monitor Forum Chatter
Monitor Your Name
Blue Team
64 / 71
Github
Brand Monitoring
Rate Employees
Most is going to be accidental
Who's your most active employee?
Monitor them closer
Blue Team
65 / 71
Github
Brand Monitoring
Rate Employees
SEIM
Alert
Correlate
Blue Team
66 / 71
Interrogator Web Application
Continuous OSINT Monitoring of Workforce
Visualize relationships with a Graph Database
Coming mid 2016!
67 / 71
68 / 71
Reccomendations Justin Seitz - @jms_dot_py
The Grugq - @thegrugq
automatingosint.com
bellingcat.com
69 / 71
Q&A
70 / 71