aggregate db lessons learned h4dip stanford 2016
TRANSCRIPT
87 Interviews
AggregateDBSponsor: Conflict and Stabilization Operations (CSO)Point of Contact: Santiago Stocker
Original Challenge:CSO needs to acquire more information about informal leaders.
Final Problem Solved:American Diplomats have this information, but it is unstructured, inefficient and difficult to access.
MS Computer Science
Software Engineer, Machine Learning
NLP & Data Mining
Meet Our Team
Shutong Zhang
BS Computer Science
Front-End, UI/UX
Software EngineerChristian Valadez
BA History
Minor Arabic
Regional ExpertNathan Hausman
Start from a real story...
A Case Study : Crisis in Algeria
21 Days until shutdown....
Helicopter Company
A Case Study : Crisis in Algeria
Embassy in Algeria
18 Days until shutdown....
Helicopter Company
A Case Study : Crisis in Algeria
Embassy in Algeria
16 Days until shutdown....
Diplomat: “Who Do I Talk To?”
Helicopter Company
A Case Study : Crisis in Algeria
Embassy in Algeria
0 Days until shutdown....
Diplomat: “Who Do I Talk To?”
Helicopter Compayny
Policy Maker: Informed Diplomat
A Case Study : Crisis in Algeria
Embassy in Algeria
0 Days until shutdown....
Diplomat: “Who Do I Talk To?”
Helicopter Company
Policy Maker: Informed Diplomat
“Real people and real companies suffer when diplomats can’t find the
right people at the right time.”
Our Journey
Understand the Real Problem
Proposed an Embassy
Portal
MVP #1
MVP 1: Connecting CSO to Embassy
MVP 1: Connecting CSO to Embassy● Reviewed With 2 policy makers, 8 diplomats
Uniform feedback from Diplomats in Africa:
“I do not have time to reupload this
information.” x10 “Just make the policy makers read my
cables.”
Our Journey
Understand the Real Problem
Proposed an Embassy
Portal
Cable Scraping
MVP #1
MVP 2: Summarizing Classified Cables
Use Natural Language Processing to “scrape” key - Names- Locations- Dates
MVP 2: Summarizing Classified Cables● Reviewed with 10 diplomats, 5 desk officers and 2
technologists
Diplomat: “Foggy Bottom is a cable monster.” x10
Desk Officer : “Cables are classified, unaggregated and unstructured.” Policy Maker: “Cables don’t show
relationships I need to know.”
MVP 2: Summarizing Classified Cables
2018
Cable Monster
Technologists Diplomats
Customer PivotFrom Policy Makers to Diplomats
Our Journey
Understand the Real Problem
Proposed an Embassy
Portal
Cable Scraping
Contact Network
Map
MVP #1 Pivot #1
MVP 3: Contact Network Map
MVP 3: Contact Network Map● Reviewed With 8 diplomats
Diplomats (Political/Economic):
“I really only need this to map my 10 closest contacts.”
“Finally, a product that addresses the needs of diplomats on the ground!
Contact Management &
Suggestion
Diplomat (Public Affairs):“Hey wait, can this
thing suggest people to invite to diplomatic functions?”
Contact Management &
Suggestion
Diplomat (Public Affairs):“Hey wait, can this
thing suggest people to invite to diplomatic functions?”
Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value PropositionKey Activities
Key Resources
Key Partners
CSO: Santiago - Writing report about Boko HaramTyler - Analyzing relationship between known members of BH and unclassified actorsWill Meeker - Travels to Nigeria for 2 years to generate contacts for CSO, get Santiago data
Embassy Staff:LES: Locally engaged staff Caroline: Economic/ Political Officer at small Embassy in stable countryJames/Steve: Economic/ Political Officer at large Embassy in an unstable country Nick: Public Diplomacy Officer at busy Embassy
S/ES& Special ProjectsKapil Gupta: Commitment to show off our demonstrate prototype John Janek:Will send our prototype to Mauritania Embassy friends w/DCM
Foreign Nationals:(OMS): Office Management Specialists and LES To implement and upkeep program at each embassy with updated information about contacts from events/meetings
Individual FSO buy-in
DCM to okay program at 1 embassy as test case and delegate time for FSO’s and LES to test/use
Kapil Gupta: Commitment to show off our demonstrate prototype
CSO Sponsor: Santiago Stocker
IDEO team leader for QDDR
A Dev Garden Consulting: TAP, SBIR, Accelerator/Combinator and Angel Funding consulting, connections, proposal writing
Beta starts in Nigeria Embassy, with Vanguard newspaper(English) and twitter hexagon of Abuja
Target 8 Countries in MENA- Saudi, Israel, Qatar... when NLP+AI perfected, final contract from DOS when product 3.0 ready for world-wide deployment
Need CSO to continue connecting us upstream w/ former clients and downstream with data sources in field
DCM initial support, for 1 or more POL/ECON sections FSO to implement first 10 names that come to mind, LES long term key holders
Source Aggregation: Gives organizations the ability to identify most important actors in rapidly evolving situations using data mined from a breadth of sources.
Hierarchy visualization: ensure people from all backgrounds can analyze and filter relevant data depending on their needs.
Extraction/Suggestion: Perform natural language processing on news articles, twitter and etc to extract categories and information for informal leaders. Suggest novel contacts for each FSO based on known network.
Improve CSO’s ability to write relevant reports by giving them tools for passive “granular” data collection w/o Embassy involvementProvide long-term contact management, node mapping software, hierarchical ranking software, news feed of recent activity of known contacts new contacts discoveryContact suggestion that cuts reliable informant acquisition time in half
Current contact management and information collection
New contact discovery and understanding
Previously published unclassified reports commissioned by DOS
Nigerian news papersgurdian, sahara Times, Punch, Business day, VanguardPolis Backbone
SV companies /DOS projects who are working in a similar space aka SalesForce and Transparency team (partnerships)
Immediate: MVP--> Prototype that integrates w/ DOS Blackberry Classic and Windows XP... sweat equity FY1: Deployment to 1 Embassy: Travel to Target Embassy, Legal Fees for DOS Cloud Review, 200 contract hours with AI & UIXUXFY2-4: NLP Non-Latin Script- LinguaSys GlobalNLP, Full Time AI expert to perfect contact suggestion, travel for sales/lobbying
Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value PropositionKey Activities
Key Resources
Key Partners
CSO: Santiago - Writing report about Boko HaramTyler - Analyzing relationship between known members of BH and unclassified actorsWill Meeker - Travels to Nigeria for 2 years to generate contacts for CSO, get Santiago data
Embassy Staff:LES: Locally engaged staff Caroline: Economic/ Political Officer at small Embassy in stable countryJames/Steve: Economic/ Political Officer at large Embassy in an unstable country Nick: Public Diplomacy Officer at busy Embassy
S/ES& Special ProjectsKapil Gupta: Commitment to show off our demonstrate prototype John Janek:Will send our prototype to Mauritania Embassy friends w/DCM
Foreign Nationals:(OMS): Office Management Specialists and LES To implement and upkeep program at each embassy with updated information about contacts from events/meetings
Individual FSO buy-in
DCM to okay program at 1 embassy as test case and delegate time for FSO’s and LES to test/use
Kapil Gupta: Commitment to show off our demonstrate prototype
CSO Sponsor: Santiago Stocker
IDEO team leader for QDDR
A Dev Garden Consulting: TAP, SBIR, Accelerator/Combinator and Angel Funding consulting, connections, proposal writing
Beta starts in Nigeria Embassy, with Vanguard newspaper(English) and twitter hexagon of Abuja
Target 8 Countries in MENA- Saudi, Israel, Qatar... when NLP+AI perfected, final contract from DOS when product 3.0 ready for world-wide deployment
Need CSO to continue connecting us upstream w/ former clients and downstream with data sources in field
DCM initial support, for 1 or more POL/ECON sections FSO to implement first 10 names that come to mind, LES long term key holders
Source Aggregation: Gives organizations the ability to identify most important actors in rapidly evolving situations using data mined from a breadth of sources.
Hierarchy visualization: ensure people from all backgrounds can analyze and filter relevant data depending on their needs.
Extraction/Suggestion: Perform natural language processing on news articles, twitter and etc to extract categories and information for informal leaders. Suggest novel contacts for each FSO based on known network.
Improve CSO’s ability to write relevant reports by giving them tools for passive “granular” data collection w/o Embassy involvementProvide long-term contact management, node mapping software, hierarchical ranking software, news feed of recent activity of known contacts new contacts discoveryContact suggestion that cuts reliable informant acquisition time in half
Current contact management and information collection
New contact discovery and understanding
Previously published unclassified reports commissioned by DOS
Nigerian news papersgurdian, sahara Times, Punch, Business day, VanguardPolis Backbone
SV companies /DOS projects who are working in a similar space aka SalesForce and Transparency team (partnerships)
Immediate: MVP--> Prototype that integrates w/ DOS Blackberry Classic and Windows XP... sweat equity FY1: Deployment to 1 Embassy: Travel to Target Embassy, Legal Fees for DOS Cloud Review, 200 contract hours with AI & UIXUXFY2-4: NLP Non-Latin Script- LinguaSys GlobalNLP, Full Time AI expert to perfect contact suggestion, travel for sales/lobbying
Value Proposition Extraction/SuggestionsPerform natural language processing on twitter and RSS feeds to extract categories and information for informal leaders. Suggest novel contacts for each FSO based
on known network.
Our Journey
Understand the Real Problem
Proposed an Embassy
Portal
Cable Scraping
Final MVP: Contact
Suggestion
Contact Network
Map
MVP #1 Pivot #1
IDEOAggregates information and facilitates sharing
- AMBITIOUS prototype
IDEO Same conclusion
State Department is in need of contact management solution.
Did IDEO overreach? IDEO wanted to gut and replace entire workflow, instead of addressing specific pain points. What we learned Do not interrupt people’s existing workflow.
1
2
3
Your team learned in 9 weeks what we did in 9
months. ”“
Final MVP: Contact Management & Suggestion
RSSFeeds
Trad.News-Papers
ContactSuggestion
Known Contacts
People You Should Connect With
Connect
Rotimi Akeredolu, Governor- Elect of Ondo State
Connect
Betty Akeredolu, founder and president of BRECAN, married to Rotimi Akeredolu
Connect
Simon Lalong, Nigerian lawyer and politician
See More ->
Natural Language Processin
g
https://github.com/USStateDept/Polis
Deploying AggregateDB to Fix Crisis in Algeria
Embassy in Algeria
Shorten to 7 Days using AggregateDB
Diplomat: “Who Do I Talk To?”
Helicopter Companies
Policy Maker: Informed Diplomat
Deploying AggregateDB to Fix Crisis in Algeria
Embassy in Algeria
Shorten to 7 Days using AggregateDB
Diplomat: “Who Do I Talk To?”
Helicopter Companies
Policy Maker: Informed Diplomat
Deploying AggregateDB To Fix Diplomacy Crises
● Deployment to target embassy - Mauritania
● Work w/ IIP for 8 embassy deployment