Download - Sentinel Week 9 H4D Stanford 2016
Team Sentinel
● Team members:
○ Jared Dunnmon
○ Darren Hau
○ Atsu Kobashi
○ Rachel Moore
● Cumulative # of interviews: 100 + 12
○ Users: 4 MDA SMEs: 3 Buyer SMEs: 5
● What we do: Enable rapid, well-informed decisions for anti-IUU and future C2 by combining effective UI with high data density, local storage, and analytics
○ Open and automated data aggregation (i.e. incorporate open source data)
○ Flexible layering and filtering with improved UI/UX
○ Enhanced intel through contextualization and easily accessible, common database
○ Identifying deviations from baseline by utilizing historical data
○ Shareable, socializable conclusions visible up and down chains of command
● Why it matters:
○ Outdated platforms not built for current generation--21st century C2 needed
○ A2/AD prevents timely deployment of traditional ISR--need rapid conclusions from available data
○ Data aggregation platforms and database access in PACOM are extremely manual
● Military Liaisons
○ --- (Colonel, US Army)
○ --- (Commander, US Navy)
● Problem Sponsor
○ --- (Lieutenant, US Navy 7th Fleet)
● Tech Mentors include:
○ --- (BAH)
QUOTE OF THE WEEK
“There are solutions out there that may work, but the biggest problem is implementation.”
Contents
1. MVP
2. Key Partners, Resources, and Activities
3. Learning about Costs
4. Experiments for Fixed Cost, Variable Cost
5. Mission Model Canvas
6. Cost Flows
7. Finance and Operations Timeline
MVP
CIC PACOM
Surface radar contact but no AIS… This is odd. Let me ALERT others.
MVP
CIC PACOM
Surface radar contact but no AIS… This is odd. Let me ALERT others.
I see an ALERT from DDG102. Lets share the C2 screen and take a look
MVP
CIC PACOM
MVP
CIC PACOM
Customer Discovery DeploymentProduct Development Navy Testing
Initial Testing
Information Assurance
Maintenance & Support
Key Partners, Resources, Activities
TRL 1 TRL 2 TRL 3 TRL 4 TRL 5 TRL 6 TRL 7 TRL 8 TRL 9
Partners Resources Incentives Risks & Obstacles
DIUx $50-100K (?), network Create + validate innovation transfer model
Spending authority is not as broad as we expect
NPS Network, access to data sources, SME on analytics, Navy systems
Inter-university collaboration Their research thrust might differ from ours
SeaVision, SkyTruth, Sea Scout
Exposure to existing tools, data sources Outside team addressing capability gaps in IUU fishing
They have stake in their own tool, proprietary or classified tools and data
Palantir Access to tools, technical advice, B2G experience Leads to Navy as potential customer
They might decide to create own tool
Lawyers IP and contracting expertise $150K (?) Finding lawyers with the right experience and investment in our problem
Sub-contractor (i.e. Lockheed Martin, OGSystems)
Expertise in contracting work with government $50K (?), potential future work with Navy
Finding the right company with solid track record and willingness to work with non-traditional
GSA/18F Highest-level information assurance Fulfill their purpose People unfamiliar w/ process = slowGSA IA route actually takes longer
OPNAV/SPAWAR/NWDC Explicit requirements statementsExpertise on procurement routesTesting in relevant environment
Improving function of Navy tools Might not be open to working with non-traditional groups
Coast Guard, 7th Fleet, 3rd Fleet, PACOM
Understanding of pain points + use casesUser feedback on MVP and productBuy-in from top levels
Bringing in C2 tools built for millennial sailors
Complexity of navigating large orgLow bandwidth / availability
Key Partners, Resources, Activities
Key Cost Experiments
Hypotheses Experiments Results Action
There are different kinds of contract vehicles worth considering (cost plus award, cost plus fixed fee, fixed fee, T&M)
-Interview with General Partner of mid-size defense (OGSystems)-Will do more validation!
Validated with caveat that, at least within IC community, there is a shift towards cost plus fixed fee b/c validating performance for award is too cumbersome (more experiments will be run for further validation)
- Talk with people within Navy who are familiar with Navy contracts to determine appropriate cost structure
Necessary to expend resources setting up quality process (i.e. AS-9100, ISO-9000)
-Interview with General Partner of mid-size defense (OGSystems)-Will do more validation!
Invalidated! These are nice-to-haves at least for signing the first contract (more experiments will be run for further validation)
- Talk with people within Navy who are familiar with Navy contracts to determine appropriate quality process for first contract
Information assurance process is avoidable and therefore we don’t have to worry about its associated costs
- Interview with problem sponsor Jason-Interview with NWDC
Invalidated, but there are potential shortcuts like the Office of Digital Services, 18F, and GSA that would hold us to a higher standard yet allow us to circumvent the long and expensive typical IA process; NWDC personnel can give us excruciating detail
- Talk with Office of Digital, 18F, and GSA or people who are familiar with these avenues as acquisition pathways
Experiments for FC and VC Validation
Experiments What we will learn Who we want to speak with
Interview commercial companies with a similar software product
- General cost breakdowns (i.e. personnel, overhead) for company with a similar product- Rough estimates for actual costs associated with each bucket
- Global Fishing Watch, Skytruth, Pew Charitable Trust- Software companies who sell C2-like tool to customers like FedEx, UPS, etc
Interview defense contractors with similar software in terms of user base and/or deployment
- General cost breakdowns (i.e. personnel, overhead) for company with a similar product- Rough estimates for actual costs associated with each bucket
- Palantir, Northrop Grumman, Scitor, SAIC
Interview experienced VC personnel with experience in developing software companies
-general approach to reasonable cost assumptions- ways to approach project management to reduce variability and mitigate unexpected costs
-BVP, etc.
Try for ourselves! Where we went wrong, what we did right
Our future selves
Data- Compile existing data resources/scope out future ones
Defining C2-F- Brainstorming what “Command and Control of the Future” would be by interviewing younger sailors
Software Development- Develop flexible data fusion/analytics algorithms, and an intuitive UI for millennials
Information Assurance
Prototype Testing/Procurement
Contracting, Acquisitions
Maintenance and Support
USN Strategic Decision Makers
USN Analysts (N/J2)
USN Operators (N/J3)
Anti-IUU Fishing Enforcers (USCG, Partner Nations, etc.)
Anti-IUU Fishing Stakeholders (NGOs, Legal Fishing)
(Commercial entities that use/would benefit from enhanced C2-type systems)
USN- Timely, accurate operational decisions- Decreased time to predict hot spots, ID & differentiate threats- Increased engagement and effectiveness of younger sailors - Up-to-date, reliable info in frontline environment
Anti-IUU Fishing- Reduction in IUU fishing worldwide due to better deterrence- Better allocation of scarce / expensive interdiction resources- Widespread engagement of operators, governments, and the public
USN- Work with fleet sponsor to get C2-F system on fleet needs list- Ensure C2-F makes it into FIMS database, engage S&T bridge personnel to talk with key decision makers- Work with NWDC, ONR S&T, PACFLT LOEs to test solution - Engage PACFLT N8/N9 shops to implement modular operational deployment & update pathways
Anti IUU Fishing- Work with NGOs, gov’t departments, USCG, operators, etc. to find key influencers/stakeholders- Deploy solution where possible,
Fixed- Existing Software tools/APIs, Data- IA process steps- Travel for site visits, pilots, interviews with sailors- R&D personnel- AWS & Distributed Computing- Overhead
Data & AnalyticsAPIs for accessing data (e.g. API for Global Fishing Watch, AIS), $$$ needed to access this
Defining C2-FIdeas/feedback from young sailorsHackathon w/ Navy and DIUx support
Software DevelopmentAWS, programmers, $$$ for both, SME on phenomenology of ships, activities
- Need commanding officer to confirm decision-making benefits
- Need intelligence officers from ONI / N2 and operators from N3 to confirm effectiveness of insights
- Need IT approvals to integrate into systems
- Need support of commercial partners if we want to leverage their platforms
-Need support of existing PMOs/S&T personnel to make sure we’re not duplicating work
Beneficiaries
Mission Achievement
Mission Budget/Costs
Buy-In
Deployment
Value Proposition
Key Activities
Key Resources
Key PartnersDataSkytruth, Pew, GFW, TerraSAR
Defining C2-F7th,3rd Fleet junior officers, sailors
Software developmentPalantir Skytruth, NPS/ONR, SeaVision, Sea Scout, Universities (e.g. Vanderbilt), NGOs
Information AssuranceGSA, NWDC
Prototype Testing/ProcurementUSFF (NWDC), NAVSEA, SPAWAR, C7F CIG, PACFLT CSIG, IA contact
Contracting, Acquisitions-IP Lawyer, subs with gov experience-DIUx, C3F N8/9, PACFLT N8/N9
Mission: Creating C2-F--Enabling Rapid Decisions from Heterogeneous Data
Information Assurance Access to personnel to provide certification / approval
Prototype Testing/AcquisitionNavy testing venue and exercise (e.g. Trident Warrior), Military Sealift Command ships, 7th Fleet experimentation ships and personnel
Contracting, AcquisitionsDomain knowledge of software contracting and IP from lawyers, subsVariable- Maintenance and Support- Integration with existing systems and processes
IMPROVE USN DECISIONS & OPS VIA C2-F WITH
IMPROVED DATA HANDLING, UI/UX,
COMMS, AND HARDWARE
(1) Rapid Strategic Decisionmaking via Improved Reporting, Coordination, Visibility
(2) Improved Tactical Decision Making via Timely, Accurate Information Sharing
(3) More Effective Analysis via Searchable, Visualizable, Source-Flexible Data Integration (Layering & Filtering)
(4) Increased Analyst Bandwidth via Predictive Intel and Alerts (e.g. Machine Learning) Flexibly Applied to Available Data
(5) Improved Collection of Existing Data Streams
(6) Increasing Morale & Engagement for Millenial Sailors
ENHANCE ANTI-IUU FISHING CAPABILITIES
(1) Improved Detection Using Data Fusion/Analytics
(2) Enhanced Enforcement via Improved Communication
(3) Lower Barriers to Engaging Civilians in Reporting IUU Fishing Activities
Cost Flows
Database ($80k)
Analytics Engine ($120k)
Translation (ETLs) ($100k)
AIS VMS Radar SAR Sat
UI ($80k)
Information Assurance
($240k)
Testing ($480k)
Maintenance and Support
(VC)
Assume 10 data streams, need cost validation on streams
$380K $240K $480K $???
Total: $1.1 MM + Var Costs
3 Year Financial/Ops/Funding Timeline
2016 2017 2018 2019
Q3 Q4 Q1 Q2
Cas
h R
eser
ves
Phase
Prod
uct
Gov
’tC
om’l
Mile
ston
es
Q1 Q2Q3 Q4 Q1 Q2Q3 Q4
TRL 1
TRL 2
TRL 3
TRL 4
TRL 5
TRL 6
TRL 7
TRL 8
TRL 9
POC
Wireframe
Prototype BetaPrototype
Marketable Product
BetaPrototype
Released to first customers (<3)
Commercial Product Launch;
2 contracts
Test in Navy env Navy-wide Deployment Maintenance and Support
V2.0 Commercial Product Launch;
5 contracts signed
Initialize System Development/Customer Relationship Development Launch at scale
Hea
dco
unt
410
20 50
15 customers; V2.5 launch
CONTRACT SIGNED CONTRACT
RENEWED
SBIR/DIUx
Series A (In-Q-Tel, Impact Investor(s))
2 com’l contracts
$250k$0
$1.25M DoD Contract
$2M
$5M
Thank you!
Any questions?
Products& Services- Timely data- Good UI/UX
for presenting data
- Streamlined reporting process
- Improved coordination across ranks
- Utilizes current tool pipeline
Customer Jobs
Gains
Pains
Gain Creators
Pain Relievers
- Good UI/UX- Platform
incorporates more data streams
- Platform is robust and can handle drop out of data streams
- Allocate assets- Identify, eliminate
threats- Predict hot spots- Safety and morale
of team- Projecting peace,
stability in region
- More informed decisions
- Faster decisions- Decisions made
on most up-to-date info
- Poor quality/lack of data
- Latency of data -> insight
Admiral/Strategic Decision Maker
Value Proposition Canvas
Customer persona:
● 3 or 4 star admiral● Born in late 1950’s● Have their own office on-base● Gives out challenge coins● Responsibility for well-being of
their entire organization (e.g. 7th Fleet)
● 30,000 ft view thinker● Spent entire professional career
in Navy (deeply ingrained culture)
Products& Services
- Contextualized, object-oriented database
- Algorithms for processing, analyzing data
- Ability to search for trends across database
- Integration of disparate data sources
- Automation of data analysis- Improved UX/UI enabling
greater engagement- Predictive notifications- Filtering and layering
features- Tool architecture is flexible
and intelligent
Customer Jobs
Gains
Pains
Gain Creators
Pain Relievers
- Compatible data format
- Incorporate multiple data streams with existing object-oriented database
- Integration into current processes is simple
- Collect & analyze data
- Communicate findings
- Piece together contextualized awareness
- More actionable insights- Faster identification & response times- Easy-to-use- Information continues to be processed
and visualized even if data streams are added/dropped (no Christmas Light effect)
- Incorporation of context is manual/mental
- Poor quality / lack of data- Latency of data -> insight- Long onboarding processes
Analyst (N2)
Value Proposition Canvas
Customer persona:● 18-35 years old, described as “19-year old from Oklahoma”● Sits in front of computer 12 hours per day● Job usually boring with bursts of activity● Regimented daily process, but schedules could change drastically
day-to-day
Products& Services
- N/A
- Actually a common operating picture!
- Data is actually synced across databases
- Tool architecture is flexible and intelligent
- UI/UX that is much more intuitive for the millennial sailor
Customer Jobs
Gains
Pains
Gain Creators
Pain Relievers
- No hardware to deploy so no risk of asset or personnel loss
- Fewer change orders- Training and integration
with current processes is simple
- Utilize assets and human capital in order to obtain ISR data on adversary or regions of interest
- Timely and enhanced allocation and deployment of assets
- Information continues to be processed and visualized even if data streams are added/dropped (no Christmas Light effect)
- Increased engagement
- High manpower, time- Operator error- Safety concern for deploying
in unfriendly territory- Struggle to redeploy systems
on short notice (<12 hours) = frustration
- Long onboarding processes
Operations (N3)
Value Proposition Canvas
Customer persona:On Ship
● 18-35 years old● Follow preset “if-then” procedures● When passing through high-risk
area, very jumpy
Customer persona:Head of N3
● 40-55 years old● General sense that N2 and N6
“work” for them● Frustrated by changing demands
- Crowd-sourced data collection
- Better COP with partner nations
- Enable fast response with better allocation of assets
Customer Jobs
Gains
Pains
Gain Creators
Pain Relievers
- Better intel = more effective use of assets
- Locate IUU fishing activity
- Apprehend the individuals involved
- Easier to communicate insights
- Increased buy-in from partner nations
- Improved deterrence
- Inability to detect “dark” targets
- Expensive interdiction vehicles sent to verify IUU fishing
Anti-IUU Fishing Enforcers (USCG)
Value Proposition Canvas
Customer persona:Intel (works in MIFC)
● 25-40 years old● Deep domain expertise - interfaces with
partner nations, NGOs● Develops internal tools● May be re-assigned by HR to unrelated roles
Products& Services
- Contextualized, object-oriented database
- Algorithms for processing, analyzing data
Customer persona:Operator (on ship)
● 20-30 years old● Is law-enforcement presence - can direct
entire Navy ship for related tasks● Domain expertise or not???
Anti-IUU Fishing Stakeholders
Value Proposition Canvas
● PLACEHOLDER● We are gathering more info on:
○ NGO’s○ Fisheries○ Partner nations○ +other potential Anti-IUU Fishing Stakeholder beneficiaries
Commercial Entities● PLACEHOLDER● We are gathering more info on:
○ Fisheries (as a customer)○ Amazon (also as they look to develop their drone delivery
capabilities)○ Commercial Shipping companies○ O&G companies
Mission Achievement
Beneficiaries Mission Achievement
Strategic Decision Makers (Admirals)
- Better, faster decisions despite uncertain information- Improved COP with analysts + operators = save
manpower/bandwidth resources associated with preparing PPT- Better understanding of what data contributed to analysis in CUB,
and reliability of that information
Analysts (N/J2) - Predictive analytics based on historical baselines eliminates tedious elements of workflow → more engagement
- Data aggregation into common database decreases time to insights
Ship CIC Operators (N3) - Up-to-date, reliable information in frontline environment- Easy to share COP with analysts + strategic decision makers- Next-generation user interface which is intuitive + leverages
familiarity with commercial tech → more engagement
Anti-IUU Fishing Enforcers (USCG, w/ support from USN)
- Better allocation of scarce / expensive interdiction resources- Faster response = better deterrent
Anti-IUU Fishing Stakeholders (Partner nations, NGOs, legal fishers)
- Ensure economic + food security- Reduce # of illegal fishers + associated IUU catch- Improve working conditions / reduce human trafficking
MVP: Modular Intake, Algorithm, and Display
MVP: Modular Intake, Algorithm, and Display
MVP: Modular Intake, Algorithm, and Display
MVP: Modular Intake, Algorithm, and Display
Customer Discovery - Get/Keep/Grow Diagram
Awareness Interest Consideration Purchase Keep Unbundling Up-sell Cross-sell Referral
Activity & People
- Evangelist & advocate from originator Flt- ???
Corey Hesselberg, CDR Jason Schwarzkopf, MIOC watch standers
- Buy-in from flag officers- ADM Swift, VADM Aucoin, RADM Piersey
- N8/9- Dave Yoshihara (PacFlt N9)- 7th Fleet ???
- Maintainers (N6)- Bob Stevenson (PacFlt N6)- 7th Fleet ???
N/A Expanding COP & intel extensions / functionality within 7th Fleet
Expanding user base within 7th Fleet
Expanding tool set to other fleets
Metrics % people who have heard of program before vs after *how to reassess?
# people who say “we want this”
Seems binary… any recommendations?
# Systems outfitted
?? ?? ?? # users within 7th Fleet using tool
# fleets using tool
Map of System Functions and Needs
QUELLFIRE
GCCS (1)
FOBM
STORAGE/COMMS
CST
GCCS (3)GCCS (2)
STORAGE/COMMS
STORAGE/COMMS
Sensors Sensors Sensors
.oth-.json Translator
Visualization
Analytics
Ship-to-Ship Sharing
Long-Term Storage
KEY NEEDSFUNCTIONS
& PROGRAMS
SHIP 2 SHIP 3SHIP 1
MVP: Software Domain Awareness
Program POC OrganizationFunction & Goals
To be used by whom?
Security Level Status Contract History Inputs
Technical Details
CSII
Insight
MTC2
Quellfire
DCGS-N Increment 2
C2PC
HAMDD
SeaVision
GCCS
EWBMRC2 (Resilient C2)
Sample In-Development Product: ONR/CTI EWBM Tool
MVP (3 weeks ago)
MVP (3 weeks ago)
MVP (3 weeks ago)
MVP (4 weeks ago)
AIS Weather
MVP (4 weeks ago)
AIS Weather
MVP (4 weeks ago)
AIS Weather
Customer Workflow
N2
N3
N2(“owns”
the intel)
N3(“owns”
the assets)
Ready-To-Use DataDeployment
Data Acquisition
Data Analysis
Data
Order/Decision
Customer Workflow
Key Acquisition Paths
● Several potential deployment strategies
○ Linking in with an existing POR (PMW-150, etc.)
■ Pros: Allocated funding, long-term integration plans
■ Cons: Long timescale, getting in the door
■ POCs: ONI, SPAWAR (Stan Kowalski), Primes
■ Source of Excitement: TBD
○ Rapid Acquisition Pathways (Limited Objective Experiments, Rapid Reaction Technology Office)
■ Pros: Speed, Close to user, Don’t have to go through Navy (other services work)
■ Cons: Limited spending authority
■ POCs: 7th Fleet (Jason Knudson), DHS (Chuck Wolf)
■ Source of Excitement: Rapid deployment, changed acquisition model
○ DARPA
■ Pros: Development mindset, existing programs (Insight) that are well-aligned, deployment authority/capability to pay for deployment to end-users
■ Cons: stepping on toes, limited number of PMs
■ POCs: Craig Lawrence (ADAPT)
■ Source of Excitement: Directly solving a problem as opposed to many-year process
Sample Deployment Path (Software, POR Path)
1. Operational testing to make sure meets military specs (engage SPAWAR for this)a. Ensure NSA-standard Information Assurance (IA)
i. Lock down system and codeii. Make sure no category 1,2,3 in code - backdoors, exceptions, etc.
b. Observe appropriate NIST protocols (TBD)2. First, limited deployment to evaluate functionality (on testbed system or specific asset)3. Then, if integrated into a POR:
a. Deployed on whatever platform is neededb. Moves into sustainment phasec. Think about disposal & replacement--we want continuous improvement!
4. IT installs where requireda. Technical support install software and make sure up and runningb. Maintains communications systems and networks
5. Personnel training for system operation and maintenancea. CTMs focus on maintaining classified systems & special collections abilities
WE WILL BE GETTING MORE DETAIL ON THIS GOING FORWARD!
Procurement + Deployment Schedule
Activity Timeline Funds People + Resources
Continue Customer Discovery w/ Gov’t May - Sept 2016
$0 Sentinel, H4D cohort, Navy, USCG, OSD
Customer Discovery with Commercial Entities May - Dec 2016
$0 Sentinel, Teaching Team, Chris Robinette, GSB network
Continue Technical Development of MVP May - Dec 2016
$0 Sentinel, AWS(?)
6 month Period of Performance SBIR/BAA/etc tailored to our MVP is released and awarded to us
Sept - Dec 2016
$0 Sentinel, Rich LeBron, NWDC(?), DHS(?)
Work on SBIR/other contract and commercial product in parallel
Jan - June 2017
$250k - $70k Sentinel
Lay framework for DoD acquisition (i.e. build relationships, iron out process, LOE, etc)
Jan - June 2017
$0 Sentinel, Rich LeBron, influencers and acquisition DoD people
Identify and develop relationships/partnerships with key early adopter/early supporter entities (i.e. commercial companies, NGO’s)
Jan - June 2017
$0 Sentinel, TBD
Close acquisition and sales deals with DoD and early adopter commercial companies!
June - August 2017
$1MM - $3MM, for each contract
Sentinel, DoD (specifics TBD), TBD
Procurement + Deployment Methods
● This references the italicized line in the schedule● For Navy, one potential deployment avenue is latch onto a program of record
already in the Fleet Experimentation Info Management System● We can also leverage SBIR contract as a way to set up for eventual acquisition● We know that the Coast Guard brings in technology for evaluation (on the path
to acquisition) relatively frequently throughout the year○ We just need to determine how we insert ourselves into that funnel
● In terms of implementation, we propose an external hard drive that has historical data pre-loaded in addition to our software solution○ In addition to the solution, this is beneficial because of bandwidth issues
and memory issues (and our customer discovery has intensely validated these needs)
○ This avoids having to deal with deploying our software on their hardware while maintaining modularity and flexibility of our solution
○ It also gives us a fully vetted avenue by which to test and roll out updates in modular fashion