apis in enterprise
TRANSCRIPT
© 2015 by Victor OlexFounder & CEO, SlashDB
@agilevic
APIs in EnterpriseUsing APIs for reporting, data science
and systems integration
2Source: Innosight - “Creative Destruction Whips Through Corporate America”
S&P 500 Churn 2002-2012
3
2002 at Amazon• All teams will henceforth expose their data and functionality
through service interfaces.• Teams must communicate with each other through these interfaces.• There will be no other form of inter-process communication allowed:
no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
• It doesn’t matter what technology they use.• All service interfaces, without exception, must be designed from the
ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
• Anyone who doesn’t do this will be fired. • Thank you; have a nice day!
“
”- Jeff Bezos
4
Flattening the Competition
5
2015 Global IT Spend $2.3TM i d d l e w a r e + S y s . I n t e g r a ti o n
$487.7B
Source: Forrester Research - Global Tech Market Outlook for 2015-2016 (after ZDNet)
ETL/Data Warehousing
6
ERP HR RISK METRICSORDERS ACCOUNTING
DATA WAREHOUSE(Single version of truth, really?)
Analytical Systems
• Data duplication• Stale data• Brittle overnight
feeds
• Central bottleneck• Does not scale out• Not easily
accessible nor searchable
7
SaaS Revenue Projections$58B worth of new data
8
Store It All in One Place?• It was hard with just
on-premises systems
• Illusory idea with today’s Cloud apps
• Try it with your contact list for starters…
ERP ORDERS HR
9
WE HAVE TO BUILD THE API,AND WE HAVE TO BUILD IT QUICKLY!
10
What is Resource Oriented Architecture (“ROA”)• “Style of software architecture and
programming paradigm for designing and developing software in the form of resources with RESTful interfaces.”
– Wikipedia• Uniform data access layer to all data
assets in their unobstructed form for reading and writing in various representations. – my take
11
What is Resource Oriented ArchitectureService Oriented• Represents Action• Transaction, Unit of Work• Message• API controlled by
functional design• Harder to adapt and scale
beyond “enterprise”• Harder to deprecate
functionality
Resource Oriented• Represents State• Addressable Resource• Update to Resource• API automatically evolves
with data• Harder to model into
complex transactions• Clients must be resilient
to change
• Single access point, but without copying data• Self-service reporting, data feeds or integrate with NoSQL
API Shell Over Data
12
Database Content as HTTP Resources
13
http://demo.slashdb.com/db/Chinook/Customer/CustomerId/1.htmlService location• On the intranet, or• In the cloud
Databasename.Supported RDBMS:• MS-SQL,• Oracle• MySQL• PostgreSQL,
and more
Table to query Field to filter and value to lookup:• Text• Number• Date
Data format• XML• JSON• HTML• CSV
Combine several
/db automatically makes hyperlinks directly to data Related records are hyperlinked thus search engine ready Filtering, drill-down, slices are natural, URLs stay nice Custom queries also possible (SQL Pass-thru)
14
Best Practices• Don’t forget about “R”
in REST– JSON isn’t the only
data format• URL should be easy
to understand– Avoid inventing mini-
query language
• Resources should be easy to discover
• Ideally every resource address should allow reading and writing
• Avoid query string to address data
15
Data & Analytics: Benefits & Challenges
Use Case: Bank - Regulatory Risk Management• Federal Reserve CCAR• Basel Independent Review• Supervisory Formula Approach (SFA)• Dodd-Frank Annual Stress Test
16
2015, Global Bank
Upwards 50% of my time goes into data reconciliation efforts.
“ The biggest pain is sharing data between Python, R, etc.
The problem is - there should be one specified entry point for data.
Consistency of column names and possible values between different versions of the data.
There are a lot of holes in the data process. I think the #1 priority would be creating a good schema. ”
Finding what you need in this zoo. (…) Currently this is done by talking to people!
17
18
Data Science Process• Data acquisition, storage, discovery and
mining, statistical learning, machine learning, predictive analytics, risk modeling
• Competencychasms at every step
19
Implemetation: SlashDB APIModel Research & Dev.
use any programming language
Reports & Visualizationdeliver now, anticipate future
Unobstructed Data Sharingstandard formats, HTTP delivery
Disparate Data Sourcesloan portfolios,
macroeconomic data, risk metrics, market data
Automatic,multi-representational,
resource-oriented,hypermedia and
search engine friendlydata API & cache.
Resource Oriented API Solves Many of the Issues• Single access point that’s easy to work with• Combines the best features of plain files
(simplicity) and databases (data integrity)• Has authentication, authorization and
encryption• Pragmatic data access for people and
programs• Search engine ready
20
21
Searchable API• Users know what they need, but may not
know where to find it• True hypermedia API should contain
hyperlinks to related resources• Search engine crawl/index is trivial when all
resources are hyperlinked• Try it yourself at:
http://demo.slashdb.com/search.html(i.e. search for: “customers from Brazil”)
Resource Oriented API is a Sensible Investment • Multiply returns on investments already
made in databases (the other ROA)• Avoid pitfalls of file-based data sharing• Avoid dangers of direct database access• Avoid opaqueness of ESB, RMI, SOAP,
CORBA, etc., etc.• Attract top developers
(they want to work on cool stuff, and they don’t know databases)
22
24
Credits & References• S&P Churn 2002-2012
“Creative Destruction Whips through Corporate America” by Richard Foster, Innosighthttp://www.innosight.com/innovation-resources/strategy-innovation/upload/creative-destruction-whips-through-corporate-america_final2015.pdf
• 2002 at Amazon“The Secret to Amazon’s Success Internal APIs” by Kin Lane, API Evangelisthttp://apievangelist.com/2012/01/12/the-secret-to-amazons-success-internal-apis/
• Flattening the CompetitionGoogle Finance, chart prepared by V. Olexhttps://www.google.com/finance?q=amzn
• 2015 Global IT Spending“Want money for that new project? Then it's time to go on a moose hunt”by Steve Ranger, ZDNethttp://www.zdnet.com/article/want-money-for-that-new-project-then-its-time-to-go-on-a-moose-hunt/
• SaaS Revenue Projections“Enterprise software spend to reach $620 billion in 2015: Forrester”by Natalie Gagliordi, ZDNethttp://www.zdnet.com/article/enterprise-software-spend-to-reach-620-billion-in-2015-forrester/
• What is Resource Oriented ArchitectureWikipediahttp://en.wikipedia.org/wiki/Resource_oriented_architecture
• Data & Analytics: Benefits & Challenges“5 Insights & Predictions On Disruptive Tech From KPMG's 2015 Global Innovation Survey”by Louis Columbushttp://www.forbes.com/sites/louiscolumbus/2015/11/08/5-insights-predictions-on-disruptive-tech-from-kpmgs-2015-global-innovation-survey/
• Data Science Processhttps://en.wikipedia.org/wiki/Data_science
• Other graphicsPhotographs of D. Trump, Flicker and public domain sources
• Logos and other trademarks are the property of their respective owners; used here for illustration purposes only, no association or endorsement implied.