sptechcon boston 2012 - sharepoint integration and interoperability

#SPTechCon @RHarbridge

SPTechConThe SharePoint Technology Conference




SharePoint 2010Integration and Interoperability:What you Need to Know


Presented By: Richard Harbridge





Who am I?

BostonWe Washington

http://www.packtpub.com/microsoft-sharepoint-for-business-executives-guide/book





Our Goal Today…

From Here To Here





What we will be talking about…1. Business Data Challenges2. SharePoint’s Important Interoperability3. What is the BCS?

• External Content Types• External Columns• External Lists• Extensibility and Tooling

5. Limitations of BCS and SharePoint 2013

4. Demonstration

Bonus!





Business Data Challenges


Business Data in an Enterprise

Customers FinancialQuality OperationsSuppliersCustomers FinancialQuality OperationsSuppliers

Users

ITHigh Overhead (Maintenance, Security)High Integration Costs (Custom)

High Training CostsHigh Dissatisfaction

Redundant Data and Processes!


Challenges with Business DataChallenge IT End User

Higher Overhead for IT to Service User Requests

Multiple Places to Search

Multiple Places to Administer and Maintain

Multiple Places to Manage Content

Multiple Places to Secure Information

High Data Integration Costs (Often 100% Custom and One Offs)

Silos/Islands of Data Lead to Redundant Data or Processes

Multiple Interfaces; Dissatisfaction and Higher Training Costs

Higher Operational and Compliance Risk

Direct ImpactIndirect Impact





The OutcomeThere are challenges that disparate and

disconnected business data systems cause.





What to watch out for…No system, product or application

can completely solve business data challenges.

People

Process

Technology





SharePoint’s Important Interoperability


“Interoperability is the ability of a system.. to work with other systems.. without special effort on the part of the customer.”

What the heck is Interoperability?

(IHN-tuhr-AHP-uhr-uh-BIHL-ih-tee)

How?


How do we achieve Interoperability?

2. By making use of a "broker" of services that can convert one system’s interface into another system’s interface "on the fly".

1. By adhering to published interface standards. Does SharePoint do this?

What about this one?

SPOILER ALERT!


SharePoint Interoperability

StorageExternal

Data

Search Platform

UI Platform

Data Access Platform

Iden

tity

Platf

orm


Is it a UI Platform?

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm


Why would we use SharePoint as a User Interface Platform?


Consistent User Experience Across Browsers

Well.. Almost.


It’s a UI Platform

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards

1. By adhering to published interface standards.


A UI Platform Bonus?

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

Bonus!

HTML Standards

Works in Office


Leverage Office and SharePoint Interface Familiarity


Is it an Identity Platform?

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards

Works in Office


Why would we use SharePoint as an Identity Platform?


Active Directory

LiveID

ASP.net Membership Trust

SharePointSTS

Client

SharePoint

Security token

4

Service token request5

Identity ProviderSecurity Token Service

(IP-STS)

SAML Based

SharePointAuthorization

ClaimsProviders

Trust

Authentication Request

3

Request Resource with service token

7

Security token response6

Request Resource

1

Authenticate Request/Redirect

2

Because Claims is Awesome

Connect with 3rd Party Authentication ProvidersConnecting with a Non Windows Integrated Authentication driven web site.

Delegation with Back-End SystemsShowing data from another server on a SharePoint web page.


Claims UsesConsistent Standards

• WS-Federation 1.1Provides the architecture for a clean separation between trust mechanisms, security tokens formats, and the protocols for obtaining tokens.

• WS-Trust 1.4Provides a method for assessing, requesting and receiving security tokens.

• SAML Token 1.1XML vocabulary used to represent claims in an interoperable way.*

• WS-Security… (various aspects) *Doesn’t actually support SAML Protocol (SAMLP)



It’s an Identity Platform

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards

Works in OfficeClaims


Is it a Search Platform?

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards



Why would we use SharePoint as a Search Platform?


Query a 3rd Party Search Engine Using SharePoint UI

Federate Queries to Multiple Search Systems…


Index Content from a Range of Content Repositories

What does all of these?

Indexes Databases, Web Services or Custom Repositories

Provides Consistent Access to External Content

Has SharePoint Designer and Visual Studio Plug-ins

B

C

S

_ _ _B C S


A Search Platform Bonus!

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards


Open Search

Bonus!


Have you heard of OpenSearch?• Standard (Creative Commons) for syndication and

aggregation of search results• Syntax for executing queries• Results returned in RSS/ATOM + extensions• Supported by Microsoft Search Server 2008

Clients: Windows 7, IE, FireFox, Chrome, SharePoint …Providers: Amazon, Flickr, Wikipedia, SharePoint…


http://technet.microsoft.com/en-us/enterprisesearch/ff727944.aspx


It is a Search Platform

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards


Open Search


Is it a Data Access Platform?

StorageExternal

Data

Search Platform

UI Platform


Iden

tity

Platf

orm

HTML Standards


Open Search


SharePoint Does a Great Job of Accessing SharePoint Content!


What about External Systems and Content?

_ _ _B C S


A Data Access Platform Bonus!

StorageExternal

Data

UI Platform


Iden

tity

Platf

orm

HTML Standards


Search Platform Open Search

Bonus! x 2

CMIS

REST


Content Management Interoperability Services

CMIS specification provides a Web services interface that: Is designed to work over existing repositories enabling

customers to build and leverage applications against multiple repositories—unlocking content they already have

Decouples Web services and content from the content management repository, enabling customers to manage content independently

Provides common Web services and Web 2.0 interfaces to dramatically simplify application development

Is development platform and language agnostic Supports composite application development and mash-up by

the business or IT analyst

A platform that allows you to build against multiple technologies without knowing anything about how all the API’s work. (If you know CMIS.)

Microsoft has been implementing this from both a client and provider perspective.

Not available “in the box”. Available via SharePoint Admin Toolkit

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=718447d8-0814-427a-81c3-c9c3d84c456e&displaylang=en


And then there’s RESTAccess to SharePoint Lists and Document Libraries…

• RESTful interface over HTTP • Uses the ADO.NET Data Services interface conventions• Feature rich: retrieve, update, browse data using

just URLs and XML/JSON• Low barrier of entry. If you have an HTTP stack

you can use it.Visual Studio Integration…

• Entirely built on top of the open RESTful interface• No client is required• Visual Studio creates proxies, visualizes schemas,

provides LINQ support etc.



SharePoint has Interoperability

StorageExternal

Data

UI Platform

Iden

tity

Platf

orm

HTML Standards


Search Platform Open Search

CMIS

REST

RBS

Some might even say it has storage interoperability or that OData is a standard… But the coolest part of interoperability?

BCS

2. By making use of a "broker" of services that can convert one system’s interface into another system’s interface "on the fly".

OData





The OutcomeSharePoint is a growing

Interoperable UI, Identity, Search, and Data Access Platform.





What to watch out for…Remember this?

“Interoperability is the ability of a system to work with other systems without special effort on the part of the customer.”

without special effort





What is the BCS?


BCS in Office Client

Persistence Store

Data Cubes

SAP ORACLE SiebelSQL

Custom .NET assembly

Identity Federation

serviceBCS in SharePoint

BCS MetaData Store

External data

AS CustomConnector

Client to Backend Direct Connection

BDCClient DataCache

DBConnector

WCF /WSConnector

.NET Assembly

Connector

BDC

SharePoint Client

You could look at this diagram…


External Data

External Content Types

External Lists

SharePoint Workspace w/ InfoPath

Forms

Outlook Forms

and Task Panes

Office Applicat

ions

Search Results

Or this simplified one…


External Data


External Lists


Forms

Outlook Forms

and Task Panes

Office Applicat

ions

Search Results

External Content TypesDescribes:• Structure of the external

system (Fields, Methods, Connection)• How this data behaves within

SharePoint and Office

Added to the BCS Service Through:• SharePoint Designer• Visual Studio 2010• Via an Import into the Business Data

Connectivity Service as part of a model

Manage and Re-Use from a Central Location


External Data


External Lists


Forms

Outlook Forms

and Task Panes

Office Applicat

ions

Search Results

External Lists

The external content type is leveraged in a variety of ways…

Such as for Exposing External Data as an External SharePoint List

Full CRUD capability Familiar UI and NavigationSort, Filter, GroupProgrammatic Access via SPList OMProfile Page for Each ItemFormAuto-generated OOBUpsize to InfoPathCan be Taken “Offline”


External Data


External Lists


Forms

Outlook Forms

and Task Panes

Office Applicat

ions

Search Results

Offline Lists

External Lists also be leveraged as Offline “Lists”

Or Connected to Outlook as Contacts, Tasks, Calendars and Posts

They can be Downloaded to a SharePoint Workspace as Lists or Word Document Libraries with External Data Columns


External Data


External Lists


Forms

Outlook Forms

and Task Panes

Office Applicat

ions

Search Results

Office Applications and Search

w/ Code

External Content Types can also be leveraged in…

Many other Office Applications (or Custom Applications)

Or Leveraged to allow for External Data Searching or Filtering


External Data


External Lists


Forms

Outlook Forms

and Task Panes

Office Applicat

ions

Search Results

Bonus! Integration Webparts

w/ Code

Bonus!Web Parts

External Data List External Data ItemExternal Data Item BuilderExternal Data Related ListExternal Data Connectivity FilterChart Web Part (New)

Integration Webparts


Didn’t those webparts work with the BDC?


So what’s different? (Besides the Chart Control?)

Read/Write CapabilityClaims AwareConnect via ADO/SQL, WCF/WS, .Net Assembly and Custom ConnectorsBatch and Bulk Operation Support

BDC is still used for search indexing connectors in 2010 they added blob, incremental crawl and item level security.

They added symmetrical server and client runtimes. Allowing them to connect from client or server directly to external data.

SQL CE database is used to cache external data which allows for “cached mode” behavior on external data basically creating a rich client cache.


How do I actually build it out?


Intermediate AdvancedSimple

Custom• Custom connectivity for

data aggregation, transformation, security, etc.

• Use Custom Code to Integrate Data into any Office Application

• Business Logic In Forms• Create Reusable

Components (UI parts, ECTs, Actions)

Out-of-box• Surface data in External

Lists

• Connect those lists to Outlook, SPW

• External Data Columns

SharePoint Designer

SharePoint SDK

DeveloperPower User

NO CODE

Advanced Developer

VisualStudio

CODE

Customize• InfoPath Forms

• Workflow

• Web Part Pages

• Outlook Taskpane and Ribbon

Where things fit.

“NO CODE”


1. Discover your Data Source• Database • Web services• .NET type

2. Map Operations to Data Source• Create, read, update, delete, lookup

3. Connect to SharePoint and/or Office • External List• Contact• Task• Appointment• Post

SharePoint Designer Process


Visual Studio Process• Upsize IT Solutions Created in SharePoint Designer• Create Connectivity Packs in Visual Studio to

• Aggregate data across multiple back-end systems• Perform custom data transformations• Execute custom business logic/rules (trigger workflow)

• Bring External Data into other Office clients with VSTO Add-Ins (Excel)

w/ Code


Suggested Life Cycle





The OutcomeThe BCS has a bunch of new capabilities

bundled with some old enhanced BDC ones.





What to watch out for…Selecting the right tool is important.

Practice makes perfect.


That’s a lot of information!





Demo!





The OutcomeYou saw how the BCS can be leveraged in

SharePoint Designer and in the SharePoint UI.





What to watch out for…Don’t assume the tools will

significantly reduce the time it takes to ‘integrate’. It typically reduces

time to ‘interface’.





What are BCSLimitations?


General BCS Limitations

• No Write support for BLOB• You cannot write back to BLOB fields using BCS

unless you write your own method.• You can access BLOB columns by defining a

StreamAccessor method and presenting the external data via the BCS Data List web part by checking the Display stream fields property.

BLOB


External List Limitations• Workflows cannot be associated with external lists

• Workaround: You can use the external data columns and manipulate it that way.

• No RSS feed support for external lists• No REST based access for external lists

• http://sharepoint/_vti_bin/ListData.svc does not support external lists• LINQ to SharePoint spmetal.exe does not support external lists• Cannot configure alerts for external lists• Cannot export external list items to Excel (using the Export to Excel

feature), Create Visio Diagram, Open with Access or Open with Project• Versioning cannot be configured on external lists• Version History is not available on external lists• Datasheet view cannot be used in external lists

• XSLT is Supported (So you can basically make your own.)

External Lists

http://sharepoint/_vti_bin/ListData.svc

http://sharepoint/_vti_bin/ListData.svc


External List Limitations Continued…

• Ratings feature is not supported for external lists• Information policies cannot be configured on external lists• Item-Level permissions are not available for external lists• No item or field level validation (without InfoPath)• Lookups don’t work on any column but ID• No attachments.

External Lists


• You cannot create a site column of type ‘External Data Column’• External data columns can be created only as list based

columns and cannot be consumed in site level content types

Bonus! External Column Limitations

Bonus!


Some Data Types Not Supported

http://msdn.microsoft.com/en-us/library/ff621599.aspx

• No support for structs that are not simple types. (Do not use non-primitive structs.)

• No nested collections (array of arrays etc)• No support for multi dimension arrays• BDC only works with .Net 3.5 - No dynamic programming

in .Net 4• No support for ICollection or IEnumerable interfaces to

represent collections in data structures (ICollection<T>, IEnumerable<T>, IList<T>…)







The OutcomeYou now know more about the

limitations of BCS and will make fewer assumptions.





What to watch out for…Some of the many limitations mentioned make sense. Think

carefully before you build a ‘workaround’.





What about SharePoint 2013?


SharePoint 2013 Main BCS Differences

There are basically four things you need to know:1. oData connector now available through Visual

Studio OOTB (WCF, SQL, .Net Assembly Remain).2. Remote Event Receiver capability – think of this as

more of a simple framework for remote event subscription.

3. App-Scoped – Instead of only Farm level external content types developers can now build isolated app scenarios.

4. More client object model support!





The OutcomeSharePoint 2013 BCS isn’t really different.

Just more flexible and capable.





What to watch out for…SharePoint 2013 introduces many

new concepts/methods that you will need to learn (app-scoped, client

object model focus etc.)





1. Business Data Challenges2. SharePoint’s Important Interoperability3. What is the BCS?

External Content TypesExternal ColumnsExternal ListsExtensibility and Tooling

4. Limitations of BCS

1. The Challenges Aren’t All Technology Challenges2. Interface, Identity, Search and Data Access Platform3. A Set of Powerful Connection Capabilities

Structured Types of External ContentLeveraging External Content with Internal ContentInteract Like SharePoint ListsExtensibility/Tool Range from UI to SPD to VS 2010

4. Good Enough and Better than Most Options

What we talked about…





Questions? Ideas? Feedback? Contact me:Twitter: @RHarbridge Blog: http://www.RHarbridge.comEmail: [email protected]: 700+ SharePoint IA Slides at.. PracticalIntranet.com 130+ SharePoint Standards at.. SPStandards.com15 Pages of Important Questions at.. SharePointDiagnostics.com

Thank You Organizers, Sponsors and You for Making this Possible.

http://www.twitter.com/rharbridge

http://www.rharbridge.com/

http://www.rharbridge.com/

mailto:[email protected]

http://www.practicalintranet.com/

http://www.spstandards.com/

http://www.sharepointdiagnostics.com/

sptechcon boston 2012 - sharepoint integration and interoperability

Technology

sptechcon sptechcon

rharbridge sptechcon

sharepoint interoperability

sharepoint asa search

sharepoint designer

sharepoint providers

search platform bonus

data access platform