Where to save my data, for devs!

Benjamin J. Steinhauser

NJ SPS 2014

1

About Me

• Benjamin J. Steinhauser• SharePoint Solution Architect

• B&R Business Solutions

• BS/MS in CS; MCP, MCTS (SP2010 Config. & Dev.)

• 12+ years as ASP.NET Application Developer

• 7+ years as SharePoint Developer

• Creator of Open Source CodePlex Projects:• SPCAMLQueryHelper (over 45,000 downloads)

• SPFileZilla (over 1,600 downloads)

• SPMMDNavigator (over 1,800 downloads)

• SharePoint 2010 and 2013 Custom Claims Authentication with Twitter

• SharePoint 2010 Claims FBA Examples with OpenID

• Author of paid products: SharePoint Metro Grid Web Part, SharePoint Swipe Gallery Web Part

• Presenter at SP User Groups and SharePoint Saturday Events2

Intro

• Where do I store my application’s data?

• Developers have many choices when building solutions that consume data in SharePoint, what are best practices?

• Who: is the consumer of the data?

• What: is the type of data being saved?

• When: is the data needed, and for how long should it persist?

• Why: is the data being stored in the first place!?

• Where: is the data being stored?

3

Consider…

• What are the advantages/disadvantages for each option?

• What are the restrictions?

• What type of data is it?

• Settings, configuration, licensing

• Structured data (employees, favorites, HR forms, etc.)

• How long should the data persist?

• Who is the data consumer?

• End user, Power User, Administrator

• Web Part, Application Page, Timer Job, Web Service, Service Application

• Is the data sensitive? Does it require encryption?

• Is Performance a factor? (always consider caching)4

Where is my data!?

1. Database

2. SharePoint List

3. SP Secure Store

4. Property Bags

5. Web Part Properties

6. Web.config

7. SPPersistedObject

5

1. Databases

• MS SQL Server, Oracle, MySQL• CRUD: create, read, update, delete• For structured data, relational data• Easy (if you know SQL), very fast• Ways to interact:

• LINQ to SQL• LINQ to Entity• Stored Procedures• Inline-SQL in .NET

• But…• Farm solutions only (not for Sandbox Solutions)• Database needs to be created (database administrator)• Permissions need to be configured (database administrator)• Contexts need to be built (when using LINQ)• Another actual “layer” to manage• Connections strings need to be saved (more data!)

• “Put it in a list, duh!”, not so fast, does it need the SP overhead?• Better for custom business applications that store lots of data, specialized data,

data that doesn’t need to be accessible via a List, Searched.

6

2. SharePoint List• ECM: Site Columns, Content Types, Custom Lists

• BCS External Lists too! Combine database, enterprise data with SP

• Provisioning: Create on Feature Activation, packaged in WSP

• Site level (root web), Web level (each web site)

• Accessible to Normal Users, Power Users, and Admins

• Sandbox Solutions supported

• Many ways to interact:• SharePoint Object Model (c#, vb.net)

• LINQ to SharePoint (SPMETAL) (c#, vb.net)

• SPQuery (with CAML)

• SPSiteDataQuery: cross site list queries

• SP Client Object Model: from JS, Silverlight, .NET (no anon access)

• REST (listdata.svc, _api) (no anon access)

• ASMX SOAP Web Services: deprecated

• CodePlex SPServices: uses the soap web services, but yes anon access!

7

Cont’d

• For saving settings, existing implementations:• My Custom Config List

• ***Codeplex: http://spconfigstore.codeplex.com

• SharePoint Enterprise Patterns and Practices

8

Cont’d• Pros:

• User Interface: Standard user interface: New, Display and Edit forms

• Change Management: Support for change management (simple one-step approvals, full workflows)

• Alerts: Users can be alerted when settings are changed

• Versioning: Versioning and the ability to roll back changes to a previous value

• Auditing: Determine who made changes and when they were made

• Security: Security trimming and the ability to set security as granular as a specific list item

• Backup/Restore: Taken care of already as part of your normal content database disaster recovery operations

• Recycle Bin: Ability to take advantage of the Recycle Bin

• Scope: Granularity can be achieved by using multiple settings lists at different levels (ex. in Central Administration for items scoped to a server farm or Web application, in the root Web site of a site collection for items scoped for the site collection, in any site for items scoped to the site itself

• Cons:

• Performance: Implications if settings are used repeatedly in a high volume environment (use cache!)

• Limitation on the types of values: Lists can only store simple values, strings, numbers, etc. They cannot store objects. Even storing serialized objects significantly limits their usefulness because it means that the standard forms are largely useless.

• Settings removal: Potential for orphaned settings if a piece of functionality is removed but the corresponding setting SPListItem does not get properly cleaned up.

• List deletion: Serious problems if the settings list is deleted. (fixes: proper security, hidden lists) 9

DEMO: Config List Reader

• Custom List created in Feature Activation

• Filled with default data

• Custom Class: ConfigListHelper

• CreateConfigList()

• GetAllValsElev()

• GetAllVals()

• Custom application page:

• Reads all config data into Hashtable

• Displays to page

10

BONUS: LINQ to SP

• Enables you to use Language Integrated Query (LINQ) queries• Translates LINQ queries into Collaborative Application Markup Language

(CAML) queries.

• Sandbox Solutions supported• Steps:

• Generate Entity Classes (using SPMETAL.exe)• Write LINQ queries in C# or VB.NET• Run solution

• Pros:• No more CAML! Natural Language syntax, or Expression syntax

• Cons:• Need to build context• SPMETAL.exe by default builds entity classes for ALL lists, content types, etc.

• Can customize this with xml parameters file, see MSDN

• Chicken and Egg problem: must create a list and deploy to run SPMETAL to build the context on it.

11

DEMO: LINQ to SP

• Feature Activation, Event Receiver:

• Create List, “Best SharePoint Blog Sites”

• Fill with sample data

• SPMETAL BAT file, creates Entity context

• SPSVB2014Linq.cs

• Custom Application Page:

• Display ListItems

• Add a ListItem

12

3. Secure Store

• Secure Store Service provides a way to map credentials and delegate access to remote resources.

• The Secure Store Service integrates seamlessly with Business Connectivity Services (BCS).

• Features an API that can be taken advantage of within custom development projects.

• Makes it an attractive option for storing configuration data.

• The Secure Store Service allows us to create Target Applications which house sets of credentials.

• The two main types are Individual and Group applications:

• Individual: credentials are mapped to individual users

• Group: all users share the same set of credentials.

• Required Assemblies:

• Microsoft.BusinessData.dll

• Microsoft.Office.SecureStoreService.dll (GAC)

• Not available from SPCOM (JSOM), not sure about Sandbox Solutions.

• Read permissions required, either explicit or through elevation/impersonation13

DEMO: Configure Secure Store

14

DEMO: Using Secure Store

• Custom Class• SecureStoreHelper.GetCredentialsFromSecureStoreService(applicationId)

• applicationId: string (ex. “SqlServerTest”)

• Custom Application Page

• Get saved settings from Secure Store

• Using “Application Id”

18

4. Property Bags

• Hashtable/Dictionary collections to store simple objects (string, int)

• Can use Serialization to save custom objects, Lists, collections, etc.

• PropertyBags are available here (everywhere!):• SPContentDatabase

• SPFarm: Farm admins only; no Sandbox; WFE read-only(?)

• TimerJob: Each timer job has its own bag; Farm admins only; no Sandbox

• SPServer

• SPWebApplication: Each web app has its own bag; Farm admins only; no Sandbox; WFE read-only(?)

• Site Collection (SPSite.RootWeb): Using root web, web admin to edit; root web read access to read; Sandbox supported

• SPWeb: Web admin to edit; web read access to read; Sandbox supported

• List (SPFolder): via root folder, List owner to edit; list read access to read; Sandbox supported

• SPListItem

• SPFile19

Cont’d

• Pros:• Easy to use (native SP functionality in SOM)• Scope: very granular because they are per-object• Backup/Restore: taken care of as part of your content database recovery

operations• Management: properties are stored and managed with their relevant object

(backing up web site saves its property bag, deleting a web site deletes its property bag)

• Cons:• Availability: certain key objects do not support a property bag (ex. SPUser),

although there are workarounds• No advanced capabilities to review, approve, audit, rollback changes• Difficult to store variations: can be difficult to store variations of a single

setting, per-locale, per-user• Not meant for large data, approx. 4-8 KB and below• No interface to CRUD, build your own, or use a plugin (codeplex), or

SPDesigner, SP Manager (2010, 2013)• http://pbs2010.codeplex.com, http://pbs2013.codeplex.com 20

DEMO: Property Bags

• Custom Class: PropBagHelper.SiteRootWeb

• SafePropBagSave(site, key, value)

• SafePropBagRead(site, key)

• SafePropBagRead(site)

• Custom Application Page:

• Get all root web keys

• Save a key

21

BONUS: Serialization

• Serialization: converting an object into a stream of bytes, to store or send the object

• Its main purpose is to save the state of an object in order to be able to recreate it when needed (hydrating).

• The reverse process is called Deserialization.• We convert bytes to strings, to save in Property Bags, ViewState,

cache, etc.• Popular Formats:

• XML Serialization• SOAP Serialization:

Conform to the SOAP specification. SOAP is a protocol based on XML, designed specifically to transport procedure calls using XML

• JSON Serialization

• Tools/API:• NewtonSoft’s Json.NET• My API 22

DEMO: Serialization

• Custom Classes:

• JsonExtensionMethod

• XmlSerialization

• Custom application page:

• Local class: Person

• On page load, Person created, Serialized to text, displayed in textboxes

• Button to Deserialize, fill Person Objects, display IDs

23

5. Web Part Properties

• The Web Part base class contains a set of properties that users can use to control its appearance and behavior (width, height)

• A custom property is a property that you create when you need additional properties to enhance your Web Part's functionality

• The custom properties you create and specify as able to be browsed are displayed automatically in the default property pane

• If the default property pane isn't rich enough in functionality, you can create a custom user interface called a tool part to manipulate your custom properties

• Available in Sandbox Solutions

• Available in Standard and Visual Web Parts

• Easy to update in custom tool pane Class, tricky to update in regular Web Part page lifecycle events

• SetPersonalizationDirty()

• ***Checkout required, Content Approval, Publishing

24

DEMO: Web Part Properties

• Web Part Feature

• Add to page

• Read web part property value

• Update web part property value

• Show in default properties pane

• Change settings, require checkout.

25

6. Web.config

• Usually the first place ASP.NET Developers look• We quickly find out that it’s a double edged sword• Pros:

• Find file, open, add key/value pair, save … profit!• 1 LOC to access data from app

• Cons:• Farm solutions only (not for Sandbox Solutions)• Have to update every WFE separately• SPWebConfigModification should be used, which may “nuke” your

custom settings added manually, or “nuke” other manually entered settings

• Need server access, admins only

• Either don’t use this, or use this for farm-wide settings that “need” to be secured in web.config

• Better yet, use a nested web.config in your applications layouts folder (saved in HIVE instead of INETPUT) 26

DEMO: Web.config

• Nested Web.config file in app “_layouts/<appname>”

• Custom Class: WebConfigHelper

• Designed to use modes, makes it easy to store different environments together (ex. dev, test, staging, prod)

• GetMode(): get the current mode

• GetKey(): get the key associated with the current mode

• Custom application page:

• Retrieves single property and displays

27

7. SPPersistedObject

• For storing settings at the Farm and Web Application level• “There is no functional difference between the hierarchical object

storage and a property bag that is scoped at the SPFarm level, so deciding which to choose is largely a matter of preference.”

• For storing multiple settings and more complex values• Create a class that inherits SPPersistedObject• Add your settings as properties• Mark them as Persisted (with attribute)• Then use SPFarm.GetChild or SPWebApplication.GetChild• Settings saved in SP Config DB• Cons:

• Requires Farm Admin to save settings• Farm solutions only (not for Sandbox Solutions)• There is no built in user interface to manage SPPersistedObjects• Must impersonate Farm admin to read and write, since access to

Config database required (RunWithElevatedPrivs doesn’t help)28

Bonus:

• Session State: not available by default in SP

• ViewState/Hidden Text Fields: use serialization, page level

• Cache: HTTPRuntime cache, AppPool level, watch out for load balancing, great for caching expensive SPList/DB queries

• Cookies: use jQuery Cookie plugin

• HTML5 Web Storage: great for SPAs

• Window.localStorage, code.sessionStorage

29

Questions?

30

Thanks!

• FREE GIVE AWAY, $100 value!• SharePoint Metro Grid Web Part,

or, SharePoint Swipe Gallery Web Part• For SharePoint 2010, 2013 (on prem. & cloud)• Free download, 30 day trial, email me for free license*

• http://www.amrein.com/apps/page.asp?Q=5827• http://www.amrein.com/apps/page.asp?Q=5853

• Email: bsteinhauser@bandrsolutions.com• Presentation and Code: http://1drv.ms/1o7sZvu

* Product licensed per front-end server, offer is for 1 free license for 1 front-end server.

33

Take a picture of this slide!