crmug uk november 2015 - data migration without tears by mike feingold

explore

engage

elevate

Data Migration Without Tears

Mike FeingoldEmpoint Ltd

Tuesday 10th November 2015

explore engage elevate

About MeMike FeingoldDynamics CRM ConsultantFocussed on data migration and integration with CRMScribe MVPBeen working with Dynamics CRM since v1.2

E: [email protected] W: www.empoint.co.ukL: http://uk.linkedin.com/in/mikefeingold

mailto:[email protected]

http://www.empoint.co.uk/

http://uk.linkedin.com/in/mikefeingold


Scenario“We're decommissioning our old SQL based CRM application and implementing Microsoft Dynamics CRM 2015.

My boss/client/manager asked me to migrate all the data from the old to the new system.

What shall I do?”


Get involved early in the projectData migration needs to be thought about during the project development stage. Don’t leave it until the end.Questions to ask:

Will there be parallel running of the old and new systems or a “big-bang” cutover?If a cutover, can this be done quickly(eg, overnight or over a weekend)?If parallel running, can the old system be made read only?Do we need ongoing integration from the old to the new system?


Tool choiceCan’t write to CRM database – must use the APICustom coded applicationTools

ScribeSSISKingswayThe are others


ScribeScribe Insight – Windows applicationScribe Online – Cloud basedHas an adapter for Dynamics CRM, and templates for getting data from other CRM systems (eg, Act, Goldmine)No coding experience requiredPrices:

Up to 15 CRM users: $300015 - 100 CRM users: $5500101 - 250 CRM users $800060d migration $2000


ScribeDemo


Scribe


Kingswaysoft - SSIS Integration Toolkit for Microsoft Dynamics CRM

SQL Server Integration Services (SSIS)Part of SQL ServerNo extra charge

KingswaysoftAdapter for CRM (online, on premise, hosted)No coding experience requiredCan be used for migration and/or ongoing integration

Price $695 for single server, one year; $1495 for perpetual licence


KingswaysoftDemo


Kingswaysoft


SSIS – Custom script componentNo 3rd party add on requiredRequires coding experience (ideally C#), knowledge of .Net and the CRM APIMaximum flexibility – everything exposed by the API is availableHighest development effort


SSIS – Custom script componentDemo


SSIS Script Component


Aspect Winner Why

Speed None Any speed bottlenecks are usually to do with the CRM web service, not the migration tool being used.

Scalability SSIS Has business data distributor component to allow multi threaded processes

Deployment SSIS Can specify all deployment parameters (eg, database connections) on

the command line. With Scribe it’s necessary to edit the DTS.‘Other’ CRM messages

SSISC# has complete access to the CRM API so can call messages that Scribe can’t (Eg: change the business unit of a user, change user manager).

ODBC SSIS Scribe needs ODBC connections (eg, to connect to files). These needs to be included in any deployment. SSIS allows a direct file connection.

Multi row operations SSIS Eg: Source of phone numbers needs to create multiple records in target

for each row in source. SSIS supports joins to source.

Debugging Scribe Row based (rather than SSIS which is set based) so easier to see what’s going on.

Logging Scribe Scribe has built in logging. SSIS need to implement manually and not as elegant.


Net change patternsWhat has changed in the source data?

DeltaVarious methods

Publish changes to a queue or table using a triggerSet a RecordModified flagUse LastModifiedDate field


Design strategyJobs must be able to be rerun without duplicating dataIf source if CRM on premise then read from filtered views in database (not API)Do any data manipulation outside of migration tool.Use a staging database (especially if source data is file format eg, CSV)If records in the source have a primary key then store this in CRM. Index this field.If source is CRM then migrate the record primary key GUID to the target. Error handling – log, don’t fail


Watch out forActivity migration can be complex

For some entity types (eg, Email), the state of the record determines if it can be updated

When migrating date time fields, watch out for BST and GMT. Users say that some dates are 1 day out and the underlying data is 1 hour out.CRM restrictions

Can’t delete usersCan’t assign roles to inactive usersCan assign records to users who don't have permissions to see them


Other considerationsMigration sequence is importantIs all data being migrated, or just active records?Users who have left the companyReconcile source and target data (esp, row counts).


Plugins, workflows and JavascriptUnderstand what these do and any impact on migrationCRM API doesn’t respect mandatory fieldsUse JavaScript for entry validation onlySynchronous plugins impact speed

Can they be disabled during data load?‘Maintenance mode’

Disable plugins and workflowsDisable email routerChange the bannerConsider user roles or read-only statusRun jobs (eg, aggregation) when load completed


Directly updating the CRM database


Directly updating the CRM databaseFields

ModifiedOn, ModifiedByCreatedBy (can use overriddencreatedon)Merged/IsMasterExchangeRateBase currency values

On premise only


SpeedDynamics CRM is a beautiful productBut it isn’t fastHow to increase migration speed

Scribe – partition the source data and run multiple instancesSSIS – use the Balanced Data Distributor

Increase CRM performanceDisable pluginsTurn off auditUse indexes


Scenario“We're decommissioning our old SQL based CRM application and implementing Microsoft Dynamics CRM 2015.

My boss/client/manager asked me to migrate all the data from the old to the new system.

What do I do?”


About MeMike FeingoldDynamics CRM Consultant

E: [email protected] W: www.empoint.co.ukL: http://uk.linkedin.com/in/mikefeingold

mailto:[email protected]

http://www.empoint.co.uk/

http://uk.linkedin.com/in/mikefeingold

crmug uk november 2015 - data migration without tears by mike feingold

Software