scribe online 01 best practices for sol performance
TRANSCRIPT
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved.
Best Practices for Scribe Online Performance
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 4
Session Abstract
In this session we will discuss the 3 “Legs” of our Performance “Stool”…How to optimize your design patterns to leverage Bulk Processing, how to reduce the excess chatter the integration can create over the wire and how to best utilize and optimize lookups in your mapping. When you leave here you should have a greater understanding of how to optimize your jobs going forward.
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 5
Agenda
• Performance Overview
• Scribe Online Technical Benefits for Performance
• Scribe Online Benchmarking data
• Performance Design Tips
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 6
Performance Overview
• Types of Performance
◦ How fast can it move the data
◦ How much data can it move
Scribe Online Technical Benefits for Performance
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 8
Scribe Online – Technical Benefits
• Scribe Online Agents
◦ Run integration as Compiled Code
◦ Leverages Bulk/Batch/Fast Load
◦ Connectors have logic to manage Timeouts/Connection issues
◦ Multi-threaded
Scribe Online Benchmarking data
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 10
Scribe Online Benchmarking Data
• Stipulations – These were done internally under controlled environments
• Data we will review
◦ Dynamics CRM
• OnPrem + OnLine
◦ Salesforce.com
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 11
Scribe Online Benchmarking Data
• Dynamics CRM
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 13
Scribe Online CRM FastLoad
CRM Connector / Scribe Online Core
Dynamics CRM API DLL
DynamicsCRM
In-Memory Array of Data
Processor Threads
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 14
Scribe Online Benchmarking Data
• Salesforce.com
Performance Design Tips
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 16
Update Process
Update
Pre-Seek on Match Criteria (Email)
Pre-Seek finds record, returns ID
Update occurs using returned ID
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 17
A – Update/Insert B – Update OR Insert
What Design Pattern is Better?
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 18
Performance Design Tips
• Don’t take the easy path
◦ Update/Insert vs Lookup/Create/Update
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 19
Performance Design Tips
• Bulk/Batch
◦ When to use which one
◦ How they differ from system to system
◦ API hits always count, Bulk can help
• Perf Killers
◦ Lookups
◦ Repetitive processing
◦ Transactional requirements
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 20
Performance Design Tips
• Stage data locally
• Leverage Database connections versus API
◦ If using Cloud endpoint, can use RS or IS map to bring key data locally
• Key Storage
• Plan for errors
◦ If you know you will get N% of errors, build logic to avoid it. Logging an error takes away from moving data
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 21
Performance Design Tips
• 360 Rows Per Second > 19 Rows Per Second
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 22
Performance Design Tips
• Take the Create or Update route…with Database lookup
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 23
Q+A
CONFIDENTIAL | © 2015 Scribe Software Corporation. All rights reserved. 24
THANK YOU