bruce perry, salesforce admin/developer
TRANSCRIPT
![Page 1: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/1.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
With special thanks to our Platinum sponsors
Dealing With DuplicatesBruce Perry, Salesforce
Admin/Developer
![Page 2: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/2.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
![Page 3: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/3.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Quiz Time
If you don’t have any duplicates,
A. Maybe you have a user adoption problem?B. Org-wide defaults set to Read-Only?C. You’re the only one with access to the database?D. Or maybe you’re just a really, really good admin?
![Page 4: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/4.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Duplicates are a data quality issue
They contribute to
● Inaccurate reports/estimates● Increased costs (time wasted, duplicated calls and mailings)● Makes your organization seem disorganized to customers/clients● User dissatisfaction
Bottom line: Duplicates can make Salesforce (and you) look bad.
![Page 5: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/5.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Where do duplicates come from?
● Users○ Lack of training (Search before you click new!)○ Bad processes
● Bulk data loads○ Import Wizard will check for duplicates(can load up to 50,000 records)○ Data Loader won’t check for duplicates○ Overlapping data sets○ Inaccuracy/inconsistency in initial data set
● Integrations/External apps that communicate with Salesforce● Attempts by alien civilizations to communicate with us via popular cloud apps
![Page 6: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/6.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Best Practice Alert!
Make a backup before you start - and put it somewhere safe.
Options:
● Back up everything (Salesforce export)● Use a report* to export a more targeted selection of fields/records● 3rd party backup tool (if you happen to have one available)
*If you use a report for this purpose, do not forget to include any relevant IDs (eg. account ID if backing up contacts)
![Page 7: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/7.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Best Practice Alert #2!
We interrupt the previous interruption to inform you that it is a best practice to have a unique identifier* for every batch of data that you upload.
It is A LOT easier to remove a batch of uploaded data if all members share a common key.
Deleting a batch of uploaded data is a very fast way to get rid of duplicates.
*doesn’t need to be fancy, your initials, the date, and maybe a number is enough.
![Page 8: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/8.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Where do we find duplicates?
● Contacts● Leads● Accounts (and sub accounts and household accounts)● Affiliations● Custom objects
![Page 9: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/9.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Steps to a duplicate-free database
1. Train your users2. Turn on duplicate detection/warning/reporting (reporting is a must)3. Develop reports on duplicates4. Make a backup!5. Merge critical duplicates manually6. Develop exact match scenarios.7. Merge duplicates from the top of your object hierarchy down.8. Develop fuzzy match scenarios.9. Merge duplicates from the top of your object hierarchy down.
![Page 10: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/10.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Preventing (or flagging) new duplicates
● Activate a standard matching rule● Or Create and Activate a custom matching rule● Activate standard duplicate rule● Or Create and Activate a custom duplicate rule
These are located under Setup
● Data○ Duplicate management
■ Matching rules
![Page 11: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/11.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Matching Rules and Duplicate Rules
Matching Rules (Standard or custom) tell Salesforce what qualifies as a duplicate.
Duplicate Rules tell Salesforce to handle any duplicates that are found.
Pro Tip - Don’t forget!!!Both types of rules need to be turned on.
If you turn on a matching rule without turning on its duplicate rule, Salesforce will do the comparisons, but take no action. Ask me how I know this.
![Page 12: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/12.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Matching Rules, part 1
Matching rules can get complicated*, You can have:
● Exact matching● Up to 10 conditions● Filter logic (AND/OR)● Fuzzy matching (multiple flavors!)
Before getting into the weeds, ask yourself, “What combination of matching fields in common use gives me a 95% chance of finding a duplicate?”
*Jaro-Winkler/Metaphone 3/Kullback Liebler complicated!
![Page 13: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/13.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
![Page 14: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/14.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Fuzzy Matching
What fields have the biggest potential benefit from fuzzy matching?
● Phone number - (123) 456 7890 is the same as 123-456-7890● First name - Bob/Rob/Robert might be the same person
Also, possibly email, but Salesforce doesn’t have this option. BTW, did you know that
[email protected] = [email protected]
It’s true!
![Page 15: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/15.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Duplicate Rules, Part 1
Duplicate rules are simpler than matching rules. The major decision is to allow or block the duplicate. You can do that both when a record is being created AND when it is being edited. You can also alert the user and flag the duplicate for reporting purposes.
You can choose to enforce sharing rules - or not. If you’re getting duplicates because users can’t see records they are duplicating, consider turning this off.
You also get the option to add filter logic. You could have a duplicate rule that applied to certain account names only or contacts created after a certain date.
![Page 16: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/16.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
New Flash!!!
If you want to do cross-object comparisons, the place you do them is the
Duplicate Rule
![Page 17: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/17.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
![Page 18: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/18.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Turn on Duplicate Management gradually
You can turn duplicate reporting on and nobody will even know you’re doing it.
You can turn on both warn and report. Users can still work as they did previously, but they will be told when they are creating a potential duplicate.
![Page 19: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/19.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
How are duplicates tracked?
Short version: background objects
Why is this useful to know?
Because you can use these objects for reports!
● Salesforce uses Duplicate Record Sets/Duplicate Record Items● DupeBlocker uses DupeBlocker Potential Duplicate● Duplicate Check uses Duplicate Check Group● Ringlead Cleanse does not keep info on matches in Salesforce
![Page 20: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/20.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Report Types for Duplicate Reporting
Generic duplicate reporting - across all object types
● Duplicate Record Set○ Duplicate Record Item
Specific object duplicate reporting:
● <Object Name>○ Duplicate record Item
![Page 21: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/21.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Report tricks for duplicate reporting, Part 1
Use a row level formula* to build a key field composed of multiple fields that you want to compare. You can then sort by the key field and see important values all in one place.
Example - <Full Name>-<City>-<State>-<Zip Code>
Maybe someday we’ll be able to group by row level formulas to make duplicates really stand out here. Not all fields available (eg. email and phone)
No added fields in your objects!
*you may need to enable these in your org first.
![Page 22: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/22.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Report Tricks, Example 1
![Page 23: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/23.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Report tricks for duplicate reporting, Part 2
Group by* columns you don’t usually use for groups (eg. email or cell phone)
The records that end up in groups might be duplicates
*Sort by works too, but Group by makes the matches stand out more
![Page 24: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/24.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Report Tricks, Example 2
![Page 25: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/25.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Report tricks for duplicate reporting, Part 2
Group by* columns you don’t usually use for groups (eg. email or cell phone)
The records that end up in groups might be duplicates
*Sort by works too, but Group by makes the matches stand out more
![Page 26: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/26.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Merging individual duplicates, Part 1
Classic
● Account - Accounts Tab | Tools | Merge Accounts● Contact - Individual Account | Contacts Section | Merge Contacts
In Salesforce Classic, accounts must have similar names to be merged
Lightning
● Account - Potential duplicates shown on account record● Contact - Potential duplicates shown on contact record
![Page 27: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/27.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Merging individual duplicates, Part 2
Lightning Limitation Alert!In Lightning, the Potential Duplicates component is only available for accounts, contacts, and leads. For any other object, you get the warning that you’re about to create a duplicate. After that warning, you have to find the duplicate record yourself.
Potential duplicates are probably based on your current rules. There’s no “Search” feature as in Classic.
![Page 28: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/28.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Merging individual duplicates, Part 3
NPSP
● Account (also Household) - Potential Duplicates pane on account (or switch to classic)
● Contact - App Menu | Merge Contacts
Remember, NPSP uses household accounts and relates contacts to accounts via primary affiliation, not a direct lookup. Households can be split and merged.
Contacts that are not on the same account can be merged (unlike Classic).
![Page 29: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/29.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Merging individual duplicates, Part 4
Merge higher level objects first, then related lower level objects. Merging the higher level objects may mean there are now duplicated lower level objects
For example, first Accounts, then Contacts.
![Page 30: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/30.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Before you merge, know your data
Are two contacts with these characteristics duplicates?
● same address, ● same birthday, ● same home phone, ● same last name, ● very similar first names
![Page 31: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/31.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Not necessarily
Meet the twins!
Frank and Frankie Smith**real world example, last name changed
Do you have a data guru? Find one before you start to de-duplicate.
![Page 32: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/32.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
![Page 33: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/33.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Quiz Time!
Are you done with duplicates after merging two accounts and any resulting contact duplicates?
![Page 34: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/34.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Not necessarily
You might be using NPSP and might have duplicated affiliations
You might have custom objects related to the contact. These can become duplicates after merging contacts.
![Page 35: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/35.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Before Account and Contact Merge
Account
Contact
Affiliation
After
Account
Contact
Affiliation
Account
Contact
AffiliationAffiliation
![Page 36: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/36.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Finding and handling existing duplicates
● Reports on flagged duplicates (report type)● Basic reports● Salesforce Merge tools - Salesforce account and
contact merges, NPSP merges● Third party apps (next slide)
![Page 37: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/37.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Spotting/Fixing Duplicates in Bulk
Salesforce duplicate jobs (Performance and unlimited Editions only)
Third party tools:
● Duplicate Check● Cloudingo● Dupe Blocker● Demand Tools● Ringlead Cleanse● CleanCRM● ZaapIT DeDupe Manager
![Page 38: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/38.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Budget Alert!
It has come to my attention that some of these tools cost money. If getting money for these kinds of add-in tools is an issue for you, keep track of how long it takes to correct bulk duplicates by:
● Exporting data● Matching the duplicates in Excel● Deciding which fields to keep● Uploading updated data to the correct record● Deleting the duplicate record
The cost of these tools should be easy to justify in terms of your time saved (for cool new projects!)
![Page 39: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/39.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Spotting/Fixing Duplicates in Bulk
Don’t forget the backup!
![Page 40: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/40.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Required Features for Bulk Duplicate Handling
● Can save your search/scenario/job for re-use● Fuzzy matching of at least first name and phone● Flags duplicates for later merging● Auto-merging (use carefully!)● User-defined rule based picking of winners/losers● oldest/newest wins rules
Custom object matching/merging and cross object matching are also useful, but you might not need them
![Page 41: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/41.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Process for bulk merging
Do a first pass using exact matching followed by one or more fuzzy passes. Merge parent records first, child objects second (eg. merge accounts, then contacts). Save your searches/jobs/scenarios (you’ll need them again)
● Parent - exact pass 1● Child - exact pass 1● Parent - fuzzy pass 2● Child - fuzzy pass 2
Note: you may well have more contact duplicates after you merge accounts.
Why? Duplicate checking looks for duplicate contacts within accounts
![Page 42: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/42.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Tools for duplicate management - details
Switch to Google sheet here – URL for sheet is in resources
![Page 43: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/43.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Resources – Page 1Link to “Duplicate Tools” Google sheet:
https://bit.ly/30zTfcd
Manage Duplicates in Salesforce (Summer 2019 version) - 44 pages full of all the details on duplicate management including exactly how all the different flavors of fuzzy compares work.
http://resources.docs.salesforce.com/220/13/en-us/sfdc/pdf/salesforce_data_quality_duplicate_prevention.pdf
Youtube playlist – matching rules, duplicate rules, and duplicate reporting (Jay Delaune, Force501.org)
https://www.youtube.com/playlist?list=PLVkg_BRPOj3z_oCzzGi39JgO6SVQGP90C
![Page 44: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/44.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
Resources – Page 2
NPSP merge/split households
https://powerofus.force.com/s/article/NPSP-Merge-or-Split-Households
Include Classic-style merging in Lightning (untested, may be unsuitable for NPSP)
https://forcepanda.wordpress.com/2019/07/30/merge-accounts-and-contact-in-lightning-like-classic/
![Page 45: Bruce Perry, Salesforce Admin/Developer](https://reader036.vdocuments.net/reader036/viewer/2022062303/62a5c6de412e260d4e215b85/html5/thumbnails/45.jpg)
Manchester, NH 2019@NEDreamin #NEDreamin
With special thanks to our Platinum sponsors!
Q & A