Education CloudIntegration with CommonApp

@salesforcedocs

Last updated: July 22, 2021

Salesforce.org and its logo are registered trademarks of Salesforce.com, inc. Other names used herein are

trademarks of Salesforce.com or their respective owners.

Contents

Contents

One Approach to Common App Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1What's Our Approach?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1How Does Common App Export Data?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2How Do We Stage Data?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Data Staging Record. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Content Document Record. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Content Document Link Record. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

How Do We Move Data to EDA?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Get Ready. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Salesforce Prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Create a Destination for the Common App ID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Configure EDA Settings and Create EDA Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Configure Name Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Install and Configure the Education Cloud Integration with Common AppPackage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Assign the Integration Permission Set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Update the Custom Metadata Type with Your Common App ID Field. . . . . . . . . . 12

Configure Your ETL Tool—Or Download the Talend Jobs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Configure Your Common App Export. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Integration Workflow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Step 1. Export Your Common App Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Step 2. Stage the Data in Salesforce with Your ETL Tool of Choice. . . . . . . . . . . . . . . . . . . . 15Step 3. Run a Batch Job to Map and Move the Data to EDA. . . . . . . . . . . . . . . . . . . . . . . . . . 16Step 4. Run a Batch Job to Delete Staging Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Troubleshoot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19EDA Mapping Details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19EDA Mapping Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

General Mapping and Record Creation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Duplicates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Personal Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Education Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Using the Applicant Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

One Approach to Common App Integration What's Our Approach?

One Approach to Common AppIntegration

If you’re a Common App member school looking to import applicant data into EducationData Architecture (EDA), you have lots of integration options. This documentation suggestsone possible approach that relies on an ETL (Extract, Transform, and Load) tool and aSalesforce batch job to move data into EDA. The tools and assets in the Education CloudIntegration with Common App package help expedite your integrations.

What's Our Approach?Our sample approach assumes that you: 

• Need to import data from Common App into EDA. 

• Need to import only current applicant data—not archived data.

Our approach starts with Common App data already exported to your local hard drive as .csvand .pdf files. You run an ETL tool against the .csv and .pdf files to create temporary stagingrecords in Salesforce. Then run a batch job to move the staging records into EDA records.After the move, the staging records are deleted. 

We’re providing Talend jobs as an example so that you can understand how an ETL toolworks with the package, but we expect that you might choose an entirely different ETL tool.As you implement your own process, don't be shy about sharing your discoveries with theEducation Data Architecture group on the Power of Us Hub community.

ImportantThe Talend jobs are provided as samples only. They provide an example of howan ETL tool can import Common App data and stage it in Salesforce. These jobsare not an official solution, are not endorsed by Salesforce, and are not officiallysupported.

1

One Approach to Common App Integration How Does Common App Export Data?

How Does Common App Export Data?Common App stores applicant responses in a .csv file, with each row representing anapplicant. Your application questions are saved to the first row, the column header.Applicants' answers to those questions are saved to cell values in a row.

Depending on what you export, your .csv file contains data for one of these export types.

Suspect / Prospect Data for suspects and prospects. Suspects add your institution to theirMy Colleges list. Prospects select a Start Term and Admissions Plan onyour Member screen.

Applicant & Writing Supplement Data for applicants who have submitted their application.

Recommendations Data from recommenders who submit information about an applicant,like a counselor or teacher.

Transfer Data for applicants who are transfer students.

Common App exports all applicant documents as .pdf files. Typically, an applicant’sCommon App ID associates each .pdf file with an applicant. Based on your settings inCommon App, .zip files are created for each PDF export job. Each day the .csv and .zip filesare made available on Common App’s SFTP server.

When you export your Common App data, use a Salesforce template. We don’t provideany PDF export templates. So make sure that .pdf files are named using the conventionsrequired by our data staging records. For more information about the required namingconventions, see Configure Your Common App Export.

For information about exporting from Common App, see Scheduled Delivery Service (SDS)Overview in the Common App Member Solutions Center.

How Do We Stage Data?You can use any ETL tool to create staging records in Salesforce. Our sample approach usesTalend. Even if you use a different tool, such as Informatica or Jitterbit, it helps to know howthe batch job in the package expects the staging records to be structured and formatted. 

Each row in a .csv file maps to a series of records. And each .pdf file also gets a series ofrecords. Your ETL tool must create these types of records.

• Data Staging contains information about the data

• Content Document stores .json or .pdf files

• Content Document Link relates Content Document records to the Data Staging record

2

One Approach to Common App Integration How Do We Stage Data?

NoteIf you don't use the sample Talend job, create these records in your ETL tool ofchoice by leveraging Salesforce APIs, such as the Bulk API, REST API, or SOAP API. 

Data Staging RecordThis record contains information about the data, but not the actual applicant responses.Think of it as the title page in a book. The Data Staging custom object is provided with thepackage. 

Each row in the .csv file must have a corresponding Data Staging record—it’s a 1:1 match.PDFs can have a 1:1 match as well. However, if there are multiple .pdf files in the sameexport for a single applicant, they can be linked to a single Data Staging record—a 1:manymatch. 

Each Data Staging record must contain these required fields.

3

One Approach to Common App Integration How Do We Stage Data?

Field Label Field API Name Field Type Required Data Sample

Batch ID BatchID__c Text Groups staging records createdby the batch. This field is notused by the batch job.

To help you track import issues,we recommend using some typeof standard formatting for thisfield, such as the data sourceplus the time stamp.

CA-1599577090189

Data StagingName

Name Text The Common App ID from eachcorresponding row in the .csvfile. The Common App ID isthe unique identifier for thatapplicant.

123456789

Data Sub Type DataSubType__c Picklist Indicates the type of contentattached to the Data Stagingrecord. For data from the .csv file(stored in your .json file), set thisvalue to Data. For .pdf files, setthis to File.

Data

Data Type DataType__c Picklist The type of data this recordcontains. Select: 

• First Year

• Recommender

• Transfer

First Year

Source Source__c Picklist The source of the data beingimported. Set this value toCommon App.

Common App

Status Status__c PIcklist Set this value to New. New

NoteFor a single applicant, you’ll have multiple Data Staging records with the same DataStaging Name.

Content Document RecordThis record stores a .json file or .pdf file. The .json file contains all the applicant’s answers tothe application in a flat JSON object. The .pdf file is the attachment the applicant uploadedto Common App, like a personal essay or a transcript. You must create separate ContentDocument records for the .json and .pdf files. Don’t store both of these files in a singleContent Document record. 

4

One Approach to Common App Integration How Do We Stage Data?

In the Salesforce app, Content Document records are Files. Salesforce automaticallycreates a Content Document record from the Content Version record. To create a ContentDocument record for a .json or .pdf file, create a Content Version record without setting theContentDocumentId field. You might need to specify the VersionData field when creating theContent Version record depending on your ETL tool. See Content Document and ContentVersion in the SOAP API Developer Guide for API requirements. Make sure to read thedetails regarding special characters and Base64 encoding. 

Also consider the daily limit on the number of files that can be added. See Salesforce CRMContent Limits and Increase the Maximum number of 'Content Versions' published per dayin Salesforce Help.

NoteTo automatically create Content Document Link records at the same time asContent Version records, specify the FirstPublishedLocationId field on theContent Version record.

JSON FilesParse and reformat all the data from the .csv row into a flat JSON object. Use each columnheader from the .csv file as the name of the JSON object property or key, and eachcorresponding row cell value as the value for the property. This pairing means that anapplication with 1,400 questions generates a .csv file containing 1,400 columns, and aJSON object containing 1,400 properties. 

Save the flat JSON object into a .json file. Each applicant has a single .json file. To uploadthe .json file to a Content Document record, create a new Content Version record. 

Let’s look more closely at creating a JSON object from .csv data. If we imagine an applicationwith only 5 questions, your .csv file might look like this.

Common

App ID

First Name Last Name Phone Number Email Address Start Year

123456789 Marla Gianni 222-555-7890 marla@example.com 2021

123456790 William Hsu 2021

456789123 Mostafa Fakour 444-555-1505 mostafa@example.com 2021

Here’s the first data row formatted as a JSON object.

{ "Common App ID" : "123456789", "First Name" : "Marla", "Last Name" : "Gianni", "Phone Number" : "222-555-7890",

5

One Approach to Common App Integration How Do We Stage Data?

"Email Address" : "marla@example.com", "Start Year" : "2021"}

NoteThe sample above is a “prettified” version of the JSON. The JSON object doesn'tneed indentation or new lines. The JSON object may be a single line, such as:

{"Common App ID":"123456789","First Name":"Marla","Last Name":"Gianni","Phone Number":"222-555-7890","Email Address":"marla@example.com","Start Year":"2021"}

JSON Do's and Don'ts• Use valid syntax. Surround keys and values with double quotes.

• All values, except for null values, must be strings. Our batch job converts the values totheir appropriate data types when converting the JSON data into EDA records and fields.

• If a cell in the .csv file is blank, such as the data for William Hsu in our sample .csv file,either eliminate that key-value pair from the JSON object for that row or include the key-value pair with null as the value, not surrounded by quotes. Here's an example:

Null Cells Removed From JSON Null Cells Included in JSON

{ "Common App ID" : "123456790", "First Name" : "William", "Last Name" : "Hsu", "Start Year" : "2021"}

{ "Common App ID" : "123456790", "First Name" : "William", "Last Name" : "Hsu", "Phone Number" : null, "Email Address" : null, "Start Year" : "2021"}

PDF FilesTo upload the .pdf file to a Content Document record, create a new Content Version record.If the .pdf files are in the same daily export and belong to the same applicant, a single DataStaging record can be shared between multiple .pdf files.

When creating your Data Staging record, the Data Staging Name field must be the CommonApp ID of the applicant who the PDF will be related to. 

File names of the .pdf files must follow this pattern:

CAID_FORM_AdditionalDetails.pdf

Where:

6

One Approach to Common App Integration How Do We Stage Data?

• CAID is the Common App ID of the applicant who the PDF will be related to. Fortransfers, use  StudentID (which pulls in the applicant’s Common App ID) instead of thedefault Transfer App ID.

• FORM is the two-letter abbreviation defined by Common App for the type of PDF(determined by the Form Type dropdown in Common App). For example, SR is for aSchool Report submitted by a counselor.

• AdditionalDetails is any other information you want to include, such as the date andterm.

For example: 12345_SR_08-29-2020_Spring21.pdf

Content Document Link RecordThis record relates a Content Document record to a Data Staging record. For everyContent Document record, there must be a matching Content Document Link. Forexample, if an applicant named Marla submits her application and provides a PDF letter ofrecommendation and a PDF transcript, these staging records are required: two Data Stagingrecords, three Content Document records, and three Content Document Link records. 

Keep in mind that each applicant is unique and their application requirements may bedifferent.

Each Content Document Link record requires a value for the LinkedEntityId field and theContentDocumentId field. Use the ID of the Data Staging record for the LinkedEntityId value.Use the ID of the Content Document for the ContentDocumentId value. To create ContentDocument Links automatically, specify the FirstPublishedLocationId field when creatingyour Content Version records.

7

One Approach to Common App Integration How Do We Move Data to EDA?

How Do We Move Data to EDA?The batch job in the package moves applicant data from staging records into EDA records.More specifically, the data in the flat JSON object stored in the .json file on the ContentDocument record moves to various EDA records based on a default mapping that ispreconfigured and can't be modified. New Content Document Link records are createdto attach the .pdf files stored on the Content Document records to the new Contact andApplication records. 

Now let’s revisit the JSON object from our previous example.  

{"Common App ID":"123456789","First Name":"Marla","Last Name":"Gianni","Phone Number":"222-555-7890","Email Address":"marla@example.com","Start Year":"2021"}

This table shows how the sample data would be mapped.

8

One Approach to Common App Integration How Do We Move Data to EDA?

Common App Question EDA Object EDA Field

Common App ID Contact Your custom text field, such asCommon_App_ID__c.

First Name Contact FirstName

Last Name Contact LastName

Phone Number Contact MobilePhone

Email Address Contact Email

Start Year Application hed__Application_JSON__c

For Common App questions that map to existing fields in standard Salesforce or EDAobjects, we've mapped it for you. We map to these objects.

• Address

• Application

• Contact

• Contact Language

• Education History

• Relationship

For Common App questions that don’t map to existing fields or if the question is moreinformational and likely not needed during the student’s life cycle, we store the data in aJSON object in one of these fields.

• Application JSON (hed__Application_JSON__c) on the Application object

• Contact JSON (hed__Contact_JSON__c) on the Contact object

• Education History JSON (hed__Education_History_JSON__c) on the Education Historyobject

For more mapping details, see EDA Mapping Details and EDA Mapping Considerations. 

If you’re using Admissions Connect, you can display data stored in the JSON objects directlyin components that your admissions staff use to verify and review applications. You caneven set up automation so that incoming PDF files are automatically associated withDocument Checklist Items. That way, your admissions staff can verify and review applicantdocuments alongside the JSON data. For more information, see Work with Applicant Dataand Documents Imported from Common App in the Admissions Connect documentation.

9

Get Ready Salesforce Prerequisites

Get Ready

To use our approach, get your Salesforce org, local environment, and Common App exporttemplates ready.

Salesforce Prerequisites

Create a Destination for the Common App IDEach applicant who uses Common App is assigned a Common App ID. This ID associatestheir data and .pdf files with their application. To import Common App data into Salesforce,create a custom field on the Contact object to save the Common App ID to. Use the Textdata type and make the field Unique. If you already have an existing custom field that youcan use for this, you don’t need to create a new one.

NoteRemember that for transfers, Common App ID is the StudentID field.

Configure EDA Settings and Create EDA RecordsMake sure your EDA settings are configured to support batch error logging. You also needto create the appropriate EDA records so that data can be moved and associated with otherrecords.

• The package's batch error logging respects EDA's error settings.

For example, if you disable Error Handling in EDA, no errors are logged when batchesto move data encounter errors. Likewise, if you disable Error Notifications, users don'treceive emails when the batch job encounters errors or successfully moves data.

We recommend that you enable and configure these EDA Settings:

Error HandlingEnsures batch job errors are logged.

Store ErrorsEnsures Error records for the batch job are created.

Error NotificationsDetermines whether notifications are sent when a batch succeeds or fails.

Error Notification Recipients

10

Get Ready Install and Configure the Education Cloud Integration with C...

Depending on what you configure, Chatter posts or emails are sent with batchresults.

For more information about EDA's error settings, see Configure Error Handling in EDA inthe EDA documentation.

• Ensure that each institution's CEEB code, including your own, is stored in the School Codefield (hed__School_Code__c) on the institution's Account record. The batch job uses theSchool Code field to find the Account record ID when creating Education History recordsfor applicants.

• Ensure that Language records exist. The batch job uses Language records to create anapplicant's Contact Language record. 

• Ensure that EDA's ADDR_Contact_TDTM trigger, which creates the default Address recordfor applicants, is active.

Configure Name SettingsFrom Setup, on the User Interface page, select Enable Middle Names for Person Namesand Enable Name Suffixes for Person Names. 

Install and Configure the Education CloudIntegration with Common App PackageThis package is not a point-and-click solution, it’s a set of tools and assets for your institutionto use during the integration process. The package delivers these pieces.

• Custom metadata type

• Custom Data Staging object 

• Permission set (provided in the package as unmanaged) 

• Batch jobs  

• Tab for the Data Staging object and an Errors list view 

• Automated emails or Chatter posts that are triggered when the batch job finishes  

• Error handling and logging that respects EDA's error settings 

The package doesn't provide an app. If you want, create an EDU Integration app and addthe Data Staging object and EDA Errors tab to it. 

After you've completed the Salesforce prerequisites, you're ready to install the package.

11

Get Ready Install and Configure the Education Cloud Integration with C...

ImportantCurrently, the Education Cloud Integration with Common App package is onlyavailable as a limited release. If you’re part of the limited release, complete thefollowing install steps.

1. Visit the limited release installer link provided by your Account Executive.

2. Log in to the correct Salesforce org.

3. Wait for the installer to retrieve information about your org.

4. Click Install. The installation may take a while. Check your email for confirmation thatthe installation was successful.

After you install the package, complete these configuration steps.

Assign the Integration Permission SetTo ensure your integration user can run the batch jobs, assign them the EducationIntegrations permission set in Setup.

We also recommend that you give your integration users the Edit Read Only Fields systempermission. Without this permission, users running the batch jobs can encounter errors.Update the Education Integrations permission set with this access or assign it separately.

Update the Custom Metadata Type with Your Common AppID Field1. In Setup, search for and select Custom Metadata Type. 

2. Click Manage Records for the Education Integration Settings record.

3. Click New. 

4. Fill out these fields.

• Label: Enter Default Integration Settings.

• EDU Integrations Setting Name: Enter Default Integration Settings.

• Contact Common App ID Field: Enter the API name of  the custom field you createdon the Contact object.

12

Get Ready Configure Your ETL Tool—Or Download the Talend Jobs

Configure Your ETL Tool—Or Download the TalendJobsFor our sample approach, we chose to use Talend jobs. You can use these jobs as is, or youcan customize them. If using a different ETL tool, we recommend using the Talend job asreference when creating your own process and ensuring that the data meets our stagingrequirements. 

To download and configure the Talend jobs, see the SalesforceFoundation/common-app-talend repo on GitHub.

ImportantThe Talend jobs are provided as samples only. They provide an example of howan ETL tool can be used to import Common App data and stage it in Salesforce.These jobs are not an official solution, are not endorsed by Salesforce, and are notofficially supported.

Configure Your Common App ExportSalesforce provides data export templates to ensure that exported data is formattedcorrectly for our batch job. Look for these templates in the Common App site whenexporting data.

• Salesforce EDU Recommendation

• Salesforce EDU Applicant or Writing Supplement

• Salesforce EDU Suspect or Prospect

• Salesforce EDU Transfer

If your institution has custom application questions, customize our templates before youexport. Make a copy of our templates, then in the Member section, specify your institution’scustom questions. For detailed steps, see How do I add school-specific questions to anexport template? in the Common App Member Solutions Center.

ImportantWhen customizing the Salesforce templates, don’t edit the header names. Werenamed some of the Common App default header names to work with our batchjob. If your column headers are different from our Salesforce templates, your data

13

Get Ready Configure Your Common App Export

is treated as custom questions and the data is stored in the Contact JSON fieldonly.

Salesforce doesn't provide PDF export templates. When exporting documentsfrom Common App, ensure .pdf files are named using the following convention:CAID_FORM_AdditionalDetails.pdf

For full details about our required naming conventions, see PDF Files. For first-yeardocuments, work with Common App to implement this naming convention. For transferdocuments, you can configure the naming convention yourself. Customize your PDF exporttemplate to use StudentID (which pulls in the applicant’s Common App ID) instead of thedefault Transfer App ID.

14

Integration Workflow Step 1. Export Your Common App Data

Integration Workflow

Step 1. Export Your Common App DataWhen exporting Common App data, select one of the Salesforce templates (or yourcustomized version of it). For first-year exports, select Include data element names in firstrow of file (only applicable to Delimited type of Export). 

ImportantIf you use a different template, keep in mind that the Talend job and batch jobwon't run as expected. 

When you export applicant data from Common App you might download a .txt file. Renamethis file to .csv. To ensure the data isn’t accidentally reformatted, don’t open the file.

Remember that .pdf files must follow the naming conventions required by our data stagingrecords. For details about the naming convention, see PDF Files.

Step 2. Stage the Data in Salesforce with Your ETLTool of ChoiceRun the Talend jobs, or your own ETL tool, to stage the data in Salesforce. Make sure that thedata meets our staging requirements. 

If you’re using our Talend jobs, see Sample Talend Jobs for the Education Cloud Integrationwith Common App Package for detailed steps on how to configure and run the jobs.

TipAs you fine-tune your own Common App ETL process, share your experience inthe  Education Data Architecture group on the Power of Us Hub community. Thecommunity is a great place to show off your awesome ideas and lessons learned.

15

Integration Workflow Step 3. Run a Batch Job to Map and Move the Data to EDA

Step 3. Run a Batch Job to Map and Move theData to EDAThe batch job provided in the package maps and moves staging records into EDA records.The batch job: 

• Finds Data Staging records with a status of New and converts the JSON object, stored inthe Content Document, to a structured Apex object with nested objects inside it. 

• Converts Common App picklist values into EDA picklist values. 

• Converts the Apex object using the preconfigured default mapping, which identifies thedata's destination and creates various EDA records for the data.

• Updates the Description field on the Content Version records with the full name of theCommon App form type (based on the abbreviation included in the filename of theattached .pdf file). 

• Creates new Content Document Link records to relate the Content Document recordsfor .pdf files to the Contact and Application records.

• Sends notifications via email or Chatter when the job finishes based on your error settingsin EDA.

The batch job creates all the EDA records for a single applicant at the same time. Wedon’t create all the Contact records, then all the Application records, and so on. After anapplicant’s records are created, you can immediately start working with them.

Run the batch script manually each time you stage more applicant data from Common App,or schedule it to run nightly. Frequency and timing are totally up to you. 

To run the batch script that kicks off the Salesforce batch job:

1. Open the Developer Console. 

2. Click Debug | Open Execute Anonymous Window to open the Enter Apex Codewindow.

3. Enter the command:

Database.executeBatch(new eduint.CommonAppImportBatchable(), 1);

4. Click Execute.

If you plan to automate how often the batch script runs, make sure to consider how yourETL tool fits in the overall workflow. If your ETL tool goes down, there won’t be stagingrecords for the batch script to process. Also consider how errors might impact your

16

Integration Workflow Step 4. Run a Batch Job to Delete Staging Records

automation. For example, if your Common App .csv file is incorrectly formatted, the batchscript might not be able to process the staging records.

If you’re processing staging records for 20,000 applicants, it can take anywhere from 8-10hours to finish creating all the EDA records. That’s about 1,000 applicants every 30 minutes.This estimate varies greatly depending on the number of attachments (per applicant andrecommender), the traffic of your org’s instance, and how much data is already in your org. 

When planning your overall scheduling, make sure to consider the time it takes your ETL toolto create the staging records and whether or not your org has other triggers, flows, or otherprocesses running. You might decide to schedule the batch jobs to run overnight if that’sconvenient. Do what works best for your institution.

Step 4. Run a Batch Job to Delete Staging RecordsAfter the data processing batch job completes and you've confirmed that the EDA recordswere created, run a batch job to delete the Data Staging records. The batch job:

• Deletes Data Staging records with a status of Processed. 

• Deletes related Content Document records that include JSON data. 

• Deletes related Content Document Link records that associated the JSON ContentDocument records and the PDF Content Document records with Data Staging records. 

To run the batch script that kicks off the Salesforce batch job:

1. Open the Developer Console. 

2. Click Debug | Open Execute Anonymous Window to open the Enter Apex Codewindow.

3. Enter the command:

Database.executeBatch(new eduint.DataStagingCleanUpBatchable(), 200);

4. Click Execute.

17

Troubleshoot Step 4. Run a Batch Job to Delete Staging Records

Troubleshoot

If an error occurs while moving the staging data into EDA records, no records are updatedor created for an applicant. In addition, the status of the applicant’s Data Staging recordupdates to Error, and the Error Message field updates with a message that tells users to lookat the Error tab for more details. To view errors quickly, use the Errors list view for the DataStaging object. The Chatter post or email we send when the batch job finishes includes errordetails, too. 

NoteEmails and Chatter posts are sent only if the following settings are turned onin EDA Error Settings: Error Handling, Error Notification, and Error NotificationRecipients.

In EDA's Error tab, records include full details of the error, a link to the Data Staging record(in the Record URL field), and if available, the error stack trace. Use this information to helpyou troubleshoot the issue. We recommend that you include the following fields on theError page layout: Context Type, Email Sent, Error Type, Datetime, Full Message, Record URL,and optionally, Posted in Chatter. For detailed steps on how to create a custom tab for EDA'sError object, see Configure Error Handling in EDA. 

After the issue is resolved, set the Data Staging record status back to New. That way, thestaging record gets picked up the next time you run the batch job. 

Consider routinely deleting the EDA Error records. The package doesn't provide anyautomated clean up for resolved errors.

18

Reference EDA Mapping Details

Reference

EDA Mapping DetailsFor full details on how we map Common App questions to EDA fields, including our picklistmappings, review the Education Cloud Integration with Common App - EDA MappingDetails document in Quip. 

We provide mapping details for standard Common App questions. We map customquestions and all other questions not identified in our mapping document to the ContactJSON field. So if you don't see mapping details for a specific question, look there.

Keep in mind that questions about the same topic might be stored in different locations.For example, the question “Date of birth” maps to the Birthdate field on the Contact object,while the Common App question “Birth city” is stored in the Contact JSON field.

EDA Mapping ConsiderationsReview the following considerations to understand how our batch job maps and moves datato EDA.

General Mapping and Record CreationCustom questions are mapped to the Contact JSON field. 

We map to some EDA and Salesforce fields for multiple Common App questions. Here’swhat we do.

• If the field is a long text field, we concatenate the values into a single string, separatedby semicolons. For example, the Common App questions Honorable Dischargeand Military Discharge Explanation both map to the Military Background field(hed__Military_Background__c). The value for this field might look like: Military Branch: Army; Honorable Discharge: Yes; Military Discharge Explanation: Injury;

• If the field can only have one value, like a date format, picklist, or boolean field, weuse the data from the most recent import. For example, if the Class Rank Scale field(hed__Class_Rank_Scale__c) is populated for an applicant after you process a .csvfile for first-year applicants, it might be updated again when you process a .csv file forrecommenders.

19

Reference EDA Mapping Considerations

• The Application JSON, Contact JSON, and Education History JSON fields each containa single JSON object. After you process a .csv file, an applicant might have data in eachof these JSON fields. If you then process another .csv file and there’s updated JSON dataavailable, the existing data is updated within the JSON object. If there’s new data, it’sadded to the existing JSON object. 

Common App stores some data for questions in a date format that doesn’t align with EDA'sMM/DD/YYYY format. We transform the Common App date value to work with EDA.

• For first-year data that uses the MMYYYY format, when creating the correspondingEDA records, we set day to the first of the month. For example, if an applicant provides06/2020 for the question Graduation Date, we set the date to 06/01/2020 on the EDArecord.

NoteSome Common App data stores the full date; we set the date as the first of themonth only when necessary

• For transfer data that uses YYYY-MM-DD or MMM DD YYYY at HH:MM, we set the value toEDA's format, which means dropping the hour and minute timestamp. 

If a Contact has multiple Application records, the batch job updates the most recentlycreated Application record. 

We map Common App picklist values to existing EDA picklist values when we can. If we finda picklist value in Common App that doesn't exist in EDA, we add the Common App picklistvalue to the picklist in EDA, and then use that one.

If you’re importing Common App data for suspects or prospects, we create Contact records,not Leads.

DuplicatesFor the most part, we don’t detect duplicates beyond checking for the Common App ID, orin some cases the CEEB code. 

NoteWe can’t match first-year applicant records to transfer records because the recordsare created using different Common App IDs.

Here are some examples of how the duplicate logic works. Let’s say a person’s data isincluded in a prospect's .csv file, and EDA records are created for them. Then a weeklater the same person finishes their application and their data is now included in the

20

Reference EDA Mapping Considerations

applicant .csv file. When the batch job runs against the applicant .csv file, the existing EDArecords (created when the person was a prospect) are updated. Or consider a case where abrother and sister apply to your institution. In this case, duplicate parent Contact records arecreated because there’s no Common App ID for them. 

We provide basic duplication prevention for parent Contact records by querying for existingRelationship records whose Contact field (hed__Contact__c) is the applicant’s Contactrecord ID. We also query for and compare existing parents’ Contact records using theparents’ first name and last name. 

To manage duplicates beyond what the package offers, use standard Salesforce duplicaterules.

Personal BackgroundWe create Contact records for each parent (including step-parents and legal guardians)the applicant provides. We also create one-way Relationship records from the applicant’sContact record to each parent’s Contact record. We set the Type field (hed__Type__c)on the Relationship record to Parent for legal guardians. If reciprocal relationships areconfigured in EDA, a two-way relationship is created. See Configure Relationship Settings inthe EDA Documentation.

NoteWe don't create Contact records for siblings, instead we map that data to theContact JSON field.

If an applicant provides language details, we create Contact Language records and fill outthe Language lookup field (hed__Language__c) using your existing Language records. Ifno matching Language record is found, the batch populates the Language Details text field(hed__Language_Details__c) instead. If the batch finds existing Contact Language recordsassociated with the Contact record, those records are deleted and new Contact Languagerecords are created based on the current JSON object. 

If an applicant doesn't provide a Preferred Phone value in the Common App, we defaultthe Preferred Phone field (hed__PreferredPhone__c) on their Contact record to MobilePhone. We then map the actual phone number to the Mobile field (MobilePhone) and anyadditional phone number is mapped to the Home Phone field (HomePhone). We use thissame mapping if a parent’s data doesn't specify a Preferred Phone value, too. 

If a transfer applicant doesn't provide a Preferred Email value in Common App,we default the Preferred Email field (hed__Preferred_Email__c) on their Contact

21

Reference EDA Mapping Considerations

record to University Email. We then map the actual email to the University Email field(hed__UniversityEmail__c).

We map address information (for applicants and parents) to the mailing fields on theContact record when possible. Then, the ADDR_Contact_TDTM trigger automatically createsdefault Address records and associates them to the Contact records. For example, if anapplicant provides a current address or only provides a permanent address, we map thatdata to the applicant’s Contact record mailing fields. The ADDR_Contact_TDTM trigger thenautomatically creates a default Address record and associates it to the applicant’s Contactrecord. If an applicant provides a current and permanent address, we map their currentaddress information to the Contact record mailing fields, and create a separate Addressrecord for their permanent address information.

Education BackgroundWe create an Education History record for each set of school information the applicantprovides, such as current school, past secondary schools, and past colleges. We useCEEB codes stored in the School Code field (hed__School_Code__c) to match the datawith Educational Institution Accounts in Salesforce. We don’t have any validation forrecord types, we match solely on the CEEB code. If we can’t find a matching Account,we leave the Educational Institution Account field (hed__Account__c) blank on theEducation History records, and store the CEEB code in the Education History JSON field(hed__Education_History_JSON__c). We don’t create new Account records when we can’tfind a matching CEEB code in your org.

Before creating a new Education History record, we check the Contact record to see if theyhave any existing Education History records with an Educational Institution Name thatmatches the CeebName provided in the data from Common App. If there’s a match, weupdate the existing Education History record with the data. If there’s no match, we create anew Education History record.

For transfers, the Education History JSON field might include JSON data for other EducationHistory records that were created for an applicant at the same time. For example, a Contactnamed Marla has two Education History records—one for Trailblazer Academy and one forConnected University. The Education History JSON field for each Education History recordincludes the same JSON data, that has information about both Trailblazer Academy andConnected University. 

We don’t create any Test or Test Score records in EDA, instead we store the data in theApplication JSON field.

First-year data for SR, OR, MR, and FR reports is stored in the Education History record forthe current school in the Education History JSON field.

22

Reference EDA Mapping Considerations

Using the Applicant DataIf your institution wants to show data from all the JSON fields, design and create a customcomponent. Likewise, if you want data for a specific Common App question that’s mappedto a JSON field to show in a custom field, create automation (such as a formula, flow, orApex code) to extract the JSON data after the EDA records are created.

Content Document records and the JSON fields on the Application, Contact, and EducationHistory objects contain PII. Make sure that you adhere to your institution’s privacy policiesand that you follow Salesforce's general GDPR guidelines. See EDA Product Information forGDPR.

23