Prerequisites to Use CDM Folders Connector

© Copyright Informatica LLC 2019. © Copyright Informatica LLC 2019. Informatica, Informatica Cloud, and the Informatica logo are trademarks or registered trademarks of Informatica LLC in the United States and many jurisdictions throughout the world. A current list of Informatica trademarks is available on the web at https://www.informatica.com/trademarks.html.

AbstractYou can use CDM Folders Connector to read data from or write data in the .csv file format to the common data model folder in the Microsoft Azure Data Lake Storage Gen2 (ADLS Gen2) storage. You can also use CDM Folders Connector to create an external dataflow on Power BI workspace to access the data stored in the common data model folder. This article explains the prerequisite tasks that you must complete before you use CDM Folders Connector.

Supported Versions• Informatica® Cloud Data Integration CDM Folders

Table of ContentsOverview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Prerequisite and Administration Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Creating an Azure Active Directory Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Creating Microsoft Azure Data Lake Storage Gen2 Storage Account. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Obtaining Tenant ID of the Azure Active Directory Account. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Creating File System and Common Data Model Folder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Rules and Guidelines for Naming File System and Common Data Model Folder. . . . . . . . . . . . . . . . . . . 8

Setting Permissions for Common Data Model Folder (Access Control List). . . . . . . . . . . . . . . . . . . . . . . 8

Creating a Power BI Workspace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

OverviewYou can use CDM Folders Connector to create a CDM Folders connection, and connect to ADLS Gen2 storage and Power BI. Before you use CDM Folders Connector to read data from or write data in the .csv file format to the common data model folder in the to the ADLS Gen2 storage or to create an external dataflow on Power BI workspace, you must complete the prerequisite tasks.

Then, you can create a CDM Folders connection and associate it with a mapping or mapping task.

For more information about CDM Folders Connector, see the Informatica Cloud Data Integration CDM Folders Connector User Guide.

Prerequisite and Administration TasksAs a user, you can use CDM Folders Connector after the Microsoft Azure Active Directory administrator performs the administration tasks and you perform the prerequisite tasks.

Reading and writing data to the common data model folder in the ADLS Gen2 storage

To read data from or write data in the .csv file format to the common data model folder present in the ADLS Gen2 storage, the following tasks must be completed:

Administration tasks

• Create an Azure Active Directory application to authenticate the users to access the ADLS Gen2 storage account.

2

• Create an ADLS Gen2 storage account or elevate the user access to create an ADLS Gen2 storage account.

• Obtain the tenant ID of the Azure Active Directory account and provide the tenant ID to the user.

• Elevate the Azure Active Directory user access to create a file system and common data model folder on Microsoft Azure Storage Explorer.

User prerequisite tasks

• Install Microsoft Azure Storage Explorer.

• Create a filesystem and folders on Microsoft Azure Storage Explorer.

• Set the Access Control List to provide the read, write, and execute permissions to the common data model folders.

Creating Power BI External Dataflow on PowerBI workspace

To create a Power BI external dataflow on the Power BI workspace, the following tasks must be completed:

Administration tasks

Procure Power BI pro license and assign it to the user account from the O365 admin portal.

User prerequisite tasks

• Create a Power BI workspace if you do not have the latest Power BI workspace.

• Generate a Power BI access token and Power BI refresh token to create a Power BI external dataflow.You must use the Informatica Power BI OAuth2 tool to generate a valid Power BI access token and refresh token. For more information about how to create the Power BI access token and refresh token, see https://marketplace.informatica.com/solutions/power_bi_oauth_tool.

The following image shows the prerequisite and administration tasks that the Microsoft Azure Active Directory administrator and user must perform:

3

Creating an Azure Active Directory ApplicationCreate an Azure Active Directory application to authenticate access to the storage account.

1. Log in to the following Azure Active Directory portal: https://portal.azure.com/

2. In the Favorites section, click Azure Active Directory.

The Azure Active Directory page appears.

3. In the Manage section, click App registrations.

The App registrations page appears.

4. Click New application registration to create a new Azure Active Directory application.

The Create page appears.

5. Perform the following steps in the Create page:

a. In the Name field, enter the application name.

For example, CDMADLSGEN2_1b. In the Application type field, select Web app / API as the type of the application.

c. In the Application type field, select Web app / API as the type of the application.

d. In the Sign-on URL field, enter the URL of the application.

For example, https://contoso.come. Click Create.

f. The newly created Azure Active Directory application page appears.

The following image shows the Azure Active Directory application details:

6. Click Settings.

7. In the GENERAL section, select Owners.

The Owners page appears.

8. Click Add owner.

The Add owner page appears.

9. In the Search field, search for the owner name or email address that you used to login to Azure Active Directory portal.

10. Select the owner name or email address and click Select.

11. In API ACCESS section, click Required permissions.

The Required permissions page appears.

4

12. Click Add.

The Add API access page appears.

13. Click 1 Select an API.

The Select an API page appears.

14. In the search field, enter Azure Storage and click Select.

The Enable Acess page appears.

15. In the DELEGATED PERMISSIONS section, select Access Azure Storage.

16. Click Select > Done.

17. In the Required permissions page, select Windows Azure Active directory and ensure that the Sign in and read user profile option is enabled in the DELEGATED PERMISSIONS section.

If Windows Azure Active directory is not listed under the Required permissions page, perform the following steps:

a. Click Add.

The Add API access page appears.

b. Click 1 Select an API.

The Select an API page appears.

c. In the search field, enter Windows Azure Active Directory and click Select.

The Enable Acess page appears.

d. In the DELEGATED PERMISSIONS section, select Sign in and read user profile.

e. Click Select > Done.

18. In the API ACCESS section, select Keys.

The Keys page appears.

19. Perform the following steps in the Keys page:

a. In the Passwords section, enter the name of the key in the Description field.

b. In the Expires field, select the duration of the key as Never Expires.

c. Click Save.

d. The value of the key is generated and displayed in the Values field.

Note: You must copy the key value as you cannot retrieve the key value once you leave the page.

Enter the Application ID in the ClientID and Key Value in the Client Secret connection properties to create a CDM Folders connection.

Creating Microsoft Azure Data Lake Storage Gen2 Storage AccountCreate an ADLS Gen2 storage account to create a file system.

1. Log in to the following Azure Active Directory portal: https://portal.azure.com/

2. In the Favorites section, click Storage accounts.

The Storage accounts page appears.

3. Click Add to create a new ADLS Gen2 storage account.

The Create storage account page appears.

5

4. In the Basics tab, perform the following steps:

a. In the PROJECT DETAILS section, select the subscription type in the Subscription field.

b. Select an existing resource group or create a new resource group where you want to create the storage account in the Resource group field.

c. In the INSTANCE DETAILS section, perform the following steps:

1. In the Storage account name field, enter the name of the storage account.

2. In the Location field, enter the location of the account.

3. In the Performance field, select Standard.

4. In the Account kind field, select StorageV2 (general purpose v2).

5. In the Replication field, select Read-access geo-redundant storage (RA-GRS).

6. In the Access tier (default) field, select Hot.

d. Click Next : Advance.

The Advanced tab appears.

5. In the Advanced tab, perform the following steps:

a. In the SECURITY section, enable the Secure transfer required option.

b. In the VIRTUAL NETWORKS section, select All networks in the Allow access from field.

c. In the DATA LAKE STORAGE GEN2 section, enable the Hierarchical namespace option.

d. Click Review + Create > Create.

6. Click on the newly created storage account name.

The storage account details appears.

7. Click Access control (IAM) > Add.

The Add role assignmentAdd role assignment page appears.

8. Perform the following steps in the Add role assignment page:

a. In the Role field, select Storage Blob Data Contributor (Preview).

b. In the Assign access to field, select Azure AD user, group or service principle.

c. In the Select field, select the Azure Active Directory account ID that requires access to access the storage account.

d. Click Save.

Note: If you want to add multiple Azure Active Directory account IDs to access the storage account, you must perform the same steps.

e. In the Role field, select Storage Blob Data Contributor (Preview).

f. In the Assign access to field, select Azure AD user, group or service principle.

g. In the Select field, select the Azure Active Directory application that you created, which requires access to access the storage account.

h. Click Save.

Enter the name of the ADLS Gen2 storage account in the Storage Account connection property to create a CDM Folders connection.

6

Obtaining Tenant ID of the Azure Active Directory AccountObtain the tenant ID of the Azure Active Directory account to authenticate access to the storage account.

1. Log in to the following Azure Active Directory portal: https://portal.azure.com/

2. In the Favorites section, click Azure Active Directory.

The Azure Active Directory page appears.

3. In the Manage section, select Properties.

The Properties page appears.

4. In the Directory properties, click the Copy icon to copy the directory ID from the Directory ID field.

The following image shows an example of a directory ID:

Enter the directory ID in the Azure TenantID connection property to create a CDM Folders connection.

Creating File System and Common Data Model FolderCreate a file system and common data model folder to store the data in the .csv file format and the associated metadata in the model.json file format.

1. Log in to the Azure Storage Explorer.

2. In the Storage Account section, select and expand the ADLS Gen2 account that you created.

3. Right-click on Blob Containers and select Create Blob Container.

4. Enter a name for the file system.

5. Click New Folder to create a new common data model folder within the file system that you created.

The New Folder page appears.

6. Enter the name of the common data model folder in the New folder name field.

7. Click OK.

Enter the name of the file system in the File System Name and folder path in the Common Data Model Folder connection properties to create a CDM Folders connection.

To obtain the common data model folder path, select the common data model folder. Then, right-click on the folder and select Properties option to copy the folder path.

For example, create a mysalesforce common data model folder within the Informatica file system name. When you select the folder, the folder path is displayed. Right-click on the folder and select Properties option to copy the folder path.

The following image shows the path of the mysalesforce common data model folder within the file system:

7

Rules and Guidelines for Naming File System and Common Data Model FolderUse the following rules and guidelines while entering the names of the file system and Common Data Model folder:

• Informatica recommends that you enter the file system name as Informatica to uniquely identify the file system that you are using with the CDM Folders Connector.

• Informatica recommends that you create at least one common data model folder within the file system. If you want to create sub-folders, you can create the sub-folders within the common data model folder that you created.

• You can add only lowercase letters, numbers, and hyphens when you enter the names of the common data model folder and file system. The names must begin and end with a letter or number.

• Do not add consecutive hyphens when you enter the names of the common data model folder and file system.

• You must enter the common data model folder name that reflects the business domain of the data that you want to store in the common data model folder.For example, if you want to create a common data model folder to store Salesforce data, enter the name of the common data model folder as mysalesforce.

• If there is only one folder in the common data model folder path, do not use Common Data Model as the folder name. You can use Common Data Model as the folder name if there is more than one folder in the common data model folder path.For example, folder1/Common Data Model or Common Data Model/folder2.

Setting Permissions for Common Data Model Folder (Access Control List)Set the Access Control List to provide permissions to the common data model folder.

1. Log in to the Azure Storage Explorer.

2. In the Storage Accounts section, select and expand the ADLS Gen2 account that you created.

3. Expand the file system.

4. Right-click on the file system that you created and select Manage Access.

The Manage Access dialog box appears.

5. In the Permission for: field, select both the Access and Default check boxes. Enable the Read, Write, and Execute permissions.

6. In the Add user or group field, enter the Azure Active Directory user name or group name and click Add.

Note: If you enter an Azure Active Directory group name, all the users within the group will have the same permissions.

7. Click Save.

When you set the permissions of a file system, all the folders within that file system will have the same permissions. However, if you create a folder within a file system before setting the permissions of the file system, you must perform the same steps to set the Access Control List for that folder.

Creating a Power BI WorkspaceCreate a latest Power BI workspace on the Power BI Online to create an external dataflow and to access data from the ADLS Gen2 storage.

1. Log in to the following Power BI portal: https://app.powerbi.com/

2. Click Workspaces > Create app workspace.

8

The Create a workspace page appears.

3. Click Try Now to create Power BI workspace.

The Create a workspace page appears.

4. Enter the name of the workspace in the Workspace name field.

5. Click Save.

Select the workspace name where you want to create the Power BI external dataflow in the Power BI Workspace Name target advanced property.

AuthorSubhashree Salam

AcknowledgementsThe author would like to acknowledge Pavan D Pothnis and Prathamesh Gade, for their technical assistance with this article.

9