Tutorial: Securing Personally Identifiable Information

Securing personally identifiable information (PII) was introduced in DHF v4.0.0. To protect PII, the PII fields must be identified in the entity model. Then QuickStart automatically generates PII security configuration files, which we will deploy to the FINAL database.

We have already loaded the Customer raw data by:

• creating the Customer entity and
• creating and running the associated input flow.

In this section, we will:

• Define the entity model by adding properties to the entity model.
• Define the source-to-entity mapping to specify which field in the dataset corresponds to the properties in the entity model.
• Create and run the Harmonize Flow.
• Deploy the configuration files.

1 - Define the Entity Model

To simplify this tutorial, we are going to harmonize only the primary key (id) and two fields that we choose to protect as PII (billing_address and shipping_address).

To define the Customer entity model,

1. In QuickStart's navigation bar, click Entities.
2. At the top of the Customer entity card, click the pencil icon to edit the Customer entity definition.
3. In the Customer entity editor, click + in the Properties section to add a new property.
   1. Set Name to id.
   2. Set Type to string.
   3. To make id the primary key, click the area in the key column for the id row.
4. Click + again to add another property.
   1. Set Name to billing_address.
   2. Set Type to string.
   3. To mark billing_address as PII, click the area in the key column for the billing_address row.
5. Click + again to add another property.
   1. Set Name to shipping_address.
   2. Set Type to string.
   3. To mark shipping_address as PII, click the area in the key column for the shipping_address row.
6. Click SAVE.
7. If prompted to update the index, click Yes.

   
8. Drag the bottom-right corner of the entity card to resize it and see the newly added properties.

   

2 - Define the Mappings

Because the information can easily be mapped between the source dataset and the entity model, we will create the following source-to-entity mappings:

To create a mapping named Customer Mapping,

1. In QuickStart’s navigation bar, click Mapping.
2. In the left panel, click the + icon for the Customer entity.
3. In the Create New Mapping form, set Mapping Name to Customer Mapping.
4. Click CREATE.

Your new mapping appears under the tab named Customer in the left panel.

The mapping editor displays a row for each property in your entity model. In each row,

• the right column displays the entity property, and
• the left column contains a dropdown list from which you can select the source field that corresponds to that entity property.

To configure the mapping,

1. For each entity property, expand the dropdown list under Source and select the source field that corresponds to that entity property.

   TIP: You can enter part of the field name to filter the dropdown list.

2. Click SAVE MAPPING.

3 - Create and Run the Harmonize Flow

Harmonization uses the data in your STAGING database to generate canonical entity instances in the FINAL database.

To create a harmonization flow for the Customer entity,

1. In QuickStart’s navigation bar, click Flows.
2. Expand the tab named Customer in the left panel.
3. Click the + for Harmonize Flows.
4. In the Create Harmonize Flow dialog, set Harmonize Flow Name to Harmonize Customers.

5. Under Mapping Generation, check “

   append: mappingcreated

   append: “ “.

6. Click CREATE.

When you create a flow with mapping, QuickStart automatically generates harmonization code based on the entity model and the mapping and then deploys the code to MarkLogic Server.

To run the harmonization flow,

1. Click the Flow Info tab.
2. Click Run Harmonize.

4 - Deploy the Configuration Files

To deploy the PII security configuration files to the FINAL database,

1. In the gradle.properties file, set mlSecurityUsername and mlSecurityPassword to your MarkLogic Server credentials.

   IMPORTANT: Your MarkLogic Server account must be assigned both manage-admin and security roles.

2. Open a command-line window, and navigate to your DHF project root directory.

3. At your project’s root folder, run the mlDeploySecurity Gradle task.

   • Unix Systems
   • Windows

   ./gradlew mlDeploySecurity

   gradlew.bat mlDeploySecurity

Only users with the pii-reader role will be able to view properties marked as PII in the documents they are allowed to view.

See Also

• Configuring Security (ml-gradle)