Harmonize Using Quickstart

Prerequisites

You must have done the following:

  • Created an entity and ingested some raw data for that entity. (See Ingest Using QuickStart.)
  • Decided which fields in the raw data to harmonize and how to name the corresponding property in the entity model. The entity model property names (not the field name in the raw dataset) will be used by apps to access that field in the datahub. If you later decide to expose more data to apps, you can add more properties to the entity model and rerun the harmonization flow.

1 - Define the entity model.

The entity model specifies the standard labels for the fields we want to harmonize.

To define an entity model,

entity properties

  1. In QuickStart's navigation bar, click Entities.
  2. At the top of the entity card, click the pencil icon to edit the entity definition.
  3. In the entity editor, click + in the Properties section to add a new property.
    1. Set Name to the name you choose for the property.
    2. Set Type to the expected data type of the property.
    3. To make this property the primary key, click the area in the key column for the row for this property.
    4. To specify that this property needs an element range index, click the area in the lightning bolt column for the row for this property.
    5. To mark this property as PII, click the area in the key column for the row for this property.
    6. To indicate that the entity can have multiple instances of this property, set Cardinality to 1..∞.
  4. Click SAVE.
  5. If prompted to update the index, click Yes.
  6. Drag the bottom-right corner of the entity card to resize it and see the newly added properties.

2 - Define the mappings.

To create your mapping,

Create mapping

  1. In QuickStart’s navigation bar, click Mapping.
  2. In the left panel, click the + icon for the entity.
  3. In the Create New Mapping form, set Mapping Name to the name you choose for your mapping.
  4. Click CREATE.

Your new mapping appears under the tab with your entity name in the left panel.

The mapping editor displays a row for each property in your entity model. In each row,

  • the right column displays the entity property, and
  • the left column contains a dropdown list from which you can select the source field that corresponds to that entity property.

To configure the mapping,

Mapping editor

  1. For each entity property, expand the dropdown list under Source and select the source field that corresponds to that entity property.

  2. Click SAVE MAPPING.

QuickStart selects one of the items ingested into the STAGING database.

  • The URI of that selected item is displayed at the top of the Source column of the mapping editor.
  • The fields of that item are presented in the dropdown lists.

To choose a different source item to map to your entity model,

  1. Get the URI of a different source item.
  2. In QuickStart’s navigation bar, click Browse Data.
  3. Select the STAGING database if not already selected.
  4. Narrow the list by clicking on the entity collection among the filters on the left.
  5. Select an item and click its copy icon to copy its URI to your clipboard.
  1. Replace the selected item in the mapping editor.
  2. In QuickStart’s navigation bar, click Mapping.
  3. In the left panel, click the mapping you just created.
  4. In the mapping editor under Source, click the pencil icon next to the URI.

3 - Create and run the harmonize flow.

Harmonization uses the data in your STAGING database to generate canonical entity instances in the FINAL database.

To create a harmonization flow for your entity,

Create Harmonize Flow form

  1. In QuickStart’s navigation bar, click Flows.
  2. Expand the tab with your entity name in the left panel.
  3. Click the + for Harmonize Flows.

  4. In the Create Harmonize Flow dialog, set Harmonize Flow Name to the name you choose for your harmonization flow.

  5. Click CREATE.

When you create a flow with mapping, QuickStart automatically generates harmonization code based on the entity model and the mapping and then deploys the code to MarkLogic Server.

To run the harmonization flow,

Run Flow form

  1. Click the Flow Info tab.
  2. Click Run Harmonize.

Next Steps

You can ingest additional data into your data hub and harmonize them. You can also add properties to your entity model as needed and rerun the harmonization flow.

When your project is ready, you can deploy it either on your own production environment or on the Data Hub Service (DHS), and then serve your data.

After deployment to your production environment, you can still run additional ingestion and harmonization flows. However, you must use other tools, because QuickStart is not intended for a production environment. See Ingest and Harmonize for a list of other tools you can use for each flow.

See Also