Publishing to Dynamics 365 Finance & Operations (DMF)

Empower supports publishing datasets from your Databricks delta lake to Dynamics 365 Finance & Operations (F&O) using the Data Management Framework (DMF). The process includes an optional (but recommended) schema validation step that checks your data structure before sending it to F&O, followed by the actual publish.

---

Prerequisites

1. Data in your Empower delta lake to publish.
2. A Dynamics 365 F&O environment with DMF enabled.
3. An Azure AD application registered and configured with F&O credentials. See how to do so below.
4. A Data Project (also called a "definition group") created in F&O for each entity you plan to publish. See the section below for details.
5. An Entity Mapping record in Empower that links each entity to its corresponding F&O Data Project.

Obtain Dynamics F&O Credentials via Azure AD Application

Your Empower deployment must be authenticated with your Dynamics F&O environment in order to publish data. The recommended strategy is to use an Azure AD application. The Azure Tenant ID, Application ID, and Client Secret are required.

📘 Tip

The same Azure application can be used to authenticate to both F&O and CRM environments if they are within the same tenant.

To configure the Azure AD application:

1. Open the Azure Portal and navigate to your AD application registration.
2. Go to API Permissions → Add a permission.
3. Select Dynamics ERP from the list of available APIs.
4. Choose the appropriate permissions and grant admin consent.
5. Under Certificates & secrets, create a new client secret and save its value — you will need it for the Key Vault setup.

The following credentials must be stored as a single JSON secret in your Azure Key Vault (accessible from Databricks):

Credential	Description
Dynamics URL	Your F&O environment URL (e.g. https://your-env.operations.dynamics.com)
Azure Tenant ID	The Azure Active Directory tenant ID
Application (Client) ID	The Azure AD app registration client ID
Client Secret	The secret value from the app registration
Legal Entity	The default F&O company code (e.g. USMF, DAT)

Create a Data Project in F&O

Before publishing an entity, a Data Project must exist in F&O for that entity. This tells F&O how to receive and process incoming data.

1. In Dynamics 365 F&O, navigate to Data Management workspace.
2. Click Import to create a new import project.
3. Give the project a name (this is the "definition group" name that Empower will reference).
4. Add the target entity (e.g. Customer groups, Workers).
5. Set the source data format to Excel.
6. Save the project.

Repeat this for every entity you intend to publish.

---

How It Works

When you publish data from Empower to F&O, the following happens behind the scenes:

1. Read — Empower reads the specified table from your delta lake and applies any filters you have configured.
2. Validate (optional but recommended) — Empower connects to F&O and compares your table's column names and data types against the entity definition in F&O. Any mismatches are flagged before data is sent.
3. Package — Empower builds a DMF-compatible import package (an Excel file with supporting metadata files, bundled into a ZIP archive).
4. Upload — The package is uploaded to F&O via a secure, temporary Azure Blob URL provided by F&O itself.
5. Import — F&O processes the package through its DMF engine, applying business logic and validation rules.
6. Track — Empower polls F&O until the import finishes, then records the outcome (success, partial success, or failure) along with any error details.

All results — including row counts, error messages, and execution timestamps — are automatically saved to an audit log table in your delta lake.

---

Steps

Step 1: Set Up Entity Mapping

Before publishing, Empower needs to know which F&O Data Project corresponds to each entity. This is configured in the Entity Mapping table.

1. In your Empower catalog, locate the dmf_entity_mapping table within the legal entity schema (e.g. USMF).
2. If the table does not exist yet, create it with two columns: entity (the F&O entity name) and definition_group_name (the F&O Data Project name).
3. Add one row for each entity you plan to publish. For example:

Entity	Definition Group Name
CustCustomerGroupEntity	CustGroup
HcmWorkerEntity	HCMWORKERENTITY_USMF
SalesOrderHeaderV4Entity	SalesOrderImport

📘 Note

If you skip this step, Empower will generate a Data Project name automatically. However, it is strongly recommended to set up explicit mappings so that your imports land in the correct, pre-configured F&O Data Project.

Step 2: Configure Column Options (If Needed)

Some F&O entities have columns that are auto-generated by F&O (for example, personnel numbers or sequence-based IDs). For these columns, you need to tell Empower to let F&O handle the value rather than expecting it from your data.

1. Locate the dmf_publish_column_options table in your legal entity schema. This table is created automatically on first publish.
2. Add a row for each column that requires special handling. Each row specifies the entity name, the column name, and which option to enable.

Common options include:

Option	When to Use
IsAutoGenerated	The column value is generated automatically by F&O (e.g. Number Sequences)
IsAutoDefault	F&O applies a default value if none is provided

📘 Tip

Most entities do not require any column options. You only need this step if F&O rejects records due to auto-generated or defaulted fields.

Step 3: Run Schema Validation (Recommended)

Before publishing, it is strongly recommended to run schema validation. This step compares your Databricks table structure against the entity definition in F&O and flags any issues — such as missing required columns, data type mismatches, or columns that F&O does not recognise.

Validation can be triggered in two ways:

• From the Empower UI — navigate to the validation section and select the entities you want to check.
• From a Databricks notebook — run the schema validation notebook directly, passing in the catalog, legal entity, Key Vault secret name, and the list of entities to validate.

Understanding the Validation Report

After validation runs, you will see a report for each entity. The report highlights:

Finding	What It Means
Type Mismatch	A column in your table has a different data type than F&O expects (e.g. a date stored as text). This must be fixed before publishing.
Unknown Column	Your table contains a column that does not exist in the F&O entity. Remove it or correct the column name.
Missing Mandatory Field	F&O requires this field but it is not present in your table. Add the column and populate it with valid values.
Skipped Column	A field that exists in F&O but is not in your table. This is usually safe to ignore — F&O treats these as optional.
DMF Not Enabled	The entity is not registered in F&O's Data Management Framework. Contact your F&O administrator.

⚠️ Important

Items marked as Type Mismatch, Unknown Column, Missing Mandatory Field, or DMF Not Enabled are blocking issues. These must be resolved before publishing. Skipped Columns are informational only and will not prevent a successful import.

Step 4: Publish Data to F&O

Once validation passes (or you are confident your data is ready), proceed with publishing.

Publishing can be triggered in two ways:

• From the Empower UI — select the target entity and source table, configure any filters, and click publish.
• From a Databricks notebook — run the publishing notebook with the appropriate parameters (catalog, legal entity, Key Vault secret, source table, and target entity name).

During publishing, Empower will:

1. Read the source table and apply any configured filters.
2. Build the DMF import package.
3. Upload the package to F&O.
4. Wait for F&O to finish processing.
5. Record the results.

Entity Dependencies and Import Order

When publishing multiple related entities, the order matters. Parent or master data must be published before child or transactional data that references it.

Recommended order:

1. Master data first — Customers, Vendors, Products, Customer Groups
2. Setup data next — Warehouses, Sites, Dimension Groups
3. Transaction headers — Sales Order Headers, Purchase Order Headers
4. Transaction lines last — Sales Order Lines, Purchase Order Lines

For example, you cannot import Sales Order Lines unless the corresponding Sales Order Headers (and the referenced Customers) already exist in F&O.

Step 5: Review Results

After publishing completes, check the results to confirm your data was imported successfully.

Possible Outcomes

Outcome	What It Means	What to Do
Succeeded	All rows were imported into F&O.	No action needed.
Partially Succeeded	Some rows were imported; others were rejected by F&O validation.	Check the execution log for error details on rejected rows. Fix the data and re-publish the failed rows.
Failed	No rows were imported.	Review the error details in the execution log. Common causes include authentication failures, missing Data Projects, or critical data issues.

Viewing the Execution Log

Every publish operation is automatically logged in the dmf_execution_logs table in your delta lake. This table contains:

• The entity that was published
• The number of rows read, written, and skipped
• Any error messages returned by F&O
• Timestamps for when the job started and finished

You can query this table at any time to review the history of all publish operations.