Skip to main content

Getting Started

Initial Setup and Configuration​

The Empower for Dynamics library provides a powerful interface for managing data synchronization with Dynamics CRM. This guide walks you through the essential setup process and core configuration concepts.

Basic Installation and Setup​

To begin using the Dynamics V2 library, first import the main Dynamics class:

from empowerspark.target.type.dynamics_v2 import Dynamics

# Create a new Dynamics client instance
dynamics_client = Dynamics()

After creating the client instance, you'll need to configure two critical database settings that determine where the library stores its operational data and logs:

# Configure the artifact schema that will hold the logs for this package
dynamics_client.__artifact_db__ = "empower_ddu"

# Set the name of the high-level batch log table for tracking execution metrics
dynamics_client.__log_table_name__ = "batch_log"

Secret Management and Authentication​

The library uses Azure Key Vault for secure credential management. Configure your client with the appropriate secrets using:

# Configure the client with your connection string secret
dynamics_client.configure("dataverse-connection-string")

Secret Structure and Requirements​

Your base secret in Azure Key Vault should contain a JSON object with the following required fields:

{
    "dynamics_url": "https://your-instance.crm.dynamics.com",
    "azure_tenant": "your-azure-tenant-id",
    "client_id": "your-client-id",
    "client_secret": "your-client-secret",
    "dynamics_erp": "your-erp-name",
    "extra_clients_secret_name": "secret-containing-additional-service-principals"
}

For enhanced performance with multiple service principals, store additional credentials in a separate secret using the format:

OAuthClientID=<client-id>;OAuthClientSecret=<client-secret>;

Performance Configuration​

Worker Thread Management​

The library uses a thread pool for parallel processing. The optimal number of worker threads depends on your service principal configuration:

  • The base calculation is: number_of_service_principals * 50
  • Each service principal supports up to 52 concurrent connections
  • For example, with 32 service principals, configure: 32 * 50 = 1,600 workers
base_config = {
    "advanced.performance.num_workers": 1600  # Adjust based on your service principals
}

Basic Usage Example​

Here's a complete example showing how to perform a basic upsert operation:

# Initialize and configure the client
dynamics_client = Dynamics()
dynamics_client.__artifact_db__ = "empower_ddu"
dynamics_client.__log_table_name__ = "batch_log"
dynamics_client.configure("dataverse-connection-string")

# Configure performance settings
dynamics_client.update_config({
    "advanced.performance.batch_size": 50_000,  # see note below
    "advanced.performance.num_workers": 1600,  # see note below
})

# Perform an upsert operation
result = dynamics_client.load(
    source_table="your_source_table",
    key_columns=["id"],  # Primary key or business key columns
    target_table="contact",  # Dynamics target table
    mode="upsert"
)

NOTE: For detailed information on configuration settings see Advanced Configuration Guide