You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: connectors/database/salesforce/index.mdx
+18-2Lines changed: 18 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -54,8 +54,24 @@ For a complete guide on managing secrets in hybrid setups, see the [Hybrid Inges
54
54
</Tip>
55
55
-**Salesforce Object Name**: Specify the Salesforce Object Name in case you want to ingest a specific object. If left blank, we will ingest all the Objects.
56
56
-**Salesforce API Version**: Follow the steps mentioned [here](https://help.salesforce.com/s/articleView?id=000386929&type=1) to get the API version. Enter the numerical value in the field, For example `42.0`.
57
-
-**Salesforce Domain**: When connecting to Salesforce, you can specify the domain to use for accessing the platform. The common domains include `login` and `test`, and you can also utilize Salesforce My Domain.
58
-
By default, the domain `login` is used for accessing Salesforce.
57
+
-**Salesforce Domain**: Specify the Salesforce domain (subdomain only) to use for authentication. This field accepts only the domain prefix, not the full URL.
58
+
59
+
**Common values:**
60
+
-`login` (default) - For production instances (resolves to `https://login.salesforce.com`)
61
+
-`test` - For sandbox instances (resolves to `https://test.salesforce.com`)
62
+
63
+
**For Salesforce My Domain:**
64
+
Enter your custom domain prefix, including all subdomain components such as `.my` or `.sandbox.my`, but without `.salesforce.com`.
65
+
66
+
**Examples:**
67
+
- If your My Domain URL is `https://mycompany.my.salesforce.com`, enter: `mycompany.my`
68
+
- If your sandbox My Domain URL is `https://mycompany--uat.sandbox.my.salesforce.com`, enter: `mycompany--uat.sandbox.my`
69
+
- If your URL is `https://example-dot-com--uat.sandbox.my.salesforce.com`, enter: `example-dot-com--uat.sandbox.my`
70
+
71
+
<Note>
72
+
**Important:** Do NOT enter the full URL or include `.salesforce.com`. Only enter the subdomain prefix as shown in the examples above.
73
+
</Note>
74
+
59
75
**SSL Configuration**
60
76
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under sslConfig which is placed in the source.
Copy file name to clipboardExpand all lines: connectors/ingestion/great-expectations.mdx
+116-3Lines changed: 116 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,6 +41,15 @@ action:
41
41
database_name: <databaseName in OM>
42
42
schema_name: <schemaName in OM>
43
43
table_name: <tableName in OM>
44
+
expectation_suite_table_config_map:
45
+
my_first_suite_name:
46
+
database_name: <databaseName in OM>
47
+
schema_name: <schemaName in OM>
48
+
table_name: <tableName in OM>
49
+
my_other_suite_name:
50
+
database_name: <databaseName in OM>
51
+
schema_name: <schemaName in OM>
52
+
table_name: <tableName in OM>
44
53
[...]
45
54
```
46
55
@@ -52,11 +61,25 @@ In your checkpoint yaml file, you will need to add the above code block in `acti
52
61
-`class_name`: this is the name of the class that will be used to execute the custom action
53
62
-`config_file_path`: this is the path to your `config.yaml` file that holds the configuration of your OpenMetadata server
54
63
-`database_service_name`: [Optional] this is an optional parameter. If not specified and 2 tables have the same name in 2 different OpenMetadata services, the custom action will fail
55
-
-`database_name`: [Optional] only required for `RuntimeDataBatchSpec` execution (e.g. run GX against a dataframe).
56
-
-`schema_name`: [Optional] only required for `RuntimeDataBatchSpec` execution (e.g. run GX against a dataframe).
57
-
-`table_name`: [Optional] only required for `RuntimeDataBatchSpec` execution (e.g. run GX against a dataframe).
64
+
-`database_name`: [Optional] The database name as it appears in OpenMetadata. For table-based validations (`SqlAlchemyDatasourceBatchSpec`), this is inferred from the batch spec. **Required** for query-based or dataframe validations (`RuntimeQueryBatchSpec`, `RuntimeDataBatchSpec`) where the table context must be explicitly specified.
65
+
-`schema_name`: [Optional] The schema name as it appears in OpenMetadata. For table-based validations, this is inferred from the batch spec. **Required** for query-based or dataframe validations. Defaults to *default* if not specified.
66
+
-`table_name`: [Optional] The table name as it appears in OpenMetadata. For table-based validations, this is inferred from the batch spec. **Required** for query-based or dataframe validations where the table cannot be automatically determined.
67
+
-`expectation_suite_table_config_map`: [Optional] A dictionary mapping expectation suite names to their target OpenMetadata tables. Required when running multi-table checkpoints, where different expectation suites should send results to different tables. Each entry specifies the `database_name`, `schema_name` and `table_name` for routing validation results.
58
68
59
69
70
+
<Info>
71
+
**Multi-Table Checkpoints**
72
+
73
+
When validating **multiple tables in a single checkpoint**, use the `expectation_suite_table_config_map` parameter to route validation results to the correct OpenMetadata tables. This is necessary because:
74
+
- Each expectation suite may target a different table
75
+
- The checkpoint action needs to know where to send each suite's results
76
+
- Without the mapping, all results would attempt to go to the same default table
77
+
78
+
**Example scenario:** You have a checkpoint validating both `users` and `orders` tables with separate expectation suites (`users_suite` and `orders_suite`). The `expectation_suite_table_config_map` ensures `users_suite` results go to the `users` table and the `orders_suite` go to the `orders` table.
79
+
80
+
For single-table checkpoints, this parameter is not needed - the table information is provided directly or inferred from the batch spec.
81
+
</Info>
82
+
60
83
**Note**
61
84
62
85
If you are using Great Expectation `DataContext` instance in Python to run your tests, you can use the `run_checkpoint` method as follows:
In v1.x.x GX introduced significant changes to their SDK. One notable change was the removal of the `great_expectations` CLI. OpenMetadata introduced support for 1.x.x version through its `OpenMetadataValidationAction1xx` class. You will need to first `pip install 'open-metadata[great-expectations-1xx]'. Below is a complete example
Start combining multiple Nodes and create a workflow.
26
+
27
+
### **Running a Periodic Batch Workflow**
28
+
29
+
To execute an on-demand workflow, click **Run Now** This immediately triggers the workflow based on its configuration.
30
+
31
+
<imgnoZoomsrc="/public/images/how-to-guides/governance/new3.png"alt="Running a Periodic Batch Workflow" />
32
+
33
+
## Best Practices
34
+
35
+
1.**Use the Right Type of Trigger**
36
+
-**Event-Based Entity Triggers** are ideal when specific fields must be automatically updated in response to a change.
37
+
38
+
*Example:* When any attribute of a Glossary Term is modified, its status should automatically update to **IN REVIEW**.
39
+
40
+
-**Periodic Batch Triggers** are best suited for bulk updates across many entities, especially for classification or enrichment workflows.
41
+
42
+
*Example:* Tables or Dashboards can be classified as Tier 1, Tier 2, or Tier 3 based on the completeness of their column descriptions.
43
+
44
+
2.**Use a Single Event-Based Workflow per Data Asset**
45
+
- Configure only one event-based entity workflow for each data asset.
46
+
47
+
Having multiple workflows attempting to update the same field (such as the status of a Glossary Term) can result in unpredictable behavior, as one workflow’s changes may override another’s.
48
+
49
+
3.**Optimize Batch Size for Periodic Workflows**
50
+
- Tune the batch size based on the number of data assets to ensure optimal performance.
51
+
- Avoid running periodic workflows across all entities without filtering. Instead, apply an inclusion filter to limit the result set and prevent performance degradation.
52
+
4.**Use User Approval Tasks Only in Event-Based Workflows**
53
+
- User Approval Tasks should be used exclusively in event-driven workflows.
54
+
55
+
Using them in periodic workflows would generate multiple approval tasks simultaneously, overloading system resources.
56
+
57
+
- If an approval step is needed in a periodic batch workflow, ensure the workflow scope is limited to a small, controlled set of entities.
58
+
59
+
## Limitations
60
+
61
+
1.**User Approval Tasks Are Limited to Assets with Reviewer Support**
62
+
- User Approval Tasks can only be used for data assets that support assigning reviewers.
63
+
64
+
Reviewer support for additional asset types will be introduced in future releases.
65
+
66
+
2.**Fallback Behavior for Entities Without Reviewers**
67
+
- For entities that do not have any reviewers configured, User Approval Tasks automatically follow the **TRUE** path as a graceful fallback.
68
+
69
+
*Example:* A Metric without a reviewer will automatically pass the approval step.
0 commit comments