Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ __The DVE offers__:
- Format normalization to Parquet for a unified data representation
- Data modelling and typecasting
- Business-rule validations executed on supported backends such as Spark and DuckDB, with the option to add custom backends
- Validate and enforce referential integrity checks with minimal configuration
- Deriving new fields and entities
- Clear validation reporting, including summary insights and record-level error messages

Expand Down
27 changes: 14 additions & 13 deletions docs/user_guidance/entity_relationships.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,26 +37,27 @@ In order to perform these validations, how to link normalised entities needs to

To allow the DVE to link between normalised assets, the following information should be provided (per linkable entity):

- parent_entity: the immediate parent of the entity
- join_fields: how to join the entity with its parent in dictionary form (parent_field_name: child_field_name)
- mandatory: whether the child entity is a mandatory field in the immediate parent
- `parent_entity`: the immediate parent of the entity
- `join_fields`: how to join the entity with its parent in dictionary form (parent_field_name: child_field_name)
- `is_root_entity`: indicates that the entity is a root node in a hierarchical model
- `mandatory`: whether the child entity is a mandatory field in the immediate parent

There is also the functionality to customise errors related to either missing parent or group rejections:

- missing_parent_id_error_code: the error code to display if a record is rejected as it has no valid parent record
- missing_parent_id_error_message: the error message to display if a record is rejected as it hs no valid parent record
- no_valid_records_error_code: the error code to display if parent records are removed due to no valid children in a mandatory field
- no_valid_records_error_message: the error message to display if parent records are removed due to no valid children in a mandatory field
- `missing_parent_id_error_code`: the error code to display if a record is rejected as it has no valid parent record
- `missing_parent_id_error_message`: the error message to display if a record is rejected as it has no valid parent record
- `no_valid_records_error_code`: the error code to display if parent records are removed due to no valid children in a mandatory field
- `no_valid_records_error_message`: the error message to display if parent records are removed due to no valid children in a mandatory field
- `empty_entity_error_code`: the error code to display if a __mandatory__ entity has any records post filtering
- `empty_entity_error_message`: the error message to display if a __mandatory__ entity has any records post filtering

!!! note
For root entities, you don't need to specify entity relationships - this will be inferred based on their absence.
But you may wish to so that error codes and messages can be customised. Ensure that for root entities the parent_entity
abd join_fields values are left blank.

Specifying root entities is __optional__. Root entities will be inferred based on their absence.
You may wish to specify root entities so that error codes and messages can be customised (e.g. empty entity).

When specifying root entities you should ensure that parent_entity and join_fields values are left blank.

## Entity Hierarchy Object



The details provided in the entity_relationships section of the dischema are used to create an EntityHierarchy object.
Please refer to [Advanced User Guidance: Entity Hierarchy](../advanced_guidance/package_documentation/entity_hierarchy.md).
29 changes: 13 additions & 16 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ mkdocs = "1.6.1"
mkdocstrings = { version = "1.0.3", extras = ["python"] }
griffelib = "2.0.1"
pymdown-extensions = "11.0.1"
zensical = "0.0.46"
zensical = "0.0.63"

[tool.ruff]
line-length = 100
Expand Down
Loading