feat(generators): translate class-level boolean expressions alike to SHACL and JSON Schema

docs/generators/json-schema.rst

@@ -172,6 +172,16 @@ LinkML supports analogous elements:
 
 Use of these elements will be translated into the appropriate JSON-Schema construct.
 
+At the class level, a class carries the expressions of its ancestors and mixins.
+An ``is_a`` member names a class every instance already belongs to, or none can,
+unless only some do, through a descendant: then the instance must match that
+class's definition, since a JSON object carries no type.
+A slot condition is unknown for an absent slot unless it decides whether the slot
+may be absent, and an instance is invalid only when an expression is definitely
+false, as described under "Class-level expressions and absent slots" in
+:doc:`Advanced features </schemas/advanced>`.
+The SHACL generator reads them the same way.
+
 Inlining
 ^^^^^^^^
 

docs/generators/shacl.rst

@@ -84,6 +84,98 @@ Example Output:
         shacl:targetClass <https://w3id.org/linkml/tests/kitchen_sink/Person> .
 
 
+Class Expressions
+^^^^^^^^^^^^^^^^^
+
+Class-level boolean expressions become the SHACL logical constraint components
+their metamodel definitions map to (`SHACL §4.6
+<https://www.w3.org/TR/shacl/#core-components-logical>`__):
+
+==================  =====================================================
+LinkML              SHACL, on the class's ``sh:NodeShape``
+==================  =====================================================
+``any_of``          ``sh:or`` over the member shapes
+``all_of``          ``sh:and`` over the member shapes
+``exactly_one_of``  ``sh:xone`` over the member shapes
+``none_of``         one ``sh:not`` per member
+==================  =====================================================
+
+Each member becomes an anonymous node shape. ``is_a`` names a class that every
+instance of the shape's class belongs to (the class itself or an ancestor), and
+then adds nothing; one that only some instances belong to, because a descendant
+of the class has it among its ancestors, and then gives ``sh:class``; or one that
+none belongs to, and then gives a shape nothing conforms to. An instance belongs
+to its class and that class's ancestors, by ``is_a`` and by ``mixins``, so a
+class that may also be another is declared as a subclass with that mixin.
+Nested expressions recurse. Each entry of ``slot_conditions`` gives an
+``sh:property`` whose path is that of the slot as induced for the class, so
+``slot_usage`` applies:
+
+* ``required``, ``value_presence`` and the cardinalities give ``sh:minCount`` /
+  ``sh:maxCount``;
+* ``minimum_value`` / ``maximum_value`` give ``sh:minInclusive`` /
+  ``sh:maxInclusive``, and ``equals_number`` gives both, so that ``5`` also
+  matches ``5.0``;
+* ``pattern`` gives ``sh:pattern``;
+* ``equals_string`` and ``equals_string_in`` give ``sh:in``; on an enum slot the
+  values are the permissible values as the enum renders them, the IRI of their
+  ``meaning`` where they have one;
+* ``range`` gives the same class, type or enum constraint as a slot's range.
+
+A shape may have at most one value of ``sh:minInclusive``, ``sh:maxInclusive``
+or ``sh:in``, and of ``sh:pattern``, whose component also takes ``sh:flags``
+(`SHACL §2.1.1 <https://www.w3.org/TR/shacl/#constraints>`__). Where one condition
+needs one of them twice, for example ``minimum_value`` next to
+``equals_number``, the second value goes into an ``sh:and`` member of the
+property shape, where it applies to the same values.
+
+A slot condition is unknown for an absent slot unless it decides whether the slot
+may be absent, and an instance violates an expression only when the expression is
+definitely false, as described under "Class-level expressions and absent slots" in
+:doc:`Advanced features </schemas/advanced>`.
+The JSON Schema generator reads them the same way. In SHACL, each expression gets
+its "not false" form, and, under ``sh:not``, its "definitely true" form, where
+such a condition also requires its slot (``sh:minCount 1``). ``exactly_one_of``
+becomes ``sh:xone`` over the "definitely true" forms of its members.
+
+.. code-block:: yaml
+
+  GeodeticReferenceSystem:
+    slots: [code, name]
+    any_of:
+      - slot_conditions:
+          code:
+            required: true
+      - slot_conditions:
+          name:
+            required: true
+
+.. code-block:: turtle
+
+    ex:GeodeticReferenceSystem a sh:NodeShape ;
+        sh:or ( [ sh:property [ sh:path ex:code ; sh:minCount 1 ] ]
+                [ sh:property [ sh:path ex:name ; sh:minCount 1 ] ] ) ;
+        ...
+
+A class expression constrains every instance of its class, so the shape of a
+class carries the expressions of its ancestors and mixins as well, as it carries
+their slots. Each is translated in the context of that class, where
+``slot_usage`` applies. A node typed only with a subclass, as ``rdflib_dumper``
+and the JSON-LD context produce it, is therefore checked without an
+``rdfs:subClassOf`` triple in the data. The ``sh:class`` that ``is_a`` gives
+recognises instances of subclasses only where the data graph states the
+``rdfs:subClassOf`` (`SHACL §4.1.1
+<https://www.w3.org/TR/shacl/#ClassConstraintComponent>`__), as it does for a
+slot's range.
+
+An operator whose members use anything else is skipped as a whole and logged as
+a warning, because leaving out one member would change what the operator
+admits. That covers, for example, ``has_member`` or a slot-level ``any_of``
+inside a slot condition, a condition on a name that is not a slot, a condition
+on the identifier slot (the node's IRI rather than a property), and
+``equals_string`` on a slot whose range does not hold strings.
+
+
 Command Line
 ^^^^^^^^^^^^
 

docs/schemas/advanced.md

@@ -64,6 +64,36 @@ The following LinkML constructs can be used to express boolean constraints:
 
 These can be applied at the class or slot level. The range of each of these is an array of *expressions*.
 
+### Class-level expressions and absent slots
+
+At the class level, each member of `any_of`, `all_of`, `exactly_one_of` and `none_of` is a class expression whose `slot_conditions` constrain the instance's slots. The expressions of a class also constrain the instances of its subclasses, and of the classes that use it as a mixin.
+
+A slot condition needs a meaning when its slot is absent. The JSON Schema and SHACL generators read it as an SQL `CHECK` constraint reads a condition on a null value: an instance is invalid only when an expression is definitely false.
+
+- A condition that decides whether its slot may be absent is true or false as usual. Such a condition sets `value_presence: PRESENT` or `ABSENT`, `required: true`, a minimum or exact cardinality of at least 1, or a maximum or exact cardinality of 0.
+- Any other condition is *unknown* when its slot is absent.
+- `any_of`, `all_of` and `none_of` combine these as "or", "and" and "not". An unknown member doesn't make `any_of` true, nor `none_of` false.
+- `exactly_one_of` holds when exactly one member is definitely true.
+
+```yaml
+classes:
+  Sample:
+    none_of:
+      - slot_conditions:
+          status:
+            equals_string: retracted
+```
+
+A `Sample` without `status` is valid: the condition is unknown, so `none_of` isn't false. To require the slot, state it with `status: {required: true}`.
+
+| Expression | `{}` | `{label: A}` | `{label: B}` |
+|---|---|---|---|
+| `any_of: [label = A]` | valid | valid | invalid |
+| `none_of: [label = A]` | valid | invalid | valid |
+| `exactly_one_of: [label = A, note = B]` | invalid | valid | invalid |
+
+[Rules](#rules) are read differently: their preconditions require their slots, and so do their postconditions unless the rule is `open_world`.
+
 ### Unions as ranges
 
 [any_of](https://w3id.org/linkml/any_of) can be used to express that a range must satisfy any of a set of ranges.

packages/linkml/src/linkml/generators/common/class_expression.py

@@ -0,0 +1,150 @@
+"""Shared semantics of slot conditions in class-level boolean expressions.
+
+The class-level operators ``any_of``, ``all_of``, ``exactly_one_of`` and
+``none_of`` combine class expressions whose slot conditions constrain slots of
+an instance.  The specification does not say what a condition means for an
+absent slot.  Generators that translate class expressions read it the same
+way, as an SQL CHECK constraint (ISO/IEC 9075) reads a condition on a null
+value: a condition that doesn't state presence is *unknown* for an absent
+slot, and an expression is violated only when it is definitely false.
+
+Each expression therefore has two forms:
+
+* its **"not false" form**, which the instance must satisfy, and in which a
+  condition holds for an absent slot unless it states presence;
+* its **"definitely true" form**, used under a negation, in which a condition
+  that doesn't state presence also requires its slot.
+
+``none_of`` takes its members in the opposite form, ``exactly_one_of`` counts
+the members that are definitely true, and ``any_of`` / ``all_of`` keep the
+form.
+
+>>> from linkml_runtime.linkml_model.meta import SlotDefinition
+>>> value_bounds(SlotDefinition("label", equals_string="A"), definite=False)
+(0, None)
+>>> value_bounds(SlotDefinition("label", equals_string="A"), definite=True)
+(1, None)
+>>> value_bounds(SlotDefinition("label", required=True, value_presence="ABSENT"), definite=True)
+(0, 0)
+>>> value_bounds(SlotDefinition("tags", minimum_cardinality=2, maximum_cardinality=3), definite=False)
+(2, 3)
+"""
+
+from linkml_runtime.linkml_model.meta import PresenceEnum, SlotDefinition
+from linkml_runtime.utils.schemaview import SchemaView
+
+_PRESENT = PresenceEnum(PresenceEnum.PRESENT)
+_ABSENT = PresenceEnum(PresenceEnum.ABSENT)
+
+
+def states_presence(condition: SlotDefinition) -> bool:
+    """Whether *condition* decides whether its slot may be absent.
+
+    Such a condition is definitely true or false for an absent slot:
+    ``value_presence: PRESENT`` or ``ABSENT``; ``required: true``, unless
+    ``value_presence`` overrides it; a minimum or exact cardinality of at least
+    1, which an absent slot fails; and a maximum or exact cardinality of 0,
+    which it satisfies.  Other bounds, ``required: false`` and ``UNCOMMITTED``
+    leave absence open, so adding one never changes the verdict on an absent
+    slot.
+
+    >>> states_presence(SlotDefinition("label", equals_string="A"))
+    False
+    >>> states_presence(SlotDefinition("label", required=False, equals_string="A"))
+    False
+    >>> states_presence(SlotDefinition("tags", maximum_cardinality=5))
+    False
+    >>> states_presence(SlotDefinition("tags", maximum_cardinality=0))
+    True
+    >>> states_presence(SlotDefinition("tags", minimum_cardinality=1))
+    True
+    """
+    if condition.value_presence is not None:
+        if condition.value_presence in (_PRESENT, _ABSENT):
+            return True
+    elif condition.required:
+        return True
+    lower = (condition.minimum_cardinality, condition.exact_cardinality)
+    upper = (condition.maximum_cardinality, condition.exact_cardinality)
+    return any(bound is not None and bound >= 1 for bound in lower) or 0 in upper
+
+
+def value_bounds(condition: SlotDefinition, definite: bool) -> tuple[int, int | None]:
+    """The least and the greatest number of values *condition* allows its slot.
+
+    ``value_presence`` takes precedence over ``required``.  In the "definitely
+    true" form (*definite*), a condition that doesn't state presence also
+    requires the slot.  The greatest number is ``None`` when unbounded.
+    """
+    lower, upper = [0], []
+    if condition.value_presence is not None:
+        if condition.value_presence == _PRESENT:
+            lower.append(1)
+        elif condition.value_presence == _ABSENT:
+            upper.append(0)
+    elif condition.required:
+        lower.append(1)
+    if definite and not states_presence(condition):
+        lower.append(1)
+    for bound, target in (
+        (condition.minimum_cardinality, lower),
+        (condition.exact_cardinality, lower),
+        (condition.maximum_cardinality, upper),
+        (condition.exact_cardinality, upper),
+    ):
+        if bound is not None:
+            target.append(int(bound))
+    return max(lower), min(upper) if upper else None
+
+
+def class_membership(sv: SchemaView, cls_name: str, target: str) -> bool | None:
+    """Whether an instance of the class *cls_name* is an instance of the class *target*.
+
+    A LinkML instance belongs to its class and to that class's ancestors, by
+    ``is_a`` and by ``mixins``.  So every instance of *cls_name* is one when
+    *target* is *cls_name* or an ancestor (``True``); some may be when *target*
+    is an ancestor of a descendant of *cls_name*, which the instance may belong
+    to (``None``); and none is otherwise (``False``).
+
+    A class carries the expressions of its ancestors, which may name any of
+    these: a ``Person`` carries its parent's ``any_of: [{is_a: Person},
+    {is_a: Organization}]``, whose first member it satisfies and whose second
+    it cannot.  A value of ``linkml:Any`` may be an instance of any class, and
+    every instance is one of ``linkml:Any``.
+
+    >>> sv = SchemaView('''
+    ... id: https://example.org/membership
+    ... name: membership
+    ... classes:
+    ...   Any: {class_uri: linkml:Any}
+    ...   Agent: {}
+    ...   Person: {is_a: Agent}
+    ...   Student: {is_a: Person}
+    ...   Organization: {is_a: Agent}
+    ... ''')
+    >>> class_membership(sv, "Student", "Person")
+    True
+    >>> print(class_membership(sv, "Agent", "Person"))
+    None
+    >>> class_membership(sv, "Person", "Organization")
+    False
+    >>> print(class_membership(sv, "Any", "Person"))
+    None
+    >>> class_membership(sv, "Person", "Any")
+    True
+    """
+    if target in sv.class_ancestors(cls_name) or _is_any(sv, target):
+        return True
+    if _is_any(sv, cls_name):
+        return None
+    if any(target in sv.class_ancestors(descendant) for descendant in sv.class_descendants(cls_name)):
+        return None
+    return False
+
+
+def _is_any(sv: SchemaView, cls_name: str) -> bool:
+    """Whether the class *cls_name* is ``linkml:Any``, the class of every value."""
+    cls = sv.get_class(cls_name)
+    if cls is None or not cls.class_uri:
+        return False
+    return cls.class_uri == "linkml:Any" or sv.expand_curie(cls.class_uri) == "https://w3id.org/linkml/Any"