fix(generators): enforce range_expression on each value alike in SHACL and JSON Schema

docs/generators/json-schema.rst

@@ -180,6 +180,11 @@ A slot condition is unknown for an absent slot unless it decides whether the slo
 may be absent, and an instance is invalid only when an expression is definitely
 false, as described under "Class-level expressions and absent slots" in
 :doc:`Advanced features </schemas/advanced>`.
+A slot's `range_expression <https://w3id.org/linkml/range_expression>`_
+constrains each of its values in the same way, with its conditions on the slots
+of the value. For a slot inlined as a dict these are the values of the dict,
+except in its compact forms ``{id: value}`` and ``{id: null}``, which the
+expression can't reach.
 The SHACL generator reads them the same way.
 
 Inlining

docs/generators/shacl.rst

@@ -106,8 +106,9 @@ then adds nothing; one that only some instances belong to, because a descendant
 of the class has it among its ancestors, and then gives ``sh:class``; or one that
 none belongs to, and then gives a shape nothing conforms to. An instance belongs
 to its class and that class's ancestors, by ``is_a`` and by ``mixins``, so a
-class that may also be another is declared as a subclass with that mixin.
-Nested expressions recurse. Each entry of ``slot_conditions`` gives an
+class that may also be another is declared as a subclass with that mixin. For a
+type or an enum, ``is_a`` gives its value constraint. Nested expressions recurse.
+Each entry of ``slot_conditions`` gives an
 ``sh:property`` whose path is that of the slot as induced for the class, so
 ``slot_usage`` applies:
 
@@ -120,7 +121,9 @@ Nested expressions recurse. Each entry of ``slot_conditions`` gives an
 * ``equals_string`` and ``equals_string_in`` give ``sh:in``; on an enum slot the
   values are the permissible values as the enum renders them, the IRI of their
   ``meaning`` where they have one;
-* ``range`` gives the same class, type or enum constraint as a slot's range.
+* ``range`` gives the same class, type or enum constraint as a slot's range;
+* ``range_expression`` gives an ``sh:node``, as described under
+  `Range Expressions`_.
 
 A shape may have at most one value of ``sh:minInclusive``, ``sh:maxInclusive``
 or ``sh:in``, and of ``sh:pattern``, whose component also takes ``sh:flags``
@@ -170,12 +173,70 @@ slot's range.
 
 An operator whose members use anything else is skipped as a whole and logged as
 a warning, because leaving out one member would change what the operator
-admits. That covers, for example, ``has_member`` or a slot-level ``any_of``
-inside a slot condition, a condition on a name that is not a slot, a condition
+admits. That covers, for example, ``has_member``, ``equals_expression`` or a
+slot-level ``any_of`` inside a slot condition, a condition on a name that is not a slot, a condition
 on the identifier slot (the node's IRI rather than a property), and
 ``equals_string`` on a slot whose range does not hold strings.
 
 
+Range Expressions
+^^^^^^^^^^^^^^^^^
+
+A slot's `range_expression <https://w3id.org/linkml/range_expression>`__
+constrains each of its values as a class-level expression constrains an
+instance: its conditions constrain the slots of the value as induced for the
+range class, and a value violates it only when it is definitely false. It
+becomes an ``sh:node`` (`SHACL §4.7.1
+<https://www.w3.org/TR/shacl/#NodeConstraintComponent>`__) on the property shape,
+next to the range's ``sh:class``, so it constrains only the values of this slot,
+and matching it doesn't make a node an instance of the range. The JSON Schema
+generator reads it the same way.
+
+.. code-block:: yaml
+
+   classes:
+     Document:
+       slots: [license]
+       slot_usage:
+         license:
+           range_expression:
+             slot_conditions:
+               category:
+                 required: true
+                 equals_string: license
+     Resource:
+       slots: [category]
+   slots:
+     license:
+       range: Resource
+     category:
+       range: string
+
+.. code-block:: turtle
+
+   ex:Document a sh:NodeShape ;
+       sh:property [ sh:path ex:license ;
+               sh:class ex:Resource ;
+               sh:node [ sh:property [ sh:path ex:category ;
+                           sh:minCount 1 ;
+                           sh:in ( "license" ) ] ] ;
+               ... ] ;
+       ...
+
+A class alternative in a slot's ``any_of`` can carry its own
+``range_expression``, and so can a slot condition, where the expression takes
+the form of the condition: under ``sh:not``, its "definitely true" form. For
+values that aren't instances of a class, an expression can combine types and
+enums with ``is_a``, for example ``any_of: [{is_a: OpenLicense}, {is_a:
+ProprietaryLicense}]``; such values have no slots for a condition to constrain.
+An expression that uses anything SHACL can't express (see above) is skipped and
+logged as a warning.
+
+A value that is a reference is checked against its description in the data
+graph, as ``sh:class`` is. The JSON Schema generator can't check it, because
+there the value is just an identifier.
+
+
 Command Line
 ^^^^^^^^^^^^
 

docs/schemas/advanced.md

@@ -92,6 +92,8 @@ A `Sample` without `status` is valid: the condition is unknown, so `none_of` isn
 | `none_of: [label = A]` | valid | invalid | valid |
 | `exactly_one_of: [label = A, note = B]` | invalid | valid | invalid |
 
+A slot's [range_expression](https://w3id.org/linkml/range_expression) constrains each value of the slot in the same way: its conditions constrain the slots of the value, as induced for the slot's range class. In a slot condition, a `range_expression` is part of the condition, so under `none_of` it must hold definitely.
+
 [Rules](#rules) are read differently: their preconditions require their slots, and so do their postconditions unless the rule is `open_world`.
 
 ### Unions as ranges

packages/linkml/src/linkml/generators/jsonschemagen.py

@@ -2,7 +2,7 @@
 import json
 import logging
 import os
-from copy import deepcopy
+from copy import copy, deepcopy
 from dataclasses import dataclass, field
 from typing import Any, cast
 
@@ -658,7 +658,7 @@ def handle_class(self, cls: ClassDefinition) -> None:
     _CLASS_OPERATOR_KEYWORDS = {"any_of": "anyOf", "all_of": "allOf", "exactly_one_of": "oneOf"}
 
     def get_subschema_for_class_operator(
-        self, cls: ClassDefinition, operator: str, members: list[AnonymousClassExpression], definite: bool
+        self, cls: ClassDefinition | None, operator: str, members: list[AnonymousClassExpression], definite: bool
     ) -> JsonSchema:
         """The subschema of the class-level boolean expression *operator* over *members*, for *cls*.
 
@@ -684,16 +684,18 @@ def get_subschema_for_class_operator(
         )
 
     def get_subschema_for_class_expression(
-        self, cls: ClassDefinition, expr: AnonymousClassExpression, definite: bool
+        self, cls: ClassDefinition | None, expr: AnonymousClassExpression, definite: bool
     ) -> JsonSchema:
         """The subschema of one member *expr* of a class-level boolean expression of *cls*.
 
         Each slot condition constrains the slot as induced for *cls*, so
-        ``slot_usage`` applies.  Its values must satisfy its value operators
-        and its ``range``; the number of values must lie within
+        ``slot_usage`` applies.  Its values must satisfy its value operators,
+        its ``range`` and its ``range_expression``, the latter in the form of
+        *expr*; the number of values must lie within
         :func:`~linkml.generators.common.class_expression.value_bounds`, where
         an empty list or dict counts as no value.  *definite* selects the form, as in
-        :meth:`get_subschema_for_class_operator`.
+        :meth:`get_subschema_for_class_operator`.  *cls* is ``None`` for values
+        that are not instances of a class.
         """
         subschema = JsonSchema()
         conjuncts: list[JsonSchema] = []
@@ -703,7 +705,17 @@ def get_subschema_for_class_expression(
                 prop_name = self._curie(slot)
             else:
                 prop_name = self.aliased_slot_name(slot)
-            values = self.get_subschema_for_slot(condition, omit_type=True, include_null=False)
+            value_condition = condition
+            if condition.range_expression is not None:
+                # translated below, in the form of *expr*
+                value_condition = copy(condition)
+                value_condition.range_expression = None
+            values = self.get_subschema_for_slot(value_condition, omit_type=True, include_null=False)
+            if condition.range_expression is not None:
+                expression = self.get_subschema_for_range_expression(
+                    condition.range or slot.range, condition.range_expression, definite
+                )
+                values = JsonSchema({"allOf": [values, expression]}) if values else expression
             lower, upper = value_bounds(condition, definite)
             if slot.multivalued and self._inlined_as_dict(slot):
                 # the values of a slot inlined as a dict are the values of the dict
@@ -742,11 +754,17 @@ def get_subschema_for_class_expression(
                 conjuncts.append(self.get_subschema_for_class_operator(cls, operator, members, definite))
         if expr.is_a is not None:
             # `is_a: <C>` requires an instance of the expression to be an instance of <C>.  Every
-            # instance of *cls* is one, or none is, unless <C> is a descendant of *cls*: then, in
-            # JSON, only the instance's structure tells, so it must match the definition of <C>.
-            membership = class_membership(self.schemaview, cls.name, expr.is_a)
+            # instance of *cls* is one, or none is, unless some are through a descendant of *cls*:
+            # then, in JSON, only the instance's structure tells, so it must match the definition
+            # of <C>.  For a type or an enum, or a value that is no class instance, the value
+            # constraint applies.
+            sv = self.schemaview
+            is_class = expr.is_a in sv.all_classes()
+            membership = class_membership(sv, cls.name, expr.is_a) if is_class and cls is not None else None
             if membership is None:
-                conjuncts.append(self.get_subschema_for_slot(AnonymousSlotExpression(range=expr.is_a, inlined=True)))
+                conjuncts.append(
+                    self.get_subschema_for_slot(AnonymousSlotExpression(range=expr.is_a, inlined=is_class))
+                )
             elif not membership:
                 conjuncts.append(JsonSchema({"not": {}}))
         if conjuncts:
@@ -761,13 +779,27 @@ def _inlined_as_dict(self, slot: SlotDefinition) -> bool:
             and get_range_associated_slots(self.schemaview, slot.range)[0] is not None
         )
 
-    def _class_expression_slot(self, cls: ClassDefinition, slot_name: str) -> SlotDefinition | None:
+    def _class_expression_slot(self, cls: ClassDefinition | None, slot_name: str) -> SlotDefinition | None:
         """The slot a condition of a class expression of *cls* names, as induced for *cls*, if any."""
         sv = self.schemaview
-        if slot_name in sv.class_slots(cls.name):
+        if cls is not None and slot_name in sv.class_slots(cls.name):
             return sv.induced_slot(slot_name, cls.name)
         return sv.get_slot(slot_name)
 
+    def get_subschema_for_range_expression(
+        self, value_range: str | None, expression: AnonymousClassExpression, definite: bool
+    ) -> JsonSchema:
+        """The subschema a value of range *value_range* must satisfy under *expression*.
+
+        A ``range_expression`` constrains each value as a class-level expression
+        constrains an instance (:meth:`get_subschema_for_class_expression`): its
+        conditions constrain the slots of the value as induced for the class
+        *value_range*, and *definite* selects the form.
+        """
+        sv = self.schemaview
+        value_class = sv.get_class(value_range) if value_range in sv.all_classes() else None
+        return self.get_subschema_for_class_expression(value_class, expression, definite)
+
     def get_subschema_for_anonymous_class(
         self, cls: AnonymousClassExpression, properties_required: bool = False
     ) -> None | JsonSchema:
@@ -938,7 +970,7 @@ def get_value_constraints_for_slot(self, slot: SlotDefinition | AnonymousSlotExp
             constraints.add_keyword("enum", slot.equals_string_in)
 
         if slot.range_expression:
-            subschema = self.get_subschema_for_anonymous_class(slot.range_expression)
+            subschema = self.get_subschema_for_range_expression(slot.range, slot.range_expression, definite=False)
             if subschema:
                 if "allOf" not in constraints:
                     constraints["allOf"] = []
@@ -990,6 +1022,7 @@ def get_subschema_for_slot(
         slot_is_boolean = any([slot.any_of, slot.all_of, slot.exactly_one_of, slot.none_of])
 
         typ = None
+        inlined_as_dict = False
         if not omit_type:
             typ, fmt, reference = self.get_type_info_for_slot_subschema(slot)
             if slot_is_inlined:
@@ -1003,6 +1036,7 @@ def get_subschema_for_slot(
                     # if the range class has an ID and the slot is not inlined as a list, then we need to consider
                     # various inlined as dict formats
                     if range_id_slot is not None and not slot.inlined_as_list:
+                        inlined_as_dict = True
                         # At a minimum, the inlined dict can have keys (additionalProps) that are IDs
                         # and the values are the range class but possibly omitting the ID.
                         additionalProps = [JsonSchema.ref_for(reference, identifier_optional=True)]
@@ -1078,6 +1112,10 @@ def get_subschema_for_slot(
             prop["items"].update(all_element_constraints)
             if any_element_constraints:
                 prop["contains"] = any_element_constraints
+        elif inlined_as_dict:
+            # The values of the slot are the values of the dict.
+            if own_constraints:
+                prop["additionalProperties"] = JsonSchema({"allOf": [prop["additionalProperties"], own_constraints]})
         else:
             prop.update(own_constraints)