hermes plugin: `observation_scopes: "shared"` is silently rejected — the engine implements it and the docs recommend it

[edetterman]

Summary

The Hermes integration plugin rejects observation_scopes: "shared" — the value your own engine implements for exactly this case, and the value the integration docs tell users to set. It normalizes the unknown keyword to None, which is the combined default, so the setting becomes a silent no-op and the user gets the fan-out they were trying to fix with no error anywhere.

Versions

• hindsight-client / API 0.10.1
• Hermes integration plugin (hindsight-integrations/hermes) 1.0.1

Where

settings.py:

_OBSERVATION_SCOPE_KEYWORDS = {"per_tag", "combined", "all_combinations"}

_normalize_observation_scopes returns None for any string not in that set:

if isinstance(value, str):
    text = value.strip()
    if text in _OBSERVATION_SCOPE_KEYWORDS:
        return text
    if text.startswith("["):
        ...
    return None          # <- "shared" lands here

and the provider only forwards the field when it is truthy, so the engine never sees it:

item.update({k: v for k, v in (("tags", merged_tags),
                               ("observation_scopes", self._observation_scopes)) if v})

The engine does support it — hindsight_api/engine/consolidation/consolidator.py:

if parsed == "shared":
    return [[]]

"""``shared`` resolves to ``[[]]`` — a single pass over the empty (untagged)
scope. ... Use it to deduplicate across volatile per-call provenance tags
(e.g. per-session ids) without dropping those tags from the source facts."""

Expected

observation_scopes: "shared" reaches the server and consolidates every memory into one untagged observation scope.

Actual

It is silently dropped. The engine falls back to combined, and in combined the scope is the memory's own tag set (combined / None -> [frozenset(memory.tags)]). The Hermes plugin retains each document with tags: ["session:<session_id>"], so on a retain-per-session workload every session becomes its own dedup scope and no two sessions' observations are ever compared.

Impact, measured

Bank of 4043 documents / 38518 memory units, one single-topic recall query (observations only, budget mid, max_tokens 4096):

• 79 results across 33 distinct scopes, every observation tagged with exactly one session:*
• 42 of the 79 were near-duplicates of an earlier result (token-set overlap ≥ 0.7)
• 0 duplicated text within a single observation

Identical retains from three sessions consolidate to 3 observations with the default, and to 1 when one scope is requested — verified both at the API level and through the plugin's own retain path.

Workaround

["shared"] (a flat tag list, i.e. one explicit scope) survives normalization and unifies the scopes, but it tags the resulting observations shared, whereas the engine's own shared keyword produces an untagged observation. Users following the docs get the silent no-op instead.

Related

There is also no supported way to re-scope observations that are already stored: POST /v1/default/banks/{bank_id}/consolidate "only processes unconsolidated memories", and a memory's observation_scopes is fixed at retain time with no update endpoint. So a bank that accumulated per-session fan-out can only be fixed forward, never repaired in place. That is a second, separate gap — happy to file it separately if you'd prefer.