Skip to content

fix: gaps in versioned objects for dataset objects - #9510

Open
pjwerneck wants to merge 5 commits into
devfrom
pjwerneck/migrations-fix-versioned-dataset-objects
Open

pjwerneck wants to merge 5 commits into
devfrom
pjwerneck/migrations-fix-versioned-dataset-objects

Conversation

@pjwerneck

Copy link
Copy Markdown
Collaborator

Summary

Two places still handled the serialization of private_metadata.yaml manually instead of going through
the protocol codec, so both ignored the dataset's protocol version. Both now call the storage layer.

Changes

  • The peer's copy of private_metadata.yaml is now serialized through the codec.
  • Dataset.private_config goes through storage.

Testing

Asana task

https://app.asana.com/1/1185126988600652/project/1210542925864934/task/1216207166877921?focus=true

get_private_dataset_files rebuilt the private config from disk_dict() and
yaml.safe_dump, skipping migrate_to_schema and the codec. A protocol-0 peer
then got canonical_name/version, which ProtocolCodecV0 strips on disk.

Route that path through DatasetStorage.wire_bytes so wire bytes match a
protocol-aware write, and test that protocol-0 wire bytes have no identity
fields.
Dataset.private_config yaml-loaded the file and built PrivateDatasetConfigV1,
skipping codec.read and migrate-to-latest. A V2 file would have come back as
V1 forever.

Read it through datasetStorage.read_private_config, and test that
a V1 file upgrades when a throwaway V2 is registered.
@pjwerneck
pjwerneck marked this pull request as ready for review September 9, 2026 09:52
@pjwerneck
pjwerneck requested a review from rasswanth-s October 9, 2026 11:32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant