Repository navigation
[python] Repro: property access detaches container returned by __getitem__ - #12359
Draft
Josiah Vinson (jovinson-ms) wants to merge 1 commit into
Conversation
… container Add failing unit tests showing that a list/dict obtained via obj["key"] is detached from the model once the attribute is accessed, because _RestField.__get__ replaces obj._data[key] with a newly deserialized container. Also add a passing guard for property-then-mutate. Co-authored-by: Copilot App <[email protected]> Copilot-Session: beceb370-8e32-49c3-9976-c8b3fb3619cf
commit: |
Josiah Vinson (jovinson-ms)
pushed a commit
to jovinson-ms/azure-sdk-for-python
that referenced
this pull request
Oct 22, 2026
Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849
Contributor
|
All changed packages have been documented.
Show changes
|
4 tasks done
Contributor
Python emitter diffBaseline No changes to generated output. Rendered diff: inline on the run summary, or the emitter-diff-html artifact. Informational check (eng/emitter-diff); does not block the PR. |
|
You can try these changes here
|
Josiah Vinson (jovinson-ms)
added a commit
to Azure/azure-sdk-for-python
that referenced
this pull request
Oct 26, 2026
…ors, add test-resources readiness probe (#49627) * [Health Deidentification] Remove stale generated files and regenerate Fixes pyright reportArgumentType (issubclass) and pylint C0123 failures in the healthdataaiservices CI pipeline. - Delete stale _serialization.py, _model_base.py, _vendor.py, aio/_vendor.py left over from an older emitter (only referenced each other). - Point tsp-location.yaml at the relocated spec (specification/healthdataaiservices/data-plane/HealthDataAIServices @ c7d9e33) and regenerate with @azure-tools/typespec-python 0.63.10. No API changes. - Emitter moves packaging to pyproject.toml (Python >=3.10, azure-core >=1.37.0) and adds types.py; bump to 1.1.0b2 and note it in CHANGELOG/README. - Tests: exclude Accept header from matching (regenerated delete_job no longer sends it) and disable the background LRO poller in the sync create/cancel/delete test, which otherwise races the test's own GETs in playback. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [Health Deidentification] Note new types module in CHANGELOG Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * Opt out of TypedDict generation for azure-health-deidentification Regenerated with generate-typeddict: false so the JSON overloads match 1.1.0b1 and the inaccurate request-body TypedDicts (types.py) are removed. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [Health Deid] Replace fixed sleep with retry/backoff in test-resources-post.ps1 Role assignment and network ACL propagation can take minutes after deployment, causing intermittent 403 AuthorizationFailure on upload. Retry container access and each blob upload with exponential backoff (5s initial, 60s cap, 5 min budget) and fail with a clear error. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 0d9b0092-996f-4265-b2da-1effbf3945c6 * [Health Deid] Keep a 30s minimum wait after the network rule update The retry loop proves the test principal's RBAC and the network rule, but not the DeID managed identity's role assignment. Restore the previous 30s wait as a minimum measured from the network rule update; time spent on retries and uploads counts toward it. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 0d9b0092-996f-4265-b2da-1effbf3945c6 * [DeID] Surface job errors in live test failures; random live job names - Add wait_for_job/wait_for_job_async test helpers that report job status, error, summary, and unsuccessful documents instead of azure-core's generic 'Operation returned an invalid status OK', and fail explicitly when the job is not terminal after the timeout (now 600s). - Generate a fresh uniquifier for every live run (including AZURE_SKIP_LIVE_RECORDING); only persist it when recording. - Samples print job.error before re-raising on job failure. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 40cdcc81-accc-4b2e-af24-fd79c6480740 * [DeID] Sync README snippet with deidentify_documents sample Co-authored-by: Copilot App <[email protected]> Copilot-Session: 40cdcc81-accc-4b2e-af24-fd79c6480740 * [DeID] Make sample job diagnostics best-effort; always close async credentials The diagnostic get_job lookup in the samples' error handling could raise (for example 404 when creation was rejected) and replace the original error. It is now wrapped so the original exception is always re-raised. The async samples close the credential via async with. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [DeID] Point tsp-location at spec commit with generate-typeddict: false Spec PR Azure/azure-rest-api-specs#47496 adds generate-typeddict: false to the Python emitter options. Regenerating from that commit produces no changes to the generated code. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [DeID] Point tsp-location at merged spec commit Azure/azure-rest-api-specs#47496 merged as 6de3840. Regenerating from it produces no changes to the generated code. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [DeID] Keep job diagnostics when document listing fails describe_job(_async) now fetches the job and lists documents in separate try blocks, so a listing failure (even on a later page) no longer discards the job status/error or documents already collected; the listing error is appended instead. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [DeID] Preserve model container identity across dict and property access The generated _RestField.__get__ replaces the backing list/dict/set in _data when a mutable field is first read as a property, detaching any reference previously returned by model["field"]. Wrap __get__ in patch_sdk() to copy the deserialized contents into the original container instead, until the emitter template is fixed. Adds unit tests for all dict/property access patterns plus a guard that fails once the generated code no longer needs the shim. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [DeID] Link microsoft/typespec#12359 as the shim removal tracker Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 * [DeID] Fix cspell failure in container-identity shim comment The shim's TODO quoted the emitter template's repository path, which contains "pygen" and fails the cspell check. Describe the template by name instead. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 --------- Co-authored-by: Josiah Vinson <[email protected]> Co-authored-by: Copilot App <[email protected]> Copilot-Session: ef2497f1-4ec3-4283-9448-f9679c8b7849 Copilot-Session: 0d9b0092-996f-4265-b2da-1effbf3945c6 Copilot-Session: 40cdcc81-accc-4b2e-af24-fd79c6480740
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In the generated model runtime (
packages/http-client-python/generator/pygen/codegen/templates/model_base.py.jinja2),_RestField.__get__deserializes list/dict/set values into a new container, writes that new container back intoobj._data[rest_name], and sets_deserialized_<rest_name>. Any container a caller already got throughobj["rest_name"]is now detached from_data, so mutations made through it are silently dropped when the model is serialized.Repro
['.txt', '.csv']. Dict access and property access share one backing container.['.txt']. The append is lost.Observed with
@azure-tools/typespec-python0.63.10 /@typespec/http-client-python0.39.0.What this PR adds (tests only, no fix)
In
tests/unit/test_model_base_serialization.py:test_list_getitem_reference_survives_attribute_access: fails (list[str], the case above)test_dict_getitem_reference_survives_attribute_access: fails (dict[str, str] variant)test_list_attribute_reference_mutation_serializes: passes. Guard: property access first, then mutate, still serializes correctly and must keep working after a fix.The failing tests are intentionally left plainly failing (not
xfail). This package's tests have noxfailconvention, and the goal of this draft is to show a red test that the fix turns green. If maintainers prefer, they can be switched to@pytest.mark.xfail(strict=True, reason=...).A
.chronuschangeset withchangeKind: internalis included becausepackages/http-client-python/tests/**is not excluded bychangedFiles.Local run
model_base.py.jinja2was rendered (unbranded,corehttp) into a temporaryspecialwords._utils.model_basepackage.unit/test_model_base_serialization.pywas then run with--noconftest, because its conftest auto-starts the Spector mock server:The guard test and all existing tests in that file pass.
Likely location and suggested fix direction
_RestField.__get__inmodel_base.py.jinja2:Suggested direction:
deserialized == itemand same type), keep the existing_datacontainer (return item) instead of replacing it.item[:] = deserializedfor lists,item.clear(); item.update(deserialized)for dicts/sets.Found during review of Azure/azure-sdk-for-python#49627.
Created by Copilot on behalf of Josiah Vinson (Josiah Vinson (@jovinson-ms))