Skip to content

python: select compatible coverage defaults and report install failures - #28

Closed
anmarchenko wants to merge 1 commit into
mainfrom
anmarchenko/python-dependency-compatibility
Closed

anmarchenko wants to merge 1 commit into
mainfrom
anmarchenko/python-dependency-compatibility

Conversation

@anmarchenko

@anmarchenko anmarchenko commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Fixes Python 3.9 installation when coverage is omitted and reports the dependency that actually failed. Default coverage becomes 7.10.7 on Python 3.9; standalone installations retain 7.13.5 on newer interpreters. A new DD_DEFAULT_COVERAGE_VERSION_PYTHON lets the GitHub Action supply its own release pin for newer Python without treating it as an explicit user override.

An explicit DD_SET_COVERAGE_VERSION_PYTHON always wins. Requesting 7.16.1 on Python 3.9 exits nonzero with Could not install coverage==7.16.1 for Python 3.9.16, retains pip's diagnostics, and emits no Python instrumentation environment variables or shared success flags.

The patch also:

  • Uses the selected virtual environment's python -m pip, separates tracer and coverage installation errors, and preserves full package paths containing spaces.
  • Runs Python setup in a function subshell so failure does not leave the caller in the instrumentation environment.
  • Emits shared enable flags after all requested language installations finish successfully.
  • Adds deterministic failure/default regression tests and a real Python 3.9–3.14 pytest/coverage matrix.
  • Proposes installer artifact v16, preserving v15 for existing checksum-pinned consumers.

Motivation

An uncoached HTTPX onboarding trial using datadog/test-visibility-github-action@v3 failed before its test planner could start. The action passed ddtrace==4.15.2 and coverage==7.16.1 to this installer's combined pip invocation. Pip correctly rejected coverage, but the installer then reported Could not install ddtrace for Python.

Failed Python 3.9 job.

The investigation confirmed:

Dependency Requires-Python Consequence
ddtrace 4.15.2 >=3.9,<3.15 Python 3.9 is supported; a cp39 wheel exists.
coverage 7.16.1 >=3.10 The action's universal coverage pin breaks Python 3.9.
coverage 7.10.7 >=3.9 Final release before Python 3.9 support was dropped.

Coverage's 7.11.0 changelog explicitly records dropping Python 3.9. The compatibility pin is therefore 7.10.7, rather than the trial's known 7.10.6 workaround or an unconditional downgrade for every interpreter.

Downloaded v15 bytes matched this repository's base 1f1cbed10074948885396f9938f95add39a63b84 and the action checksum e64a4a45e0b171f4294b9f949f7fc99946192f952a2d463c61fad3cb8b956a58.

Additional Notes

Coordinated action PR: DataDog/test-visibility-github-action#73

The action proposal preserves ddtrace 4.15.2 and coverage 7.16.1 for Python 3.10–3.14. Its omitted coverage input forwards an empty explicit override and supplies the modern pin through the new default variable. This installer owns the frozen Python 3.9 compatibility boundary; normal action library bumps must not replace it.

Release sequence:

  1. Merge and validate this installer change, then publish v16 through the existing manual GitLab deployment job.
  2. Download the published script and verify its bytes against the reviewed source. The current proposed SHA256 is be1de997fab7862640146b6836c96cbf13f4e9fa577cb501e80c37c5a5c60768.
  3. Verify/update the action's URL and checksum together, run its hosted Python matrix, and release the action.

The proposed v16 URL returned HTTP 404 on 7 October 2026. The local checksum is provisional until matching bytes are deployed and verified. If source changes, recompute it. Do not overwrite v15: immutable action releases depend on those existing bytes and their checksum. This PR does not deploy or publish the artifact.

Possible Drawbacks / Trade-offs

The fixed Python 3.9 pin intentionally stops receiving newer coverage releases because those releases no longer support the interpreter. Future changes to supported Python versions require reviewing the defaults and running the interpreter matrix. Explicit incompatible overrides fail rather than silently selecting another version or disabling instrumentation.

Shared enable flags now appear after language-specific outputs. A multi-language failure still exits nonzero and may leave earlier successful language outputs, but does not advertise overall enablement. The existing Go decision matrix passes with this ordering change.

Windows was excluded from this investigation. After opening this PR, the hosted Ubuntu Python checks passed on all six versions, 3.9–3.14, and the Go matrices passed on Ubuntu and macOS. Datadog event delivery and the original hosted HTTPX action run remain unverified. HTTPX's Twine metadata and trio warning failures, and hidden .coverage.* upload handling, are separate issues outside this change.

Describe how to test/QA your changes

Completed locally on macOS arm64, with fresh application and instrumentation virtual environments:

Python ddtrace Coverage default Instrumented pytest fixture Statement/branch coverage
3.9.16 4.15.2 7.10.7 2 passed 100%
3.10.16 4.15.2 7.16.1 supplied by action 2 passed 100%
3.11.17 4.15.2 7.16.1 supplied by action 2 passed 100%
3.12.9 4.15.2 7.16.1 supplied by action 2 passed 100%
3.13.15 4.15.2 7.16.1 supplied by action 2 passed 100%
3.14.8 4.15.2 7.16.1 supplied by action 2 passed 100%

Each runtime fixture asserts that the active Datadog TestOptPlugin is registered. A dummy API key and closed localhost tracing endpoint were used; this validates active instrumentation without claiming backend delivery. The 100% result applies only to this small branch fixture, not HTTPX's application suite.

Additional results:

  • Python 3.9 explicit coverage 7.10.6 stayed exact, ran 2 instrumented tests, and produced 100% statement/branch coverage.
  • Python 3.9 explicit 7.16.1 failed with a coverage-specific diagnostic and no instrumentation environment output.
  • 7 deterministic tests passed, covering interpreter defaults, explicit overrides, standalone modern pin, coverage/tracer/venv failures, and package paths with spaces.
  • 16 existing Go regression scenarios passed on final source.
  • Bash syntax, ShellCheck at error severity, and workflow YAML parsing passed.
  • The actual action download/run shell step was replayed against these checksum-verified local bytes on Python 3.9 and 3.14; its incompatible Python 3.9 override left GITHUB_ENV empty. Download and optional /github/workspace creation were stubbed; this was not a hosted action run.

Reproduce on the desired interpreter:

python -B -m unittest discover -s tests -p test_python_install.py -v
tests/run_python_smoke.sh /absolute/path/to/python
tests/run_go_matrix.sh

The new hosted installer workflow covers Python 3.9–3.14 directly from source and has passed on all six interpreters. The action matrix must run against the published artifact after v16 deployment.

@anmarchenko
anmarchenko requested a review from a team as a code owner October 7, 2026 16:11
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T16:12:48.731800Z 33a90d6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static validation supports the Python 3.9 coverage default, explicit-override precedence, and dependency-specific failure handling. Real-package compatibility across Python 3.9–3.14 remains unverified in this review.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 33a90d6 · @DataDog review to ask questions

@anmarchenko anmarchenko closed this Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant