Repository navigation
Remove the deprecated nwis module - #457
Draft
thodson-usgs wants to merge 3 commits into
Draft
thodson-usgs wants to merge 3 commits into
thodson-usgs wants to merge 3 commits into
Conversation
Remove dataretrieval.nwis ahead of its 2027-05-06 horizon. USGS is decommissioning the waterservices endpoints it called, and every getter has a waterdata (or nwdc) replacement. - Delete dataretrieval/nwis.py, its tests and fixtures, and its reference page. - Remove NoSitesError (only the waterservices path raised it) and the "No sites/data" body check in _querying; remove REMOVALS["nwis"] and the now-unused _validation.require_any_of. - Drop nwis from .importlinter (layers + nwis-quarantine contract), pyproject, pre-commit comments, and the pyscn clone baseline. - Port demos: the defunct hydroshare water-use notebook is rewritten on nwdc.get_wateruse (USGS_NWDC_WaterUse_Examples); the nwqn_data_pull scripts use waterdata.get_monitoring_locations / get_daily; copy-edit notebooks that still described nwis as current. - Docs: add missing nblinks for USGS_WaterUse_Examples and USGS_NLDI_Examples; update README, NEWS, CONTEXT, AGENTS, CONTRIBUTING, the architecture overview, and mark ADR 0005 completed. BREAKING CHANGE: `import dataretrieval.nwis` raises ModuleNotFoundError, and dataretrieval.NoSitesError / exceptions.NoSitesError are removed.
Prose only (Markdown, notebook Markdown cells, docstrings, and comments); no identifiers, runtime strings, or doctest blocks changed. - 'while' for contrast -> whereas/although (STA8 p. 60); 'since' for cause -> because (p. 45); 'data' plural (p. 48); 'and/or' -> 'or both' (p. 44) - Hyphenate unit modifiers: water-quality, surface-water (p. 28) - Numbers: thousands separators, 'from X to Y' ranges, words for isolated small counts (pp. 145-146) - State names spelled out at first use and when standing alone (p. 127); 'State' capitalized for a U.S. State and 'U.S.' as an adjective (GPO) - Fix typos in notebooks and docstrings
Prose only; no identifiers, runtime strings, or doctest blocks changed.
- Capitalize 'State' consistently for a U.S. State, including
attributive uses ('State code', 'State FIPS'); lowercase for Mexican
states and Canadian provinces
- More 'since'/'while' -> because/whereas (STA8 pp. 45, 60); 'data'
plural (p. 48)
- Commas: before 'whereas' and nonrestrictive 'which' (pp. 22, 35),
serial comma (p. 26), after the year in a full date (p. 145)
- Related numbers in figures (p. 146); 'that is' for i.e. in running
text (p. 125); water-quality unit modifier (p. 28)
- American spelling (behavior, judgment, neighboring), 'gage', Water
Quality Portal capitalization, and a garbled sentence in
configuration_test
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This removes
dataretrieval.nwisbefore its published 2027-05-06 horizon, as part of the next major release. USGS is decommissioning the waterservices endpoints the module called, and every getter in it already has awaterdataornwdcreplacement.Breaking:
import dataretrieval.nwisnow raisesModuleNotFoundError.NoSitesErroris removed, bothdataretrieval.NoSitesErrorandexceptions.NoSitesError. Only the waterservices path raised it. The modern getters return an empty DataFrame when nothing matches.Code
nwis.py,tests/nwis_test.py,tests/waterservices_test.py, and 9 fixtures used only by those tests.REMOVALS["nwis"], the"No sites/data"body check in_querying, and_validation.require_any_of, whose only caller was nwis..importlinter: dropped nwis from the layers contract and deleted thenwis-quarantinecontract. Also updated the pyproject comment, the pre-commit comments, and the pyscn clone baseline.utils.query, the deprecation mechanism) so they call that code directly.rdb.read_rdb's HTML-response message used to tell callers to migrate to waterdata. Onlywaterdata.get_ratingsreaches it now, so the message asks them to report the moved endpoint instead.Notebooks and demos
hydroshare/USGS_NWIS_WaterUse_Examples.ipynbwas defunct and couldn't run. It is rewritten onnwdc.get_wateruseand renamedUSGS_NWDC_WaterUse_Examples.ipynb.nwqn_data_pullscripts now usewaterdata.get_monitoring_locationsandget_daily. Approval filtering happens on the server (approval_status="Approved") instead of matching the old NWIS_cdstrings.USGS_WaterUse_Examplesandhydroshare/USGS_NLDI_Examples. Updatedexamples/index.rst.Prose
userguide/dataportals.rst.Other deprecations evaluated
None of the other published horizons is earlier than May 2027, so they all stay. They are listed in the NEWS entry:
get_time_series_metadatav0 filtersget_cql(service=)waterusealiasogc.interruptionsThe camelCase
get_sampleskwargs have no horizon published.Remaining "NWIS" mentions (intentional)
providers=NWIS, NLDInwissite/nwisgw, NGWMNorganization_type="NWIS", the USGS NWIS time-zone code list, and docstrings saying site numbers are assigned by NWIS.readNWIS*calls quoted in the R-vignette notebook comments.Testing
ruff check,ruff format --check,lint-imports(7 kept),xenon, andcomplexipyall pass, as do the pre-commit hooks.pytest: 1650 passed. Branch coverage is 99.07%, abovefail_under98.9.mypy --python-version 3.12is clean. Locally, the default 3.10 target fails only inside numpy's installed stub (typestatement), which is unrelated to this change. CI should confirm.make htmlsucceeds and the executed notebooks render. The only cell errors areNetworkErrors on WQP calls inrvignettesandDiscreteSamples, caused by a local SSL problem that also reproduces with a barehttpx.get.retrieve_nwqn_streamflow.map_retrievalon one site with lithops and S3 stubbed out.