Skip to content

request_headers for downloads; DuckDB converter downloads archives also without a cache - #63

Merged
m-mohr merged 4 commits into
mainfrom
duckdb-temp-download
Oct 23, 2026
Merged

m-mohr merged 4 commits into
mainfrom
duckdb-temp-download

Conversation

@ivorbosloper

Copy link
Copy Markdown
Collaborator

Two converter improvements that came up in fiboa/cli#345, where the JP converter had to override both.

  • request_headers on a converter is sent with every download, next to avoid_range_request. Some servers refuse clients that do not look like a browser.
  • Without a cache, the DuckDB converter used to hand every source to DuckDB, which can only read parquet in place. Sources it cannot read, such as archives, are now downloaded to a temporary folder first, as the GeoDataFrame-based converter already does. Remote parquet without a cache is still read in place.

🤖 Generated with Claude Code

@m-mohr
m-mohr requested a balanced review from Copilot October 23, 2026 18:28

This comment was marked as resolved.

ivorbosloper and others added 2 commits October 23, 2026 22:42
# Conflicts:
#	CHANGELOG.md
#	tests/test_convert_duckdb.py
…geoparquet URLs in place

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This comment was marked as resolved.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused changes address prior feedback, include targeted regression coverage, and have no unresolved blocking findings.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@m-mohr
m-mohr merged commit 963e580 into main Oct 23, 2026
8 checks passed
@m-mohr
m-mohr deleted the duckdb-temp-download branch October 23, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants