Repository navigation
Conversation
Also: - fixed role change text - fixed remove all roles - allow intervals and copying from same column in aliased table
You can't copy or anchor from nullable (or via nullable) to non-nullable. You can't copy from an FK to an FK that points to a different table. Bugs fixed.
A subdirectory of parquet files (e.g. Hive-partitioned yr=2023/p00001.parquet) is now one table named after the directory, instead of one table per part file. All files in a directory must share columns, types and partition keys, otherwise make-tables exits with an error recommending the data be reorganised. Partition keys become TEXT columns. DuckDB sees each dataset through a temporary view so source databases are not modified. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Differing column names or partition keys remain a hard error. Differing column types are now a single warning per column, using the first file's type, and the comparison ignores case so int32 equals Int32. Pandas nullable types such as Int32 now map to the SQL type of their numpy dtype. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
parquet_dir is read back from orm.yaml as text, so create_db_engine and find_parquet_tables now declare Path | str. Drops the mypy-style type: ignore in the test, which ty does not honour. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Read partitioned parquet directories as single tables
Add column-level evaluation layer and rank proposals in 'propose' command
…into copy_proposer
Collaborator
Author
|
This is set to merge into the wrong branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #130
Also adds a copy proposer that allows the copying of a column that has a "source" role. This can be used for all the OMOP person_id columns that should all copy from each other.