Skip to content

Restore qualified Cython module name in out-of-tree builds - #534

Open
bocklund wants to merge 1 commit into
symengine:mainfrom
bocklund:fix-out-of-tree-module-name
Open

bocklund wants to merge 1 commit into
symengine:mainfrom
bocklund:fix-out-of-tree-module-name

Conversation

@bocklund

Copy link
Copy Markdown

Cython infers a module's qualified name from the structure implied by __init__.py. Old CMake build copied the whole symengine/ package into the build tree (file(COPY symengine/ ...) in the top-level CMakeLists.txt), so those files were present.

Since the move to scikit-build-core builds compile the wrapper with the module name symengine_wrapper instead of symengine.lib.symengine_wrapper and Symbol('x').__module__ == "symengine_wrapper" that breaks pickles across process.

This only occurs on non-editable installs and builds outside the source tree that have __init__.py files in the right structure and my proposed fix is to create dummy __init__.py files to produce the correct import structure if they don't already exist (which they do for in-tree builds).

To reproduce (before the fix):

# create.py
import pickle
import symengine
with open('x.pkl', 'wb') as fp:
  fp.write(pickle.dumps(symengine.Symbol('x')))
# read.py
import pickle
with open("x.pkl", "rb") as fp:
  print(pickle.load(fp))

Running

> python create.py
> python read.py
Traceback (most recent call last):
  File "/.../read.py", line 3, in <module>
    print(pickle.load(fp))
          ~~~~~~~~~~~^^^^
ModuleNotFoundError: No module named 'symengine_wrapper'

I'm not sure how to test this in CI since CI does in-tree builds and the module name resolves correctly.

It'd be great to get this in before 0.15 since we depend on this for PyCalphad and have been stuck pinned <0.14 for awhile

Cython infers a module's qualified name from the structure implied by `__init__.py`. Old CMake build copied the whole symengine/ package into the build tree (`file(COPY symengine/ ...)` in the top-level CMakeLists.txt), so those files were present.

Since the move to scikit-build-core builds compile the wrapper with the module name `symengine_wrapper` instead of `symengine.lib.symengine_wrapper` and `Symbol('x').__module__ == "symengine_wrapper"` that breaks pickles across process.

This only occurs on non-editable installs and builds outside the source tree that have `__init__.py` files in the right structure and my proposed fix is to create dummy `__init__.py` files to produce the correct import structure if they don't already exist (which they do for in-tree builds).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant