[srimanachanta]

Closes #3256.

Summary

A pure-Python package cannot give host_launch a GIL-free host function: a ctypes callback built from Python takes the GIL, and libc memcpy does not match CUhostFn. This PR adds one in cuda.core, as agreed in #3256.

• cuda.core.HOST_COPY_FN: a ctypes CUhostFn pointer to a noexcept nogil function that copies nbytes from src to dst. It works as fn in host_launch, the graph callback methods, and update().
• cuda.core.pack_host_copy_args(dst, src, nbytes) -> bytes: packs the record for user_data. It accepts ints, Buffer, and buffer-protocol objects, and checks bounds, overlap, null addresses, and the host accessibility of a Buffer's memory resource. It makes no driver calls, so a Python host callback can use it. dst and src are not retained.

The record layout is public: uintptr_t dst, uintptr_t src, size_t nbytes, the same as struct.pack("@PPN", ...). A caller can pre-pack records and pass an address as an int user_data. The host callback resolver rejects HOST_COPY_FN with no user_data, a zero address, or bytes of the wrong length, so a bad call raises instead of crashing the driver thread.

The HOST_COPY_FN docs note that the driver may run all host functions of a context on one thread, so a long copy delays other host functions, including Python callbacks. They recommend chunks of a few MiB.

Changes

• cuda_core/cuda/core/_host_callback.pyx: the copy function, HOST_COPY_FN, pack_host_copy_args, and the resolver check. Exported from cuda.core.
• cuda_core/docs/source/api.rst: HOST_COPY_FN and pack_host_copy_args. The host_launch and graph callback docstrings point to them. A release-note entry for 1.3.0.
• cuda_core/examples/host_copy.py: a chunked device-to-NumPy copy through two pinned slots, one stream per slot.
• cuda_core/tests/test_host_copy.py: stream ordering, int user_data, graph replay, zero-byte copies, the packed layout, and each validation error.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/cuda-python
• Branch: main
• File: .coderabbit.yaml
• Commit: 01037b1
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 5132e545-1e90-4bf6-b3da-06a4e1f3b17f

Commits

Reviewing files that changed from the base of the PR and between 2794230 and 73897d7.

Files selected for processing: 7

• cuda_core/cuda/core/_host_callback.pyi
• cuda_core/cuda/core/_host_callback.pyx
• cuda_core/cuda/core/_launcher.pyi
• cuda_core/cuda/core/_launcher.pyx
• cuda_core/docs/source/api.rst
• cuda_core/docs/source/examples.rst
• cuda_core/docs/source/release/1.3.0-notes.rst

Files skipped from review as they are similar to previous changes: 2

• cuda_core/docs/source/examples.rst
• cuda_core/docs/source/release/1.3.0-notes.rst

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Summary

Summary by CodeRabbit

• New Features
  • Added a host-copy callback and argument-packing helper for copying data through host_launch and graph host callbacks without requiring a compiled extension.
  • Added an example showing chunked transfers from a device buffer into a NumPy array using pinned staging buffers.
• Documentation
  • Expanded API guidance on callback arguments, validation errors, and how long host copies may affect other callbacks.

Adds HOST_COPY_FN, a GIL-free host-memory copy callback, and pack_host_copy_args for creating its user-data record. Exports both from cuda.core and documents their use with host launch and graph callbacks.

Changes: Host callback and copy support

Layer / File(s)	Summary
Copy callback and argument packing cuda_core/cuda/core/_host_callback.pyx, cuda_core/cuda/core/_host_callback.pyi, cuda_core/tests/test_host_copy.py	Adds the callback pointer and native-width argument packer. The packer accepts integer addresses and host buffers, checks buffer properties and address ranges, and raises documented errors for invalid inputs. Tests cover record layout and input validation.
Host launch and graph callback integration cuda_core/cuda/core/__init__.py, cuda_core/cuda/core/_launcher.*, cuda_core/cuda/core/graph/*, cuda_core/docs/source/api.rst	Exports the callback and packer. Host launch and graph callback code use the shared resolver, and their documentation describes the packed-record requirement for HOST_COPY_FN.
Copy execution and example cuda_core/tests/test_host_copy.py, cuda_core/examples/host_copy.py, cuda_core/docs/source/examples.rst, cuda_core/docs/source/release/1.3.0-notes.rst	Tests cover launch ordering, graph replay, and zero-byte copies. The example copies device data into a NumPy array in chunks using two pinned staging slots. The example index and release notes describe the new APIs.

Assessment against linked issues

Objective	Addressed	Explanation
Provide a CUhostFn-signature host-copy function that runs without the GIL and copies dst, src, and nbytes from packed user data. [#3256]	✅
Expose the callback as a function pointer and provide a helper to pack arguments for host launch and graph callbacks. [#3256]	✅

Possibly related PRs

• NVIDIA/cuda-python#3258: Adds the host-launch path that this change uses for HOST_COPY_FN.
• NVIDIA/cuda-python#2925: Covers callback contracts reused by this change, and the PR relocates the shared graph callback resolver.

Suggested labels: feature

Suggested reviewers: andy-jost

Change: Feature

Merge Risk: Minimal · up to 73897

No identified issue currently prevents merging after normal checks.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

[Andy-Jost]

/ok to test b11ec0f

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-3269/
https://nvidia.github.io/cuda-python/pr-preview/pr-3269/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-3269/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-3269/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[Andy-Jost]

Thanks, this implements #3256 as agreed. I reviewed only the three commits on top of #3258. _host_copy is noexcept nogil, reads the documented 24-byte @PPN record, and skips memcpy for zero bytes; _resolve_host_callback rejects None, 0, empty, and wrong-length records before it allocates, and the packed bytes ride the existing user_data ownership on the stream, builder, definition, and update() paths. pack_host_copy_args makes no driver call on any built-in path: it reads _memory_resource.is_host_accessible directly and so avoids the cuPointerGetAttributes that Buffer.is_host_accessible falls back to. The example orders its pinned slots correctly. Since the branch carries #3258, it merges after #3258.

Three changes before merge:

1. The Notes of pack_host_copy_args say a Python host callback can use it "to enqueue the next copy". Enqueueing is a CUDA call, which host_launch forbids inside a host function. Say instead that a host callback can call it, for example to fill a record that a later HOST_COPY_FN reads through an int user_data.
2. A negative or oversized nbytes or int address raises OverflowError (and a test pins it), but Raises lists only TypeError, ValueError, and RuntimeError. Document it, in the .pyx and the .pyi, or convert it to ValueError.
3. dst and src have no annotations; the public-API rule wants int | Buffer | BufferProtocol, as Buffer.fill uses.

Smaller points:

• Only a Python int is an address. True packs as address 1, and a NumPy integer scalar (np.uintp(addr)) or a ctypes.c_void_p exports the buffer protocol, so it packs the address of its own 8 bytes rather than the address it holds. Reject bool and these scalars with a TypeError that says to pass int(addr), or say in the docstring that only an int is an address.
• "The caller must keep both alive" is not quite enough for a bytearray: it must also not be resized until the copy has run.
• HostCallbackNode.update and ExecutableHostCallbackNode.update raise the same HOST_COPY_FN record error but their docstrings do not say so.
• Tests: a Buffer.from_handle(ptr, n) with no memory resource would pin the no-query path (GPU-free), and the overlap test covers only dst > src; add the src > dst case and its adjacent boundary.

[srimanachanta]

@Andy-Jost I have rebased this PR on top of main as #3258 has landed, and addressed your previous feedback!