Repository navigation
Conversation
Implement the subset of the NeXus API used by McCode and the Monitor_nD libraries (NXopen/close, make/open/closegroup, (comp)makedata(64), open/close/putdata, putslab64, putattr, getinfo64, getgroupinfo, getgroupID, makelink, error reporting) directly on HDF5 in mccode-r.c, with the same names so components using nxhandle keep compiling. The written layout (groups, NX_class, string types, chunking, shuffle + deflate level, links and 'target' attributes) matches libNeXus 4.4.3, verified with h5dump -p/h5diff on serial, list-flush, --IDF and MPI (VDS) runs for McStas and McXtrace. The open groups are a stack of HDF5 ids: no fixed path buffers, a failed open does not change the state, over-closing is reported, nested error suppression nests, and chunks are kept below the 4 GiB HDF5 limit. Fixes in the NeXus writers: - nxstr used the value as printf format: string parameters, titles or instrument text with '%' were garbled (e.g. "a%sb" -> "aa%sbb") or UB. Values are now passed as "%s", without the 1024-char buffer. - Monitor_nD BINS output wrote uninitialised MCDETECTOR fields (Date, xylimits, ... seen as stack garbage in MPI runs); zero it. - uninitialised NXlink used for /entry/data links when a group failed to open; unbalanced closegroup in mcdatainfo_out_nexus and the IDF group. - existing files were opened with mode NXACC_CREATE5|NXACC_RDWR (7). - nxhandle was defined in the header (duplicate symbol with -fno-common when the runtime is not embedded); siminfo_out() wrote to the NeXus handle as a FILE*. - drop a debug printf in mcdetector_out_3D_nexus and the duplicated zmin/zmax geometry attributes in Monitor_nD. Output changes: no root NX_class is written (as libNeXus 4.4.3), the file_time offset is now correct during DST, NeXus_version stays "4.4.2". Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
They belong to the UB_matrix work on ub-matrix-injector and were picked up here by accident from a shared working tree. Nothing on this branch uses them. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Test_NeXus_output.instr with test_nexus_output.py, run by the basictest workflows after the MPI list test. It runs the instrument with --format=NeXus, serial and with MPI, and checks mccode.h5 with h5py: - string values containing '%' are stored as given (instrument parameter and component parameter literal) - all string attributes and data sets are printable (no uninitialised memory, as in the Monitor_nD BINS output) - no debug output on stdout - a monitor file name of 1000 characters does not crash the run - --append adds entry2 to mccode.h5 With the current napi-based writer the first four fail. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Conflicts with the napi fixes of mccode-dev#2889 in mccode-r.c/mccode-r.h.in are resolved in favour of the native HDF5 layer, which already contains them. The napi-only workarounds merged from mccode-dev#2889 are dropped here: the native layer has no 1024-character path buffers (long monitor names are written, not skipped) and NXcompmakedata64 already keeps chunks below 4 GiB. Co-Authored-By: Claude Opus 5.5 <[email protected]>
willend
marked this pull request as ready for review
October 26, 2026 18:14
NeXus output needs HDF5 only since the native HDF5 writer: - conda: hdf5 instead of libnexus in mccode-create-conda-yml (used by the conda CI jobs, so these now build without libNeXus present); drop 'nexus' from environment.yml and the Windows meta-pkg environments (they already list hdf5) - remove 3rdparty/nexus-package (it packaged libNeXus) with its sync.sh clone and the GhostBSD mkdist line - mccode-build-conda: no -DNEXUSLIB/-DNEXUSINCLUDE, the conda NEXUSFLAGS already use $CONDA_PREFIX/include and lib - basictest workflows: install libhdf5-dev (Linux) / hdf5 (Homebrew) instead of libnexus-dev / nexusformat, no -DNEXUSLIB/-DNEXUSINCLUDE - toolchains: Debian ones set NEXUSFLAGS fully, drop the unused NEXUSLIB/NEXUSINCLUDE; macOS/MinGW NEXUSINCLUDE points at the conda include dir holding hdf5.h instead of include/nexus - code generator: hint '-DUSE_NEXUS -lhdf5' instead of '-lNeXus' - manuals (McStas, McXtrace), ILL_H16_IN5_Mantid and the macOS dependency README: HDF5 instead of libNeXus, -lhdf5 instead of -lNeXus Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
McCode's NeXus output stops using libNeXus. The NeXus C API (
napi.h) has been frozen and deprecated upstream. It is replaced by about 300 lines of native HDF5 code in the runtime, keeping the sameNX*function names, so--format=NeXusneeds only HDF5.This is the full replacement that #2889 was the pre-study for. #2889 fixed the napi-based writer and added
Test_NeXus_output. This PR keeps those fixes and passes the same test.The libNeXus dependency is removed from packaging, devel tools, CI, toolchains and docs (commit
1318d48ff); the conda-forge recipes are the remaining follow-up.What changes
mccode-r.h.inno longer includesnapi.h. It declares the napi subset McCode and the Monitor_nD libraries use:NXhandle,NXstatus,NXlink,NX_OK/NX_ERROR,NX_CHAR/NX_FLOAT64/NX_INT32,NX_UNLIMITED,NXACC_RDWR/NXACC_CREATE5, andNX_COMP_NONE/NX_COMP_LZW/NX_COMP_LZW_LVL0..9. Existing-DNX_COMPRESSION=NX_COMP_LZW_LVL1flags keep working.mccode-r.cimplements the 19 functions used, on HDF5:NXopen,NXclose;NXmakegroup,NXopengroup,NXclosegroup;NXmakedata,NXcompmakedata,NXcompmakedata64,NXopendata,NXclosedata,NXputdata,NXputslab64;NXputattr,NXgetinfo64,NXgetgroupinfo;NXgetgroupID,NXmakelink;NXMDisableErrorReporting,NXMEnableErrorReporting.Components that use
nxhandle/NX*directly keep compiling.Build:
NEXUSFLAGS(inPlatformDefaults.cmake, the Debian toolchains,MCUtil.cmakeandInstallMCCODE.cmake) link only HDF5. Debian packages depend onlibhdf5-devinstead oflibnexus-dev. An existingmccode_config.jsonthat still passes-lNeXuskeeps working: the runtime's ownNX*take precedence.Design
NX_classattribute;NX_COMPRESSIONsets it;targetattribute;NeXus_version=4.4.2.NXcompmakedata64.Output differences against the libNeXus build
The new and old builds were compared with
h5dump -pandh5diff, with timestamps masked. Covered: serial runs, list-buffer flushes (2.5M events),--IDF, MPI with 2 ranks (node files and the VDS),Test_NeXus_comp_par_runtime, and a McXtrace instrument. Group and data-set layout, chunking, filters and data were identical, down to byte-identical file sizes. The remaining differences:/entry1/databecauseNXlinkkeeps napi's 1024-charactertargetPath; this is reported.NX_classis written; the libNeXus 4.4.3 builds in use never wrote one.file_timeoffset is now correct during daylight saving time; napi was one hour off.NeXus error: ….Limitations for component authors
napi.hitself, will not build.NXopengroupdoesn't checkNX_class.NXgetgroupinfofills name and class only at the root.NXputdatawrites the whole data set; useNXputslab64to append.CI (willend/McCode,
workflow_dispatch, merge commit1e3c36752)mcstas-basictest and mcxtrace-basictest are both green. The NeXus output regression test passed in all 12 classic jobs: Linux gcc-13, clang, Debian and conda; macOS clang; Windows MSVC. The antlr jobs skip it by design.
Follow-ups (not in this PR)
nexus/libnexusfromenvironment.yml,meta-pkgs/windows/*.ymlandmccode-create-conda-yml(nowhdf5), remove3rdparty/nexus-packagelibnexusbyhdf5libhdf5-dev/ Homebrewhdf5instead oflibnexus-dev/nexusformat, no nexus CMake dirs-DUSE_NEXUS -lhdf5; manuals and examples updatedNEXUSINCLUDEpoints at the HDF5 headers (macOS, MinGW);mccode-build-condano longer passes nexus dirs🤖 Generated with Claude Code