Repository navigation
BUG: Fix OOC Review Findings - #1764
Draft
jmarquisbq wants to merge 81 commits into
Draft
jmarquisbq wants to merge 81 commits into
jmarquisbq wants to merge 81 commits into
Conversation
jmarquisbq
force-pushed
the
fix/ooc-review-findings
branch
from
October 6, 2026 15:22
476e193 to
9420aa3
Compare
imikejackson
force-pushed
the
fix/ooc-review-findings
branch
2 times, most recently
from
October 7, 2026 21:26
027776a to
7f49132
Compare
This was referenced Oct 8, 2026
imikejackson
self-requested a review
October 13, 2026 14:18
7 of 8 tasks
imikejackson
force-pushed
the
fix/ooc-review-findings
branch
from
October 16, 2026 19:54
1336ee8 to
9e87010
Compare
jmarquisbq
force-pushed
the
fix/ooc-review-findings
branch
from
October 19, 2026 15:48
9e87010 to
3e376fe
Compare
10 tasks
Keep parallel execution disabled after an earlier storage requirement or explicit disable. Preserve the explicit enable call as the reset point. Add regression cases for both helpers, mixed calls, empty/null inputs, and the multicore enable/reset behavior. Signed-off-by: Jessica Marquis <[email protected]>
Retain storage location when cloning an array parameter and accept explicit matching in-memory and out-of-core stores. Keep metadata-only and string-array behavior unchanged, with readable mismatch diagnostics. Add the location and clone regression matrix. Signed-off-by: Jessica Marquis <[email protected]>
Use one cluster by default and reject explicit zero during preflight and both algorithm entry points before output changes or data access. Update parameter help and contracts, and add literal one-cluster and unchanged-output regressions for the direct and scanline paths. Signed-off-by: Jessica Marquis <[email protected]>
Add const checked flush methods for numeric stores, list stores, data objects, and data structures while retaining the existing void APIs. Forward each object once and retain ordered warnings, backend error codes, object IDs, and every object path. Classify allocation failures as -272 and preserve invalid results that have no error diagnostic. Add regression coverage for legacy adapters, missing stores, shared stores, multiple parents, exceptions, diagnostic ordering, and later-object visits. Signed-off-by: Jessica Marquis <[email protected]>
Use checked flushes at filter completion and retain original diagnostics. Prepare snapshots before replacing prior results, report final snapshot failures, and preserve completion failure during cancellation and retry. Carry warnings through the HDF5 writer chain and expose root errors to CLI observers. Add focused pipeline and command-line regression cases. Signed-off-by: Jessica Marquis <[email protected]>
* Add a nonallocating chunk-shape entry point with explicit output and suffix storage so callers can include policy scratch in their budget. * Keep the existing vector API on the same arithmetic and cover both policies, literal shapes, and rejected span boundaries. Signed-off-by: Jessica Marquis <[email protected]>
* Reject invalid ranks, bounds, products, and input spans before selecting or writing a dataset region, preserving existing values on rejection. * Keep empty simple selections, surplus input prefixes, primitive conversions, and valid unlinked-open dataset writes supported. * Guard local dataspaces and report file, dataset, selection, and native failure details outside the HDF5 API lock. * Cover all exported primitive types, boundary and native-write errors, and real sparse metadata that reaches both arithmetic overflow guards. Validation: paired Release simplnx_test builds; 18 focused cases and 3 additional lock controls in each configuration. The final stronger arithmetic assertions also pass in both builds. Signed-off-by: Jessica Marquis <[email protected]>
Read policy-neutral cached metadata, stage the complete selection, and resolve destination storage before loading selected values. Preserve per-path copies for multi-parent sources and use checked identifiers and exact-object cleanup when publication fails. Add cache, filter, legacy, finalizer, identifier, and rollback regressions. Full paired consumer builds passed, with 98 in-memory cases and 432 OOC cases across the coordinated import and storage verification. Signed-off-by: Jessica Marquis <[email protected]>
Report failed writes, flushes, and closes before publishing an Avizo export. Close the output stream on failed reads and exceptions, while preserving earlier errors and warnings and the existing file format. Cover both real filter wrappers and injected output failures, including destination preservation and one consuming close on each exit path. Signed-off-by: Jessica Marquis <[email protected]>
* Validate caller-owned extents and native chunk metadata before reading. * Bound codec buffers, decoder state and diagnostics under one allowance while preserving the existing reader and compressed-stream behavior. * Coalesce contiguous source/output copies and cover bounds, failures, sparse data, physical component splits and payload ownership. Signed-off-by: Jessica Marquis <[email protected]>
* Reject invalid threshold values before changing preferences and repair only invalid saved fields with their original value and default context. * Retain correction batches for one startup consumer. Preserve exact JSON values and key presence in shared, algorithm and benchmark scopes. * Report CLI corrections before overrides and preflight, with owned-profile checks for malformed values, zero, maximum values and normal saving. Signed-off-by: Jessica Marquis <[email protected]>
* Reuse XY transfers across up to eight YZ slices within the admitted extra memory, with a one-slice fallback when no extra memory is free. * Preserve completed slices and ordered error details on cancellation or failure, and reject invalid slice sizes before allocation. * Add focused output, ownership, failure, and opt-in timing checks. Signed-off-by: Jessica Marquis <[email protected]>
Preserve the existing no-op callback while fixing the unsupported implicit conversion from a lambda reported by Clang. Signed-off-by: Jessica Marquis <[email protected]>
Rename the write callback parameter while retaining its signature, values, and overflow assertions. This fixes Clang shadow-all builds. Signed-off-by: Jessica Marquis <[email protected]>
Use the factories' concrete DataStructure return type in templated helpers, so member template calls do not depend on auto deduction. Keep test data, scenarios, assertions, and benchmark settings unchanged. Signed-off-by: Jessica Marquis <[email protected]>
Check the generated stream and compare all records through matching EOF, retaining the two intentional metadata exclusions. Normalize only CRLF terminators and test missing, extra and changed records beyond 1,024 with LF/CRLF and malformed-tail controls. Validated with paired Release target builds and focused CTest selections. Signed-off-by: Jessica Marquis <[email protected]>
* Check complete five-feature lists with literal 5/NaN/10 expectations. * Verify resident and genuine HDF5 numeric/list backing stores. * Preserve all input lists and require the average output to be absent regardless of object type. Validated with paired Release builds, ten focused resident regressions per configuration and the OOC contract group. Signed-off-by: Jessica Marquis <[email protected]>
* Check complete five-feature lists with literal 5/NaN/10 expectations. * Verify resident and genuine HDF5 numeric/list backing stores. * Preserve all input lists and require the average output to be absent regardless of object type. Validated with paired Release builds, ten focused resident regressions per configuration and the OOC contract group. Signed-off-by: Jessica Marquis <[email protected]>
Check both ID forms through reads, writes, failures and cancellation. Verify exact submitted and stored connectivity, transfer order, and unchanged inputs for the ordinary buffered utility. Validated with paired Release builds and 14 resident / 22 OOC selected CTest entries. Signed-off-by: Jessica Marquis <[email protected]>
Exercise the actual geometry utility with four literal face normals. Check the float arithmetic contract, preserved geometry and pre-cancel sentinels across the resident algorithm scenarios. Validated with paired Release builds and 14 resident / 22 OOC selected CTest entries. Signed-off-by: Jessica Marquis <[email protected]>
Check full-block and distinct tail values using an independent normal oracle. Cover exact failed reads and writes, completed prefixes, cancellation, and accepted empty geometry without changing the filter. Paired Release target builds and six resident / seven OOC CTest entries passed, including the existing STL and conversion controls. Signed-off-by: Jessica Marquis <[email protected]>
Check full blocks and a distinct tail against literal acute, obtuse, collinear and repeated-vertex results. Verify exact failed transfers, completed prefixes, cancellation and accepted empty geometry. Paired Release target builds and six resident / seven OOC CTest entries passed, including the existing correctness and conversion controls. Signed-off-by: Jessica Marquis <[email protected]>
Run the existing Rodrigues ordering case with named resident algorithm scenarios and target-only dispatch checks. Check all four input and both output stores while preserving both input orders and the independent 45-degree quaternion oracle. Paired Release OrientationAnalysisUnitTest builds and the selected ordering case passed in both configurations. Signed-off-by: Jessica Marquis <[email protected]>
Add a tiny resident scenario test and shared independent file oracle. Check signed components, float scalars, all header fields and values, matching EOF, unchanged inputs and geometry. Retain binary and cancellation coverage. Paired Release target builds and three resident / four OOC selected CTest entries passed. Signed-off-by: Jessica Marquis <[email protected]>
Add a registered metadata-only input and test the exact -79047 error. Use a narrow test guard that restores original storage-manager pointers and capabilities after normal exit and a controlled exception. Preserve input metadata and require no output object. Paired Release target builds and the combined four resident / six OOC preflight controls passed. The fixtures contain metadata, not disk data. Signed-off-by: Jessica Marquis <[email protected]>
Reuse the scoped registry guard for the plain watershed -79048 branch. Check exact error context, unchanged input metadata, absent output and complete manager restoration; retain the provider-present controls. Paired Release target builds and the combined four resident / six OOC preflight controls passed. The fixtures contain metadata, not disk data. Signed-off-by: Jessica Marquis <[email protected]>
Check complete unit-spacing area/volume and optional sphericity outputs for isolated and paired cells through both resident algorithm scenarios. Preserve existing archive controls and verify inputs and storage. Signed-off-by: Jessica Marquis <[email protected]>
Apply the storage scenario before archive import and check the contour input, final vertices, faces and normals. Preserve all archived values. Signed-off-by: Jessica Marquis <[email protected]>
Check complete tetrahedron winding for both accepted ID forms. Preserve existing normal values and identity, or require no normal object to be created when none exists. Verify all vertices, IDs and resident stores. Signed-off-by: Jessica Marquis <[email protected]>
Check NaN positions and component counts without changing the explicit skip-NaN option or tolerance policy. Keep bounded reads and input precision. Add real-array regression checks for mismatches, transfer failures, partial batches, warnings and complete source preservation. Signed-off-by: Jessica Marquis <[email protected]>
Point the analytical V&V inventory to its current test source and names. Separate its 31 registrations from the broader filter suite and preserve the dated sign-off and unresolved historical revision. Record the fresh 31/31 Windows checks in both builds with their resident-data scope. Signed-off-by: Jessica Marquis <[email protected]>
Remove the capture/upload steps and helper from temporary commit 3d7d593 after the native library compatibility review. Restore the exact prior workflow content without changing either build matrix or the general CI fixes. Parsed YAML and blob comparisons verify the scoped restoration. Signed-off-by: Jessica Marquis <[email protected]>
* Add atomic all-or-nothing admission against working-memory quota and registered cache headroom, retaining existing reservation behavior. * Cover exact limits, contention, pending release, budget reduction, and token lifetime without invoking eviction callbacks. Signed-off-by: Jessica Marquis <[email protected]>
Use the unique largest empty gap to unwrap occupied cell coordinates, then average all cells with their actual populations. Keep the ordinary mean when the gap is tied or the axis is fully occupied. Bound occupancy storage with working-memory reservations and sequential batches. Propagate warnings and I/O errors, and check cancellation before output. Add independent numerical oracles, small-grant and overflow checks, failure regressions, and an actual HDF5-store witness. Update the filter description while preserving the historical verification record. Signed-off-by: Jessica Marquis <[email protected]>
Read faces, IDs, and vertex coordinates through reserved bulk buffers. Keep float32 operations and feature addition order unchanged. Write each changed face block once with complete tuples and retain the resident path. Add resident equivalence, disabled-normal, and float32-order regressions. Paired Windows Release and OOC Debug winding tests pass, including the companion full-filter HDF5 tests in SimplnxOoc. Signed-off-by: Jessica Marquis <[email protected]>
Check each triangle vertex before computing a coordinate offset. Return the face number, invalid index, and valid vertex count when input is bad. Keep valid normal and angle calculations unchanged. Add regression cases for each corner, first and later faces, the vertex count boundary, and the largest index. Verify errors retain the input and leave the failing output block unpublished. Signed-off-by: Jessica Marquis <[email protected]>
Keep coordinate caches only when their buffers fit the working allowance. Otherwise gather coordinates in bounded blocks. Plan useful requests and size buffers from the actual grant to limit unnecessary cache eviction. Use temporary XYZ records for low-memory smoothing. Keep calculation order and completed-pass cancellation behavior. Release compute buffers before an imported array makes its writable copy during publication. Keep the resident smoothing path and its older working arrays unchanged. Signed-off-by: Jessica Marquis <[email protected]>
* Use fixed uint64 arrays for both centroid memory-budget loops so Linux GCC and Clang do not infer conflicting unsigned integer types. * Keep all five budgets and the existing numerical assertions unchanged. Validation: full Windows in-memory and OOC Release builds; eight centroid cases in each configuration; two real HDF5 cases with 402 assertions; before/after compiler probes, clang-format 19.1.1, and scoped clang-tidy. Native Linux builds remain for CI after publication. Signed-off-by: Jessica Marquis <[email protected]>
Correct the same output-type assumption in the companion resident test. Preserve its Direct and Scanline scenarios and all numerical, ID, seed, and tuple assertions. Validated with full Windows in-memory and OOC Release builds, the focused tests, and the selected KMeans/Abaqus regressions. Production behavior is unchanged. Signed-off-by: Jessica Marquis <[email protected]>
* Give each 65,537-cell scan a complete unit-spaced row with constant Y. * Keep independent coordinate checks at the transfer and scan boundaries, while preserving the distinct tail values in all other data columns. * Keep reader validation and production behavior unchanged. Validation: full Windows in-memory and OOC Release builds; 20 in-memory and 21 OOC H5Oina cases; the formerly failing hidden batch-tail case; clang-format 19.1.1 and configured changed-line clang-tidy. Signed-off-by: Jessica Marquis <[email protected]>
jmarquisbq
force-pushed
the
fix/ooc-review-findings
branch
from
October 20, 2026 18:24
543cb50 to
b16b292
Compare
* Enable /bigobj for simplnx_test on MSVC to avoid the Debug object section limit. * Remove three unused headers from the surface-area tests. Signed-off-by: Jessica Marquis <[email protected]>
Describe admitted coordinate caches and bounded gathers. Complete the parameter and result tags for gather, planning, and smoothing publication helpers. These changes do not change executable code. Signed-off-by: Jessica Marquis <[email protected]>
Admit lookup and face buffers together. Cache complete tables only when the shared working budget allows it; otherwise gather complete tuples through bounded pages. Reuse the IPF orientation helpers across blocks. Keep the in-memory calculations and prior phase validation unchanged. Add resident tests for the working-memory contract. Paired HDF5 tests cover literal colors and angles, sparse lookups, failures, and cancellation. Signed-off-by: Jessica Marquis <[email protected]>
Replace full resident output copies with one admitted labels/counts block. Calculate disjoint resident rows in parallel with read-only map lookups, then write each block serially. Release earlier transfer buffers first. Preserve uncomputed rows on cancellation, existing ID randomization, store diagnostics, and partial-write behavior. Add hand-counted coverage for feature-face IDs, label pairs, counts, and the reserved zero row. Signed-off-by: Jessica Marquis <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Correct storage contracts and error reporting used by out-of-core (OOC) pipelines, reduce repeated small mesh and image transfers, and add regression coverage for resident and disk-backed execution. Also correct periodic feature centroids, KMeans count validation, and numeric growth initialization. 189 files changed, +28,581 / −1,452 lines.
Coordinated companion PRs:
Storage and pipeline contracts
nxrunner, instead of reporting successful completion after a storage failure.Filter correctness and OOC performance
Regression coverage and portability
Test plan
Laplacian smoothing still has approximately 32 bytes per vertex of older resident working arrays. The new buffers are bounded, but the whole algorithm is not yet bounded by the OOC working allowance. The broader all-filter OOC coverage audit remains a follow-up.
Related Issue
Related to #1532 — expands filter and failure-path coverage; does not complete the coverage inventory.
Related to #481 — adds import, geometry-reference, and persistence tests; does not complete all HDF5 coverage.
Related to #1756 — cluster-count validation is fixed; the initial-sampling hang remains deferred.
Related to #1768 — includes compatibility repairs; replacement of library-version gates remains deferred.
Related to #1835 — bounds the newer mesh buffers; the remaining 32 bytes per vertex is separate work.