Repository navigation
[fix] Handle zero-width horizontal crops - #2173
Conversation
felixdittrich92
left a comment
There was a problem hiding this comment.
Hi @AryanRG 👋
Thanks for the fix.
Tests: please merge them into the existing ones
1. test_extract_rcrops_zero_width → fold into test_extract_rcrops
The new test runs 16 parametrized cases (4 boxes × 2 paths × 2 image types), and much of it checks things this PR doesn't touch:
- half the cases run the oriented path (
assume_horizontal=False), which is unchanged; - the grayscale/float32 image, the input-not-modified checks and the comparison against single-box crops all cover existing behaviour, not the fix.
test_extract_rcrops is already parametrized over assume_horizontal, so a few lines there cover the regression on both paths. Add this just before the # No box check:
# Zero-width boxes yield an empty crop, without shifting the other crops
collapsed_boxes = np.array(
[[[x, 0.25], [x, 0.25], [x, 0.75], [x, 0.75]] for x in (0.0, 0.5, 1.0)] + [[[0.5, 0.5]] * 4],
dtype=np.float32,
)
croped_imgs = geometry.extract_rcrops(
doc_img, np.concatenate([rel_boxes, collapsed_boxes]), assume_horizontal=assume_horizontal
)
assert len(croped_imgs) == num_crops + len(collapsed_boxes)
assert all(crop.size > 0 for crop in croped_imgs[:num_crops])
assert all(crop.shape == (0, 0, 3) and crop.dtype == doc_img.dtype for crop in croped_imgs[num_crops:])Then remove test_extract_rcrops_zero_width.
2. test_predictors_drop_zero_width_quadrilaterals → parametrize test_predictors_keep_objectness_scores_aligned
The new test is about 70 lines and largely repeats test_predictors_keep_objectness_scores_aligned: the same fixed detection predictor, the same dummy recognition predictor, the same OCR/KIE parametrization and the same [0.9, 0.8] objectness assertions. The only new thing is the rotated-box input. Adding an assume_straight_pages parameter to the existing test covers it.
Some parts of the new test can be dropped:
- The "all-collapsed" cases only exercise the generic empty-crop filtering in
_prepare_crops, which this PR doesn't change. - The sort by minimum x isn't needed: words come back in box order and
word.geometry[0][0]works for polygons too. - The reco call counting and the
crop.size > 0check inside the recognizer are already implied by the word and score assertions. - The
type(result) is ...check is already covered bytest_predictors_on_empty_batch. detect_orientation=False,straighten_pages=False,resolve_lines=False,resolve_blocks=Falseare either defaults or don't affect the result.
Suggested replacement for test_predictors_keep_objectness_scores_aligned:
@pytest.mark.parametrize("assume_straight_pages", [True, False])
@pytest.mark.parametrize("predictor_cls", [OCRPredictor, KIEPredictor])
def test_predictors_keep_objectness_scores_aligned(predictor_cls, assume_straight_pages):
# The middle box lies on the right border of the page and has no width: its crop is empty,
# so it is dropped before recognition
if assume_straight_pages:
boxes = np.array(
[[0.1, 0.1, 0.3, 0.2, 0.9], [1.0, 0.5, 1.0, 0.6, 0.1], [0.5, 0.1, 0.7, 0.2, 0.8]],
dtype=np.float32,
)
else:
# Polygons with the objectness score in the last row
boxes = np.array(
[
[[0.1, 0.1], [0.3, 0.1], [0.3, 0.2], [0.1, 0.2], [0.9, 0.9]],
[[1.0, 0.5], [1.0, 0.5], [1.0, 0.6], [1.0, 0.6], [0.1, 0.1]],
[[0.5, 0.1], [0.7, 0.1], [0.7, 0.2], [0.5, 0.2], [0.8, 0.8]],
],
dtype=np.float32,
)
class _FixedDetPredictor(nn.Module):
def forward(self, pages, **kwargs):
return [{CLASS_NAME: boxes.copy()} for _ in pages]
class _DummyRecoPredictor(nn.Module):
def forward(self, crops, **kwargs):
return [(f"word{idx}", 1.0) for idx in range(len(crops))]
# Without page orientation, rotated boxes are cropped with `assume_horizontal=True`
predictor = predictor_cls(
_FixedDetPredictor(),
_DummyRecoPredictor(),
assume_straight_pages=assume_straight_pages,
disable_page_orientation=True,
disable_crop_orientation=True,
)
page = predictor([np.full((100, 200, 3), 255, dtype=np.uint8)]).pages[0]
if predictor_cls is KIEPredictor:
words = page.predictions[CLASS_NAME]
else:
words = [word for block in page.blocks for line in block.lines for word in line.words]
# Each remaining word keeps the objectness score of its own box
assert [word.geometry[0][0] for word in words] == pytest.approx([0.1, 0.5])
assert [word.objectness_score for word in words] == pytest.approx([0.9, 0.8])disable_crop_orientation=True also keeps the test from loading the pretrained crop orientation model. Then remove test_predictors_drop_zero_width_quadrilaterals.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2173 +/- ##
=======================================
Coverage 97.61% 97.61%
=======================================
Files 170 170
Lines 10740 10744 +4
=======================================
+ Hits 10484 10488 +4
Misses 256 256
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thanks @felixdittrich92, folded both regressions into the existing tests and removed the duplicates. Tests and checks pass locally. |
Summary
Handle zero-width quadrilaterals in horizontal crop extraction without raising
IndexError. Return the existing empty-crop placeholder so predictor filtering keeps crops, boxes and objectness scores aligned. The oriented crop path is unchanged.The regression cases are folded into the existing geometry and OCR/KIE score-alignment tests, covering both straight and rotated boxes.
Boxes with NaN x coordinates also produce an empty crop; broader non-finite polygon validation is outside this patch.
Validation
Focused geometry, OCR/KIE alignment and adjacent builder/utility tests pass locally, along with pre-commit, Ruff and mypy. The consolidated regressions reproduce the original failure on upstream code. The full local suite was not rerun for this test-only revision.