feat(scan): Support split and parallel scans - Part-1: validated FileScanTask byte ranges

[kamcheungting-db]

Summary

Allow FileScanTask to represent a validated half-open byte range instead of always representing an entire data file. Standalone and REST task JSON now preserve those ranges.

• add FileScanTask::MakeSplit, start, length, and is_split
• validate negative, empty, out-of-file, and overflowing ranges
• preserve ranges when parsing and writing standalone and REST scan-task JSON
• include ranges in REST task equality
• report each split byte size and estimate row counts additively without overflow
• reject partial tasks in FileScanTaskReader until range-aware file reading is implemented

Range behavior

start is a byte offset, and length is the number of bytes to scan, not the ending offset. For a 10-byte file, these two tasks cover the file without overlap:

{"start": 0, "length": 4}
{"start": 4, "length": 6}

A full-file range is canonicalized to the existing whole-file task representation. A zero-length range is valid only for a zero-byte file: start=0, length=0.

Both C++ JSON parsers return JsonParseError for invalid supplied ranges. Standalone JSON requires both fields. REST JSON defaults to the whole file only when both fields are absent.

C++ and Java comparison

Valid standalone ranges behave the same in C++ and Java. C++ additionally validates range bounds. REST range transport differs: Java currently ignores these fields and creates whole-file tasks.

The tables assume a 10-byte data file and otherwise valid JSON, except where a zero-byte file is explicitly mentioned. Java behavior was checked at commit db43b88.

Standalone task JSON

This is the self-contained format with embedded schema, partition spec, and delete files.

Input or operation	C++ with this PR	Java
Serialize a task	Writes actual start and length	Same
Both fields missing	Rejects	Rejects
Only one field present	Rejects	Rejects
Both present, either is null	Rejects	Rejects
String, boolean, floating-point number (including 1.0), or integer outside signed 64-bit range	Rejects	Rejects
Whole file: (0,10)	Accepts and preserves range	Same
Valid split: (0,9) or (4,6)	Accepts and preserves range through a round-trip	Same
Negative start or length	Rejects	Stores the values without checking bounds
Beyond-file range: (1,10)	Rejects	Stores the values without checking bounds
Zero-length range in a nonempty file	Rejects	Stores the values without checking bounds
End offset would overflow, although each field fits in signed 64 bits	Rejects without overflowing	Stores the values without checking bounds
(0,0) for a zero-byte file	Accepts	Accepts

All C++ rejections above return JsonParseError. Error wording can differ from Java. Java accepting a range during parsing does not establish that the range is valid or readable.

Sources: Java FileScanTaskParser, BaseFileScanTask.SplitScanTask, and JsonUtil.getLong.

REST scan-task JSON

This format uses external partition specs and references a shared delete-file list.

Input or operation	C++ with this PR	Java
Serialize a task	Writes actual start and length	Omits both fields
Both fields missing	Whole-file task (0,10)	Same
Only one field present	Rejects	Ignores the field; whole-file task
Both present, either null, wrong type, or outside signed 64-bit range	Rejects	Ignores the fields; whole-file task
Whole-file range (0,10)	Whole-file task	Whole-file task
Valid split (4,6)	Preserves the split	Ignores the range; whole-file task
Negative, beyond-file, empty partial, or overflowing range	Rejects	Ignores the range; whole-file task

All C++ rejections above return JsonParseError. The current Java REST parser and REST FileScanTask schema do not define range handling. These fields are therefore a C++ extension, and this PR does not establish interoperable REST split transport with Java. Sending two partial tasks for the same file to that Java implementation can turn them into two whole-file tasks.

Sources: Java RESTFileScanTaskParser and the REST FileScanTask schema.

Scope

This PR models, validates, and transports byte ranges. It does not enable partial file reads yet. Returning NotSupported prevents a split task from silently rereading the entire file; range-aware format readers will follow separately.

Testing

• Existing task-model and reader checks: ctest --test-dir build-clang -R "^(scan_test|data_test|rest_catalog_test)$" --output-on-failure
• Standalone JSON and REST suites pass with the final changes: json_serde_test and rest_catalog_test
• Standalone tests cover whole-file and split round-trips, delete-file and residual-filter preservation, empty files, and negative/out-of-bounds/overflow ranges
• pre-commit passed on the five changed JSON source/header/test files
• git diff --check passed

[zhjwpku]

Generally LGTM but you seems to miss the ToJson part in src/iceberg/json_serde.cc(added in #939 ), json[kStart] is still hard coded to 0, and json[kLength] is the file size.

[zhjwpku]

Other than the two minor issues, LGTM.

[wgtmac]

Overall LGTM.

[wgtmac]

IMHO, this file_size field is duplicated with that in the data_file. Can we remove this to avoid duplication and possible inconsistency in the future?