Skip to content

fix: upgrade to apache-arrow 24.0.0 and enable CI run success - #922

Merged
SYaoJun merged 3 commits into
apache:mainfrom
SYaoJun:fix_arrow_24
May 11, 2026
Merged

SYaoJun merged 3 commits into
apache:mainfrom
SYaoJun:fix_arrow_24

Conversation

@SYaoJun

@SYaoJun SYaoJun commented May 8, 2026 •

Copy link
Copy Markdown
Contributor

Reason for this PR

This commit fixes deprecation errors when compiling with Apache Arrow 24.0.0+.

close: #920

What changes are included in this PR?

  1. replace deprecated ReadTable(Table**) with arrow::Result version in cpp
  2. replace pull_request_target with pull_request in pre-commit.yml, which is a bug
  3. python-wheel cannot upload to pypi since it not meet the apache way requirements https://github.com/apache/incubator-graphar/actions/runs/25500338422
  4. initialize arrow-compute and datasets when upgrade to apache-arrow 24.0.0
  5. introducing apache-arrow to 24.0.0 to all CI jobs

Are these changes tested?

yes

Are there any user-facing changes?

no

@codecov-commenter

codecov-commenter commented May 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.09091% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.55%. Comparing base (bcc5980) to head (5e990f0).

Files with missing lines Patch % Lines
cpp/src/graphar/expression.cc 60.00% 6 Missing ⚠️
cpp/src/graphar/filesystem.cc 92.85% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #922      +/-   ##
============================================
+ Coverage     76.80%   80.55%   +3.74%     
  Complexity      615      615              
============================================
  Files            84       94      +10     
  Lines          8886    10742    +1856     
  Branches       1054     1060       +6     
============================================
+ Hits           6825     8653    +1828     
- Misses         1821     1849      +28     
  Partials        240      240              
Flag Coverage Δ
cpp 70.85% <84.09%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@SYaoJun

SYaoJun commented May 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Pre-commit Checks / Pre-commit Checks (pull_request_target)Pre-commit Checks / Pre-commit Checks (pull_request_target) the old configuration not install pre-commit, so it failed. After this PR merged, the problem will automaticly resolve.

@SYaoJun
SYaoJun requested a review from yangxk1 May 8, 2026 12:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades GraphAr’s C++/CI integration to Apache Arrow 24.0.0 and updates the codebase and workflows to address Arrow 24 deprecations and CI failures (including initialization changes for Arrow compute/dataset usage).

Changes:

  • Update C++ (and C++ tests) to use Arrow/Parquet APIs compatible with Arrow 24 (e.g., ReadTable() returning arrow::Result).
  • Add initialization calls for Arrow compute and dataset execution paths to avoid runtime/compile issues with newer Arrow.
  • Update GitHub Actions workflows to install/use Arrow 24 across CI and adjust Python wheel publishing/build steps.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
cpp/test/test_builder.cc Switch Parquet ReadTable usage to Arrow 24 Result API in tests.
cpp/test/test_arrow_chunk_writer.cc Switch Parquet ReadTable usage to Arrow 24 Result API in tests.
cpp/src/graphar/reader_util.cc Add Arrow compute initialization before evaluating/inspecting filter expressions.
cpp/src/graphar/high-level/graph_reader.cc Add dataset/compute initialization in EdgesCollection::Make.
cpp/src/graphar/filesystem.cc Add dataset/compute initialization helper; migrate to Arrow 24 ReadTable() API.
cpp/src/graphar/expression.h Ensure compute initialization happens before creating literals inline.
cpp/src/graphar/expression.cc Add centralized compute initialization helper and call it from expression evaluators.
cpp/src/graphar/arrow/chunk_writer.cc Initialize compute/dataset prior to using Acero plan-based sorting.
cpp/src/graphar/arrow/chunk_reader.cc Initialize compute before Cast usage in reader utilities.
cpp/src/CMakeLists.txt Add LibXml2 linkage for static Arrow builds on non-Apple/non-MSVC.
cpp/CMakeLists.txt Update Arrow version threshold logic for selecting C++20.
cpp/cmake/apache-arrow.cmake Bump default Arrow version to build from source to 24.0.0.
.github/workflows/rust.yaml Install Arrow 24 packages in Rust CI job.
.github/workflows/python.yml Install Arrow 24 packages and update checkout action version.
.github/workflows/python-wheel-workflow.yml Adjust macOS wheel build environment/setup; pin PyPI publish action.
.github/workflows/pre-commit.yml Switch trigger to pull_request, update checkout action, add pre-commit installation steps.
.github/workflows/ci.yml Install Arrow 24 packages; update checkout action version; remove clang-format step from main CI job.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -41,9 +41,15 @@ jobs:
runs-on: ubuntu-slim
Comment on lines 171 to 183
- name: Install Arrow (macOS)
if: matrix.os == 'macos'
shell: bash
run: |
set -euxo pipefail
mamba install -y -c conda-forge arrow-cpp
# Note: CONDA_PREFIX may be unset unless conda is activated in this shell.
# setup-miniconda exports CONDA (base install prefix), which is sufficient here.
echo "CMAKE_PREFIX_PATH=$CONDA" >> "$GITHUB_ENV"
brew install apache-arrow
echo "CMAKE_PREFIX_PATH=/usr/local" >> "$GITHUB_ENV"
# Optional sanity check: ensure Arrow dylib isn't built for a newer macOS than deployment target.
if command -v otool >/dev/null 2>&1; then
ls -lah "$CONDA/lib" || true
if [ -f "$CONDA/lib/libarrow.dylib" ]; then
otool -l "$CONDA/lib/libarrow.dylib" | (grep -A3 -E 'LC_BUILD_VERSION|LC_VERSION_MIN_MACOSX' || true)
ls -lah /usr/local/lib || true
if [ -f "/usr/local/lib/libarrow.dylib" ]; then
otool -l "/usr/local/lib/libarrow.dylib" | (grep -A3 -E 'LC_BUILD_VERSION|LC_VERSION_MIN_MACOSX' || true)
fi
CIBW_MANYLINUX_AARCH64_IMAGE: ${{ matrix.os == 'linux' && format('manylinux{0}', matrix.manylinux) || '' }}
CIBW_ENVIRONMENT_WINDOWS: DISTUTILS_USE_SDK=1 MSSdk=1
CIBW_ENVIRONMENT_MACOS: ${{ matrix.os == 'macos' && format('MACOSX_DEPLOYMENT_TARGET={0} CMAKE_OSX_DEPLOYMENT_TARGET={0} CFLAGS=-mmacosx-version-min={0} CXXFLAGS=-mmacosx-version-min={0} LDFLAGS=-mmacosx-version-min={0}', matrix.deployment-target) || '' }}
CIBW_ENVIRONMENT_MACOS: ${{ matrix.os == 'macos' && 'MACOSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion) CMAKE_OSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion) CFLAGS=-mmacosx-version-min=$(sw_vers -productVersion) CXXFLAGS=-mmacosx-version-min=$(sw_vers -productVersion) LDFLAGS=-mmacosx-version-min=$(sw_vers -productVersion)' || '' }}
Comment on lines +25 to +31
Status EnsureComputeInitialized() {
static bool initialized = false;
if (!initialized) {
RETURN_NOT_ARROW_OK(arrow::compute::Initialize());
initialized = true;
}
return Status::OK();
Comment on lines +98 to +110
Status EnsureDatasetScannerInitialized() {
static bool initialized = false;
static Status init_status = Status::OK();
if (!initialized) {
auto st = arrow::compute::Initialize();
if (!st.ok()) {
init_status = Status::ArrowError(st.ToString());
} else {
arrow::dataset::internal::Initialize();
}
initialized = true;
}
return init_status;
Comment on lines +945 to +950
static bool initialized = false;
if (!initialized) {
RETURN_NOT_ARROW_OK(arrow::compute::Initialize());
arrow::dataset::internal::Initialize();
initialized = true;
}
Comment on lines +48 to +52
static bool initialized = false;
if (!initialized) {
RETURN_NOT_ARROW_OK(arrow::compute::Initialize());
initialized = true;
}
Comment on lines 73 to +80
Status GeneralCast(const std::shared_ptr<arrow::Array>& in,
const std::shared_ptr<arrow::DataType>& to_type,
std::shared_ptr<arrow::Array>* out) {
static bool initialized = false;
if (!initialized) {
RETURN_NOT_ARROW_OK(arrow::compute::Initialize());
initialized = true;
}
Comment on lines 115 to +122
Status CastTableWithSchema(const std::shared_ptr<arrow::Table>& table,
const std::shared_ptr<arrow::Schema>& schema,
std::shared_ptr<arrow::Table>* out_table) {
static bool initialized = false;
if (!initialized) {
RETURN_NOT_ARROW_OK(arrow::compute::Initialize());
initialized = true;
}
@SYaoJun
SYaoJun merged commit b621490 into apache:main May 11, 2026
12 of 13 checks passed
@SYaoJun
SYaoJun deleted the fix_arrow_24 branch May 11, 2026 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++][CI]: C++ code compile failed

4 participants