Package icebug + algo extension, test GDS_PAGE_RANK on all platforms - #2
Merged
Merged
Conversation
Ship PageRank (and other GDS functions) as Cypher UDFs by bundling the icebug 13.2 prebuilt and the Ladybug algo extension with the app: - scripts/download_icebug.sh: fetch platform prebuilt into ./icebug/ - scripts/download-liblbug.sh: fetch shared liblbug (algo extension dlopens it; static liblbug.a gets dead-stripped so LOAD EXTENSION fails with missing GDSFunction symbols) - .cargo/config.toml: ICEBUG_DIR + rpaths covering dist, dev, .app, and --target triple-dir test binary layouts - build.rs: -rdynamic for LOAD EXTENSION (ELF), icebug presence check, Windows networkit_state link - Cargo: icebug 13.2 behind default icebug-analytics feature, arrow 55->56 to match; tempfile dev-dep for the new test - src/backend.rs: auto-LOAD the bundled algo extension per connection (ExtensionManager is per-Database; BUGSCOPE_ALGO_EXTENSION override) + in-process graphr_page_rank cross-check - scripts/vendor_arrow.sh + stage_macos_bundle.sh: vendor libarrow/ libomp next to libnetworkit so the downloaded extension's @rpath/libarrow resolves; bundle dylib/.so/.dll in every archive - tests/gds_page_rank.rs: INSTALL/LOAD the prebuilt algo extension and assert GDS_PAGE_RANK works (skips only when offline) - README: document the new analytics path; .gitignore the locally built extension binary (dev artifact, not shipped)
- Download icebug prebuilt + shared liblbug in CI and release jobs - Vendor arrow/openmp runtimes next to libnetworkit (Unix) so the INSTALL'd algo extension resolves @rpath/libarrow - Add Test GDS_PAGE_RANK step (cargo test --lib --test gds_page_rank) with LD_LIBRARY_PATH on Unix and icebug/lib + liblbug + vcpkg bin on PATH for Windows - Bundle icebug shared libs + algo extension into every release archive (macOS .app via stage_macos_bundle.sh, Linux tarball with best-effort libomp/libarrow vendoring, Windows zip with DLLs + vcpkg arrow DLLs) - Linux/macOS deps: LLVM libomp + Apache Arrow repos, cmake, libarrow-dev/libomp-20-dev, brew apache-arrow - Windows: vcpkg manifest (arrow, pkgconf), lukka/run-vcpkg setup with pinned commit, NuGet GitHub Packages binary caching, PKG_CONFIG_PATH/CMAKE_PREFIX_PATH/CXXFLAGS env for the icebug crate's pkg-config probe
- Bound apt mirror stalls on Linux: wget --timeout/--tries, Acquire timeouts + retries, timeout-minutes on dep steps - Authenticate vcpkg NuGet with classic PAT (VCPKG_NUGET_PAT) instead of GITHUB_TOKEN: repo-scoped token 403s pushing new packages to the org feed so the binary cache never populated - Put vcpkg pkgconf on PATH for the icebug crate's pkg-config probe (convert the whole PATH with cygpath -p so later pwsh steps keep a resolvable Windows-form PATH) - Add icebug/lib/networkit to PATH for the Windows GDS test (networkit_state.dll lives in that subdir on the win prebuilt)
adsharma
force-pushed
the
icebug-gds-packaging
branch
from
October 2, 2026 02:46
c050224 to
3619bfa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bundles the icebug prebuilt (libnetworkit 13.2) and the Ladybug
algoextension with the app — same approach as bugscope-tauri's icebug packaging — so PageRank etc. run as Cypher UDF/GDS functions.Packaging
scripts/download_icebug.sh: platform prebuilt →./icebug/.cargo/config.toml:ICEBUG_DIR+ rpaths for dist/dev/.applayouts (RPATH-not-RUNPATH on Linux so transitive arrow/omp resolve)build.rs:-rdynamicforLOAD EXTENSION(ELF only; breaks Apple linking), icebug presence check, Windowsnetworkit_statelinkscripts/build_algo_extension.sh: cmake build ofLadybugDB/extensionsalgo (pinned SHA) against the ladybug tag matching thelbugcrate + the shared icebug prebuiltscripts/stage_macos_bundle.sh: Frameworks staging +@rpathrewrite (incl. the dlopen'd extension)Runtime
BUGSCOPE_ALGO_EXTENSIONoverride); missing bundle degrades gracefullybackend::graphr_page_rank: in-process icebug PageRank cross-checkicebug =13.2.0behind defaulticebug-analyticsfeature; arrow 55→56 to matchTests
tests/gds_page_rank.rsportsGDSPageRankStar(exact ranks, sum-to-1, scan-fallback equality) andGDSPageRankStaleProjectionfromalgo/test/test_files/gds_page_rank.test; skips without the bundleICEBUG_ENABLED=ON)Watch items for first CI run: Windows has no upstream precedent for building the algo extension (MSVC+vcpkg arrow path is extrapolated), and the Linux libomp-20 soname assumption is unverified.