Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3cf0e81
build: consolidate Rust crates into a Cargo workspace + extract nativ…
timsaucer Jun 12, 2026
fd4dd74
build: inherit crate version/edition from workspace; publish datafusi…
timsaucer Jun 12, 2026
2e50d6e
build: drop forward-referencing RAT excludes from foundation split
timsaucer Jun 12, 2026
9d2096c
build: drop overbroad **/*.md RAT exclude
timsaucer Jun 12, 2026
2514f56
docs: add README for datafusion-jni-common
timsaucer Jun 12, 2026
3ff4fda
fix: scope foundation docs/excludes to what 1/6 ships
timsaucer Jun 12, 2026
bb023eb
build: point release scripts at workspace rust-target/ dir
timsaucer Jun 12, 2026
9e9de7d
build: mark datafusion-jni-common publish = false
timsaucer Jun 12, 2026
e87fd64
feat: add plain-C scan ABI crate over Arrow C Data/Stream
timsaucer Jun 15, 2026
2598322
feat: add scan_config / scan_request protobuf for the C scan ABI
timsaucer Jun 15, 2026
cfaac8d
build: compile scan protobufs into datafusion-scan-ffi
timsaucer Jun 15, 2026
4cded33
feat: add datafusion.listing provider decoding ScanConfig
timsaucer Jun 15, 2026
f95191d
feat: add JNI shim over the plain-C scan ABI
timsaucer Jun 15, 2026
bc91ff9
test: end-to-end JVM scan through the JNI shim
timsaucer Jun 15, 2026
436e2b3
test: projection and filter pushdown from Java
timsaucer Jun 15, 2026
d5eefee
feat: wire row limit through the scan ABI
timsaucer Jun 15, 2026
0b24d7c
feat: add Spark DataSourceV2 connector
timsaucer Jun 15, 2026
1a1d0ec
feat: columnar reader on Spark 4.0 with provided Arrow
timsaucer Jun 15, 2026
573b438
feat: push Spark limit into the scan (SupportsPushDownLimit)
timsaucer Jun 15, 2026
a378bb4
docs: add DataFusion-Spark DataSource design doc
timsaucer Jun 15, 2026
301fd13
docs: add ADBC reuse analysis to Spark design doc
timsaucer Jun 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# Keep Cargo's workspace output out of `target/` so `mvn clean` (which deletes
# the root `target/`) does not nuke the Rust build cache.
[build]
target-dir = "rust-target"
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ jobs:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-${{ hashFiles('native/Cargo.lock') }}
rust-target
key: ${{ runner.os }}-cargo-${{ hashFiles('Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-

- name: Build native and run tests
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
run: ./mvnw -q spotless:check

- name: Check Rust formatting
run: cd native && cargo fmt --all -- --check
run: cargo fmt --all -- --check

clippy:
name: Clippy
Expand All @@ -81,9 +81,9 @@ jobs:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-clippy-${{ hashFiles('native/Cargo.lock') }}
rust-target
key: ${{ runner.os }}-clippy-${{ hashFiles('Cargo.lock') }}
restore-keys: ${{ runner.os }}-clippy-

- name: Run clippy
run: cd native && cargo clippy --all-targets -- -D warnings
run: cargo clippy --workspace --all-targets -- -D warnings
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
target/
rust-target/
*.class
.idea/
.vscode/
Expand Down
Loading