feat(store): implement pluggable cloud storage for HStore - #3081
feat(store): implement pluggable cloud storage for HStore#3081vaijosh wants to merge 24 commits into
Conversation
…age with S3 support out of the box
apache#3081 -added multipart upload for the S3 storage provider
6a3f496 to
8b6d597
Compare
apache#3081 -added multipart upload for the S3 storage provider
8b6d597 to
4275426
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3081 +/- ##
============================================
- Coverage 39.19% 36.05% -3.14%
- Complexity 264 462 +198
============================================
Files 770 774 +4
Lines 65779 65881 +102
Branches 8726 8736 +10
============================================
- Hits 25779 23756 -2023
- Misses 37247 39483 +2236
+ Partials 2753 2642 -111 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
apache#3081 - Improved test coverage.
VGalaxies
left a comment
There was a problem hiding this comment.
Review summary
- Blocking: yes
- Summary: The change cannot currently guarantee recoverable RocksDB state, may silently disable cloud storage, and contains S3 retry and deployment defects.
- Evidence:
- Static analysis of
git diff origin/master...HEAD - shell syntax and POM XML checks passed
git diff --checkreported whitespace-only errors
- Static analysis of
apache#3081 - Improved review comments. - Synced METADATA, CURRENT AND OPTIONS to cloud so that we can recover the DB after cluster crash scenarios. - Improved the resiliancy and reduced the data loss possibilities.
apache#3081 - Implemented review comments, Delete or tombstone the remote database generation during database destruction and make hydration honor that generation marker. Updated the runbooks and documentation for this. - Fixed the test coverage issues reported by previous run
apache#3081 - Implemented review comment Cloud upload blocks the RocksDB event thread, S3 service failures bypass the provider retry contract,Multipart wrapping erases the direct-DLQ marker,Release metadata lists different dependency versions, improved test reporting.
apache#3081 - Added few missing configs in hugegraph-store/hg-store-dist/src/assembly/static/conf/application.yml
apache#3081 - Fixed UT failures and code coverage issues.
VGalaxies
left a comment
There was a problem hiding this comment.
🚨 Review summary
Important
The cloud recovery path has blocking consistency, configuration, deletion-safety, release-metadata, and verification gaps.
📊 Risk dashboard
| Signal | Result |
|---|---|
| 🚦 Review gate | Blocked |
| 🔎 Actionable findings | 9 (8 High, 1 Medium) |
| 🧪 Verification coverage | 5 checks |
| 2 / 1 |
🔬 Coverage details
⚠️ S3 failure semantics — Confirmed
Trace: S3CloudStorageProvider upload, download, delete, and exception-classification paths, CloudStorageEventListener purge and hydration flows, AWS SDK 2.33.8 retry-code and file-transformer behavior
Conclusion: Confirmed ignored per-object deletion errors, retryable error-code misclassification, and non-atomic download recovery risks; lower-impact lifecycle leads were dropped.
⚠️ Release and E2E integrity — Confirmed
Trace: install-dist release LICENSE and known-dependencies inventory, cloud-storage Docker artifact build and recovery workflow, Maven provider packaging and test activation
Conclusion: Confirmed the release dependency inventory mismatch and E2E false-green paths caused by suppressed build and wipe failures.
🟡 Focused unit suites — Limited
Trace: hg-store-common cloud configuration, provider, and factory tests, hg-store-cloud-s3 exception-classification tests, hg-store-node listener, retry, metrics, tracker, configuration, and callback tests, hg-store-core BusinessHandlerImplTest
Conclusion: Surefire reports recorded 184 tests with 0 failures, 0 errors, and 0 skipped. The suites do not cover concurrent metadata publication, comma-separated data roots, or partial S3 batch deletion.
✅ Maven reactor — Clear
Trace: Full 44-project reactor with the cloud-s3 profile, hugegraph-store module ordering and dependency graph
Conclusion: Maven validate completed successfully for all 44 reactor projects; the suspected duplicate cloud-s3 module activation did not reproduce.
✅ Shell syntax — Clear
Trace: pd-entrypoint.sh, store-entrypoint.sh, test-graph-queries-and-sst.sh
Conclusion: bash -n returned successfully for all three changed shell entrypoints and workflows.
Warning
Verification limits
- No live Docker and MinIO recovery drill was run, so cluster recovery and purge behavior were not exercised end to end.
- The test environment used Java 17; the legacy JaCoCo agent emitted unsupported class-version instrumentation errors, so coverage instrumentation was not verified even though test assertions passed.
- S3SingleLargeFileE2ETest was intentionally not run because its default path generates a 20 GiB file.
- The independent recovery-consistency run timed out after 900 seconds and produced no usable result.
🤖 Codex review · GPT-5.6 Sol · effort: xhigh
apache#3081 - Implemented review comments.
Hi @VGalaxies. |
apache#3081 - Improved code coverage.
apache#3081 - Improved code coverage.
|
@VGalaxies Not sure why the coverage is suddenly dropped at project level to 30.60% (-11.72%) compared to e960cc5 codecov/projectFailing after 1s — 30.60% (-11.72%) compared to e960cc5 |
There was a problem hiding this comment.
Pull request overview
Implements a pluggable cloud storage layer for HugeGraph Store (HStore), integrating with RocksDB lifecycle events to offload/hydrate SSTs via an SPI-based provider model (with an initial S3 provider), plus local Docker/MinIO tooling and expanded test coverage.
Changes:
- Introduces the cloud storage SPI (
CloudStorageProvider,CloudStorageProviderFactory) and config model, plus S3 provider module (hg-store-cloud-s3). - Hooks cloud lifecycle into RocksDB open/truncate and read-miss hydration paths, and adds metrics + tracking utilities for durability/sync observability.
- Adds local dev/integration environment under
docker/cloud-storage/and updates release docs (LICENSE/NOTICE/known-dependencies) for new third-party deps.
Reviewed changes
Copilot reviewed 56 out of 58 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| pom.xml | Excludes cloud-storage runtime artifacts/generated dirs from checks. |
| install-dist/scripts/dependency/regenerate_known_dependencies.sh | Adds helper script to regenerate known runtime dependencies list. |
| install-dist/scripts/dependency/known-dependencies.txt | Updates runtime dependency inventory for new cloud/S3 deps. |
| install-dist/release-docs/NOTICE | Adds AWS SDK v2 notice block. |
| install-dist/release-docs/licenses/LICENSE-reactive-streams-1.0.4.txt | Adds MIT-0 license text for reactive-streams. |
| install-dist/release-docs/LICENSE | Updates third-party license inventory (AWS SDK v2, Netty, etc.). |
| hugegraph-store/pom.xml | Adds hg-store-cloud-s3 module and a build profile stanza. |
| hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBSessionTest.java | Test updates/imports related to new RocksDB behaviors. |
| hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBFactoryTest.java | Adds tests for metadata checkpoint flushing + hydration/open ordering + retryability. |
| hugegraph-store/hg-store-rocksdb/src/main/java/org/apache/hugegraph/rocksdb/access/SessionOperatorImpl.java | Adds read-miss hook to trigger cloud hydration and retry reads. |
| hugegraph-store/hg-store-rocksdb/src/main/java/org/apache/hugegraph/rocksdb/access/RocksDBSession.java | Adds pre-open hydration hook, listener registration, truncate notifications, metadata checkpoint capture. |
| hugegraph-store/hg-store-node/src/test/java/org/apache/hugegraph/store/node/metrics/CloudStorageMetricsTest.java | Adds unit tests for cloud storage metrics registration/lifecycle. |
| hugegraph-store/hg-store-node/src/test/java/org/apache/hugegraph/store/node/cloud/CloudSyncTrackerTest.java | Adds unit tests for SST confirmation tracking bitmap logic. |
| hugegraph-store/hg-store-node/src/test/java/org/apache/hugegraph/store/node/cloud/CloudRecoveryIntegrationTest.java | Adds end-to-end recovery + JNI boundary failure integration tests with in-memory provider. |
| hugegraph-store/hg-store-node/src/test/java/org/apache/hugegraph/store/node/AppConfigCloudStorageTest.java | Adds Spring binding + stable cloud-scope resolution tests. |
| hugegraph-store/hg-store-node/src/main/resources/application.yml | Adds default cloud.storage config (disabled by default) for node. |
| hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/FailedUploadTask.java | Adds DLQ task model for failed uploads with epoch safety. |
| hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/CloudSyncTracker.java | Adds confirmed-SST tracking + epoch-based stale callback protection. |
| hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/CloudStorageMetricsConst.java | Adds metric name/tag constants for cloud storage monitoring. |
| hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/CloudStorageMetrics.java | Adds Micrometer metric registration and per-DB meter cleanup helpers. |
| hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/AppConfig.java | Wires cloud storage initialization/shutdown, metrics binding, stable scope persistence. |
| hugegraph-store/hg-store-node/pom.xml | Adds cloud S3 module + RoaringBitmap dependency. |
| hugegraph-store/hg-store-dist/src/assembly/static/conf/application.yml | Documents and adds full cloud.storage configuration block. |
| hugegraph-store/hg-store-core/src/test/java/org/apache/hugegraph/store/business/BusinessHandlerImplTest.java | Adds/merges tests (including truncate callback expectations). |
| hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/business/BusinessHandlerImpl.java | Fires truncate callbacks around key-range delete. |
| hugegraph-store/hg-store-common/src/test/resources/META-INF/services/org.apache.hugegraph.store.cloud.CloudStorageProvider | Adds test SPI registration for discovery/duplicate-name testing. |
| hugegraph-store/hg-store-common/src/test/java/org/apache/hugegraph/store/cloud/TestDuplicateNamedProvider.java | Adds test-only provider for SPI duplicate-name behavior. |
| hugegraph-store/hg-store-common/src/test/java/org/apache/hugegraph/store/cloud/CloudStorageProviderTest.java | Adds tests for provider default method behaviors (list/deletePrefix). |
| hugegraph-store/hg-store-common/src/test/java/org/apache/hugegraph/store/cloud/CloudStorageProviderFactoryTest.java | Adds tests for factory init/disable/swap behavior and SPI discovery paths. |
| hugegraph-store/hg-store-common/src/test/java/org/apache/hugegraph/store/cloud/CloudStorageNonRetryableExceptionTest.java | Adds tests for non-retryable exception type. |
| hugegraph-store/hg-store-common/src/main/java/org/apache/hugegraph/store/cloud/CloudStorageProviderFactory.java | Introduces SPI discovery + provider lifecycle management. |
| hugegraph-store/hg-store-common/src/main/java/org/apache/hugegraph/store/cloud/CloudStorageProvider.java | Introduces provider SPI interface with default bulk-delete helpers. |
| hugegraph-store/hg-store-common/src/main/java/org/apache/hugegraph/store/cloud/CloudStorageNonRetryableException.java | Adds marker exception for non-retryable cloud failures. |
| hugegraph-store/hg-store-common/src/main/java/org/apache/hugegraph/store/cloud/CloudStorageConfig.java | Adds cloud storage configuration POJO and common knobs. |
| hugegraph-store/hg-store-cloud-s3/src/test/java/org/apache/hugegraph/store/cloud/s3/S3CloudStorageProviderTest.java | Adds S3 provider unit tests (exception classification, pagination safety, lifecycle). |
| hugegraph-store/hg-store-cloud-s3/src/main/resources/META-INF/services/org.apache.hugegraph.store.cloud.CloudStorageProvider | Registers S3 provider via SPI. |
| hugegraph-store/hg-store-cloud-s3/src/main/java/org/apache/hugegraph/store/cloud/s3/S3CloudStorageConfig.java | Adds S3 provider config POJO and keys/defaults. |
| hugegraph-store/hg-store-cloud-s3/pom.xml | Adds S3 provider module deps (AWS SDK v2) and test configuration. |
| docker/README.md | Documents selecting an alternate Store Dockerfile via env var. |
| docker/docker-compose.dev.yml | Makes Store Dockerfile configurable via HG_STORE_DOCKERFILE. |
| docker/cloud-storage/images/store.Dockerfile | Adds cloud-storage local Store image build for MinIO/dev. |
| docker/cloud-storage/images/pd.Dockerfile | Adds cloud-storage local PD image build. |
| docker/cloud-storage/entrypoints/store-entrypoint.sh | Adds Store entrypoint wiring Spring JSON + cloud env vars. |
| docker/cloud-storage/entrypoints/pd-entrypoint.sh | Adds PD entrypoint wiring Spring JSON env vars. |
| docker/cloud-storage/docker-compose.yml | Adds MinIO + PD + 3 Store nodes + Server local integration stack. |
| docker/cloud-storage/.gitignore | Ignores local artifacts/logs/data under cloud-storage dev setup. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
apache#3081 - Implemented review comments
apache#3081 - Added test to improve patch coverage.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 57 out of 59 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/business/BusinessHandlerImpl.java:1042
- The new truncate() block is not indented like the rest of the file (4-space indent), which makes the method hard to read and diverges from surrounding code style. Please re-indent the try-with-resources and its inner blocks consistently.
try (RocksDBSession dbSession = getSession(graphName, partId)) {
String dbName = dbSession.getGraphName();
String dbPath = dbSession.getDbPath();
factory.notifyTruncateBegin(dbName, dbPath);
boolean truncated = false;
try {
dbSession.sessionOp().deleteRange(keyCreator.getStartKey(partId, graphName),
keyCreator.getEndKey(partId, graphName));
// Release map ID
keyCreator.delGraphId(partId, graphName);
truncated = true;
} finally {
if (truncated) {
factory.notifyTruncate(dbName, dbPath);
} else {
factory.notifyTruncateAbort(dbName, dbPath);
}
}
}
}
hugegraph-store/hg-store-common/src/main/java/org/apache/hugegraph/store/cloud/CloudStorageConfig.java:156
- The Javadoc for providerProperties ends mid-sentence (trailing comma), which makes the public configuration docs look incomplete. Please finish the sentence and clarify what belongs in this map.
/**
* Provider-specific properties, mapped from {@code cloud.storage.<provider>.*} in the configuration.
* These are passed verbatim to the provider implementation and may include credentials,
*/
private Map<String, String> providerProperties = new HashMap<>();
apache#3081 - Implemented minor changes suggested by Copilot agent review.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 57 out of 59 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (9)
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/SessionOperatorReadMissTest.java:1
- The test creates a RocksDBSession but never closes it, relying on factory destroy/cleanup. This can leak native RocksDB resources and can make temp-dir deletion flaky (especially on Windows). Use try-with-resources for
session(or explicitlysession.close()beforedestroyGraphDB).
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBFactoryTest.java:1 - Like the read-miss test, this test creates a
RocksDBSessionthat is not closed on the success path. Please wrap the session in try-with-resources (or close it infinally) to avoid leaking native handles and to reduce flakiness when deleting the temp directory.
hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/CloudStorageMetricsConst.java:1 - This metric is registered as a Gauge in
CloudStorageMetrics, but the name suffix_totalis conventionally reserved for Counters in Prometheus/Micrometer ecosystems. Consider renaming it to something gauge-appropriate (e.g.,_confirmed_filesor_confirmed_files_count) to avoid confusion and incorrect dashboard assumptions.
hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/CloudStorageMetrics.java:1 - Catching
IllegalArgumentExceptionas a control-flow mechanism is brittle here: Micrometer throwsIllegalArgumentExceptionfor multiple reasons (not only 'already registered'), and a partial registration could leaveperDbMetersmissing/incorrect (makingunregisterDatabaseMetrics()unable to remove all meters). A safer approach is to guard viaperDbMeters.containsKey(dbName)/computeIfAbsentand only register once per dbName, and only swallow the specific duplicate-registration scenario if you can reliably detect it.
hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/cloud/CloudStorageMetrics.java:1 - Catching
IllegalArgumentExceptionas a control-flow mechanism is brittle here: Micrometer throwsIllegalArgumentExceptionfor multiple reasons (not only 'already registered'), and a partial registration could leaveperDbMetersmissing/incorrect (makingunregisterDatabaseMetrics()unable to remove all meters). A safer approach is to guard viaperDbMeters.containsKey(dbName)/computeIfAbsentand only register once per dbName, and only swallow the specific duplicate-registration scenario if you can reliably detect it.
hugegraph-store/hg-store-node/src/test/java/org/apache/hugegraph/store/node/AppConfigCloudStorageTest.java:1 - The last assertion is ineffective:
readProviderProperties()strips thecloud.storage.<provider>.prefix, so the map will never contain a key likecloud.storage.s3.bucketregardless of implementation correctness. To validate isolation, assert something observable in the returned map (e.g., thatbucketequalsgcs-bucketand nots3-bucket, and/or that only the expected gcs keys are present).
hugegraph-store/hg-store-common/src/test/java/org/apache/hugegraph/store/cloud/CloudStorageNonRetryableExceptionTest.java:58 - This test currently passes even if the
serialVersionUIDfield is missing or inaccessible because the catch block swallows the failure. The catch should fail the test (e.g., viafail(...)) so that regressions in serialization metadata are actually detected.
public void testSerialVersionUID() {
try {
java.lang.reflect.Field field =
CloudStorageNonRetryableException.class.getDeclaredField("serialVersionUID");
field.setAccessible(true);
Long serialVersionUID = (Long) field.get(null);
assertEquals(Long.valueOf(1L), serialVersionUID);
} catch (NoSuchFieldException | IllegalAccessException e) {
// serialVersionUID field exists as expected
}
}
hugegraph-store/hg-store-common/src/main/java/org/apache/hugegraph/store/cloud/CloudStorageConfig.java:157
- Since
providerPropertiesis a mutableMap, API consumers can accidentally share and mutate it across components (and tests in this PR even rely on aliasing). For safer config handling, consider defensively copying the map in the setter/getter (or returning an unmodifiable view) so provider configuration is stable after initialization.
/**
* Provider-specific properties, mapped from {@code cloud.storage.<provider>.*} in the
* configuration. These are passed verbatim to the provider implementation and may include
* credentials, endpoint overrides, regions, or SDK tuning keys expected by that provider.
*/
private Map<String, String> providerProperties = new HashMap<>();
docker/cloud-storage/docker-compose.yml:30
- Using
minio/minio:latestmakes the dev/integration environment non-reproducible and can break unexpectedly when MinIO releases change behavior. Pin to a specific MinIO version tag (similar to howminio/mcis pinned below) to keep CI/dev validation deterministic.
minio:
image: minio/minio:latest
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 57 out of 59 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (6)
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBFactoryTest.java:1
- This test hard-asserts an implementation detail of RocksDB checkpoints (that they flush memtables and emit SST-created events), and uses time-based waiting +
System.out.printf, which can make CI runs flaky across platforms/RocksDB versions. Consider making the test assert only the project-level contract you need (e.g., that your chosen metadata-sync path does/doesn’t trigger file-created callbacks under your own code), or convert this into a non-failing diagnostic/assumption-style probe and remove stdout printing (use logging if needed).
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBFactoryTest.java:1 - This test hard-asserts an implementation detail of RocksDB checkpoints (that they flush memtables and emit SST-created events), and uses time-based waiting +
System.out.printf, which can make CI runs flaky across platforms/RocksDB versions. Consider making the test assert only the project-level contract you need (e.g., that your chosen metadata-sync path does/doesn’t trigger file-created callbacks under your own code), or convert this into a non-failing diagnostic/assumption-style probe and remove stdout printing (use logging if needed).
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBFactoryTest.java:1 - This test hard-asserts an implementation detail of RocksDB checkpoints (that they flush memtables and emit SST-created events), and uses time-based waiting +
System.out.printf, which can make CI runs flaky across platforms/RocksDB versions. Consider making the test assert only the project-level contract you need (e.g., that your chosen metadata-sync path does/doesn’t trigger file-created callbacks under your own code), or convert this into a non-failing diagnostic/assumption-style probe and remove stdout printing (use logging if needed).
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/SessionOperatorReadMissTest.java:1 RocksDBSessionis never explicitly closed here. Relying ondestroyGraphDB()for cleanup can leak native resources if the session remains referenced or if destroy fails, and it can also cause temp directory deletion failures on some platforms. Consider using try-with-resources forRocksDBSession(or closing it in the finally block before destroy) to make the test cleanup deterministic.
hugegraph-store/hg-store-node/src/test/java/org/apache/hugegraph/store/node/metrics/CloudStorageMetricsTest.java:1- The static reset helper clears several fields but does not reset the
*_supplierstatic fields (retryQueueSizeSupplier,dlqPersistenceHealthySupplier,deleteMarkerHealthySupplier). Because those suppliers can capture references to prior test objects, leaving them untouched can introduce hidden inter-test coupling and memory leaks. Consider explicitly resetting them to their documented defaults (0/1) as part ofresetStaticState().
docker/cloud-storage/docker-compose.yml:31 - Using
minio/minio:latestmakes the local integration environment non-reproducible and can cause sudden CI/dev breakages when MinIO changes behavior. Pin to a specific MinIO release tag (similar to howminio/mcis pinned) so recovery/hydration tests remain stable over time.
image: minio/minio:latest
container_name: cloud-storage-minio
apache#3081 - Implemented minor review comments by Copilot
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 58 out of 60 changed files in this pull request and generated no new comments.
Suppressed comments (1)
hugegraph-store/hg-store-rocksdb/src/test/java/org/apache/hugegraph/rocksdb/access/RocksDBSessionTest.java:24
- This test file now has unused imports (e.g., static import
fail,java.nio.file.Files/Path, andorg.junit.Test). In Java, unused imports fail compilation, and this class currently has no active@Testmethods (the annotations are commented out), so these imports should be removed (or the tests re-enabled).
apache#3081 - Removed unused imports suggested by copilot review.
Removed unused imports. |
Purpose
This PR introduces a cloud storage architecture for HugeGraph Store (HStore). In cloud-native environments, storage nodes are ephemeral, making reliance on local disk storage a single point of failure. This implementation decouples the storage layer from local disk dependencies, allowing SST files to be offloaded to durable, scalable cloud storage providers.
Key Changes
Pluggable Architecture
CloudStorageProviderSPI interface to enable extensible storage backends.CloudStorageProviderFactoryfor seamless provider discovery and lifecycle management.S3 Provider Implementation
hg-store-cloud-s3module, leveraging AWS SDK v2 for production-ready S3/S3 compatible storage interaction.Lifecycle & Event Integration
CloudStorageEventListenerwithRocksDBFactory. This hooks into critical SST lifecycle events (onTableFileCreated,onTableFileDeleted) to ensure synchronized state between local RocksDB and cloud storage.onDBCreated) to backfill pre-existing files and read-miss on-demand hydration to ensure data availability.Infrastructure & Testing
docker/cloud-storage/using MinIO.test-graph-queries-and-sst.sh) to verify end-to-end data durability and query consistency.Implementation Highlights
META-INF/services.RocksdbEventListenerwithinRocksDBFactoryto intercept file operations without modifying core RocksDB logic.onReadMissto prevent redundant hydration requests during high-concurrency read scenarios.Verifying These Changes
CloudStorageConfigTest), factory registration, and event listener logic.docker-composesetup with MinIO.application.ymlbindings for credentials, bucket management, and sync intervals.Impact
LICENSEandNOTICEaccordingly.cloud.storagenamespace toapplication.yml(disabled by default).Documentation
hugegraph-store/docs/pluggable-cloud-storage-architecture.md.