nessie

Spring boot Spark Iceberg Demo

Transfer data from relational database tables, views, and query results to Apache Iceberg tables

An end-to-end streaming lakehouse demo using Kafka, Flink, Iceberg, MinIO, Nessie, and Streamlit. Real-time events flow through Flink into Iceberg tables, with metadata managed by Nessie. Streamlit provides an interactive UI for exploring data, branches, commits, and generated alerts.

IoT lakehouse on Kafka 4 (KRaft) + Flink 2.2 + Iceberg 1.10 + Nessie + MinIO. Streaming end-to-end with one docker compose up.

This project demonstrates an end-to-end Lakehouse architecture built entirely with open-source technologies. A Spark application extracts data from PostgreSQL, writes it as Apache Iceberg tables to MinIO object storage, and registers metadata in Nessie. Dremio queries the Iceberg tables directly and exposes them to downstream analytics and BI tool

Trino CatalogStore plugin that loads Iceberg catalogs from PostgreSQL at runtime — no static .properties files, no coordinator restart.