Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
df78787
Parquet extract module for KDB-X
ConorSwainDI Jul 24, 2026
62f1b9c
Remove dependency on qmamba
ConorSwainDI Aug 18, 2026
6509036
Fix typos
ConorSwainDI Aug 18, 2026
1e8abb5
Fix error in documentation
ConorSwainDI Aug 18, 2026
a0d121b
Modify comment
ConorSwainDI Aug 19, 2026
3b38b5c
Enlist syms in calcsize; Reference timecol opt in manifest records
ConorSwainDI Aug 19, 2026
36a657a
Changes based off review feedback
ConorSwainDI Aug 19, 2026
dd5dc01
Change to directory/file delete/create behaviour
ConorSwainDI Aug 19, 2026
1588e93
Newline to end file
ConorSwainDI Aug 19, 2026
5193417
Remove unnecessary getdefault
ConorSwainDI Aug 19, 2026
651e13e
Return manifest entries from extract
ConorSwainDI Aug 20, 2026
5322a79
Changes in docs to reflect new behavhiour
ConorSwainDI Aug 20, 2026
9229af5
Implement one sym per file functionality
ConorSwainDI Aug 27, 2026
f3db658
Add dictcol functionality
ConorSwainDI Aug 28, 2026
48edb18
Virtual table functionality
ConorSwainDI Aug 31, 2026
f0f76fc
Type check on dictcols
ConorSwainDI Aug 31, 2026
2a70c96
Minor change remove key check
ConorSwainDI Sep 1, 2026
1ef2a37
Rename dictcols to virtualcols
ConorSwainDI Sep 1, 2026
e6850c5
Change manifest output to JSON
ConorSwainDI Sep 1, 2026
361466d
Docs changes
ConorSwainDI Sep 1, 2026
64f395c
Docs changes
ConorSwainDI Sep 1, 2026
70e1d3b
Add single row assumption
ConorSwainDI Sep 1, 2026
0077dcd
Add version
ConorSwainDI Sep 2, 2026
f853c6e
Add docs on codecs
ConorSwainDI Sep 2, 2026
2f85412
Fix codec docs
ConorSwainDI Sep 2, 2026
9290061
Add logic to catch bad inputs for buildvirtualtable
ConorSwainDI Sep 2, 2026
936118e
Variable rename
ConorSwainDI Sep 2, 2026
b5ace39
Hide castvirtualcols
ConorSwainDI Sep 3, 2026
48135a5
Modify behaviour of buildvirtualtable to accept types for the virtual…
ConorSwainDI Sep 3, 2026
8bbe23f
Fix logging and success path
ConorSwainDI Sep 3, 2026
d31e3dc
Missing semicolon
ConorSwainDI Sep 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions di/pqx/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.1.0
20 changes: 20 additions & 0 deletions di/pqx/init.q
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/ KDB-X Parquet extract module to save kdb+ data to parquet storage convention

arrow:use`kx.arrow
pq:use`kx.pq
pqt:use`kx.pq.t

\l ::pqx.q

/ module version, read from the VERSION file (one plain-text file to bump per release). read
/ module-relative at load (`:::` resolves to di/pqx) and BEFORE export, since export:([...])
/ evaluates each name; version stays in the export so di.depcheck reads it from the export dict.
/ trim, and fail LOUD on a missing/unreadable/empty VERSION, rather than a bare `first read0`:
/ a raw OS error names no module, and an empty or whitespace-padded value is worse than an error -
/ di.depcheck compares versions as STRINGS, so padding silently breaks the comparison and an empty
/ value reads as "exports no version", failing every dependent module's check for a reason that
/ points nowhere near the real cause.
version:@[{trim first read0 x};`:::VERSION;{'"di.pqx: VERSION file missing or unreadable"}];
if[0=count version;'"di.pqx: VERSION file is empty"];

export:([init;extract;getmanifest;checkandconvertcols;estimate;plan;writefile;readfile;tryfn;buildvirtualtable;checkvirtuallevels;checkvirtualtypes;version])
Loading