Skip to content

Optionally keep the binary a sample was submitted as (#95) - #178

Open
r0ny123 wants to merge 2 commits into
danielplohmann:mainfrom
r0ny123:feat/95-keep-submitted-binaries
Open

Optionally keep the binary a sample was submitted as (#95)#178
r0ny123 wants to merge 2 commits into
danielplohmann:mainfrom
r0ny123:feat/95-keep-submitted-binaries

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Closes #95 (option to store submitted binaries in MCRIT).

Changes

  • STORAGE_KEEP_SUBMITTED_BINARIES (default off): when set, Worker.addBinarySample stores the raw bytes after the sample is indexed and reports binary_stored in the job result. Nothing changes for SMDA report submissions, which carry no binary.
  • Storage: storeSampleBinary(), getSampleBinary(), deleteSampleBinary() on the interface. The MongoDB backend keeps them in a GridFS bucket sample_binaries of the storage database, keyed by metadata.sample_id (indexed), replaces on re-store, removes them with deleteSample and drops the bucket with clearStorage. The memory backend keeps them in a dict.
  • GET /samples/{id}/binary answers the bytes as application/octet-stream, 404 for an unknown sample or one without a stored binary; McritClient.getSampleBinary() answers bytes or None.

Verification

  • tests/testSampleBinaries.py (4 tests): the route serves octets and 404s both ways; the client answers bytes or None; the worker stores the binary for the indexed sample only when configured.
  • tests/testStorage.py (both backends): store, read back, replace, delete, and the binary goes with its sample.
  • Full suite: 209 passed; ruff check, ruff format --check, ty check clean.
  • Live verification on the MongoDB-backed instance follows in a comment.

Binaries submitted through /samples/binary were disassembled and discarded (danielplohmann#95). With STORAGE_KEEP_SUBMITTED_BINARIES (default off) addBinarySample stores the raw bytes in a GridFS bucket of the storage database, keyed by sample id and removed with the sample; GET /samples/{id}/binary and McritClient.getSampleBinary hand them out, 404/None when none was kept.
@r0ny123

r0ny123 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Live verification on the MongoDB-backed instance with STORAGE_KEEP_SUBMITTED_BINARIES = True (server and worker restarted on this change):

  • Submitted /usr/bin/cat (39,384 bytes) through McritClient.addBinarySample. The job result carried binary_stored: True for the new sample, and one file sat in the sample_binaries bucket for it.
  • getSampleBinary(sample_id) answered bytes equal to the submitted file.
  • getSampleBinary(0), a sample that was submitted as an SMDA report, answered None (404).
  • deleteSample(sample_id) removed the sample and its bucket file with it (0 files afterwards).

@r0ny123 r0ny123 mentioned this pull request Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Option to store submitted binaries in MCRIT

1 participant