Optionally keep the binary a sample was submitted as (#95) - #178
Open
r0ny123 wants to merge 2 commits into
Open
Conversation
Binaries submitted through /samples/binary were disassembled and discarded (danielplohmann#95). With STORAGE_KEEP_SUBMITTED_BINARIES (default off) addBinarySample stores the raw bytes in a GridFS bucket of the storage database, keyed by sample id and removed with the sample; GET /samples/{id}/binary and McritClient.getSampleBinary hand them out, 404/None when none was kept.
…ob in the tuning guide
Contributor
Author
|
Live verification on the MongoDB-backed instance with
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #95 (option to store submitted binaries in MCRIT).
Changes
STORAGE_KEEP_SUBMITTED_BINARIES(default off): when set,Worker.addBinarySamplestores the raw bytes after the sample is indexed and reportsbinary_storedin the job result. Nothing changes for SMDA report submissions, which carry no binary.storeSampleBinary(),getSampleBinary(),deleteSampleBinary()on the interface. The MongoDB backend keeps them in a GridFS bucketsample_binariesof the storage database, keyed bymetadata.sample_id(indexed), replaces on re-store, removes them withdeleteSampleand drops the bucket withclearStorage. The memory backend keeps them in a dict.GET /samples/{id}/binaryanswers the bytes asapplication/octet-stream, 404 for an unknown sample or one without a stored binary;McritClient.getSampleBinary()answers bytes or None.Verification
tests/testSampleBinaries.py(4 tests): the route serves octets and 404s both ways; the client answers bytes or None; the worker stores the binary for the indexed sample only when configured.tests/testStorage.py(both backends): store, read back, replace, delete, and the binary goes with its sample.ruff check,ruff format --check,ty checkclean.