You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
create_or_update calls new_record.save / existing_record.save with no rescue, and this loop has
no rescue either. Any unhandled exception raised while persisting or triggering a single record —
whether from a before_save-phase save trigger (row not yet persisted) or an after_commit-phase
save trigger (row already persisted, e.g. create_reference, add_tracker, generate_document) —
propagates all the way up through store to retrieve_validate_store, which logs the error to the
job request and re-raises, aborting the entire pull run.
This means a single bad/edge-case REDCap record (e.g. one that fails a downstream save-trigger
action such as create_reference due to unexpected data, or simply fails field validations) currently blocks every other record in
that pull from being processed, rather than being skipped/reported individually.
Requirement
Add an option (opt-in, not a change of default behavior) that allows individual record
exceptions during a REDCap pull's store phase to be caught and recorded, so the pull continues
processing the remaining records instead of aborting the whole run.
Proposed Behavior
Add a project-admin-configurable option (e.g. on Redcap::ProjectAdmin / records_request_options, following existing patterns like verify_file_fields) to enable
per-record exception isolation during store.
When enabled, wrap the per-record create_or_update call in store's loop in a rescue that:
Catches the exception for that record only.
Records it in errors (consistent with the existing { id:, errors:, action: } shape used
elsewhere in this class) so it's visible in the job request result.
Allows the loop to continue to the next record.
When disabled (or by default, to preserve current behavior for callers relying on
fail-fast semantics), retain today's behavior: an unhandled exception aborts the whole pull.
Clarify and document, for a record that fails via a before_save trigger exception, that its
row is not persisted (save aborted), vs. failing via an after_commit trigger exception, where
the row is already committed but subsequent trigger actions for that record did not complete —
both cases should be caught and reported the same way under this option.
Ensure capture_files' existing per-field rescue behavior is unaffected (it already isolates
failures without this option).
Acceptance Criteria (tests)
Spec demonstrating that with the option disabled (default), a single record whose save
trigger raises still aborts the whole store run (current/documented behavior, regression guard).
Spec demonstrating that with the option enabled, a record whose save trigger raises is
recorded in errors and other records in the same pull are still created/updated.
Spec covering both a before_save-phase trigger failure and an after_commit-phase trigger
failure under the new option.
Follow Rspec project coding standards; add a
comment at the top of the new/modified spec file(s) explaining their purpose.
Related context
Related issue: #1382 "Add test coverage for save_trigger before_save execution and exception semantics".
Both stem from investigating whether save trigger failures during REDCap pulls roll back the
individual record or the entire pull (they currently do neither correctly — the record's own
persistence is unaffected by after_commit-phase failures, but the entire pull is aborted
regardless of which phase raised).
Summary
Currently,
Redcap::DataRecords#storeprocesses retrieved records in a loop(app/models/redcap/data_records.rb):
create_or_updatecallsnew_record.save/existing_record.savewith no rescue, and this loop hasno rescue either. Any unhandled exception raised while persisting or triggering a single record —
whether from a
before_save-phase save trigger (row not yet persisted) or anafter_commit-phasesave trigger (row already persisted, e.g.
create_reference,add_tracker,generate_document) —propagates all the way up through
storetoretrieve_validate_store, which logs the error to thejob request and re-raises, aborting the entire pull run.
This means a single bad/edge-case REDCap record (e.g. one that fails a downstream save-trigger
action such as
create_referencedue to unexpected data, or simply fails field validations) currently blocks every other record inthat pull from being processed, rather than being skipped/reported individually.
Requirement
Add an option (opt-in, not a change of default behavior) that allows individual record
exceptions during a REDCap pull's store phase to be caught and recorded, so the pull continues
processing the remaining records instead of aborting the whole run.
Proposed Behavior
Redcap::ProjectAdmin/records_request_options, following existing patterns likeverify_file_fields) to enableper-record exception isolation during
store.create_or_updatecall instore's loop in a rescue that:errors(consistent with the existing{ id:, errors:, action: }shape usedelsewhere in this class) so it's visible in the job request result.
fail-fast semantics), retain today's behavior: an unhandled exception aborts the whole pull.
before_savetrigger exception, that itsrow is not persisted (save aborted), vs. failing via an
after_committrigger exception, wherethe row is already committed but subsequent trigger actions for that record did not complete —
both cases should be caught and reported the same way under this option.
capture_files' existing per-field rescue behavior is unaffected (it already isolatesfailures without this option).
Acceptance Criteria (tests)
trigger raises still aborts the whole
storerun (current/documented behavior, regression guard).recorded in
errorsand other records in the same pull are still created/updated.before_save-phase trigger failure and anafter_commit-phase triggerfailure under the new option.
comment at the top of the new/modified spec file(s) explaining their purpose.
Related context
Related issue: #1382 "Add test coverage for save_trigger before_save execution and exception semantics".
Both stem from investigating whether save trigger failures during REDCap pulls roll back the
individual record or the entire pull (they currently do neither correctly — the record's own
persistence is unaffected by
after_commit-phase failures, but the entire pull is abortedregardless of which phase raised).