feat(connectors): let sinks control when offsets are committed

[kriti-sc]

Closes #2928, #2927
Related: #3203

Rationale

Sinks must have an option to decide exactly when it is safe to advance the offset, as only the sink knows when a write is truly durable from the target's perspective.

What changed?

This PR adds an offset_commit setting to sink configs:

• after_polling (default) - existing behaviour of offset being committed when messages are polled. Nothing changes for existing configs.
• after_consuming - the offset is stored only once the sink reports that it wrote the batch.

The offset commit decision hinges upon the i32 returned by the sink.

This also required binding the i32 returned by the sink, which was previous discarded. Now, the sink exits if i32 != 0, regardless of offset commit configuration, which is the correct behaviour as a sink returns error for failures lying outside its purview.

This PR also introduces integration tests that inject failures into a running sink and verify the desired commit behaviour.

[github-actions]

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

• /ready - back to S-waiting-on-review after addressing feedback
• /author - flip to S-waiting-on-author while you finish changes
• /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.03%. Comparing base (e362c0f) to head (9766124).

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3954      +/-   ##
============================================
+ Coverage     84.98%   85.03%   +0.04%     
  Complexity     1402     1402              
============================================
  Files          1226     1225       -1     
  Lines        180701   179798     -903     
  Branches     147005   146102     -903     
============================================
- Hits         153575   152889     -686     
+ Misses        23070    22897     -173     
+ Partials       4056     4012      -44     

Components	Coverage Δ
Rust Core	85.94% <ø> (+0.06%)	⬆️
Java SDK	67.29% <ø> (ø)
C# SDK	75.37% <ø> (ø)
Python SDK	90.06% <ø> (ø)
PHP SDK	85.65% <ø> (ø)
Node SDK	96.22% <ø> (-0.03%)	⬇️
Go SDK	69.35% <ø> (+0.03%)	⬆️

Files with missing lines	Coverage Δ
core/connectors/runtime/src/configs/connectors.rs	41.07% <ø> (ø)
core/connectors/runtime/src/error.rs	26.66% <ø> (ø)
core/connectors/runtime/src/main.rs	85.76% <ø> (ø)
core/connectors/runtime/src/manager/sink.rs	94.16% <ø> (ø)
core/connectors/runtime/src/sink.rs	78.81% <ø> (+2.96%)	⬆️

... and 45 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either @core on Discord or by mentioning them directly here on the PR.

Thank you for your contribution!