feat(go): add stream/topic purge API with BDD tests and fix server purge bug by chengxilo · Pull Request #3608 · apache/iggy

chengxilo · 2026-07-04T22:28:21Z

Which issue does this PR address?

Relates to #2148

Rationale

Go SDK doesn't have the purge topic/ purge stream API, so this PR implement them. To test it, I added an new BDD test scenario, and implemented both Go and Rust BDD tests.

What changed?

The Go SDK was missing PurgeStream and PurgeTopic methods. Added them along with a shared BDD scenario (stream_topic_purge.feature) and implementations in both Go and Rust.

While testing, discovered that purge_all_segments only cleared on-disk segments but left stale messages in the journal and in-flight tiers. Polling after purge would return messages that should have been deleted. Fixed by calling journal.reset() and clear_in_flight() during partition re-initialization. I am not famliar with Rust server code so during the code review this part worth more attention

Local Execution

Passed
Pre-commit hooks ran

AI Usage

DeepSeek pro v4, Claude Opus 4.6
DeepSeek pro v4 for BDD test implementation, Claude Opus 4.6 for server bug fixing(bug doesn't exists anymore now, 9/2/2026).
I reviewed them with my best effert, the bdd scripts are executed and actually works, and they found that there is a bug in the server so I believe test test works.
I hope so.

github-actions · 2026-07-04T22:28:35Z

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

/ready - back to S-waiting-on-review after addressing feedback
/author - flip to S-waiting-on-author while you finish changes
/request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

codecov · 2026-07-04T22:29:49Z

Codecov Report

❌ Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.13%. Comparing base (f45f9dd) to head (13df54c).
⚠️ Report is 5 commits behind head on master.

Files with missing lines	Patch %	Lines
foreign/go/internal/command/stream.go	0.00%	4 Missing ⚠️
foreign/go/internal/command/topic.go	0.00%	4 Missing ⚠️
foreign/go/client/tcp/tcp_stream_management.go	0.00%	3 Missing ⚠️
foreign/go/client/tcp/tcp_topic_management.go	0.00%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3608      +/-   ##
============================================
- Coverage     85.15%   85.13%   -0.02%     
  Complexity     1402     1402              
============================================
  Files          1230     1226       -4     
  Lines        181317   181327      +10     
  Branches     147611   147607       -4     
============================================
- Hits         154392   154376      -16     
- Misses        22874    22907      +33     
+ Partials       4051     4044       -7

Components	Coverage Δ
Rust Core	`86.06% <ø> (-0.02%)`	⬇️
Java SDK	`67.29% <ø> (ø)`
C# SDK	`75.53% <ø> (ø)`
Python SDK	`90.06% <ø> (ø)`
PHP SDK	`85.65% <ø> (ø)`
Node SDK	`96.24% <ø> (ø)`
Go SDK	`69.19% <0.00%> (-0.12%)`	⬇️

Files with missing lines	Coverage Δ
foreign/go/client/tcp/tcp_stream_management.go	`78.04% <0.00%> (-6.17%)`	⬇️
foreign/go/client/tcp/tcp_topic_management.go	`87.27% <0.00%> (-5.04%)`	⬇️
foreign/go/internal/command/stream.go	`85.00% <0.00%> (-9.45%)`	⬇️
foreign/go/internal/command/topic.go	`83.01% <0.00%> (-3.26%)`	⬇️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

hubcio · 2026-07-09T10:54:10Z

@chengxilo i'll check this during the weekend. could you please rebase?

numinnex · 2026-07-09T11:31:35Z

I've checked whether there is potential concurrency issue due to interleaving of flush + purge commands as this is what I've reckon from the changes could potentially be an issue and did not find any problems with that. Let's wait for @hubcio review and we can merge it.

chengxilo · 2026-07-14T20:11:50Z

/ready

chengxilo · 2026-07-14T20:16:07Z

nothing has been changed, just because I rebased onto master locally so it was forced pushed() I shouldn't do that tho.

hubcio · 2026-08-05T11:24:05Z

@chengxilo could you please rebase again? i will review this today

chengxilo · 2026-08-06T16:03:24Z

@chengxilo could you please rebase again? i will review this today

done. Currently, we get the pre-purge messages back for roughly > a second. I'm not sure if we should justify it in BDD test or fix it. Previously for the non-VSR it was fixed. But not for the VSR enabled one. (since according to the comments, it was intended)

github-actions · 2026-08-22T00:17:22Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either @core on Discord or by mentioning them directly here on the PR.

Thank you for your contribution!

slbotbm · 2026-08-23T21:17:06Z

@chengxilo Seems the rust bdd test is failing?

/author

slbotbm · 2026-08-23T21:25:24Z

+    And I poll messages from stream "purge-test-stream", topic "purge-topic", partition 0 starting from offset 0
+    Then I should receive 0 messages


Could we wait for purge convergence before polling here? The purge command commits a new purge generation, but the partition reconciler clears the messages on a later tick. This immediate poll can therefore still see pre-purge data and make this scenario flaky (faced this while writing tests for cpp). Consider adding bounded retry/wait logic before asserting the message count. The same applies to the other immediate post-purge polls in this feature.

chengxilo · 2026-08-23T22:27:38Z

@chengxilo Seems the rust bdd test is failing?

/author

Yes, I am wondering if this is a behavior that need to be fixed. Though in the comment in server side, it seems to be expected. (I mean like... I personally don't like this behavior.

github-actions · 2026-08-31T00:45:54Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either @core on Discord or by mentioning them directly here on the PR.

Thank you for your contribution!

# Conflicts: # bdd/go/tests/basic_messaging.go # bdd/go/tests/leader_redirection.go

github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jul 4, 2026

chengxilo changed the title ~~test(sdk): add stream/topic purge scenario, go/rust implemented, one server purge bug fixed~~ feat(go): add stream/topic purge API with BDD tests and fix server purge bug Jul 4, 2026

chengxilo mentioned this pull request Jul 5, 2026

BDD Test Coverage Roadmap #3609

Open

hubcio added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 9, 2026

chengxilo added 6 commits July 13, 2026 22:00

feat(go): implement purge stream/topic function

c3efcff

test(sdk): add stream_topic_purge scenario

c410303

test(sdk): update the ./scripts/run-bdd-tests.sh for new scenario

25478a7

test(go): implement stream_topic_purge scenario for go

737b29f

fix(server): clear journal and in-flight data on partition purge

78f08f1

test(rust): implement stream_topic_purge bdd test for rust

f5d74d9

chengxilo force-pushed the feat/go-sdk-purge-stream-topic branch from 8bc60d8 to f5d74d9 Compare July 14, 2026 02:08

github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jul 14, 2026

Merge branch 'master' into feat/go-sdk-purge-stream-topic

a604653

chengxilo force-pushed the feat/go-sdk-purge-stream-topic branch from 9b5e79c to a604653 Compare July 25, 2026 22:29

chengxilo and others added 2 commits August 1, 2026 23:28

Merge branch 'master' into feat/go-sdk-purge-stream-topic

2521c74

Merge branch 'master' into feat/go-sdk-purge-stream-topic

f68bf01

Merge branch 'master' into feat/go-sdk-purge-stream-topic

2064d1d

Merge branch 'master' into feat/go-sdk-purge-stream-topic

2943e47

chengxilo mentioned this pull request Aug 13, 2026

test(bdd): add stream CRUD coverage for Rust and Java #3875

Merged

Merge branch 'master' into feat/go-sdk-purge-stream-topic

b1564fd

github-actions Bot added the S-stale Inactive issue or pull request label Aug 22, 2026

Merge branch 'master' into feat/go-sdk-purge-stream-topic

23c8dc9

github-actions Bot removed the S-stale Inactive issue or pull request label Aug 22, 2026

fix for lint

9ee0b89

github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Aug 23, 2026

slbotbm requested changes Aug 23, 2026

View reviewed changes

chengxilo added 2 commits August 26, 2026 02:11

Merge branch 'master' into feat/go-sdk-purge-stream-topic

59121fd

check whether the bdd supports stream_topic_purge.feature

348e6db

github-actions Bot added the S-stale Inactive issue or pull request label Aug 31, 2026

chengxilo added 3 commits August 31, 2026 09:34

Merge branch 'master' into feat/go-sdk-purge-stream-topic

9c3cdee

# Conflicts: # bdd/go/tests/basic_messaging.go # bdd/go/tests/leader_redirection.go

Merge branch 'master' into feat/go-sdk-purge-stream-topic

d0da8f0

fix(bdd): update the argument for rust SDK change

adf11d3

github-actions Bot removed the S-stale Inactive issue or pull request label Sep 1, 2026

chengxilo added 3 commits September 1, 2026 15:56

Merge branch 'master' into feat/go-sdk-purge-stream-topic

2cf7f02

fix(bdd): rust bdd should fail when skip

9f8da25

fix(bdd): match purge poll step in Then position

13df54c

		And I poll messages from stream "purge-test-stream", topic "purge-topic", partition 0 starting from offset 0
		Then I should receive 0 messages

Uh oh!

Conversation

chengxilo commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR address?

Rationale

What changed?

Local Execution

AI Usage

Uh oh!

github-actions Bot commented Jul 4, 2026

Uh oh!

codecov Bot commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

hubcio commented Jul 9, 2026

Uh oh!

numinnex commented Jul 9, 2026

Uh oh!

chengxilo commented Jul 14, 2026

Uh oh!

chengxilo commented Jul 14, 2026

Uh oh!

hubcio commented Aug 5, 2026

Uh oh!

chengxilo commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 22, 2026

Uh oh!

slbotbm commented Aug 23, 2026

Uh oh!

slbotbm Aug 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chengxilo commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

chengxilo commented Jul 4, 2026 •

edited

Loading

codecov Bot commented Jul 4, 2026 •

edited

Loading

chengxilo commented Aug 6, 2026 •

edited

Loading

chengxilo commented Aug 23, 2026 •

edited

Loading