Skip to content

[fix][broker] The offline consumer is not removed from the consumerList #14970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lordcheng10 wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[fix][broker] The offline consumer is not removed from the consumerList #14970

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4d3a3ae
remove all consumer from consumerList
lordcheng10 Mar 31, 2022
eb75a3c
1.do a check in addConsumer;
lordcheng10 Apr 1, 2022
fc1d0b5
check style
lordcheng10 Apr 1, 2022
e1c9596
1.use ConsumerBusyException;
lordcheng10 Apr 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion ...va/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,13 @@ public synchronized void addConsumer(Consumer consumer) throws BrokerServiceExce
throw new ConsumerBusyException("Subscription reached max consumers limit");
}

if (consumerList.contains(consumer)) {

@Jason918 Jason918 Apr 1, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide the case of duplicated consumer?
I wonder if we should replace the old consumer with the new one. The older consumer maybe somewhat expired.

@lordcheng10 lordcheng10 Apr 1, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes so I think we should write a method like org.apache.pulsar.broker.service.Producer#isSuccessorTo to judge whether the previous consumer object should be overwritten by consumerEpoch，like this:

public boolean isSuccessorTo(Consumer other){
return Objects.equals(cnx.clientAddress(), other.cnx.clientAddress()) && consumerId == other.consumerId
&& other.consumerEpoch < consumerEpoch;
}

@lordcheng10 lordcheng10 Apr 1, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide the case of duplicated consumer?

OK, I will try to find duplicate cases.

log.warn("[{}] Consumer with the same id is already created:"
+ " consumerId={}, consumer={}",
consumer.cnx().clientAddress(), consumer.consumerId(), consumer);
throw new ConsumerBusyException("Consumer with the same id is already created!");
}

consumerList.add(consumer);
consumerList.sort(Comparator.comparingInt(Consumer::getPriorityLevel));
consumerSet.add(consumer);
Expand All @@ -168,7 +175,7 @@ public synchronized void removeConsumer(Consumer consumer) throws BrokerServiceE
// decrement unack-message count for removed consumer
addUnAckedMessages(-consumer.getUnackedMessages());
if (consumerSet.removeAll(consumer) == 1) {
consumerList.remove(consumer);
consumerList.removeIf(consumer::equals);
log.info("Removed consumer {} with pending {} acks", consumer, consumer.getPendingAcks().size());
if (consumerList.isEmpty()) {
cancelPendingRead();
Expand Down