Repository navigation
Pub/Sub : Subscriber Exceptions #2480
Description
Activity
I believe you can do something like this
Logger.getLogger("com.google.cloud.pubsub").setLevel(Level.OFF);
to disable logging from package right? You should be able to set it to other levels as well.
FWIW, newer versions should, I think, spam a little less since streaming pull makes fewer calls.
How are you setting the logging to stackdriver. Did you register stackdriver as logging handler?
Thanks for your answer. Ideally we would not want to set them off completely as we wouldn't want to miss important messages if they ever occur (do they ?). I will try different levels, though I imagine this means all messages are logged with the same severity.
The example I gave is meant to be a
WARNINGfrom the looks of it. I would like it to be logged as such if at all possible but am ready to live with limitations. I am trying with a different logging level at the moment but it sometimes takes a long time before it starts giving error messages.How are you setting the logging to stackdriver. Did you register stackdriver as logging handler?
We run on a google cloud container with logging enabled so we didn't have to setup the agent or anything else really. We simply log using log4j using the following Appender :
<Appenders> <Console name="STDOUT" target="SYSTEM_OUT"> <GCloudLayout /> </Console> </Appenders>GCloudLayoutextendsAbstractStringLayoutand doesn't do anything exotic. Simply serializes our in-house LogEntry objects to JSON. I can provide details if needed.Just ran a few tests and unfortunately adding the initializer in the entry point of the app doesn't work.
static { Logger.getLogger("com.google.cloud.pubsub").setLevel(Level.INFO); }
The message still gets logged at the error level in the console.
@hmigneron I posted PR that should make new versions spam you less.
I think the reason you're still seeing the logs is that
setLevelmeans the logger will only generate logs that that level or higher. Since the logger is set toINFO, it will continue to logWARNINGsince warnings are considered higher level than infos. In the linked PR, I made retryable errorsFINEso that loggers set atINFOwon't pick them up.@jabubake I'm not sure why logging is picking up each level of stack trace separately though. Do you have any context here?
Given these logs are being directed to stdout, it is picking up individual lines as log entries.
I believe there is a way to structure the log via appender as JSON to avoid this problem. @saturnism may be able to help.Another option is to use the Stackdriver logback appender / java.util.logging handler.
Thanks to both of you for the PR and for your help.
It took me a while, but I ultimately understood that in our case the logs were being direct to stderr (not sure if this is a default but I don't think we configured anything for that) which is why every single line was considered an error on stack driver.
The solution I came up with is to replace the default ConsoleHandler with a custom one :
Logger globalLogger = Logger.getLogger(""); Handler[] handlers = globalLogger.getHandlers(); for (Handler handler : handlers) { if (handler instanceof ConsoleHandler) { globalLogger.removeHandler(handler); } } globalLogger.addHandler(new CustomHandler());
The
CustomHandlerderives fromHandlerand only really "translates" the message to the format we use with stack driver. Something like :@Override public void publish(LogRecord record) { String message = record.getMessage(); if (record.getThrown() != null) { message += System.lineSeparator() + Helper.getStackTrace(record.getThrown()); } LogManager.getLogger(record.getLoggerName()).log(getLog4jLevel(record), logEntry); }
Probably not the cleanest solution but it does work for me because I get to keep the logging level of the messages which I definitely prefer.
Would be curious to hear about the appender solution.
This solution works for us right now, so unless there is something really wrong with it you can probably close this ticket. Thanks again !
Hope this helps : https://cloud.google.com/solutions/customizing-stackdriver-logs-fluentd
- added a commit that references this issue
on Feb 20, 2026
Using version
0.20.1-betaof the google-cloud-pubsub lib.We use
com.google.cloud.pubsub.v1.Subscriberobjects that live for a long time (multiple hours / days). Our logs are filled with the same errors that occur over and over :From what I can tell, this is fine. The errors are recoverable and we do get all the messages. The problem really is that it fills up our logs and triggers a bunch of alerts and we can't seem to catch the exceptions ourselves to lower the logging level. The format is weird too, we use StackDriver and in this case, it seems like every single line becomes its own log entry (all at the error level) :
How can we keep the library from logging so many messages are the error level ? Any way to catch the exceptions ourselves and / or change the logging level ?
Our subscribers are created this way (if it makes a difference) :