Can't prevent Sentry SDK from exiting the process even with onFatalError

[crunchtime-ali]

@sentry/node version: 4.1.1
Node.js version: 10.9.0

Sentry exits my process after an uncaught exception even I provide my own implementation of onFatalError. How can I prevent that?

Here is a minimal working example. Without sentry.init I get to the 2nd throw. With it the process finisches after the 1st one.

const sentry = require('@sentry/node');

sentry.init(
  { dsn: 'https://key@mysentry.mycompany/2', 
    integrations: [new sentry.Integrations.OnUncaughtException({
      onFatalError: error => {
        console.log('an error occured', error);
      }
    })]
  }
);

process.on('uncaughtException', err => {
  console.log(err, 'Uncaught Exception thrown');
  // process.exit(1);
});

console.log("before throwing the first time")
setTimeout(() => { throw 42 }, 2000)
setTimeout(() => { console.log('still running') }, 10000)
throw 42;

[HazAT]

@dj-hedgehog Does this help?
https://docs.sentry.io/learn/draining/?platform=node

[kamilogorek]

onFatalError is only meant to perform a cleanup before we exit the process, not fully prevent it. This is how Node.js is behaving by default (always crash the process and let it restart).

If you want to have a complete control over it, you have to turn off this integration and add handler yourself.

Sentry.init({
  dsn: 'https://key@mysentry.mycompany/2',
  integrations(integrations) {
    return integrations.filter(integration => integration.id !== 'OnUncaughtException')
  }
});

global.process.on('uncaughtException', (error) => {
  const hub = Sentry.getCurrentHub();
  hub.withScope(async (scope) => {
    scope.setLevel('fatal');
    hub.captureException(error, { originalException: error });
  });
});

[crunchtime-ali]

@kamilogorek Thank you for your quick response. The following code exits the app after the first throw nonetheless:

const sentry = require('@sentry/node');

sentry.init(
  { dsn: 'https://key@mysentry.mycompany/2', 
    integrations(integrations) {
      return integrations.filter(integration => integration.id !== 'OnUncaughtException');
    }
  }
);

global.process.on('uncaughtException', (error) => {
  const hub = sentry.getCurrentHub();
  hub.withScope(async (scope) => {
    scope.setLevel('fatal');
    hub.captureException(error, { originalException: error });
  });
});

console.log("before throwing the first time");
setTimeout(() => { throw "one"}, 1000)
setTimeout(() => { throw "two" }, 2000);
setTimeout(() => { throw "three" }, 4000);
setTimeout(() => { console.log('still running') }, 10000);

[kamilogorek]

@dj-hedgehog it should be integration.name !== 'OnUncaughtException' not integration.id !== 'OnUncaughtException'. Second format is still unreleased, sorry about that 😅

[crunchtime-ali]

That solved the issue, thanks a lot :)

[JamesHagerman]

onFatalError is only meant to perform a cleanup before we exit the process, not fully prevent it. This is how Node.js is behaving by default (always crash the process and let it restart).

I just ran into this. It would be wonderful to document the intent spelled out by @kamilogorek somewhere.

Maybe here?: https://docs.sentry.io/platforms/node/configuration/integrations/default-integrations/#onuncaughtexception

[mayfield]

I hate to necrobump this 2018 issue, but I just ran into this in 2022 and it was very surprising to have my applications behavior changed by the use of Sentry like this. It also cost me several hours of hair pulling out trying to figure out what was happening.

I have always thought one of the design goals of Sentry is to have no meaningful side effects.

Any maintainers care to comment?

[kamilogorek]

What was exactly your case? We are following all the defaults from node's implementation written down in the docs - https://nodejs.org/api/process.html#event-uncaughtexception
We also mention what we do there in our own docs - https://docs.sentry.io/platforms/node/configuration/integrations/default-integrations/#onuncaughtexception

We also had no reports of this being not an intended behavior in almost 4 years, thus my question here.

cc @getsentry/team-web-backend

[mayfield]

Hi @kamilogorek,

This is in the context of an electron app which also registers and handles uncaughtException with a dialog. Meaning node doesn't kill the process.

I think maybe sentry is assuming that there is no other listener and doing what node would do in that case, ie. exit.

Just a thought, but with node 13+ they have uncaughtExceptionMonitor

https://nodejs.org/api/process.html#event-uncaughtexceptionmonitor

Works for promises and normal exceptions and doesn't have side effects. Almost like, how it should have been in the first place! 😄

[kamilogorek]

We do that for browser SDK, not sure why we decide to skip it here tbh. Might be just something that we missed -

sentry-javascript/packages/utils/src/instrument.ts

Lines 583 to 621 in cc44957

	let _oldOnErrorHandler: OnErrorEventHandler = null;
	/** JSDoc */
	function instrumentError(): void {
	_oldOnErrorHandler = global.onerror;
	global.onerror = function (msg: any, url: any, line: any, column: any, error: any): boolean {
	triggerHandlers('error', {
	column,
	error,
	line,
	msg,
	url,
	});
	if (_oldOnErrorHandler) {
	// eslint-disable-next-line prefer-rest-params
	return _oldOnErrorHandler.apply(this, arguments);
	}
	return false;
	};
	}
	let _oldOnUnhandledRejectionHandler: ((e: any) => void) | null = null;
	/** JSDoc */
	function instrumentUnhandledRejection(): void {
	_oldOnUnhandledRejectionHandler = global.onunhandledrejection;
	global.onunhandledrejection = function (e: any): boolean {
	triggerHandlers('unhandledrejection', e);
	if (_oldOnUnhandledRejectionHandler) {
	// eslint-disable-next-line prefer-rest-params
	return _oldOnUnhandledRejectionHandler.apply(this, arguments);
	}
	return true;
	};
	}

TIL about uncaughtExceptionMonitor, thanks! We still do need to support older versions, so we'd need to do something similar to browser implementation instead.
I wonder if it can handle async code though, which we need to flush remaining events from the buffer -

sentry-javascript/packages/node/src/integrations/utils/errorhandling.ts

Line 32 in cc44957

}),

There's a possibility that monitor will trigger events, and let it fall through to uncaughtException handler, which will in turns kill the process before letting requests out.

I'll notify the team to backlog this issue :)

[ibc]

Our Node app sets a process.on('uncaughtException') listener that avoids process termination when an uncaught error happens. Then we enable Sentry and Sentry decides that the process should terminate on uncaught exception ¯_(ツ)_/¯

TIL about uncaughtExceptionMonitor, thanks! We still do need to support older versions, so we'd need to do something similar to browser implementation instead.

Within the uncaughtException listener that Sentry installs, it could count the number of event listeners for uncaughtException and if >1 then do NOT exit the process.

[AbhiPrasad]

Within the uncaughtException listener that Sentry installs, it could count the number of event listeners for uncaughtException and if >1 then do NOT exit the process.

We could also expose an option here for full backwards compatibility. Re-opening and adding to the backlog. PRs welcome!