lib/*.js (benchmark): expose benchmark functionality as a module

[zombieleet]

Draft PR to expose the benchmark functionality as part of the core nodejs module.

It's still WIP

[Trott]

@nodejs/benchmarking @nodejs/tsc

I'm not sure our current benchmark implementation is something that we'd want to expose, but I'm also not sure it isn't. I've always imagined it as kind of ragged and for our own use. But maybe I don't give it enough credit.

[aduh95]

I don't have an opinion whether this should be exposed or not, however in order to land there are a couples of things that need to change on this PR:

• Please remove from lib/ the files whose names starts with an underscore, and move them to lib/internal/benchmark instead.
• Please make sure that only the new module can only be loaded with the node: prefix (require('node:benchmark')). require('benchmark') should still refer to userland (e.g. node_modules/benchmark/index.js). To achieve that, add it to that set:

  node/lib/internal/bootstrap/loaders.js

  Lines 124 to 127 in 472edc7

  	// Modules that can only be imported via the node: scheme.
  	const schemelessBlockList = new SafeSet([
  	'test',
  	]);

• This needs documentation.

[mscdex]

I agree with @Trott, the existing benchmarking system is not something we should just dump into a public API. If we decide node core needs to be exposing this sort of functionality, we should instead create something that's designed to be more user/developer-friendly (along with real unit tests) since the existing system was designed more around node's own needs.

One of the big issues with doing benchmarking in an environment like this is getting the statistical side right, using javascript. I know there have been attempts at this, but AFAIK using R to do the calculations is still the most reliable way of doing so, so we would need someone familiar enough with R to be able to convert the existing benchmark/*.R scripts safely to JS. I think part of the solution for this would include utilizing BigInts to simulate a "BigDecimal" type for the number crunching in order to maintain precision and accuracy.

[targos]

I know there have been attempts at this, but AFAIK using R to do the calculations is still the most reliable way of doing so, so we would need someone familiar enough with R to be able to convert the existing benchmark/*.R scripts safely to JS.

I did that in https://github.com/targos/node-benchmark-compare. The JS version gives the exact same numeric results as the R one. The issue isn't really the language (R also uses IEEE 754 floating point numbers), but that we need to be able to compute some statistical distributions. My package ends up using https://github.com/nearform/node-cephes (WASM)

[zombieleet]

@mscdex @Trott
The existing implementation is actually okay but not user/developer friendly (some parts of it also makes use of BigInt), I just need to make it more user/developer friendly, if it needs to be rewritten, I am fine doing that as well.

[anonrig]

Is there an open issue to discuss the changes stated in this pull request?

[zombieleet]

Is there an open issue to discuss the changes stated in this pull request?

@anonrig sorry for the late reply, here is an issue I created #45037