[feature request] Automatic chunking of data files by file_packager and FS (and potentially wasm files as well?)

[vadimkantorov]

It would be nice if file_packager / FS supported configurable chunking of produced data files. This would be beneficial for browser caching and for dealing with compressed files >2Gb (all individual files are < 2Gb).

This is currently possible by manual chunking of directories, which is already not bad, but would be better if this was automatic / configurable, or at least if there was some recipe for this in docs.

TensorFlow.js does it (at least couple years ago) for distribution of the models

[vadimkantorov]

This should also help with prefetching.

Even if I prefetch <link rel="preload" href="/dist/busytex.wasm" as="fetch" type="application/octet-stream" />, and then fetch() it manually, if the original request was still in-flight (which is reasonable if wasm/data are large - many megabytes), seems the new request doesn't seem to reuse cached bytes in chrome.

[vadimkantorov]

Chunking support would also be welcome because in case of serving off GitHub Pages, committed files must be <100Mb in size