wasm thread: TypeError: [object Int32Array] is not a shared typed array.

[Wrufesh]

Describe the bug

Looks like current wasm build script produces binary which expects SharedArrayBuffer as memory buffer type but is configured to use ArrayBuffer.

Reproduction

I am using the build_wasm.sh in hf_xet_wasm using.
The error occurs when trying to upload file:
const pf = await xetSession.uploadFileFromBlob(BigInt(0), file);

Logs

Following is the full trace:

simple.js:669 INFO examples/simple.rs:25 Starting init hf_xet_wasm...
simple.js:669 INFO examples/simple.rs:27 Done
(index):19 crossOriginIsolated: true
(index):20 Memory buffer type: ArrayBuffer
(index):35 getting auth info...
(index):65 casEndpoint: https://xet-cas.huggingface.co
simple.js:1108 Uncaught TypeError: [object Int32Array] is not a shared typed array.
    at Atomics.waitAsync (<anonymous>)
    at imports.wbg.__wbg_waitAsync_61f0a081053dd3c2 (simple.js:1108:29)
    at simple-51e1a399f2d66a01.wasm.__wbg_waitAsync_61f0a081053dd3c2 externref shim (simple_bg.wasm:0xedf3f)
    at simple-51e1a399f2d66a01.wasm.wasm_bindgen_futures::task::multithread::Task::run::h90b43039ae522598 (simple_bg.wasm:0xa7b09)
    at simple-51e1a399f2d66a01.wasm.wasm_bindgen_futures::queue::QueueState::run_all::hb943435ae21ba736 (simple_bg.wasm:0xcf25e)
    at simple-51e1a399f2d66a01.wasm.wasm_bindgen_futures::queue::Queue::new::{{closure}}::h9610788f355f5ba6 (simple_bg.wasm:0xed66c)
    at simple-51e1a399f2d66a01.wasm.<dyn core::ops::function::FnMut<(:5000/A,)>+Output = R as wasm_bindgen::closure::WasmClosure>::describe::invoke::h225d01fc207b6c58 (http://127.0.0.1:5000/target/simple_bg.wasm)
    at simple-51e1a399f2d66a01.wasm.closure697 externref shim (simple_bg.wasm:0xef12d)
    at __wbg_adapter_52 (simple.js:281:10)
    at real (simple.js:121:20)
imports.wbg.__wbg_waitAsync_61f0a081053dd3c2 @ simple.js:1108
$__wbg_waitAsync_61f0a081053dd3c2 externref shim @ simple_bg.wasm:0xedf3f
$wasm_bindgen_futures::task::multithread::Task::run::h90b43039ae522598 @ simple_bg.wasm:0xa7b09
$wasm_bindgen_futures::queue::QueueState::run_all::hb943435ae21ba736 @ simple_bg.wasm:0xcf25e
$wasm_bindgen_futures::queue::Queue::new::{{closure}}::h9610788f355f5ba6 @ simple_bg.wasm:0xed66c
$<dyn core::ops::function::FnMut<(A,)>+Output = R as wasm_bindgen::closure::WasmClosure>::describe::invoke::h225d01fc207b6c58 @ simple_bg.wasm:0xed640
$closure697 externref shim @ simple_bg.wasm:0xef12d
__wbg_adapter_52 @ simple.js:281
real @ simple.js:121

System info

using repo:huggingface/xetcore commit-id:85b5ba5

[Wrufesh]

Not sure if this was an effect of rust-lang/rust#147225.

I managed to make it work by modifying build_wasm.sh to following:

#!/bin/sh
set -ex

# Build std with atomics and shared memory support enabled.
# This is required by wasm_thread for multi-threading via Web Workers.

RUSTFLAGS="-C target-feature=+atomics,+bulk-memory,+mutable-globals \
  -C link-arg=--shared-memory \
  -C link-arg=--max-memory=1073741824 \
  -C link-arg=--import-memory \
  -C link-arg=--export=__wasm_init_tls \
  -C link-arg=--export=__tls_size \
  -C link-arg=--export=__tls_align \
  -C link-arg=--export=__tls_base \
  --cfg getrandom_backend=\"wasm_js\"" \
cargo +nightly build \
    --example simple \
    --target wasm32-unknown-unknown \
    --release \
    -Z build-std=std,panic_abort

# Generate JS/WASM glue for web targets
RUSTFLAGS='--cfg getrandom_backend="wasm_js"' \
wasm-bindgen \
    target/wasm32-unknown-unknown/release/examples/simple.wasm \
    --out-dir ./examples/target/ \
    --typescript \
    --target web

PS: I'm still a beginner with both Rust and WASM. It would be great if someone can verify it.

[seanses]

@Wrufesh Thank you for figuring this out! Would you like to submit a PR so we can test?