Fix Node.js compatibility for CJS consumers and WASM initialization#8
Fix Node.js compatibility for CJS consumers and WASM initialization#8kaleababayneh wants to merge 5 commits intoLumeraProtocol:masterfrom
Conversation
Re-reviewed after 13d2d6b. The
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
src/wasm/raptorq-proxy.ts
Outdated
| async function getWasmSource(): Promise<any> { | ||
| if (typeof window === "undefined") { |
There was a problem hiding this comment.
getWasmSource() checks typeof window === "undefined" to decide between the Node.js and browser code paths. But getRqModule() (line 17) sets globalThis.window = globalThis as a shim for Node.js. If initialize() fails after getRqModule() has already been called and cached (e.g., the WASM init itself throws), initPromise is reset to null, allowing a retry. On that retry, getWasmSource() now sees window as defined and takes the browser path (import("rq-library-wasm/rq_library_bg.wasm?url")), which will fail in Node.js.
Consider using a check that is not affected by the shim, such as typeof process !== "undefined" && process.versions?.node or checking before the shim is applied.
Fix it with Roo Code or mention @roomote and request a fix.
package-lock.json
Outdated
| @@ -0,0 +1,8242 @@ | |||
| { | |||
There was a problem hiding this comment.
This project uses pnpm as its package manager (there is already a pnpm-lock.yaml in the repo). Committing a package-lock.json (8,200+ lines) introduces a conflicting lockfile that can cause dependency resolution confusion. This file should be removed from the PR and added to .gitignore.
Fix it with Roo Code or mention @roomote and request a fix.
src/wasm/raptorq-proxy.ts
Outdated
| export async function getFileSize(...args: Parameters<typeof import("rq-library-wasm").getFileSize>) { | ||
| const rq = await getRqModule(); | ||
| return rq.getFileSize(...args); | ||
| } |
There was a problem hiding this comment.
These re-exported utility functions (getFileSize, syncDirExists, etc.) were previously synchronous re-exports from rq-library-wasm. Wrapping them in async functions changes their return type from T to Promise<T>. Any downstream consumer that was calling, for example, getFileSize(path) synchronously (without await) will now silently receive a Promise object instead of a number. This is a breaking change to the public API surface. If these functions are not currently consumed externally, this is fine, but it should be called out in release notes or the version should be bumped accordingly.
Fix it with Roo Code or mention @roomote and request a fix.
…window for Node.js, remove package-lock.json
| coverage/package-lock.json | ||
| package-lock.json |
There was a problem hiding this comment.
The original file ended with coverage/ and no trailing newline. When package-lock.json was appended, it got concatenated onto that line, turning coverage/ into coverage/package-lock.json. No functional impact since coverage/ is already ignored on line 171, but this wasn't the intended result.
| coverage/package-lock.json | |
| package-lock.json | |
| coverage/ | |
| package-lock.json |
Fix it with Roo Code or mention @roomote and request a fix.
mateeullahmalik
left a comment
There was a problem hiding this comment.
Thanks for the PR — the intent is good and I appreciate the effort to improve Node compatibility + CJS consumption.
I went through the changes and validated locally. Before we can merge, there are a few blocking issues to address:
- CJS runtime break:
import.meta.urlis still emitted into CJS output fromsrc/wasm/raptorq-proxy.ts, which causesSyntaxError: Cannot use 'import.meta' outside a modulewhen requiring the CJS build. - Public API behavior drift: re-exported FS helpers now become async wrappers (e.g.
getFileSize,dirExists,syncDirExists). This changes consumer-facing behavior/signatures and is a breaking change unless explicitly versioned + documented. - Current branch test status:
tests/wasm/raptorq-proxy.spec.tsis failing in this branch and should be green before merge.
Please fix the above and I’ll re-review quickly. Happy to approve once these are addressed 🙌
src/wasm/raptorq-proxy.ts
Outdated
| } else { | ||
| const { createRequire } = await import("node:module"); | ||
| // @ts-ignore - import.meta.url is only available in ESM | ||
| const req = createRequire(import.meta.url); |
There was a problem hiding this comment.
import.meta.url here is problematic for the CJS build. In the emitted dist/cjs output this causes a hard parse/runtime failure (Cannot use 'import.meta' outside a module) for require() consumers. We should avoid emitting import.meta into CJS-targeted code paths.
src/wasm/raptorq-proxy.ts
Outdated
| const rq = await getRqModule(); | ||
| return rq.readFileChunk(...args); | ||
| } | ||
| export async function getFileSize(...args: Parameters<typeof import("rq-library-wasm").getFileSize>) { |
There was a problem hiding this comment.
This changes getFileSize from a direct export to an async wrapper returning a Promise. That is a behavior/signature change for public API consumers. Please preserve the original sync/async contract (or treat as explicit breaking change with versioning/docs).
src/wasm/raptorq-proxy.ts
Outdated
| const rq = await getRqModule(); | ||
| return rq.dirExists(...args); | ||
| } | ||
| export async function syncDirExists(...args: Parameters<typeof import("rq-library-wasm").syncDirExists>) { |
There was a problem hiding this comment.
Similar concern for syncDirExists/dirExists wrappers: converting these to async wrappers changes external behavior. Can we keep export semantics aligned with previous API surface to avoid regressions for existing callers?
…nc FS exports, fix tests. Added a node example
|
The existing example used Worth considering creating a |
Problem
The SDK doesn't work properly in Node.js environments. Two issues block usage:
window is not defined(screenshot 1)Importing the SDK immediately crashes because
rq-library-wasmuses a top-level import that eagerly evaluatesbrowser_fs_mem.js, a browser-only file that assigns towindow. This occurs even if the consumer never uses any RaptorQ functionality.ERR_UNKNOWN_FILE_EXTENSION: ".wasm"(screenshot 2)Node's ESM loader does not recognize
.wasmfile extensions, causing WASM initialization to fail.Fix
Lazy-load the WASM module — Instead of importing
rq-library-wasmat the top of the file (which immediately executes browser-only code), the module is now loaded on demand viaimport()only when RaptorQ is actually invoked. AglobalThis.windowshim is applied before loading so that the browser FS code does not throw in Node.js.Correct the CJS build paths —
package.jsonpointed todist/cjs/index.cjs, buttscoutputsindex.js. Updated the paths and added a{"type": "commonjs"}marker indist/cjs/so Node resolves the module system correctly.✅ Built and verified locally in Node.js environments.