Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions handwritten/spanner-driver/spannerlib-node/BUILD_AND_RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Node.js Spanner Wrapper Build and Release Architecture

This document describes the compilation pipeline, dual-publishing methodology, and native bridge linking process for the `spannerlib-node` driver.

## The 3-Layer Pipeline

Bridging JavaScript to the native Go SDK involves a sequential compilation pipeline mapping V8 types to Go-compatible memory pointers across three separate layers:

```
[TypeScript Source] ---> (Babel/TSC) ---> [ESM & CJS JavaScript]
|
(node-gyp bridge)
v
[C++ Native Addon]
|
(cgo linker bridge)
v
[Go Shared Library]
```

## Compilation Phases

### Phase 1: Compiling the Go Shared Library (CGO Link)

Before building the Node.js Addon, the underlying Go codebase must be compiled into an object format that C/C++ can link against.
* **Trigger:** Executed via `npm run build:go` which runs `bash scripts/build-shared-lib.sh`.
* **Action:** The build script invokes the Go compiler with the `-buildmode=c-shared` flag, targeting the primary C-shared entry point located at [shared_lib.go](../../shared/shared_lib.go).
* **Outputs:** Generates a platform-specific native shared library (e.g., `libspannerlib.dylib` on macOS, `.so` on Linux, `.dll` on Windows) along with the corresponding C header file (`libspannerlib.h`). Both files are placed into the `spannerlib/shared/` directory.

### Phase 2: Compiling the Native Bridge (node-gyp)

Once the Go shared library is generated, the Node.js C++ wrapper is compiled using `node-gyp` to map V8 engine objects into Go pointers.
* **Trigger:** Executed as part of `npm run build` which invokes `node-gyp rebuild`.
* **Action:** Reads the `gyp` build instructions in [binding.gyp](./binding.gyp) to locate the Go header files, and dynamically links the bridge against the generated Go shared object. It compiles the bridge source file [addon.cc](./src/cpp/addon.cc) using the local OS C++ compiler toolchain (e.g., Clang on macOS, GCC on Linux, MSVC on Windows).
* **Output:** Generates the native Node.js binary file at `build/Release/spanner_napi.node`.
* **Post-build Link Patch (macOS Only):** To ensure portability on macOS without requiring root or global library installs, `npm run postbuild` invokes the `install_name_tool`. This command alters the dynamic linker search path in the `.node` file to use `@loader_path/libspannerlib.dylib`, ensuring Node.js locates the Go dynamic library relatively from the compiled C++ bridge binary path.

### Phase 3: TypeScript Compilation & Dual-Publishing (ESM / CJS)

Finally, the TypeScript layer (which handles user-facing API classes, JavaScript's `FinalizationRegistry` garbage collection mapping, and Protobuf serialization) is compiled for consumer consumption. The build is configured to output both **ES Modules (ESM)** and **CommonJS (CJS)** simultaneously, ensuring compatibility across modern and legacy Node.js environments.
* **Trigger:** Executed via `npm run compile`.
* **Action:**
1. **ESM Setup:** `npm run compile:esm` runs the standard `tsc` (TypeScript Compiler) using the primary [tsconfig.json](./tsconfig.json) file. Outputs ES6 modules into `build/esm/`.
2. **CJS Setup:** `npm run compile:cjs` compiles the codebase for legacy support using [tsconfig.cjs.json](./tsconfig.cjs.json) and pipes the JS output through `@babel/cli` to translate modern `import`/`export` keywords into standard CommonJS `require()` bindings.
3. **Extension Fixer:** Because Node.js requires explicit file extensions when using ESM but Babel strips them or expects `.cjs` in CommonJS modes, the post-compilation script `node scripts/fix-extensions.cjs` rewrites the internal import paths across the `build/cjs/` folder to use `.cjs` extensions.
* **Output:** Generates two parallel directory trees under `build/esm` and `build/cjs`, making the distributed npm package natively dual-consumable using the `exports` configuration in `package.json`.

## End-to-End Local Builds

To run the entire pipeline end-to-end and generate a fully runnable local build, invoke the top-level script:
```bash
npm run build
```
This builds the underlying Go shared library, links the C++ bridge layer via `node-gyp`, patches dynamic linker paths, and outputs the final dual ESM/CJS JavaScript distributions.
55 changes: 55 additions & 0 deletions handwritten/spanner-driver/spannerlib-node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Node-API Wrapper for Spanner Shared Library

This package provides a high-performance Node-API (N-API) bridge to the Go-based Spanner shared library. It offers superior stability and performance compared to traditional FFI approaches.

## Prerequisites

- Node.js >= 20.0.0
- Go compiler (to build the underlying shared library, if not pre-built)
- C++ toolchain (GCC/Clang or MSVC)

## Installation & Building

To build the native addon and compile the driver for development, run:

```bash
npm install
```

For a comprehensive architectural breakdown of the 3-layer compilation pipeline (Go Link, C++ node-gyp bridge, and Dual-publishing ESM/CJS), refer to the [BUILD_AND_RELEASE.md](file:///Users/gargsurbhi/Work/Github/go-sql-spanner/spannerlib/wrappers/spannerlib-node/BUILD_AND_RELEASE.md) documentation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The link to BUILD_AND_RELEASE.md uses a hardcoded local absolute file:/// URL pointing to the author's local directory. This link will be broken for anyone else reading the documentation. Please change it to a relative link.

Suggested change
For a comprehensive architectural breakdown of the 3-layer compilation pipeline (Go Link, C++ node-gyp bridge, and Dual-publishing ESM/CJS), refer to the [BUILD_AND_RELEASE.md](file:///Users/gargsurbhi/Work/Github/go-sql-spanner/spannerlib/wrappers/spannerlib-node/BUILD_AND_RELEASE.md) documentation.
For a comprehensive architectural breakdown of the 3-layer compilation pipeline (Go Link, C++ node-gyp bridge, and Dual-publishing ESM/CJS), refer to the [BUILD_AND_RELEASE.md](BUILD_AND_RELEASE.md) documentation.


## Usage

```javascript
const { Pool, Connection } = require('spannerlib-node');

async function run() {
const pool = await Pool.create(connectionString);

const conn = await pool.createConnection();
const rows = await conn.executeSql('SELECT 1');

while (await rows.next()) {
// process rows
}

await rows.close();
await conn.close();
await pool.close();
}
```

## Architecture

The wrapper consists of:
1. **`src/cpp/addon.cc`**: C++ Node-API bridge that handles thread boundaries and type conversions between V8 and C.
2. **`src/ffi/utils.ts`**: Helper functions to invoke native methods asynchronously using Promises.
3. **`src/lib/`**: JavaScript classes (`Pool`, `Connection`, `Rows`) that provide a clean object-oriented interface.

### Component Interaction

The wrapper operates by executing calls through the low-level native bridge (`addon.cc`). The user-facing classes (`Pool`, `Connection`, `Rows`) in the `lib` folder map execution requests into Protobuf payloads and pass them to the bridge via the asynchronous dispatcher in `utils.ts`. Each successful execution returns memory ID handles that are tracked inside the singleton state registry (`spannerlib.ts`) to link native garbage collection triggers.

### Component Interaction & Memory Management

When a JavaScript object (like a `Pool` or `Connection`) is created, it holds an ID referencing a pinned Go object in memory. The `spannerLib` singleton maintains a **`FinalizationRegistry`**. This registry allows Node.js to listen for when the JavaScript object is garbage collected. When GC occurs, the registry automatically triggers a cleanup call to the native layer to release the corresponding Go object, preventing native memory leaks even if the developer forgets to call `.close()` explicitly.
68 changes: 68 additions & 0 deletions handwritten/spanner-driver/spannerlib-node/binding.gyp
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
{
'targets': [
{
'target_name': 'spanner_napi',
'sources': [ 'src/cpp/addon.cc' ],
'include_dirs': [
'<!@(node -p "require(\'node-addon-api\').include")',
'./shared',
],
'dependencies': [
'<!(node -p "require(\'node-addon-api\').gyp")'
],
'cflags!': [ '-fno-exceptions' ],
'cflags_cc!': [ '-fno-exceptions' ],
'xcode_settings': {
'MACOSX_DEPLOYMENT_TARGET': '10.15',
'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
'CLANG_CXX_LIBRARY': 'libc++',
},
'msvs_settings': {
'VCCLCompilerTool': { 'ExceptionHandling': 1 },
},
'conditions': [
['OS=="mac"', {
'libraries': [
'<(module_root_dir)/./shared/libspanner.dylib'
],
'xcode_settings': {
'OTHER_LDFLAGS': [
'-Wl,-rpath,@loader_path'
]
},
'copies': [
{
'destination': '<(PRODUCT_DIR)',
'files': [ '<(module_root_dir)/./shared/libspanner.dylib' ]
}
]
}],
['OS=="linux"', {
'ldflags': [
'-Wl,-rpath,$$ORIGIN'
],
'libraries': [
'<(module_root_dir)/./shared/libspanner.so'
],
'copies': [
{
'destination': '<(PRODUCT_DIR)',
'files': [ '<(module_root_dir)/./shared/libspanner.so' ]
}
]
}],
['OS=="win"', {
'libraries': [
'<(module_root_dir)/./shared/libspanner.dll'
],
'copies': [
{
'destination': '<(PRODUCT_DIR)',
'files': [ '<(module_root_dir)/./shared/libspanner.dll' ]
}
]
}]
]
}
]
}
15 changes: 15 additions & 0 deletions handwritten/spanner-driver/spannerlib-node/eslint.config.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
let customConfig = [];
let hasIgnoresFile = false;
try {
require.resolve('./eslint.ignores.cjs');
hasIgnoresFile = true;
} catch {
// eslint.ignores.js doesn't exist
}

if (hasIgnoresFile) {
const ignores = require('./eslint.ignores.cjs');
customConfig = [{ignores}];
}

module.exports = [...customConfig, ...require('gts')];
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = ['build/']
64 changes: 64 additions & 0 deletions handwritten/spanner-driver/spannerlib-node/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"name": "spannerlib-node",
"version": "0.1.0",
"main": "./build/cjs/src/index.cjs",
"module": "./build/esm/src/index.js",
"types": "./build/cjs/src/index.d.ts",
"type": "module",
"exports": {
".": {
"import": {
"types": "./build/esm/src/index.d.ts",
"default": "./build/esm/src/index.js"
},
"require": {
"types": "./build/cjs/src/index.d.ts",
"default": "./build/cjs/src/index.cjs"
}
}
},
"engines": {
"node": ">=20.0.0"
},
"scripts": {
"build:go": "bash scripts/build-shared-lib.sh",
"build": "npm run build:go && node-gyp rebuild && npm run compile",
"postbuild": "node -e \"if (process.platform === 'darwin') require('child_process').execSync('install_name_tool -change libspanner.dylib @loader_path/libspanner.dylib ./build/Release/spanner_napi.node')\"",
"compile:esm": "tsc -p .",
"compile:cjs": "tsc -p ./tsconfig.cjs.json && babel build/cjs --out-dir build/cjs --out-file-extension .cjs && node scripts/fix-extensions.cjs",
"compile": "npm run compile:esm && npm run compile:cjs",
"test:esm": "mocha build/esm/test/**/*.js",
"test:cjs": "mocha build/cjs/test/**/*.cjs",
"test": "npm run test:esm && npm run test:cjs",
"lint": "gts lint",
"clean": "gts clean",
"fix": "gts fix",
"prepare": "npm run compile",
"pretest": "npm run compile"
},
"files": [
"build/esm/src",
"build/cjs/src",
"build/Release/*.node"
],
"dependencies": {
"node-addon-api": "^8.0.0",
"bindings": "^1.5.0",
"@google-cloud/spanner": "^8.7.1"
},
"devDependencies": {
"mocha": "^11.0.1",
"typescript": "^6.0.0",
"@types/node": "^24.0.0",
"@types/bindings": "^1.5.0",
"@types/mocha": "^10.0.6",
"@babel/core": "^8.0.0",
"@babel/cli": "^8.0.0",
"sinon": "^22.0.0",
"@types/sinon": "^21.0.0",
"gts": "^7.0.0",
Comment on lines +51 to +59

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Several devDependencies specify non-existent or hallucinated major versions (e.g., typescript v6.0.0, @types/node v24.0.0, @babel/core v8.0.0, @babel/cli v8.0.0, sinon v22.0.0, @types/sinon v21.0.0, and gts v7.0.0). These versions do not exist on the npm registry and will cause npm install to fail. Please downgrade them to the latest stable, existing versions.

Suggested change
"typescript": "^6.0.0",
"@types/node": "^24.0.0",
"@types/bindings": "^1.5.0",
"@types/mocha": "^10.0.6",
"@babel/core": "^8.0.0",
"@babel/cli": "^8.0.0",
"sinon": "^22.0.0",
"@types/sinon": "^21.0.0",
"gts": "^7.0.0",
"typescript": "^5.4.5",
"@types/node": "^20.14.2",
"@types/bindings": "^1.5.0",
"@types/mocha": "^10.0.6",
"@babel/core": "^7.24.5",
"@babel/cli": "^7.24.5",
"sinon": "^18.0.0",
"@types/sinon": "^18.0.0",
"gts": "^6.0.2",
References
  1. Verify the existence of a package version on the npm registry (e.g. npmjs.com) before flagging it as a typo or non-existent.
  2. When managing dependencies in package.json, ensure that @types packages (e.g., @types/sinon) are kept in sync with the major version of their corresponding main packages (e.g., sinon).

"@grpc/grpc-js": "^1.12.0",
"@grpc/proto-loader": "^0.8.0"
},
"gypfile": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/bin/bash
set -e

# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Builds the shared library and places it in the shared directory.
# This script handles OS detection to use the correct file extension.

log() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}

log "Starting Spannerlib Shared Library Build for Node Wrapper..."

# Resolve absolute paths
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SHARED_LIB_DIR="$(cd "$SCRIPT_DIR/../shared" && pwd)"

log "Script Directory: $SCRIPT_DIR"
log "Shared Lib Directory: $SHARED_LIB_DIR"

# Auto-detect OS
case "$(uname -s)" in
Linux*) OS="Linux";;
Darwin*) OS="macOS";;
CYGWIN*|MINGW*|MSYS*) OS="Windows";;
*) OS="Unknown";;
esac
log "Auto-detected OS: $OS"

if [ "$OS" == "macOS" ]; then
echo "Building for macOS..."
go build -C "$SHARED_LIB_DIR" -o libspanner.dylib -buildmode=c-shared shared_lib.go
elif [ "$OS" == "Linux" ]; then
echo "Building for Linux..."
go build -C "$SHARED_LIB_DIR" -o libspanner.so -buildmode=c-shared shared_lib.go
elif [ "$OS" == "Windows" ]; then
echo "Building for Windows..."
go build -C "$SHARED_LIB_DIR" -o libspanner.dll -buildmode=c-shared shared_lib.go
else
echo "Unsupported operating system: $OS"
exit 1
fi

echo "Build complete."
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright 2026 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

const fs = require('fs');
const path = require('path');

const cjsDir = path.join(__dirname, '../build/cjs');

function traverseDir(dir) {
fs.readdirSync(dir).forEach(file => {
const fullPath = path.join(dir, file);
if (fs.statSync(fullPath).isDirectory()) {
traverseDir(fullPath);
} else if (fullPath.endsWith('.cjs')) {
let content = fs.readFileSync(fullPath, 'utf8');
// Replace require paths starting with . / or .. / and ending with .js
content = content.replace(/require\(['"]((\.|\.\.)\/[^'"]+)\.js['"]\)/g, "require('$1.cjs')");
// Fix import.meta.url syntax error in CommonJS
content = content.replace(/import\.meta\.url/g, '""');
fs.writeFileSync(fullPath, content);
} else if (fullPath.endsWith('.js')) {
// Delete the original .js file generated by tsc
fs.unlinkSync(fullPath);
}
});
}

if (fs.existsSync(cjsDir)) {
traverseDir(cjsDir);
console.log('Fixed extensions in .cjs files and removed .js files in build/cjs');
}
Loading
Loading