Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/content/docs/guides/sync-and-async.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Global queries join the default connection's JavaScript queue when the name was

Without a default managed connection, global `execute`, `executeAsync`, `executeRaw`, and `executeRawAsync` call the native methods directly and still need an open native database handle. Global `prepare`, batches, and transactions need a default connection opened through `open()`.

`NitroSQLite.native` exposes the underlying [`NitroSQLiteNative` hybrid object](/api/react-native-nitro-sqlite/hybrid-objects/NitroSQLiteNative). Its methods take a database name and return raw results without the connection helper's `rows` container. They also throw native errors without converting them to `NitroSQLiteError`. Its `executeAsync()` accepts an optional `onRows` callback that receives leading rows in batches, in order; the resolved `results` then holds only the rows after the last batch.
`NitroSQLite.native` exposes the underlying [`NitroSQLiteNative` hybrid object](/api/react-native-nitro-sqlite/hybrid-objects/NitroSQLiteNative). Its methods take a database name and return raw results without the connection helper's `rows` container. They also throw native errors without converting them to `NitroSQLiteError`. Its `executeAsync()` accepts an optional `onRows` callback that receives leading rows in batches, in order; the resolved `results` then holds only the rows after the last batch. The managed API sends those batches to one handler set with `setRowBatchHandler()` instead of passing a callback per query, so don't replace that handler while you also use the managed API.

```ts
import { NitroSQLite } from 'react-native-nitro-sqlite'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,8 @@ NitroSQLiteQueryResult HybridNitroSQLite::execute(const std::string& dbName, con

std::shared_ptr<Promise<NitroSQLiteQueryResult>> HybridNitroSQLite::executeAsync(const std::string& dbName, const std::string& query,
const std::optional<SQLiteQueryParams>& params,
const std::optional<SQLiteRowBatchHandler>& onRows) {
const std::optional<SQLiteRowBatchHandler>& onRows,
std::optional<double> rowBatchId) {
const auto copiedParams = copyArrayBufferParamsForBackground(params);
SQLiteConnectionPtr connection;
try {
Expand All @@ -194,12 +195,21 @@ std::shared_ptr<Promise<NitroSQLiteQueryResult>> HybridNitroSQLite::executeAsync
return Promise<NitroSQLiteQueryResult>::rejected(std::current_exception());
}

SQLiteRowBatchHandler rowBatches = onRows.value_or(nullptr);
if (!rowBatches && rowBatchId && _rowBatchHandler) {
rowBatches = [handler = _rowBatchHandler, id = *rowBatchId](SQLiteQueryResults rows) { (*handler)(id, std::move(rows)); };
}

return enqueueConnectionOperation<NitroSQLiteQueryResult>(
connection, [connection, query, copiedParams, onRows = onRows.value_or(nullptr)]() -> NitroSQLiteQueryResult {
return sqliteExecute(connection, query, copiedParams, onRows);
connection, [connection, query, copiedParams, rowBatches = std::move(rowBatches)]() -> NitroSQLiteQueryResult {
return sqliteExecute(connection, query, copiedParams, rowBatches);
});
};

void HybridNitroSQLite::setRowBatchHandler(const std::function<void(double, SQLiteQueryResults)>& handler) {
_rowBatchHandler = std::make_shared<const std::function<void(double, SQLiteQueryResults)>>(handler);
}

NitroSQLiteRawQueryResult HybridNitroSQLite::executeRaw(const std::string& dbName, const std::string& query,
const std::optional<SQLiteQueryParams>& params) {
return sqliteExecuteRaw(_connections.get(dbName), query, params);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ class HybridNitroSQLite : public HybridNitroSQLiteSpec {

private:
DatabaseConnections _connections;
// Set and read on the JavaScript thread; queries copy the pointer for their worker.
std::shared_ptr<const std::function<void(double, SQLiteQueryResults)>> _rowBatchHandler;

public:
// Methods
Expand Down Expand Up @@ -66,12 +68,17 @@ class HybridNitroSQLite : public HybridNitroSQLiteSpec {
NitroSQLiteQueryResult execute(const std::string& dbName, const std::string& query,
const std::optional<SQLiteQueryParams>& params) override;

/** Execute one SQL statement on a background thread.
/** Execute one SQL statement on a background thread. Leading row batches go to @p onRows, or else to
* the handler from setRowBatchHandler, tagged with @p rowBatchId.
* @return A promise of the native query result.
*/
std::shared_ptr<Promise<NitroSQLiteQueryResult>> executeAsync(const std::string& dbName, const std::string& query,
const std::optional<SQLiteQueryParams>& params,
const std::optional<SQLiteRowBatchHandler>& onRows) override;
const std::optional<SQLiteRowBatchHandler>& onRows,
std::optional<double> rowBatchId) override;

/** Set the handler that receives row batches for executeAsync calls made with a row batch ID. */
void setRowBatchHandler(const std::function<void(double, SQLiteQueryResults)>& handler) override;

/** Execute one SQL statement on the calling thread, returning rows as arrays of column values. */
NitroSQLiteRawQueryResult executeRaw(const std::string& dbName, const std::string& query,
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions packages/react-native-nitro-sqlite/src/__mocks__/nitro.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ export const HybridNitroSQLite = {
detach: jest.fn(),
execute: jest.fn(),
executeAsync: jest.fn(),
setRowBatchHandler: jest.fn(),
executeRaw: jest.fn(),
executeRawAsync: jest.fn(),
prepare: jest.fn(),
Expand Down
55 changes: 49 additions & 6 deletions packages/react-native-nitro-sqlite/src/__tests__/execute.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ const dbName = 'execute-test'
const query = 'SELECT ? AS value'
const params = [7]

// The managed API registers its row batch handler on the first async query.
let deliverRowBatch:
| Parameters<typeof HybridNitroSQLite.setRowBatchHandler>[0]
| undefined
jest
.mocked(HybridNitroSQLite.setRowBatchHandler)
.mockImplementation((handler) => {
deliverRowBatch = handler
})

beforeEach(() => jest.clearAllMocks())

afterEach(() => {
Expand Down Expand Up @@ -101,19 +111,22 @@ describe('execute', () => {
dbName,
query,
params,
expect.any(Function),
undefined,
expect.any(Number),
)
expect(HybridNitroSQLite.executeAsync).toHaveBeenCalledTimes(3)
})

it('joins rows delivered in batches before the remaining result rows', async () => {
jest
.mocked(HybridNitroSQLite.executeAsync)
.mockImplementationOnce(async (_dbName, _query, _params, onRows) => {
onRows?.([{ value: 1 }, { value: 2 }])
onRows?.([{ value: 3 }])
return nativeResult([{ value: 4 }])
})
.mockImplementationOnce(
async (_dbName, _query, _params, _onRows, rowBatchId) => {
deliverRowBatch!(rowBatchId!, [{ value: 1 }, { value: 2 }])
deliverRowBatch!(rowBatchId!, [{ value: 3 }])
return nativeResult([{ value: 4 }])
},
)

const result = await executeAsync(dbName, query, params)
expect(result.results).toEqual([
Expand All @@ -127,6 +140,36 @@ describe('execute', () => {
expect(result.rows.item(3)).toEqual({ value: 4 })
})

it('passes a row callback to native builds without setRowBatchHandler', async () => {
let olderNative = HybridNitroSQLite
let olderExecuteAsync = executeAsync
jest.isolateModules(() => {
olderNative =
jest.requireMock<typeof import('../nitro')>(
'../nitro',
).HybridNitroSQLite
delete (olderNative as Partial<typeof olderNative>).setRowBatchHandler
olderExecuteAsync = jest.requireActual<
typeof import('../operations/execute')
>('../operations/execute').executeAsync
})
jest
.mocked(olderNative.executeAsync)
.mockImplementationOnce(async (_dbName, _query, _params, onRows) => {
onRows?.([{ value: 1 }])
return nativeResult([{ value: 2 }])
})

const result = await olderExecuteAsync(dbName, query)
expect(result.results).toEqual([{ value: 1 }, { value: 2 }])
expect(olderNative.executeAsync).toHaveBeenCalledWith(
dbName,
query,
undefined,
expect.any(Function),
)
})

it('keeps the native rows when no batch was delivered', async () => {
const native = nativeResult([{ value: 7 }])
jest.mocked(HybridNitroSQLite.executeAsync).mockResolvedValueOnce(native)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ describe('open', () => {
dbName,
'SELECT 2',
undefined,
expect.any(Function),
undefined,
expect.any(Number),
)

db.close()
Expand Down Expand Up @@ -376,13 +377,15 @@ describe('open', () => {
'transaction-2',
'SELECT 3',
undefined,
expect.any(Function),
undefined,
expect.any(Number),
)
expect(HybridNitroSQLite.executeAsync).not.toHaveBeenCalledWith(
'transaction-1',
'SELECT 2',
undefined,
expect.any(Function),
undefined,
expect.any(Number),
)

gate.resolve()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ describe('transaction', () => {
dbName,
'SELECT ?',
[2],
expect.any(Function),
undefined,
expect.any(Number),
)
expect(HybridNitroSQLite.execute).toHaveBeenLastCalledWith(
dbName,
Expand Down Expand Up @@ -222,7 +223,8 @@ describe('transaction', () => {
dbName,
'BEGIN EXCLUSIVE TRANSACTION',
undefined,
expect.any(Function),
undefined,
expect.any(Number),
)
expect(HybridNitroSQLite.execute).toHaveBeenCalledTimes(1)
expect(HybridNitroSQLite.execute).toHaveBeenCalledWith(
Expand Down
68 changes: 58 additions & 10 deletions packages/react-native-nitro-sqlite/src/operations/execute.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,30 +93,78 @@ export function executeAsyncNative<Row extends QueryResultRow = never>(
): Promise<QueryResult<Row>> {
// Not `async`: every extra promise layer is a measurable part of a short query's cost.
// Large results arrive in batches, so rows are converted while SQLite reads the rest.
let leadingRows: Row[] | undefined
if (!hasRowBatchHandler) {
return executeAsyncWithRowCallback<Row>(dbName, query, params)
}
const rowBatchId = nextRowBatchId()
let nativeResult: Promise<NitroSQLiteQueryResult>
try {
nativeResult = HybridNitroSQLite.executeAsync(
dbName,
query,
params,
(rows) => {
if (leadingRows === undefined) {
leadingRows = rows as Row[]
} else {
Array.prototype.push.apply(leadingRows, rows as Row[])
}
},
undefined,
rowBatchId,
)
} catch (error) {
return Promise.reject(NitroSQLiteError.fromError(error))
}
return nativeResult.then(
(result) => buildJSQueryResult<Row>(result, leadingRows),
rejectWithNitroSQLiteError,
(result) =>
buildJSQueryResult<Row>(result, takeLeadingRows<Row>(rowBatchId)),
(error: unknown) => {
leadingRowBatches.delete(rowBatchId)
return rejectWithNitroSQLiteError(error)
},
)
}

// Leading row batches of async queries in flight, by row batch ID. One handler, set on the first
// query, receives every batch, so a query passes a number to native code instead of a new callback.
const leadingRowBatches = new Map<number, QueryResultRow[]>()
// Older native builds have no handler and get a callback per query instead.
const hasRowBatchHandler =
typeof HybridNitroSQLite.setRowBatchHandler === 'function'
let lastRowBatchId = 0

function nextRowBatchId(): number {
if (lastRowBatchId === 0) {
HybridNitroSQLite.setRowBatchHandler((rowBatchId, rows) => {
leadingRowBatches.set(
rowBatchId,
appendRows(leadingRowBatches.get(rowBatchId), rows),
)
})
}
lastRowBatchId++
return lastRowBatchId
}

function takeLeadingRows<Row>(rowBatchId: number): Row[] | undefined {
const rows = leadingRowBatches.get(rowBatchId)
if (rows !== undefined) leadingRowBatches.delete(rowBatchId)
return rows as Row[] | undefined
}

function executeAsyncWithRowCallback<Row extends QueryResultRow>(
dbName: string,
query: string,
params?: SQLiteQueryParams,
): Promise<QueryResult<Row>> {
let leadingRows: Row[] | undefined
return callNativeAsync(() =>
HybridNitroSQLite.executeAsync(dbName, query, params, (rows) => {
leadingRows = appendRows(leadingRows, rows as Row[])
}),
).then((result) => buildJSQueryResult<Row>(result, leadingRows))
}

function appendRows<Row>(leadingRows: Row[] | undefined, rows: Row[]): Row[] {
if (leadingRows === undefined) return rows
Array.prototype.push.apply(leadingRows, rows)
return leadingRows
}

/** Execute one SQL statement synchronously by database name, returning rows as arrays.
* Uses the default managed connection's queue when the database was opened through `open()`.
* Otherwise, calls the open native database directly.
Expand Down
13 changes: 13 additions & 0 deletions packages/react-native-nitro-sqlite/src/specs/NitroSQLite.nitro.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,14 +101,27 @@ export interface NitroSQLite
* @param onRows Receives leading rows in batches, in order, while SQLite still reads the rest, so
* JavaScript can convert a batch while the next one is read. Batched rows are not repeated in the
* result's `results`, which then holds only the rows after the last batch.
* @param rowBatchId Without `onRows`, sends the same batches to the handler set with
* `setRowBatchHandler`, tagged with this ID. Unlike `onRows`, no function crosses to native code
* for each query.
* @returns A promise of the native query result.
*/
executeAsync(
dbName: string,
query: string,
params?: SQLiteQueryParams,
onRows?: (rows: NitroSQLiteQueryRows) => void,
rowBatchId?: number,
): Promise<NitroSQLiteQueryResult>
/** Set the function that receives row batches for `executeAsync` calls made with a `rowBatchId`.
* The managed API sets it on its first async query and relies on it, so replacing it drops leading
* rows from managed async results.
* @param handler Receives the query's row batch ID and its next batch of leading rows, in order,
* before that query's promise settles.
*/
setRowBatchHandler(
handler: (rowBatchId: number, rows: NitroSQLiteQueryRows) => void,
): void
/** Execute one SQL statement synchronously on the calling thread, returning rows as arrays of column values.
* @param dbName Name of an open database.
* @param query SQL statement with optional positional placeholders.
Expand Down
Loading