Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,18 @@ for full provenance and the per-version index.

## What's new on the living fork

- **Serializable Snapshot Isolation (SSI)** — an opt-in `DB_TXN_SNAPSHOT_SAFE`
transaction mode that detects dangerous read/write dependency structures and
- **Serializable Snapshot Isolation (SSI)** — the `DB_TXN_SNAPSHOT`
transaction mode is now serializable: it detects dangerous read/write
dependency structures and
aborts the pivot with `DB_SNAPSHOT_CONFLICT`. Both of Cahill's rw-conflict
detection mechanisms are implemented: the lock-table path (a concurrent
writer meeting a reader's SIREAD marker) and the MVCC version-chain path in
`mp_fget` (a reader handed an older version than one a concurrent writer
committed). SIREAD markers are reclaimed incrementally and bounded (not only
at checkpoint); the commit-time pivot check is race-free against concurrent
edge recording; and `DB_TXN_SNAPSHOT_SAFE` is rejected with `prepare()`/2PC.
edge recording; and a `DB_TXN_SNAPSHOT` transaction is rejected with `prepare()`/2PC.
(This fork makes `DB_TXN_SNAPSHOT` serializable and removed the earlier
separate `DB_TXN_SNAPSHOT_SAFE` flag — a deliberate ABI break.)
The SIREAD marker/locker/detail lifetime is hardened for concurrent writers:
a family of pre-existing use-after-free bugs (most importantly a lock object
reclaimed while it still held SIREAD markers) was found with TSan/ASan and
Expand Down
3 changes: 1 addition & 2 deletions build_windows/db.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 1 addition & 2 deletions dist/api_flags
Original file line number Diff line number Diff line change
Expand Up @@ -275,8 +275,7 @@ DbEnv.txn_begin
DB_TXN_NOWAIT # Do not wait for locks
DB_TXN_FAMILY # Cursors and child txns are
# independent but lock-compatible
DB_TXN_SNAPSHOT # Snapshot isolation
DB_TXN_SNAPSHOT_SAFE __PIN=0x00000800 # Serializable snapshot isolation (SSI)
DB_TXN_SNAPSHOT # Serializable snapshot isolation (SSI)
DB_TXN_SYNC # Always sync log on commit
DB_TXN_WAIT # Always wait for locks in this txn
DB_TXN_WRITE_NOSYNC # Write the log but don't sync
Expand Down
20 changes: 11 additions & 9 deletions docs_src/_migrate/flag-reconcile.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ environment / database flag surfaces the SSI work touched. Sources of truth:
| DB_TXN_FAMILY | **MISSING** | **added** (txnbegin.md) |
| DB_TXN_NOSYNC | doc | doc |
| DB_TXN_NOWAIT | doc | doc |
| DB_TXN_SNAPSHOT | doc | doc |
| DB_TXN_SNAPSHOT_SAFE | **MISSING** | **added** (txnbegin.md, +guides, +return codes) |
| DB_TXN_SNAPSHOT | doc | doc (now = SSI) |
| DB_TXN_SNAPSHOT_SAFE | *removed* | flag removed — folded into DB_TXN_SNAPSHOT |
| DB_TXN_SYNC | doc | doc |
| DB_TXN_WAIT | doc | doc |
| DB_TXN_WRITE_NOSYNC | doc | doc |
Expand Down Expand Up @@ -58,10 +58,11 @@ Every `txn_begin` flag the engine accepts is now documented.
| DB_YIELDCPU | doc |
| DB_HOTBACKUP_IN_PROGRESS | doc |

Note: `DB_TXN_SNAPSHOT_SAFE` is deliberately **not** a valid `set_flags`
flag — SSI is a per-transaction property only (the env `OK_FLAGS` mask accepts
plain `DB_TXN_SNAPSHOT` but not `_SAFE`). `envset_flags.md` now states this
explicitly under `DB_TXN_SNAPSHOT`.
Note: `DB_TXN_SNAPSHOT` is now serializable snapshot isolation (SSI); the
separate `DB_TXN_SNAPSHOT_SAFE` flag was **removed** and its behavior folded
into `DB_TXN_SNAPSHOT`, which is a valid `set_flags` env flag (SSI can be set
environment-wide or per-transaction). `envset_flags.md` states this under
`DB_TXN_SNAPSHOT`.

## Public return codes (src/dbinc/db.in) — SSI-era

Expand All @@ -75,9 +76,10 @@ return *code* an SSI transaction may get, not a `txn_begin` option.

## What this change fixed

- `DB_TXN_SNAPSHOT_SAFE` (0x800) — the SSI flag our engine work added:
documented on `api/c/txnbegin.md`, cross-noted on `api/c/envset_flags.md`
(per-transaction only), and explained in the transactions guide
- `DB_TXN_SNAPSHOT` — now serializable snapshot isolation (SSI); the earlier
separate `DB_TXN_SNAPSHOT_SAFE` (0x800) flag was removed and folded in:
documented on `api/c/txnbegin.md`, cross-noted on `api/c/envset_flags.md`,
and explained in the transactions guide
(`guides/gsg_txn/isolation.md`) and the programmer's reference
(`guides/programmer_reference/transapp_read.md`).
- `DB_SNAPSHOT_UNSAFE` / `DB_SNAPSHOT_CONFLICT` return codes documented in
Expand Down
2 changes: 1 addition & 1 deletion docs_src/api/c/envset_flags.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ The **flags** parameter must be set by bitwise inclusively **OR**'ing together o

The DB_TXN_SNAPSHOT flag may be used to configure Berkeley DB at any time during the life of the application.

Note that only plain snapshot isolation can be configured environment-wide. The serializable-snapshot-isolation flag DB_TXN_SNAPSHOT_SAFE is a per-transaction flag only: it is accepted by <a href="txnbegin.md" class="xref" title="DB_ENV-&gt;txn_begin()">DB_ENV-&gt;txn_begin()</a> but not by `DB_ENV->set_flags()`, so SSI must be requested for each transaction individually.
Note that snapshot transactions configured environment-wide with this flag receive serializable snapshot isolation (SSI); `DB_TXN_SNAPSHOT` is now the SSI mode (there is no separate plain-SI flag). The same behavior can be requested per-transaction via <a href="txnbegin.md" class="xref" title="DB_ENV-&gt;txn_begin()">DB_ENV-&gt;txn_begin()</a>.

- `DB_TXN_WRITE_NOSYNC`

Expand Down
12 changes: 4 additions & 8 deletions docs_src/api/c/txnbegin.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,17 +75,13 @@ The **flags** parameter must be set to 0 or by bitwise inclusively **OR**'ing to

- `DB_TXN_SNAPSHOT`

This transaction will execute with <a href="../../guides/programmer_reference/transapp_read.md" class="olink">snapshot isolation</a>. For databases with the <a href="dbopen.md#dbopen_DB_MULTIVERSION" class="link">DB_MULTIVERSION</a> flag set, data values will be read as they are when the transaction begins, without taking read locks. Silently ignored for operations on databases with <a href="dbopen.md#dbopen_DB_MULTIVERSION" class="link">DB_MULTIVERSION</a> not set on the underlying database (read locks are acquired).
This transaction will execute with **serializable snapshot isolation** (SSI): snapshot isolation plus serializable conflict detection (the Cahill SSI algorithm). For databases with the <a href="dbopen.md#dbopen_DB_MULTIVERSION" class="link">DB_MULTIVERSION</a> flag set, data values are read as they are when the transaction begins, without taking read locks; for databases without DB_MULTIVERSION, read locks are acquired. On top of that consistent snapshot, Berkeley DB tracks read/write anti-dependencies between concurrent snapshot transactions and, when it detects a dependency structure that could produce a non-serializable schedule, aborts one of the transactions so the committed history is equivalent to some serial order.

The error `DB_LOCK_DEADLOCK` will be returned from update operations if a snapshot transaction attempts to update data which was modified after the snapshot transaction read it.
A transaction that hits such a potential anomaly is aborted with one of two Berkeley DB-specific error returns instead of committing: <a href="../../guides/programmer_reference/program_errorret.md" class="olink">DB_SNAPSHOT_UNSAFE</a> (a potential serializable-snapshot anomaly was detected via a read/write anti-dependency) or <a href="../../guides/programmer_reference/program_errorret.md" class="olink">DB_SNAPSHOT_CONFLICT</a> (a conflicting snapshot update was detected). On either return the application must abort the transaction and may retry it. The error `DB_LOCK_DEADLOCK` may also be returned from update operations if a snapshot transaction attempts to update data modified after it read the snapshot.

- `DB_TXN_SNAPSHOT_SAFE`
A `DB_TXN_SNAPSHOT` (SSI) transaction **cannot be prepared for two-phase commit**: <a href="txnprepare.md" class="xref" title="DB_TXN-&gt;prepare()">DB_TXN-&gt;prepare()</a> returns `EINVAL` for such a transaction, because SSI's conflict status is not frozen at prepare time and a later-detected anomaly could not be honored after the transaction had entered the prepared state.

This transaction will execute with **serializable snapshot isolation** (SSI). This is a Berkeley DB extension beyond `DB_TXN_SNAPSHOT`: it implies `DB_TXN_SNAPSHOT` (the transaction reads a consistent snapshot as of its start, as above) and additionally enables serializable conflict detection (the Cahill SSI algorithm). Berkeley DB tracks read/write anti-dependencies between concurrent snapshot-safe transactions and, when it detects a dependency cycle that could produce a non-serializable schedule, aborts one of the transactions so the committed history is equivalent to some serial order.

A transaction that hits such a potential anomaly is aborted with one of two Berkeley DB-specific error returns instead of committing: <a href="../../guides/programmer_reference/program_errorret.md" class="olink">DB_SNAPSHOT_UNSAFE</a> (a potential serializable-snapshot anomaly was detected via a read/write anti-dependency) or <a href="../../guides/programmer_reference/program_errorret.md" class="olink">DB_SNAPSHOT_CONFLICT</a> (a conflicting snapshot update was detected). On either return the application must abort the transaction and may retry it. As with `DB_TXN_SNAPSHOT`, snapshot behavior applies to databases opened with <a href="dbopen.md#dbopen_DB_MULTIVERSION" class="link">DB_MULTIVERSION</a>.

A `DB_TXN_SNAPSHOT_SAFE` (SSI) transaction **cannot be prepared for two-phase commit**: <a href="txnprepare.md" class="xref" title="DB_TXN-&gt;prepare()">DB_TXN-&gt;prepare()</a> returns `EINVAL` for such a transaction, because SSI's conflict status is not frozen at prepare time and a later-detected anomaly could not be honored after the transaction had entered the prepared state.
> **Note:** In this fork, `DB_TXN_SNAPSHOT` provides *serializable* snapshot isolation. In stock Oracle Berkeley DB, `DB_TXN_SNAPSHOT` provided only plain (non-serializable) snapshot isolation, and the earlier `DB_TXN_SNAPSHOT_SAFE` flag has been removed — there is no separate non-serializable snapshot mode.

- `DB_TXN_SYNC`

Expand Down
6 changes: 3 additions & 3 deletions docs_src/guides/gsg_txn/isolation.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,9 +396,9 @@ main(void)

#### Serializable Snapshot Isolation

Berkeley DB also offers *serializable snapshot isolation* (SSI), a stronger mode selected with the `DB_TXN_SNAPSHOT_SAFE` flag. Plain snapshot isolation (`DB_TXN_SNAPSHOT`, above) gives each transaction a consistent view as of its start and avoids read locks, but it permits a small class of anomalies (write skew) that a fully serializable schedule would not. SSI closes that gap: it runs the transaction under snapshot isolation *and* tracks read/write anti-dependencies between concurrent snapshot-safe transactions, aborting a transaction whenever it detects a dependency cycle that could produce a non-serializable outcome.
In this fork, the `DB_TXN_SNAPSHOT` flag provides *serializable snapshot isolation* (SSI). Plain (non-serializable) snapshot isolation gives each transaction a consistent view as of its start and avoids read locks, but it permits a small class of anomalies (write skew) that a fully serializable schedule would not. SSI closes that gap: it runs the transaction under snapshot isolation *and* tracks read/write anti-dependencies between concurrent snapshot transactions, aborting a transaction whenever it detects a dependency structure that could produce a non-serializable outcome. (Stock Oracle Berkeley DB gave `DB_TXN_SNAPSHOT` only plain SI, with a separate `DB_TXN_SNAPSHOT_SAFE` flag for SSI; that flag has been removed and `DB_TXN_SNAPSHOT` is now always SSI.)

`DB_TXN_SNAPSHOT_SAFE` is a per-transaction flag; it implies `DB_TXN_SNAPSHOT` and is passed to <a href="../../api/c/txnbegin.md" class="olink">DB_ENV-&gt;txn_begin()</a> the same way. Unlike `DB_TXN_SNAPSHOT`, it cannot be enabled environment-wide through `DB_ENV->set_flags()` — each transaction that wants serializable snapshot isolation must request it.
`DB_TXN_SNAPSHOT` (SSI) is passed to <a href="../../api/c/txnbegin.md" class="olink">DB_ENV-&gt;txn_begin()</a>, or enabled environment-wide through `DB_ENV->set_flags()`.

Because SSI aborts transactions to preserve serializability, a serializable-snapshot transaction may fail to commit with one of two Berkeley DB-specific return codes:

Expand All @@ -407,4 +407,4 @@ Because SSI aborts transactions to preserve serializability, a serializable-snap

In both cases the application must abort the transaction and may retry it, exactly as it would for `DB_LOCK_DEADLOCK`. See <a href="../../guides/programmer_reference/program_errorret.md" class="olink">Error Returns to Applications</a> for the return-code descriptions.

One restriction applies: a `DB_TXN_SNAPSHOT_SAFE` transaction cannot be prepared for two-phase commit. <a href="../../api/c/txnprepare.md" class="olink">DB_TXN-&gt;prepare()</a> returns `EINVAL` for such a transaction, because SSI's conflict status is not frozen at prepare time and a later-detected anomaly could not be honored once the transaction had entered the prepared state.
One restriction applies: a `DB_TXN_SNAPSHOT` (SSI) transaction cannot be prepared for two-phase commit. <a href="../../api/c/txnprepare.md" class="olink">DB_TXN-&gt;prepare()</a> returns `EINVAL` for such a transaction, because SSI's conflict status is not frozen at prepare time and a later-detected anomaly could not be honored once the transaction had entered the prepared state.
4 changes: 2 additions & 2 deletions docs_src/guides/programmer_reference/program_errorret.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ The DB_SECONDARY_BAD error is returned if a secondary index has been corrupted.

**DB_SNAPSHOT_CONFLICT**

The DB_SNAPSHOT_CONFLICT error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT_SAFE</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a conflicting snapshot update — an attempt to commit an update that conflicts with another concurrent snapshot-safe transaction. The affected transaction must be aborted; the application may then retry it. This return is specific to serializable snapshot isolation and does not occur for plain <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT</a> transactions.
The DB_SNAPSHOT_CONFLICT error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a conflicting snapshot update — an attempt to commit an update that conflicts with another concurrent snapshot transaction. The affected transaction must be aborted; the application may then retry it. This return is specific to serializable snapshot isolation.

**DB_SNAPSHOT_UNSAFE**

The DB_SNAPSHOT_UNSAFE error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT_SAFE</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a potential serializability anomaly through a read/write anti-dependency: the transaction sits as the pivot of a dangerous structure (it is both the read end and the write end of anti-dependency edges among committed and running transactions) that could otherwise produce a non-serializable schedule. The affected transaction is aborted rather than allowed to commit; the application may then retry it. As with DB_SNAPSHOT_CONFLICT, this return is specific to serializable snapshot isolation.
The DB_SNAPSHOT_UNSAFE error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a potential serializability anomaly through a read/write anti-dependency: the transaction sits as the pivot of a dangerous structure (it is both the read end and the write end of anti-dependency edges among committed and running transactions) that could otherwise produce a non-serializable schedule. The affected transaction is aborted rather than allowed to commit; the application may then retry it. As with DB_SNAPSHOT_CONFLICT, this return is specific to serializable snapshot isolation.
Loading
Loading