[alloc] Add `BStackVec` for typed, growable vector backed by a `BStack` allocation by williamwutq · Pull Request #2 · williamwutq/bstack

williamwutq · 2026-05-20T18:32:12Z

BStackVec

Description: Typed, growable vector backed by a BStack allocation.

New Feature: Yes; Public Facing
Important Feature: Yes
Type: Container
Feature Flags: alloc, set
Breaking change: No
New Types: BStackVecIter, BStackVec
Rust Only: No
Fuzz: Not needed

williamwutq · 2026-05-20T18:32:44Z

Original entry in PLANNED.md:

Adding `BStackVec<T>` for typed vector storage

Feature flag: set and alloc
Breaking change: No (additive; new type only)

Motivation

While BStackSlice provides a flexible byte-oriented view of allocated blocks, many applications would benefit from a typed vector abstraction that manages element layout, length tracking, and safe access patterns. A BStackVec<T> type would provide a convenient API for storing and manipulating sequences of typed data on a BStack, while still leveraging the underlying durability and crash-safety guarantees.

Design

BStackVec<T> mirrors Rust's standard Vec<T> API, but backed by a BStack allocation. It would manage its own length and capacity metadata, stored in the block header or a reserved prefix.

Memory layout

The allocation layout would consist of:

Header (16 bytes): length (u64) + capacity (u64)
Elements: inline typed data, stored as [T; capacity]

The header occupies the first 16 bytes of the block, with element data following. This ensures that the metadata is always present and can be recovered even if the BStackVec handle is lost.

API surface

Core methods mirroring Vec<T>:

impl<'a, T: Copy, A: BStackAllocator> BStackVec<'a, T, A> {
    // Creation and destruction
    pub fn new(alloc: &'a A) -> Result<Self, A::Error>;
    pub fn with_capacity(capacity: u64, alloc: &'a A) -> Result<Self, A::Error>;
    pub fn from_slice(slice: &[T], alloc: &'a A) -> Result<Self, A::Error>;
    
    // Accessors
    pub fn len(&self) -> Result<u64, A::Error>;
    pub fn capacity(&self) -> Result<u64, A::Error>;
    pub fn is_empty(&self) -> Result<bool, A::Error>;
    
    // Element access
    pub fn get(&self, index: u64) -> Result<Option<T>, A::Error>;
    pub fn as_slice(&self) -> Result<BStackSlice<'a, A>, A::Error>;
    
    // Modification
    pub fn push(&mut self, value: T) -> Result<(), A::Error>;
    pub fn pop(&mut self) -> Result<Option<T>, A::Error>;
    pub fn truncate(&mut self, len: u64) -> Result<(), A::Error>;
    pub fn clear(&mut self) -> Result<(), A::Error>;
    pub fn reserve(&mut self, additional: u64) -> Result<(), A::Error>;
    pub fn resize(&mut self, new_len: u64, value: T) -> Result<(), A::Error>;
    
    // Iteration
    pub fn iter(&self) -> Result<BStackVecIter<'a, T, A>, A::Error>;
}

Growth strategy

When capacity is exceeded, BStackVec would use the BStack's realloc to grow the block, similar to Vec's doubling strategy (or a configurable policy). The metadata prefix would be preserved and updated with new capacity.

Guarding support

BStackVec<T, A> should integrate with BStackGuardedSlice by providing a as_guarded_slice() method that wraps the underlying slice with a guard, enabling transparent transforms across the entire vector.

Serialization considerations

BStackVec should be serializable to persistent storage by writing only the populated portion (length, not capacity) and reconstructable on load by reading back the metadata from the block header.

Open questions

Initial capacity: Should new() create an empty vector with zero capacity, or should it allocate a small default capacity (e.g., 4 or 8 elements) to avoid immediate reallocations on first push?
Generic bounds on T: Should T be required to implement Copy, or should a Drop implementation be provided for types with destructors? A Drop impl would be complex, as it must be called on remaining elements when the vec is deallocated.
Error handling: Should methods return Result for all operations, or should some (e.g., len(), capacity()) be infallible? Returning Result allows for better error propagation but may be more verbose for simple accessors.
Initialization of new elements: When growing the vector, should new elements be zero-initialized, left uninitialized (i.e. MaybeUninit), or should we require T to be Default and call default()? Zero-initialization is safer and guranteed for newly allocated space, but may be unnecessary overhead for some types.
Zeroing on deallocation: Should empty parts of the vector be zeroed on deallocation for security, or should this be left to the caller? Zeroing adds overhead but can prevent data leakage.

williamwutq · 2026-05-20T18:33:55Z

The Feature Checklist

API Design
- Define new public types, functions, and traits.
- API is consistent with the crate's design and follows Rust conventions.
- All invariants are documented.
- "Raw" accessors are provided and documented as such.
- Methods do not panic unless justified; all panics are documented.
- Invalid input is handled gracefully via Result or Option, not panics.
- Methods that may compromise memory safety, whether that of BStack or the heap memory, are marked unsafe and documented.
- Relevant traits are implemented (e.g., Debug, Clone, Copy, PartialEq, Read/Write).
- Breaking changes are avoided where possible; if unavoidable, documented with migration guidance.
- Forward compatibility and extensibility are considered.
- If a plan exists, it addresses the Design section of the relevant issue, with all decisions resolved or documented.
- Use the correct feature flags.
- No additional dependencies.
Implementation
- All invariants are maintained.
- No logical errors or overlooked edge cases.
- No hidden assumptions.
- Safe abstractions are preferred where practical.
- Public APIs are re-exported in lib.rs and/or mod.rs as appropriate.
- Implementation is reviewed.
- If a plan exists, it addresses the Open Questions section of the relevant issue, with all questions resolved or documented.
- No TODOs, FIXMEs, todo! macros, or unimplemented! macros remain in the code.
- Interoperability with existing BStack APIs is considered and tested.
- If not a Rust only feature, C version is implemented and tested.
Safety
- All unsafe code is justified, minimal, and well-documented.
- Safety invariants are clearly stated in documentation for all unsafe items.
- All methods are power-fail safe by default: interruption leaves BStack in a consistent state. Any exceptions or data-loss consequences (e.g., a block lost mid-allocation) are documented.
- Thread safety is documented for all structs and methods; required synchronization is implemented.
- In places where atomicity is relevant, document whether operations are atomic, and if not, what the implications are for concurrent usage.
Optimisation
- Implementation is optimised for time complexity, memory usage, and minimal BStack operations.
- Trade-offs from any optimisations are documented.
- Batching is considered.
- Suggestions for further optimisations are added to PLANNED.md if not implemented now.
Testing
- Tests cover all functionality, edge cases, and error conditions.
- Integration tests for overall behavior, if necessary.
- New allocator implementations are added to alloc_fuzz_tests.rs and pass fuzz tests.
- CI passes.
- CodeQL passes without warnings.
Documentation
- All public items document their purpose, parameters, return values, and any panics or side effects.
- cargo doc --all-features builds without warnings except unused manifest key.
- Doc tests pass.
- New feature flags have meaningful documentation.
- Changelog entry added.
- For breaking changes, concise migration guidance provided in the changelog.
- README and lib.rs updated if the feature changes public-facing behavior or usage.
- New files have a module-level doc comment describing their purpose.
- Important new features have an example in examples/ showing realistic usage.
- New BStack usage patterns have a README section, lib.rs documentation, and an examples/ entry.
- Allocators and safety-sensitive methods include a markdown table. Example columns: name, atomicity, single-operation, and relevant notes.

williamwutq · 2026-05-20T18:42:26Z

@copilot check the The Feature Checklist according to the PR description. Comment on this Github PR with an edited version of the checklist where items that are satisfied are marked. Give additional comments on how to implement/satisfy some of the items that might take effort, if necessary.

Copilot · 2026-05-20T18:47:45Z

@copilot check the The Feature Checklist according to the PR description. Comment on this Github PR with an edited version of the checklist where items that are satisfied are marked. Give additional c...

Reviewed on 9fd7606 (no new code changes). Here is the edited checklist for this PR:

API Design

Implementation

Safety

unsafe code is minimal/justified/documented.
Safety invariants are stated for unsafe items.
Power-fail/crash-consistency behavior is documented.
Thread safety requirements are explicitly documented for this type.
Atomicity expectations for concurrent usage are explicitly documented.

Optimisation

Core implementation has reasonable complexity and growth behavior.
Main trade-offs are documented.
Batching is considered/documented where relevant.
Further optimization suggestions added to PLANNED.md.

Testing

Tests cover all BStackVec functionality, edge cases, and errors.
Integration tests as needed.
New allocator fuzz-test requirement is N/A (this is a container, not a new allocator).
CI passes.
CodeQL passes.

Documentation

High-effort follow-ups I recommend next:

Add dedicated BStackVec tests (constructor/header recovery, reserve/resize overflow paths, truncation/pop zeroing, iter snapshot semantics, raw-block roundtrip, edge case with zero-sized T policy).
Add a realistic examples/bstack_vec.rs using LinearBStackAllocator and reopen/recover flow.
Add explicit thread-safety + atomicity docs for BStackVec operations.
Panic audit: either remove internal unwrap assumptions or document panic conditions explicitly.
Clarify PR metadata on Rust/C scope (either implement C equivalent or mark this as Rust-only feature).

williamwutq · 2026-05-20T20:21:51Z

+/// | `push` (no realloc) | write element → increment `len` | Crash after element write: element on disk but `len` not updated; slot is effectively invisible. Re-running `push` with the same value recovers correctly. |
+/// | `push` (with realloc) | `realloc` → write `cap` → write element → increment `len` | Crash at any point: header re-read on next open reflects the committed state; worst case is a leaked block (with `LinearBStackAllocator`) or a stale cap value. |
+/// | `pop` | read element → zero slot → decrement `len` | Crash after zero but before `len` decrement: element is zeroed on disk but `len` still counts it; `get(len-1)` returns a zero-bit `T`. Decrementing `len` on the next `pop` call removes it cleanly. |
+/// | `truncate` | zero removed slots → write `len` | Crash after zeroing but before `len` write: logically removed elements are on disk as zeros; `len` still points past them. Calling `truncate` again to the same target is idempotent. |
+/// | `resize` (grow) | `reserve` → write elements → write `len` | Same as `push` repeated; elements between the old and new `len` may be partially written. |
+/// | `clear` | (delegates to `truncate(0)`) | See `truncate`. |
+/// | `reserve` | `realloc` → write `cap` | Crash between the two: cap field may reflect the old value; the block is larger than cap indicates. Harmless — the next `push` re-checks and may realloc again unnecessarily. |


Do not refer to concrete allocators such as LinearBStackAllocator in these comments

williamwutq · 2026-05-20T20:59:23Z

+    /// Allocate a `BStackVec` and populate it from a Rust slice.
+    ///
+    /// The resulting vec has `len == capacity == data.len()`.
+    pub fn from_slice(data: &[T], alloc: &'a A) -> io::Result<Self> {
+        let len = data.len() as u64;
+        let slice = alloc.alloc(Self::block_size(len)?)?;
+        let vec = Self {
+            slice,
+            _phantom: PhantomData,
+        };
+        if len > 0 {
+            // Write len and cap; elements are written individually below.
+            vec.write_header(len, len)?;
+            for (i, &item) in data.iter().enumerate() {
+                vec.write_elem_at(i as u64, item)?;
+            }
+        }
+        Ok(vec)
+    }


We have from_slice, we might need to add read_all or read_vec that reads this as a rust Vec<T>.

Copilot

Pull request overview

This PR introduces BStackVec, a public, typed, growable vector abstraction backed by a BStack allocation (behind alloc + set), alongside documentation and examples to support the new container API.

Changes:

Adds BStackVec and BStackVecIter implementation under src/alloc/vec.rs and re-exports under the appropriate feature gates.
Updates crate/module/README documentation and CHANGELOG to describe the new API and its crash-recovery model.
Adds a bstack_vec example and updates Cargo.toml example feature requirements.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`src/lib.rs`	Documents `BStackVec` under feature flags and re-exports it when `alloc + set` are enabled.
`src/alloc/vec.rs`	New `BStackVec`/`BStackVecIter` implementation plus unit tests.
`src/alloc/mod.rs`	Adds `vec` module and exports `BStackVec`/`BStackVecIter` behind `set` (within `alloc`).
`README.md`	Extends feature documentation and adds a `BStackVec` section with layout/usage.
`PLANNED.md`	Removes the now-implemented “Adding `BStackVec<T>`” planned item.
`examples/bstack_vec.rs`	New example demonstrating persistence/reopen flow using `BStackVec`.
`CHANGELOG.md`	Adds an Unreleased entry describing `BStackVec`.
`Cargo.toml`	Registers examples with `required-features`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+/// reconstruct the handle after a reopen without any additional recovery logic.
+///
+/// ## Feature flags
+///
+/// Requires both the `alloc` and `set` Cargo features.
+pub struct BStackVec<'a, T: Copy, A: BStackSliceAllocator> {
+    /// The full block: header (16 B) followed by element data.
+    slice: BStackSlice<'a, A>,
+    _phantom: PhantomData<T>,
+}


williamwutq · 2026-05-20T21:13:35Z

+    /// Remove and return the last element, or `None` if empty.
+    ///
+    /// The vacated slot is zeroed before `len` is decremented.
+    pub fn pop(&mut self) -> io::Result<Option<T>> {
+        let (len, _) = self.read_header()?;
+        if len == 0 {
+            return Ok(None);
+        }
+        let value = self.read_elem_at(len - 1)?;
+        self.zero_elem_at(len - 1)?;
+        self.write_len_field(len - 1)?;
+        Ok(Some(value))
+    }
+
+    /// Shorten the vec to `new_len` elements.
+    ///
+    /// No-op when `new_len >= len`.  Removed slots are zeroed in a single
+    /// [`BStackSlice::zero_range`] call; capacity is unchanged.
+    pub fn truncate(&mut self, new_len: u64) -> io::Result<()> {
+        let (len, _) = self.read_header()?;
+        if new_len >= len {
+            return Ok(());
+        }
+        let start = Self::elem_offset(new_len);
+        let removed_bytes = (len - new_len) * Self::elem_size();
+        self.slice.zero_range(start, removed_bytes)?;
+        self.write_len_field(new_len)
+    }


The recommendation is to reverse the order so len change comes before zeroing so post-crash never deserialize zeroed slots that should be cleaned

+    /// Return the underlying block slice (header + all allocated element space).
+    ///
+    /// This is the original allocation handle and may be passed to
+    /// [`crate::BStackAllocator::realloc`] or [`crate::BStackAllocator::dealloc`].
+    pub fn raw_block(&self) -> BStackSlice<'a, A> {
+        self.slice
+    }
+


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

williamwutq added 5 commits May 20, 2026 10:05

Add more details to plan

d2b7dcd

Basic implemenation of Vec

efcee97

Add to CHANGLOG

e32badd

Relevant documentation

4b7387a

Cargo autofix

13197fc

williamwutq self-assigned this May 20, 2026

Merge branch 'master' into vec

9fd7606

Copilot started work on behalf of williamwutq May 20, 2026 18:42 View session

Copilot finished work on behalf of williamwutq May 20, 2026 18:48

williamwutq added 6 commits May 20, 2026 12:27

Improve documentation

3491678

Add tests to vec

159d1dc

Remove planned section

06e4490

Add example

2bc5e39

Fix warnings by unwrapping result

a5b6449

Build example with feature flags

49359c2

williamwutq changed the title ~~Add BStackVec for typed, growable vector backed by a BStack allocation~~ [alloc] Add BStackVec for typed, growable vector backed by a BStack allocation May 20, 2026

williamwutq added 2 commits May 20, 2026 13:04

Minor improvements

e3c541c

Improvements for safety and add integration tests

e7b7f38

williamwutq commented May 20, 2026

View reviewed changes

Fix table

e441a52

williamwutq requested a review from Copilot May 20, 2026 21:03

Copilot started reviewing on behalf of williamwutq May 20, 2026 21:03 View session

williamwutq marked this pull request as ready for review May 20, 2026 21:09

Copilot AI reviewed May 20, 2026

View reviewed changes

Fix hashmap gating

99de6a2

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

williamwutq force-pushed the vec branch from 2da6cbe to c3d9919 Compare May 20, 2026 21:23

Add batching

89e5e66

williamwutq force-pushed the vec branch from c3d9919 to 89e5e66 Compare May 20, 2026 21:24

williamwutq added 2 commits May 20, 2026 20:16

Merge branch 'master' into vec

7d84e87

Merge branch 'master' into vec

735b109

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[alloc] Add `BStackVec` for typed, growable vector backed by a `BStack` allocation#2

[alloc] Add `BStackVec` for typed, growable vector backed by a `BStack` allocation#2
williamwutq wants to merge 19 commits into
masterfrom
vec

williamwutq commented May 20, 2026 •

edited

Loading

Uh oh!

williamwutq commented May 20, 2026

Uh oh!

williamwutq commented May 20, 2026 •

edited

Loading

Uh oh!

williamwutq commented May 20, 2026

Uh oh!

Copilot AI commented May 20, 2026 •

edited by williamwutq

Loading

Uh oh!

Uh oh!

williamwutq May 20, 2026

Uh oh!

Uh oh!

williamwutq May 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

williamwutq May 20, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

williamwutq commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

BStackVec

Uh oh!

williamwutq commented May 20, 2026

Adding BStackVec<T> for typed vector storage

Motivation

Design

Memory layout

API surface

Growth strategy

Guarding support

Serialization considerations

Open questions

Uh oh!

williamwutq commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The Feature Checklist

Uh oh!

williamwutq commented May 20, 2026

Uh oh!

Copilot AI commented May 20, 2026 • edited by williamwutq Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

williamwutq May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

williamwutq May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

williamwutq May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

williamwutq commented May 20, 2026 •

edited

Loading

Adding `BStackVec<T>` for typed vector storage

williamwutq commented May 20, 2026 •

edited

Loading

Copilot AI commented May 20, 2026 •

edited by williamwutq

Loading