Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
107 changes: 72 additions & 35 deletions .github/tools/azurite/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion .github/workflows/job-arrow-extension.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
uses: ./.github/actions/rust-extension-toolchain

- name: Cache cargo registry and build
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
with:
# ext-php-rs bindings are generated against the PHP headers, so builds are per PHP version
key: php${{ matrix.php }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/job-flow-php-extension.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
uses: ./.github/actions/rust-extension-toolchain

- name: Cache cargo registry and build
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
with:
# ext-php-rs bindings are generated against the PHP headers, so builds are per PHP version
key: php${{ matrix.php }}
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ phpunit.xml
phpbench.json
benchmarks/datasets/
benchmarks/var/
.env
.env
/.github/tools/azurite/node_modules/
3 changes: 0 additions & 3 deletions benchmarks/materialise.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@

require __DIR__ . '/../bootstrap.php';

// Datasets::reset() (benchmarks/bootstrap.php) wipes benchmarks/var and refuses any path that does
// not contain /benchmarks/var. Fixtures live in benchmarks/datasets, so nothing here interacts with it.

$rows = BenchmarkRows::count();
$windowRows = BenchmarkRows::windowCount();

Expand Down
6 changes: 1 addition & 5 deletions benchmarks/src/BenchmarkRows.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,7 @@
use function getenv;

/**
* The row counts every bench provider and materialise.php resolve through. They must agree: a fixture
* set materialised for a different count leaves the first timed run generating one.
*
* phpbench's own --parameters cannot carry this - it replaces a subject's whole parameter set rather
* than merging into it, so any subject with a second provider would lose that axis.
* Providers and materialise.php must agree; phpbench --parameters replaces rather than merges a parameter set.
*/
final class BenchmarkRows
{
Expand Down
3 changes: 1 addition & 2 deletions benchmarks/src/Datasets/DigestCache.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@
namespace Flow\Benchmarks\Datasets;

/**
* One source-tree walk per tree per process. Warmed from a #[Bench\BeforeMethods] method so no
* timed window ever pays for it.
* Warm from #[BeforeMethods] so no timed window pays for the tree walk.
*/
final class DigestCache
{
Expand Down
8 changes: 0 additions & 8 deletions benchmarks/src/Datasets/FixtureFormat.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,6 @@
*/
enum FixtureFormat: string
{
/**
* Every derived fixture is from_parquet(root) -> to_X(), and the root itself is
* FakeRandomOrdersExtractor -> to_parquet, so these six trees are in the byte-producing
* chain of every fixture regardless of its format. Floe lives inside src/core/etl/src.
*
* benchmarks/src/Datasets is in the chain because TextDataset and SellersDataset build their
* rows in PHP rather than delegating to an adapter, so their bytes change with this tree.
*/
private const CHAIN = [
'src/core/etl/src',
'src/core/etl/tests/Flow/ETL/Tests/Double/FakeRandomOrdersExtractor.php',
Expand Down
3 changes: 1 addition & 2 deletions benchmarks/src/Datasets/FixturePath.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@
use function glob;

/**
* The single place a fixture path is built. The fingerprint lives in the filename rather than in a
* sidecar so it can never drift from the bytes it describes.
* Fingerprint lives in the filename so it cannot drift from the bytes.
*/
final readonly class FixturePath
{
Expand Down
3 changes: 1 addition & 2 deletions benchmarks/src/Partitioning/PartitionCardinality.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@
use function Flow\ETL\DSL\ref;

/**
* The orders fixture spreads rows over 5 sellers and a full year of created_at, so these two keys
* give ~5 and ~365 partitions. There is no synthetic fixture behind a third point and none is invented.
* Orders fixture gives ~5 (seller) and ~365 (created_at day) partitions; no third fixture exists.
*/
enum PartitionCardinality: string
{
Expand Down
7 changes: 1 addition & 6 deletions benchmarks/src/Partitioning/PartitionedTree.php
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,7 @@
use function substr;

/**
* A partitioned tree is a derived fixture, so it lives in the fixture cache under a fingerprinted
* name rather than in benchmarks/var - which Datasets::reset() wipes at every process start, making
* every phpbench iteration rebuild the whole tree before it could measure a read.
*
* The read path must be a glob. from_csv() over a bare partitioned directory silently reads 0 rows,
* and a following partition filter() then fails at bind: "Schema definition for entry ... not found."
* Lives in fixture cache (benchmarks/var is wiped per process); read path must be a glob or from_csv reads 0 rows.
*/
final class PartitionedTree
{
Expand Down
12 changes: 1 addition & 11 deletions benchmarks/src/Pipeline/CanonicalPipeline.php
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,7 @@
use function uniqid;

/**
* The one pipeline shape every source is measured through, so a source's number differs only by its
* read leg. Every subject that claims to be comparable with the others must run through here.
*
* withColumn is the only verb that adds a column, so it forces schema widening at plan-bind time.
* filter is deliberately non-selective - created_at is never null - so the row count reaching the loader
* cannot move for a second reason. select re-converges every source onto the same five columns, so the
* loader does identical work whatever was read. Floe stores typed values, adding no stringification the
* source had not already paid.
*
* The output path is unique per call because a fixed path re-run across iterations and revolutions would
* measure overwrite behaviour.
* Every comparable subject runs through here so sources differ only by their read leg.
*/
final readonly class CanonicalPipeline
{
Expand Down
4 changes: 1 addition & 3 deletions benchmarks/src/Pipeline/LimitPushdownScenario.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,7 @@
use function Flow\ETL\DSL\data_frame;

/**
* A null limit runs the same pipeline with no limit() in the plan, so the pair prices what a
* limit left in the plan costs on a local source. The remote half of the question needs
* push-down, which this engine does not have yet.
* Null limit = same plan without limit(); the pair prices a limit on a local source (no push-down yet).
*/
final readonly class LimitPushdownScenario
{
Expand Down
6 changes: 1 addition & 5 deletions benchmarks/src/Pipeline/NodePipelineScenario.php
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,7 @@
use function uniqid;

/**
* XML and Text do not infer and cannot carry the orders schema, so they form their own family: the
* same four verbs in the same order as the canonical shape, over one column.
*
* These two numbers are comparable to each other and to nothing else. The loader is to_text because a
* one-column string frame in Floe would measure Floe's footer rather than the pipeline.
* XML/Text cannot carry the orders schema: comparable only to each other; to_text avoids measuring Floe's footer.
*/
final readonly class NodePipelineScenario
{
Expand Down
11 changes: 1 addition & 10 deletions benchmarks/src/Pipeline/OrdersSchema.php
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,7 @@
use function Flow\Types\DSL\type_structure;

/**
* The WEAK schema — for each source, exactly what that source's own inference produces today.
* Declaring it prices the sampling pass and nothing else; a stronger schema would additionally price
* type strength, which is a different question. Frozen as literals rather than computed at bench
* time, so inference drift fails loudly instead of quietly measuring something new. Arms that
* coincide today are still independent facts — do not fold them into a shared helper.
*
* Every column of every source infers nullable, so there is no separate nullability rule.
* SourceExtractorTest asserts every Source arm still equals what that source infers. The two
* ServiceSource arms have no such guard: proving them needs a live database, so it cannot live in
* the unit suite.
* Frozen literals of each source's own inference (SourceExtractorTest guards them; ServiceSource arms unguarded).
*/
final readonly class OrdersSchema
{
Expand Down
5 changes: 1 addition & 4 deletions benchmarks/src/Pipeline/PlanDepthScenario.php
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,7 @@
use function Flow\ETL\DSL\ref;

/**
* schema() answers from the plan without executing it, so this prices Planner::plan() alone (optimize, translate,
* bind, split) rather than $depth withColumn calls per row.
*
* One call plans PLANS identical plans, because a single plan is far below timer resolution.
* schema() plans without executing, so this prices Planner::plan() alone.
*/
final readonly class PlanDepthScenario
{
Expand Down
Loading
Loading