Skip to content

Preserve compound value types across user function calls - #404

Closed
luisleo526 wants to merge 5 commits into
mainfrom
fix/issue-391-compound-value-types
Closed

luisleo526 wants to merge 5 commits into
mainfrom
fix/issue-391-compound-value-types

Conversation

@luisleo526

Copy link
Copy Markdown
Contributor

Valid Pine v6 scripts could generate C++ that did not compile because an array, matrix, UDT or string was inferred as double at a function return, untyped parameter or reassigned local.

Infer compound types through branch returns, nested calls, tuple arguments and local assignments. Keep separate callable variants for different array element types and share inference work within an expression query.

Reproduction tests:

  • tests/test_callable_compound_type_flow.py::test_callable_boundaries_keep_the_value_type[array-block-return]
  • tests/test_callable_compound_type_flow.py::test_untyped_array_calls_keep_distinct_element_types
  • tests/test_callable_compound_type_flow.py::test_reassigned_matrix_local_keeps_its_initializer_type

Fixes #391

Use the established integer storage and matrix alias in type-flow assertions.
Check array specializations, mutations, matrix values and copy independence.
Share operand and value inference while an expression query is active, then
release cached results before bindings can change. Preserve ordinary matrix
member initialization when the declaration expression is still available.
Merge the current main while retaining the feature correction.

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; formal verification proved 0 of 16 changed function(s) (3 sampled, not proven, 13 not verified).

Formal verification. PR-changed functions: 3/16 verified (0 proven, 3 may-equivalent, 0 distinguished) · 13 not verified (13 unsupported).

Not verified on this run: \_\_init\_\_ (unsupported), \_analyze (unsupported), \_operator\_operand\_type (unsupported), \_propagate\_call\_site\_counts (unsupported), \_untyped\_param\_family\_conflicts (unsupported), \_visit\_FuncDef (unsupported), \_visit\_TupleAssign (unsupported), \_handle\_user\_func\_call (unsupported), \_\_init\_\_ (unsupported), \_visit (unsupported), and 3 more.


Graphify review — findings

Adds structured type propagation for user functions: array, object and string results survive branch returns and history clones, and untyped arguments keep their collection element types through nested calls and tuple bindings. Binds exact parameter and return profiles to the callsite variants the emitter uses, and treats a structured mismatch such as a vector<double> passed to a vector<int>& parameter as a conflicting argument family rather than sharing one signature. Routes operand type resolution through a per-expression OperatorTypeQuery memo so deeply nested calls share inference work, including when a return type stays unresolved, and reassigned matrix locals keep their initializer's type.

Worth a look

  • Adjacent expressions make base.py unparsable — pineforge_codegen/analyzer/base.py · Escalate · high
    • unverified hypothesis — survived the adversarial refute pass; needs human review
  • Assignment-inferred types are not carried into later reads — pineforge_codegen/analyzer/types.py:1491 · Escalate · medium
    • unverified hypothesis — survived the adversarial refute pass; needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 5116 functions depend on the 1147 functions this change touches.

Health — grade A; 10 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

  • Analyzer — 107 callers, 15 callees (high)
  • _generate() — 131 callers, 9 callees (high)
  • CodeGen — 68 callers, 14 callees (high)
  • _generate_raw() — 29 callers, 9 callees (high)
  • _generate() — 9 callers, 25 callees (high)
  • _analyze() — 30 callers, 6 callees (high)
  • _check() — 31 callers, 3 callees (high)
  • _codegen() — 10 callers, 7 callees (high)
  • …and 2 more

Verification — 5116 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 4161 function(s) in the blast radius were not formally verified this run

Cross-repo impact

Cross-repo impact

This change impacts 2 symbol(s) across 1 peer repo(s) — blast radius over derived cross-repo edges:

  • pineforge-4pass/pineforge-hpo — python/pineforge_hpo/transpile.py (via import_moniker, heuristic), tests/python/test_transpile.py (via import_moniker, heuristic)

1 peer repo(s) in this group are not yet indexed — cross-repo impact may be incomplete.

Cross-repo linkage is derived (SCIP-moniker matches are sound; name/contract matches are heuristic) and ADVISORY — it never affects this check's conclusion.

Test selection

Test selection

528 of 528 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety (noncode-change, uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • tests/test_action_pins.py — full-run-safety
  • tests/test_allow_entry_in_qualifier.py — impact, full-run-safety
  • tests/test_allow_entry_in_request_const.py — impact, full-run-safety
  • tests/test_analyzer.py — impact, full-run-safety
  • tests/test_analyzer_matrix_inference.py — impact, full-run-safety
  • tests/test_analyzer_ta_return_types.py — impact, full-run-safety
  • tests/test_anchor_tf_change.py — impact, full-run-safety
  • tests/test_anchor_time.py — impact, full-run-safety
  • tests/test_anchor_vwap.py — impact, full-run-safety
  • tests/test_array_bounds_insert_fill_slice.py — impact, full-run-safety
  • tests/test_array_checked_access.py — impact, full-run-safety
  • tests/test_array_empty_aggregates.py — impact, full-run-safety
  • tests/test_array_history.py — impact, full-run-safety
  • tests/test_array_missing_integer_index.py — impact, full-run-safety
  • tests/test_array_new_color_fill_and_na_size.py — impact, full-run-safety
  • tests/test_array_new_defaults_and_order.py — impact, full-run-safety
  • tests/test_array_reference_semantics.py — impact, full-run-safety
  • tests/test_array_retained_alias.py — impact, full-run-safety
  • tests/test_array_slice_view.py — impact, full-run-safety
  • tests/test_array_sort_indices.py — impact, full-run-safety
  • tests/test_array_terminal_returns.py — impact, full-run-safety
  • tests/test_arrseg_coded_run_stops.py — impact, full-run-safety
  • tests/test_arrseg_review_fixes.py — impact, full-run-safety
  • tests/test_binop_operand_order.py — impact, full-run-safety
  • tests/test_block_shadow_primitive_types.py — impact, full-run-safety
  • tests/test_block_var_plain_siblings.py — impact, full-run-safety
  • tests/test_bool_na_refusal.py — impact, full-run-safety
  • tests/test_bool_na_refusals_2.py — impact, full-run-safety
  • tests/test_bool_na_values.py — impact, full-run-safety
  • tests/test_bool_string_inference.py — impact, full-run-safety
  • tests/test_builtin_name_shadow.py — impact, full-run-safety
  • tests/test_builtin_named_locals_compile.py — impact, full-run-safety
  • tests/test_builtin_named_locals_price_refusal.py — impact, full-run-safety
  • tests/test_builtin_named_locals_receipt.py — impact, full-run-safety
  • tests/test_builtin_named_locals_scope.py — impact, full-run-safety
  • tests/test_builtin_named_parameters.py — impact, full-run-safety
  • tests/test_builtin_named_parameters_reference.py — full-run-safety
  • tests/test_calc_on_order_fills_codegen.py — impact, full-run-safety
  • tests/test_calendar_lambda_locals.py — impact, full-run-safety
  • tests/test_call_argument_order.py — impact, full-run-safety
  • tests/test_callable_compound_type_flow.py — impact, changed-test, full-run-safety
  • tests/test_callable_local_histories.py — impact, full-run-safety
  • tests/test_callable_series_param_types.py — impact, full-run-safety
  • tests/test_callable_string_series_type.py — impact, full-run-safety
  • tests/test_callable_ta_forwarded_length.py — impact, full-run-safety
  • tests/test_callable_var_first_reach.py — impact, full-run-safety
  • tests/test_checked_settings.py — impact, full-run-safety
  • tests/test_ci_engine_main_touched.py — full-run-safety
  • tests/test_closed_trade_fold.py — full-run-safety
  • tests/test_closed_trade_fold_emission.py — impact, full-run-safety
  • … and 478 more

non-code file(s) changed (CHANGELOG.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (CHANGELOG.md, pineforge_codegen/analyzer/call_handlers.py, pineforge_codegen/codegen/emit_top.py) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Docs that may be stale (advisory)

Formal verification

Could not verify: Could not verify \_\_init\_\_.

The verifier did not have enough to check \_\_init\_\_, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_analyze.

The verifier did not have enough to check \_analyze, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_operator\_operand\_type.

The verifier did not have enough to check \_operator\_operand\_type, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_propagate\_call\_site\_counts.

The verifier did not have enough to check \_propagate\_call\_site\_counts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_untyped\_param\_family\_conflicts.

The verifier did not have enough to check \_untyped\_param\_family\_conflicts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_visit\_FuncDef.

The verifier did not have enough to check \_visit\_FuncDef, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_visit\_TupleAssign.

The verifier did not have enough to check \_visit\_TupleAssign, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_handle\_user\_func\_call.

The verifier did not have enough to check \_handle\_user\_func\_call, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_\_init\_\_.

The verifier did not have enough to check \_\_init\_\_, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

No difference found (not proven): No behavior difference found in \_selection\_type\_spec (not a proof).

The verifier ran both versions of \_selection\_type\_spec on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

No difference found (not proven): No behavior difference found in \_type\_spec\_from\_expr (not a proof).

The verifier ran both versions of \_type\_spec\_from\_expr on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_visit.

The verifier did not have enough to check \_visit, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_build\_func\_instances.

The verifier did not have enough to check \_build\_func\_instances, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_register\_global\_aggregate\_member\_types.

The verifier did not have enough to check \_register\_global\_aggregate\_member\_types, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_emit\_func\_def\_body.

The verifier did not have enough to check \_emit\_func\_def\_body, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

No difference found (not proven): No behavior difference found in \_type\_spec\_from\_expr\_raw (not a proof).

The verifier ran both versions of \_type\_spec\_from\_expr\_raw on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

@luisleo526

Copy link
Copy Markdown
Contributor Author

Superseded by #406, which carried this change and is merged on main as b7379dd. #391 is fixed there.

@luisleo526 luisleo526 closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant