Skip to content

Preserve reference types across fields and function calls - #403

Closed
luisleo526 wants to merge 4 commits into
mainfrom
fix/issue-390-reference-handle-types
Closed

luisleo526 wants to merge 4 commits into
mainfrom
fix/issue-390-reference-handle-types

Conversation

@luisleo526

Copy link
Copy Markdown
Contributor

Valid Pine v6 scripts could generate C++ that did not compile when assigning na to a drawing or array field, passing a drawing handle through a user function or tuple, or keeping a polyline handle in a local or UDT field.

Preserve the destination storage type for missing reference values, pass drawing handles as values across function and tuple boundaries, and retain polyline handle types for declarations and deletion calls.

Reproduction tests:

  • tests/test_reference_handle_lowering.py::test_reference_field_na_uses_the_field_type[line-Line]
  • tests/test_reference_handle_lowering.py::test_array_field_na_default_has_collection_storage
  • tests/test_reference_handle_lowering.py::test_handle_parameter_alias_and_tuple_are_values_at_each_call
  • tests/test_reference_handle_lowering.py::test_polyline_array_element_keeps_its_handle_type
  • tests/test_reference_handle_lowering.py::test_polyline_field_and_cast_preserve_typed_reads_and_delete

Fixes #390

Lower missing reference values to their destination storage and retain drawing
handle types through user functions and tuple returns. Support declared
polyline types, fields and deletion arguments.
Read optional function metadata safely when inferring reference casts, so bare
emitters retain matrix element types without raising an AttributeError.
Merge the current main while retaining the feature correction.

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; formal verification proved 0 of 19 changed function(s) (4 sampled, not proven, 15 not verified).

Formal verification. PR-changed functions: 4/19 verified (0 proven, 4 may-equivalent, 0 distinguished) · 15 not verified (15 unsupported).

Not verified on this run: \_\_init\_\_ (unsupported), \_generate\_cpp (unsupported), \_is\_default\_drawing\_initializer (unsupported), \_is\_omitted\_udt\_field (unsupported), \_is\_skip\_expr (unsupported), \_detect\_drawing\_usage (unsupported), \_drawing\_call\_return\_cpp (unsupported), \_emit\_drawing\_method (unsupported), \_spec\_mentions\_drawing (unsupported), \_emit\_func\_def\_body (unsupported), and 5 more.


Graphify review — findings

Adds typed opaque handles for polyline locals, array elements and UDT fields, so default construction and bare na produce a missing handle and copies keep its identity. polyline.new and functional or method-form delete stay visual-only but now evaluate their arguments through declarations and calls, with the opaque ID counter joining script reset and rollback; the counter allocation and the keyword-argument ordering in _emit_polyline_call are where that evaluation contract could go wrong. Fixes reference-typed UDT fields to keep their type when initialized or reassigned with na, including array defaults, and routes named casts like label(na) through a shared na_reference_cast_kind helper in both type inference and codegen. Drawing handles now cross user-function and tuple boundaries by value, and the new cast helper is the piece that has to be wired into compilation for those paths to hold. Narrows the omitted-field and skip-receiver logic to table only, since polyline now has a C++ representation, and gates the _pf_polyline_next_id_ member on a new _uses_polyline flag so non-polyline strategies emit unchanged C++.

Worth a look

  • Polyline ID allocation is not thread-safe — pineforge_codegen/codegen/drawing.py:376 · Escalate · medium
    • unverified hypothesis — survived the adversarial refute pass; needs human review
  • Polyline constructor reorders keyword-argument side effects — pineforge_codegen/codegen/drawing.py:372 · Escalate · medium
    • unverified hypothesis — survived the adversarial refute pass; needs human review
  • Named NA reference casts are not wired into compilation — pineforge_codegen/reference_casts.py:11 · Escalate · medium
    • unverified hypothesis — survived the adversarial refute pass; needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 5256 functions depend on the 1331 functions this change touches.

Health — grade A; 10 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

  • _generate() — 131 callers, 9 callees (high)
  • CodeGen — 68 callers, 14 callees (high)
  • _generate_raw() — 29 callers, 9 callees (high)
  • _generate() — 9 callers, 25 callees (high)
  • _check() — 31 callers, 3 callees (high)
  • _codegen() — 10 callers, 7 callees (high)
  • _generate() — 8 callers, 8 callees (high)
  • _generator() — 9 callers, 7 callees (high)
  • …and 2 more

Verification — 5256 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 4325 function(s) in the blast radius were not formally verified this run

Cross-repo impact

Cross-repo impact

This change impacts 2 symbol(s) across 1 peer repo(s) — blast radius over derived cross-repo edges:

  • pineforge-4pass/pineforge-hpo — python/pineforge_hpo/transpile.py (via import_moniker, heuristic), tests/python/test_transpile.py (via import_moniker, heuristic)

1 peer repo(s) in this group are not yet indexed — cross-repo impact may be incomplete.

Cross-repo linkage is derived (SCIP-moniker matches are sound; name/contract matches are heuristic) and ADVISORY — it never affects this check's conclusion.

Test selection

Test selection

528 of 528 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety (noncode-change, uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • tests/test_action_pins.py — full-run-safety
  • tests/test_allow_entry_in_qualifier.py — impact, full-run-safety
  • tests/test_allow_entry_in_request_const.py — impact, full-run-safety
  • tests/test_analyzer.py — impact, full-run-safety
  • tests/test_analyzer_matrix_inference.py — impact, full-run-safety
  • tests/test_analyzer_ta_return_types.py — impact, full-run-safety
  • tests/test_anchor_tf_change.py — impact, full-run-safety
  • tests/test_anchor_time.py — impact, full-run-safety
  • tests/test_anchor_vwap.py — impact, full-run-safety
  • tests/test_array_bounds_insert_fill_slice.py — impact, full-run-safety
  • tests/test_array_checked_access.py — impact, full-run-safety
  • tests/test_array_empty_aggregates.py — impact, full-run-safety
  • tests/test_array_history.py — impact, full-run-safety
  • tests/test_array_missing_integer_index.py — impact, full-run-safety
  • tests/test_array_new_color_fill_and_na_size.py — impact, full-run-safety
  • tests/test_array_new_defaults_and_order.py — impact, full-run-safety
  • tests/test_array_reference_semantics.py — impact, full-run-safety
  • tests/test_array_retained_alias.py — impact, full-run-safety
  • tests/test_array_slice_view.py — impact, full-run-safety
  • tests/test_array_sort_indices.py — impact, full-run-safety
  • tests/test_array_terminal_returns.py — impact, full-run-safety
  • tests/test_arrseg_coded_run_stops.py — impact, full-run-safety
  • tests/test_arrseg_review_fixes.py — impact, full-run-safety
  • tests/test_binop_operand_order.py — impact, full-run-safety
  • tests/test_block_shadow_primitive_types.py — impact, full-run-safety
  • tests/test_block_var_plain_siblings.py — impact, full-run-safety
  • tests/test_bool_na_refusal.py — impact, full-run-safety
  • tests/test_bool_na_refusals_2.py — impact, full-run-safety
  • tests/test_bool_na_values.py — impact, full-run-safety
  • tests/test_bool_string_inference.py — impact, full-run-safety
  • tests/test_builtin_name_shadow.py — impact, full-run-safety
  • tests/test_builtin_named_locals_compile.py — impact, full-run-safety
  • tests/test_builtin_named_locals_price_refusal.py — impact, full-run-safety
  • tests/test_builtin_named_locals_receipt.py — impact, full-run-safety
  • tests/test_builtin_named_locals_scope.py — impact, full-run-safety
  • tests/test_builtin_named_parameters.py — impact, full-run-safety
  • tests/test_builtin_named_parameters_reference.py — full-run-safety
  • tests/test_calc_on_order_fills_codegen.py — impact, full-run-safety
  • tests/test_calendar_lambda_locals.py — impact, full-run-safety
  • tests/test_call_argument_order.py — impact, full-run-safety
  • tests/test_callable_local_histories.py — impact, full-run-safety
  • tests/test_callable_series_param_types.py — impact, full-run-safety
  • tests/test_callable_string_series_type.py — impact, full-run-safety
  • tests/test_callable_ta_forwarded_length.py — impact, full-run-safety
  • tests/test_callable_var_first_reach.py — impact, full-run-safety
  • tests/test_checked_settings.py — impact, full-run-safety
  • tests/test_ci_engine_main_touched.py — full-run-safety
  • tests/test_closed_trade_fold.py — full-run-safety
  • tests/test_closed_trade_fold_emission.py — impact, full-run-safety
  • tests/test_closed_trade_text_equals.py — impact, full-run-safety
  • … and 478 more

non-code file(s) changed (CHANGELOG.md, docs/reference/support-contracts.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (CHANGELOG.md, docs/reference/support-contracts.md, pineforge_codegen/codegen/drawing.py, pineforge_codegen/codegen/emit_top.py, pineforge_codegen/codegen/visit_stmt.py) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

No difference found (not proven): No behavior difference found in \_type\_spec\_from\_expr (not a proof).

The verifier ran both versions of \_type\_spec\_from\_expr on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_\_init\_\_.

The verifier did not have enough to check \_\_init\_\_, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_generate\_cpp.

The verifier did not have enough to check \_generate\_cpp, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_is\_default\_drawing\_initializer.

The verifier did not have enough to check \_is\_default\_drawing\_initializer, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_is\_omitted\_udt\_field.

The verifier did not have enough to check \_is\_omitted\_udt\_field, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_is\_skip\_expr.

The verifier did not have enough to check \_is\_skip\_expr, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_detect\_drawing\_usage.

The verifier did not have enough to check \_detect\_drawing\_usage, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

No difference found (not proven): No behavior difference found in \_drawing\_call\_is\_void (not a proof).

The verifier ran both versions of \_drawing\_call\_is\_void on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_drawing\_call\_return\_cpp.

The verifier did not have enough to check \_drawing\_call\_return\_cpp, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_emit\_drawing\_method.

The verifier did not have enough to check \_emit\_drawing\_method, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_spec\_mentions\_drawing.

The verifier did not have enough to check \_spec\_mentions\_drawing, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_emit\_func\_def\_body.

The verifier did not have enough to check \_emit\_func\_def\_body, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_emit\_includes.

The verifier did not have enough to check \_emit\_includes, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

No difference found (not proven): No behavior difference found in \_infer\_type (not a proof).

The verifier ran both versions of \_infer\_type on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_slot\_scalar\_cpp\_type.

The verifier did not have enough to check \_slot\_scalar\_cpp\_type, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

No difference found (not proven): No behavior difference found in \_type\_spec\_from\_expr\_raw (not a proof).

The verifier ran both versions of \_type\_spec\_from\_expr\_raw on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_visit\_func\_call.

The verifier did not have enough to check \_visit\_func\_call, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_drawing\_target\_cpp\_type.

The verifier did not have enough to check \_drawing\_target\_cpp\_type, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

Could not verify: Could not verify \_visit\_assignment.

The verifier did not have enough to check \_visit\_assignment, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)

@luisleo526

Copy link
Copy Markdown
Contributor Author

Superseded by #405, which merged this change to main in 0ba632a.

@luisleo526 luisleo526 closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant