Repository navigation
Preserve reference types across fields and function calls - #403
luisleo526 wants to merge 4 commits into
Conversation
Lower missing reference values to their destination storage and retain drawing handle types through user functions and tuple returns. Support declared polyline types, fields and deletion arguments.
Read optional function metadata safely when inferring reference casts, so bare emitters retain matrix element types without raising an AttributeError.
Merge the current main while retaining the feature correction.
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.
Not checked: tests were not run; formal verification proved 0 of 19 changed function(s) (4 sampled, not proven, 15 not verified).
Formal verification. PR-changed functions: 4/19 verified (0 proven, 4 may-equivalent, 0 distinguished) · 15 not verified (15 unsupported).
Not verified on this run: \_\_init\_\_ (unsupported), \_generate\_cpp (unsupported), \_is\_default\_drawing\_initializer (unsupported), \_is\_omitted\_udt\_field (unsupported), \_is\_skip\_expr (unsupported), \_detect\_drawing\_usage (unsupported), \_drawing\_call\_return\_cpp (unsupported), \_emit\_drawing\_method (unsupported), \_spec\_mentions\_drawing (unsupported), \_emit\_func\_def\_body (unsupported), and 5 more.
Graphify review — findings
Adds typed opaque handles for polyline locals, array elements and UDT fields, so default construction and bare na produce a missing handle and copies keep its identity. polyline.new and functional or method-form delete stay visual-only but now evaluate their arguments through declarations and calls, with the opaque ID counter joining script reset and rollback; the counter allocation and the keyword-argument ordering in _emit_polyline_call are where that evaluation contract could go wrong. Fixes reference-typed UDT fields to keep their type when initialized or reassigned with na, including array defaults, and routes named casts like label(na) through a shared na_reference_cast_kind helper in both type inference and codegen. Drawing handles now cross user-function and tuple boundaries by value, and the new cast helper is the piece that has to be wired into compilation for those paths to hold. Narrows the omitted-field and skip-receiver logic to table only, since polyline now has a C++ representation, and gates the _pf_polyline_next_id_ member on a new _uses_polyline flag so non-polyline strategies emit unchanged C++.
Worth a look
- Polyline ID allocation is not thread-safe —
pineforge_codegen/codegen/drawing.py:376· Escalate · medium- unverified hypothesis — survived the adversarial refute pass; needs human review
- Polyline constructor reorders keyword-argument side effects —
pineforge_codegen/codegen/drawing.py:372· Escalate · medium- unverified hypothesis — survived the adversarial refute pass; needs human review
- Named NA reference casts are not wired into compilation —
pineforge_codegen/reference_casts.py:11· Escalate · medium- unverified hypothesis — survived the adversarial refute pass; needs human review
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 5256 functions depend on the 1331 functions this change touches.
Health — grade A; 10 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):
_generate()— 131 callers, 9 callees (high)CodeGen— 68 callers, 14 callees (high)_generate_raw()— 29 callers, 9 callees (high)_generate()— 9 callers, 25 callees (high)_check()— 31 callers, 3 callees (high)_codegen()— 10 callers, 7 callees (high)_generate()— 8 callers, 8 callees (high)_generator()— 9 callers, 7 callees (high)- …and 2 more
Verification — 5256 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 4325 function(s) in the blast radius were not formally verified this run
Cross-repo impact
Cross-repo impact
This change impacts 2 symbol(s) across 1 peer repo(s) — blast radius over derived cross-repo edges:
- pineforge-4pass/pineforge-hpo —
python/pineforge_hpo/transpile.py(via import_moniker, heuristic),tests/python/test_transpile.py(via import_moniker, heuristic)
1 peer repo(s) in this group are not yet indexed — cross-repo impact may be incomplete.
Cross-repo linkage is derived (SCIP-moniker matches are sound; name/contract matches are heuristic) and ADVISORY — it never affects this check's conclusion.
Test selection
Test selection
528 of 528 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_action_pins.py— full-run-safetytests/test_allow_entry_in_qualifier.py— impact, full-run-safetytests/test_allow_entry_in_request_const.py— impact, full-run-safetytests/test_analyzer.py— impact, full-run-safetytests/test_analyzer_matrix_inference.py— impact, full-run-safetytests/test_analyzer_ta_return_types.py— impact, full-run-safetytests/test_anchor_tf_change.py— impact, full-run-safetytests/test_anchor_time.py— impact, full-run-safetytests/test_anchor_vwap.py— impact, full-run-safetytests/test_array_bounds_insert_fill_slice.py— impact, full-run-safetytests/test_array_checked_access.py— impact, full-run-safetytests/test_array_empty_aggregates.py— impact, full-run-safetytests/test_array_history.py— impact, full-run-safetytests/test_array_missing_integer_index.py— impact, full-run-safetytests/test_array_new_color_fill_and_na_size.py— impact, full-run-safetytests/test_array_new_defaults_and_order.py— impact, full-run-safetytests/test_array_reference_semantics.py— impact, full-run-safetytests/test_array_retained_alias.py— impact, full-run-safetytests/test_array_slice_view.py— impact, full-run-safetytests/test_array_sort_indices.py— impact, full-run-safetytests/test_array_terminal_returns.py— impact, full-run-safetytests/test_arrseg_coded_run_stops.py— impact, full-run-safetytests/test_arrseg_review_fixes.py— impact, full-run-safetytests/test_binop_operand_order.py— impact, full-run-safetytests/test_block_shadow_primitive_types.py— impact, full-run-safetytests/test_block_var_plain_siblings.py— impact, full-run-safetytests/test_bool_na_refusal.py— impact, full-run-safetytests/test_bool_na_refusals_2.py— impact, full-run-safetytests/test_bool_na_values.py— impact, full-run-safetytests/test_bool_string_inference.py— impact, full-run-safetytests/test_builtin_name_shadow.py— impact, full-run-safetytests/test_builtin_named_locals_compile.py— impact, full-run-safetytests/test_builtin_named_locals_price_refusal.py— impact, full-run-safetytests/test_builtin_named_locals_receipt.py— impact, full-run-safetytests/test_builtin_named_locals_scope.py— impact, full-run-safetytests/test_builtin_named_parameters.py— impact, full-run-safetytests/test_builtin_named_parameters_reference.py— full-run-safetytests/test_calc_on_order_fills_codegen.py— impact, full-run-safetytests/test_calendar_lambda_locals.py— impact, full-run-safetytests/test_call_argument_order.py— impact, full-run-safetytests/test_callable_local_histories.py— impact, full-run-safetytests/test_callable_series_param_types.py— impact, full-run-safetytests/test_callable_string_series_type.py— impact, full-run-safetytests/test_callable_ta_forwarded_length.py— impact, full-run-safetytests/test_callable_var_first_reach.py— impact, full-run-safetytests/test_checked_settings.py— impact, full-run-safetytests/test_ci_engine_main_touched.py— full-run-safetytests/test_closed_trade_fold.py— full-run-safetytests/test_closed_trade_fold_emission.py— impact, full-run-safetytests/test_closed_trade_text_equals.py— impact, full-run-safety- … and 478 more
non-code file(s) changed (
CHANGELOG.md,docs/reference/support-contracts.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
CHANGELOG.md,docs/reference/support-contracts.md,pineforge_codegen/codegen/drawing.py,pineforge_codegen/codegen/emit_top.py,pineforge_codegen/codegen/visit_stmt.py) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
No difference found (not proven): No behavior difference found in \_type\_spec\_from\_expr (not a proof).
The verifier ran both versions of \_type\_spec\_from\_expr on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_\_init\_\_.
The verifier did not have enough to check \_\_init\_\_, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_generate\_cpp.
The verifier did not have enough to check \_generate\_cpp, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_is\_default\_drawing\_initializer.
The verifier did not have enough to check \_is\_default\_drawing\_initializer, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_is\_omitted\_udt\_field.
The verifier did not have enough to check \_is\_omitted\_udt\_field, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_is\_skip\_expr.
The verifier did not have enough to check \_is\_skip\_expr, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_detect\_drawing\_usage.
The verifier did not have enough to check \_detect\_drawing\_usage, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
No difference found (not proven): No behavior difference found in \_drawing\_call\_is\_void (not a proof).
The verifier ran both versions of \_drawing\_call\_is\_void on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_drawing\_call\_return\_cpp.
The verifier did not have enough to check \_drawing\_call\_return\_cpp, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_emit\_drawing\_method.
The verifier did not have enough to check \_emit\_drawing\_method, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_spec\_mentions\_drawing.
The verifier did not have enough to check \_spec\_mentions\_drawing, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_emit\_func\_def\_body.
The verifier did not have enough to check \_emit\_func\_def\_body, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_emit\_includes.
The verifier did not have enough to check \_emit\_includes, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
No difference found (not proven): No behavior difference found in \_infer\_type (not a proof).
The verifier ran both versions of \_infer\_type on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_slot\_scalar\_cpp\_type.
The verifier did not have enough to check \_slot\_scalar\_cpp\_type, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
No difference found (not proven): No behavior difference found in \_type\_spec\_from\_expr\_raw (not a proof).
The verifier ran both versions of \_type\_spec\_from\_expr\_raw on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: the synth_receiver tier. A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_visit\_func\_call.
The verifier did not have enough to check \_visit\_func\_call, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_drawing\_target\_cpp\_type.
The verifier did not have enough to check \_drawing\_target\_cpp\_type, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify \_visit\_assignment.
The verifier did not have enough to check \_visit\_assignment, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Valid Pine v6 scripts could generate C++ that did not compile when assigning
nato a drawing or array field, passing a drawing handle through a user function or tuple, or keeping a polyline handle in a local or UDT field.Preserve the destination storage type for missing reference values, pass drawing handles as values across function and tuple boundaries, and retain polyline handle types for declarations and deletion calls.
Reproduction tests:
tests/test_reference_handle_lowering.py::test_reference_field_na_uses_the_field_type[line-Line]tests/test_reference_handle_lowering.py::test_array_field_na_default_has_collection_storagetests/test_reference_handle_lowering.py::test_handle_parameter_alias_and_tuple_are_values_at_each_calltests/test_reference_handle_lowering.py::test_polyline_array_element_keeps_its_handle_typetests/test_reference_handle_lowering.py::test_polyline_field_and_cast_preserve_typed_reads_and_deleteFixes #390