diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c11b388..6fcc1e7d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,8 +50,33 @@ supported as exact pairs; on the 0.x line they are independent. See the that overlap collection-history or requested-context storage. Persistent declarations in sibling blocks also retain distinct storage when a script uses their generated spelling. Declarations, calls and history storage use - the same allocated identities. Regenerate C++ and relink affected strategies. - + the same allocated identities, including when selecting reachable variants + for emission. Regenerate C++ and relink affected strategies. +- Persistent declarations accept a type qualifier after `var` or `varip`. + Lower-timeframe requests return independently typed arrays for inline and + helper-returned tuple payloads. +- Complete nested indentation levels can combine tabs and spaces. Requested + `na`/`nz` helper parameters retain their bindings beside time operations; + global tuple declarations and their member histories share one requested + sample. Uncalled forwarding helpers no longer allocate unresolved TA buffers + for their unreachable call variants (#372). Bare tuple-element history + payloads retain their located refusal. Tuple histories rendered through the + expression visitor, including reads through globals, retain their chart + series and warning. Resolvable variants of a reachable helper retain their + TA state even when registered through an uncalled caller. Requested builtin + tuples reuse their existing sampled results, and mixed-clock tuple history + uses the registered history diagnostic. + +- Built-in size, position, line/label style and shape constants retain their + string values in variables, selections, function arguments and returns, and + string input defaults/options. This fixes ambiguous C++ assignments, NUL + strings, unsupported size inputs and null-pointer string construction (#389). + Sort-order constants retain their distinct `sort_order` identity through + defaults and comparisons; invalid string conversions are diagnosed at their + Pine location, including individual elements of string input options. + `array.sort_indices` honors a supplied sort direction. + Scalar function overloads no longer crash during type checking. + Regenerate C++ and relink affected strategies. - A zone name the engine rejects now makes calendar-form `timestamp()` stop the run at its first evaluation, instead of silently calculating a UTC value. The paired engine reports a typed timezone error identifying the failed bar. diff --git a/docs/codegen-coverage-gaps.md b/docs/codegen-coverage-gaps.md index 99a30e73..cb1d6eae 100644 --- a/docs/codegen-coverage-gaps.md +++ b/docs/codegen-coverage-gaps.md @@ -70,7 +70,7 @@ tables (`HARD_REJECT_FUNC`, `HARD_REJECT_NAMESPACE`, `NOT_YET_FUNC`, | `visit_expr.py:443` | `session.` → `"false"` | Known members being enumerated does not justify a value for an unknown member. The catch-all is a placeholder defect. | | `visit_expr.py:448` | `syminfo.` → `"0"` | The historical whitelist blocked unknown members upstream; the emitter placeholder is still a defect. | | `visit_expr.py:459` | `color.` → `"0"` | The historical whitelist covered known constants; an unknown-member placeholder is still a defect. | -| `visit_expr.py:461` | `SKIP_NAMESPACES` (table/label/line/box/polyline/chart/linefill/display/size/position) → `"0"` | The audit treated these as visual omissions. Substituting zero for a missing value lowering is a defect; each supported member needs its semantics. | +| `visit_expr.py:461` (historical audit) | Legacy visual namespace fallbacks → `"0"` | Inventoried size, position, line/label style and shape constants now use typed string values from `builtin_constants.py` before these fallbacks. Remaining missing value lowerings are defects; rendering omissions do not justify substituting zero for a value. | | `visit_expr.py:484-507` | `strategy.oca.`, `strategy.direction.`, `strategy.commission.`, `strategy.closedtrades.`, `strategy.opentrades.` catch-alls | Known enum values do not define unknown members. The catch-all defaults are placeholder defects. | | `visit_call.py:357` | `map.(...)` → `"0"` | The historical whitelist rejected unknown functions upstream; the emitter placeholder is still a defect. | | `visit_call.py:404` | `array.(...)` → `"0"` | The historical whitelist rejected unknown functions upstream; the emitter placeholder is still a defect. | diff --git a/docs/reference/file-map.md b/docs/reference/file-map.md index 364fec29..922a25ae 100644 --- a/docs/reference/file-map.md +++ b/docs/reference/file-map.md @@ -52,6 +52,9 @@ pineforge_codegen/ │ (after generation; the C++ unchanged). ├── session_reads.py The session.[k] reads the generated │ C++ holds (analyzer and codegen share it). +├── builtin_constants.py Typed scalar constant values shared by +│ analysis, emission and input proofs; +│ nominal builtin enums retain TypeSpec identity. ├── signatures.py Pine v6 builtin signature registry │ (TA / math / str / strategy / input / │ map / built-ins) — single source of diff --git a/docs/reference/support-contracts.md b/docs/reference/support-contracts.md index cf55d2f5..57353c27 100644 --- a/docs/reference/support-contracts.md +++ b/docs/reference/support-contracts.md @@ -1,5 +1,29 @@ ## Support contracts +Persistent typed declarations parse their storage prefix separately from their +qualifier: `var const float`, `varip const int`, and the corresponding `simple` +and `series` forms retain both annotations and their existing initialization. + +`request.security_lower_tf` tuple payloads, written inline or returned by a +helper, produce one array per element. Each intrabar evaluates the tuple once +before distributing values to the independently typed arrays; clearing the +request clears every array together. Tuple-bound arrays retain collection +metadata in global and callable scopes. + +A global tuple declaration read by `request.security` is sampled once in the +requested evaluator, in declaration order. Scalar and history reads share that +sample, including stateful helper results. Member history commits on completed +requested bars even when a lazy payload does not read the member on that bar. +Helper parameter bindings remain active through `na`/`nz` builtin emission in +both evaluator modes. These rules supersede the older tuple refusal limitations +described in the historical helper row below. + +Complete mixed tab/space nested block prefixes use four columns per tab plus +the spaces. Previously accepted incomplete mixed prefixes, skipped first-body +widths and pending logical-line continuations retain their earlier handling +pending additional TradingView observations; they are not newly refused as part +of the nested-block correction. + Builtin `array.slice` on a resolved typed array in the script, a function or method parameter, or a callable-local array opens the view gate and lowers the script's array storage to typed handles: @@ -277,6 +301,17 @@ This exception covers only constructs PineForge refuses today. A form that compi The emission invariant (`AGENTS.md#emission-invariant`) states this rule for every emission path. The notes in this file describe existing behavior: a documented approximation or unsupported fallback is an implementation gap, not an exception to the invariant. Input defaults use the input family's Pine type and qualifier contract. +Inventoried `size.*`, `position.*`, `line.style_*`, `label.style_*` and +`shape.*` members have string values independently of a visual call's +rendering support. `builtin_constants.py` supplies their value and semantic +type to analysis, emission and string-input proofs. Line styles also supply +box border styles. Assignments, selections and callable variants preserve +those types. `order.ascending` and `order.descending` instead retain nominal +`sort_order` TypeSpecs; their runtime storage does not permit a Pine string +conversion. Typed slots and operator contracts check that identity. +`tests/test_builtin_style_values.py` covers the value paths, named bindings +and nominal restrictions, including order arguments of collection sorting. + An explicit `const color` carrier with a proven initializer supplies its packed value to the getter and settings descriptor before `on_bar`; the manifest reads that descriptor. `codegen/constant_values.py` resolves lexical diff --git a/pineforge_codegen/analyzer/base.py b/pineforge_codegen/analyzer/base.py index 5f99998b..ae29864b 100644 --- a/pineforge_codegen/analyzer/base.py +++ b/pineforge_codegen/analyzer/base.py @@ -32,6 +32,7 @@ ) from ..errors import SourceLocation, Diagnostic, CompileError, Level, Phase from ..limits import TimeBudget, iter_ast_nodes, syntax_children +from ..builtin_constants import BUILTIN_ENUM_STORAGE, contains_builtin_enum, is_builtin_enum from ..input_keys import assign_input_keys from ..loop_types import FLOAT_REASSIGNMENT, LoopTypeSources, for_counter_type from ..security_contexts import ( @@ -3656,6 +3657,8 @@ def path_counts(stateful: set[str]) -> dict[str, int]: @staticmethod def _primitive_pine_type_from_spec(spec) -> PineType: + if is_builtin_enum(spec): + return BUILTIN_ENUM_STORAGE[spec.name] if spec is None or getattr(spec, "kind", None) != "primitive": return PineType.UNKNOWN return { @@ -4485,7 +4488,10 @@ def _visit(self, node: ASTNode | None) -> PineType: # Convert CamelCase to snake_case for method lookup visitor = getattr(self, method_name, None) if visitor is not None: - return visitor(node) + result = visitor(node) + if isinstance(node, FuncCall): + self._check_builtin_enum_arguments(node) + return result # Fallback: try generic visit return PineType.VOID @@ -5057,6 +5063,11 @@ def _record_parameter_default_scalar_types( for value in (definition.annotations or {}).get("param_defaults", ()) if value is not None } + for name, spec, value in zip( + definition.params, self._param_type_specs_from_def(definition), + (definition.annotations or {}).get("param_defaults", ()), + ): + self._check_nominal_slot(value, spec, f"default of parameter '{name}'") def _check_typed_parameter_mismatches( self, @@ -5079,6 +5090,8 @@ def _check_typed_parameter_mismatches( # only limits the existing refusal. default_types = self._parameter_default_scalar_types.get(id(definition), {}) for name, spec, value in zip(definition.params, specs, values): + if value is not None and spec is not None: + self._check_nominal_slot(value, spec, f"parameter '{name}' of '{function_name}'") if ( value is not None and spec is not None @@ -5093,6 +5106,91 @@ def _check_typed_parameter_mismatches( value.loc, ) + def _check_nominal_slot(self, value, expected, context, parameter_types=None): + """A builtin enum fits only its own nominal domain, never its storage.""" + if (value is None or expected is None or self._pine_version != 6 + or self._in_v5_library_definition(value)): + return + if (expected.kind == "array" and expected.element is not None + and isinstance(value, TupleLiteral)): + for item in value.elements: + self._check_nominal_slot(item, expected.element, context, parameter_types) + return + actual = self._operator_operand_type(value, parameter_types) + if not (contains_builtin_enum(actual) or contains_builtin_enum(expected)): + return + if (actual is None or actual == TypeSpec.primitive("na") + or join_value_types(actual, expected, invariant=True) is not None): + return + self._error( + f"Cannot use {actual} as {expected} for {context}.", + value.loc, + ) + + def _check_builtin_enum_arguments(self, node, parameter_types=None): + """Bind nominal argument contracts from the intrinsic signatures.""" + if self._pine_version != 6 or self._in_v5_library_definition(node): + return + callee = node.callee + namespace = None + arguments = list(node.args) + if isinstance(callee, Identifier): + name = callee.name + if name in self._func_defs: + return + elif isinstance(callee, MemberAccess) and isinstance(callee.object, Identifier): + name = callee.member + receiver = callee.object + fields = self._udt_field_type_specs.get(receiver.name) + if name == "new" and fields is not None: + for index, (field_name, field_spec) in enumerate(fields.items()): + value = (arguments[index] if index < len(arguments) + else node.kwargs.get(field_name)) + self._check_nominal_slot( + value, field_spec, f"field '{field_name}' of '{receiver.name}'", + parameter_types, + ) + return + receiver_spec = ( + self._operator_operand_type(receiver, parameter_types) + if parameter_types is not None else self._type_spec_from_expr(receiver) + ) + binding = self._value_bindings.declaration_bindings.get(id(receiver)) + if binding is None: + namespace = receiver.name + elif receiver_spec is not None and receiver_spec.kind in {"array", "matrix", "map"}: + owner_name = method_receiver_type_name(receiver_spec) + if f"{owner_name}.{name}" in self._method_signatures: + return + namespace = receiver_spec.kind + arguments.insert(0, receiver) + else: + return + else: + return + if namespace is None and name in {"int", "float", "bool", "string"}: + for argument in arguments: + self._check_nominal_slot( + argument, TypeSpec.primitive(name), f"cast '{name}'", parameter_types, + ) + intrinsic = sigs.get_intrinsic(namespace, name) + if intrinsic is None: + return + # Signature types for polymorphic collection values describe storage. + # Their element arguments retain the existing TypeSpec-aware checks. + parameters = max(intrinsic.signatures, key=lambda signature: len(signature.params)).params + for index, parameter in enumerate(parameters): + value = (arguments[index] if index < len(arguments) + else node.kwargs.get(parameter.name)) + if value is None: + continue + if namespace in {"array", "matrix", "map"} and parameter.type_spec is None: + continue + expected = parameter.type_spec or TypeSpec.primitive(parameter.pine_type.value) + self._check_nominal_slot(value, expected, f"argument '{parameter.name}' of " + f"'{namespace + '.' if namespace else ''}{name}'", + parameter_types) + def _record_script_block_binding(self, sym: Symbol, outer: Symbol | None) -> None: if not self._block_node_stack or self._collection_scope_stack: return @@ -5166,6 +5264,11 @@ def _visit_VarDecl(self, node: VarDecl) -> PineType: ) # Infer type from the value expression val_type = self._visit(node.value) + if node.type_hint: + self._check_nominal_slot( + node.value, self._type_spec_from_hint(node.type_hint), + f"declaration '{node.name}'", + ) if (node.type_hint == "int" and isinstance(node.value, NumberLiteral) and isinstance(node.value.value, float) @@ -5455,6 +5558,9 @@ def _visit_VarDecl(self, node: VarDecl) -> PineType: def _visit_Assignment(self, node: Assignment) -> PineType: # Visit the value first val_type = self._visit(node.value) + self._check_nominal_slot( + node.value, self._type_spec_from_expr(node.target), "assignment", + ) if node.op in ("+=", "-=", "*=", "/=", "%="): self._check_operator_operands( node, node.op[:-1], (node.target, node.value), @@ -5627,6 +5733,9 @@ def _selection_tuple_shape( def _visit_TupleAssign(self, node: TupleAssign) -> PineType: val_type = self._visit(node.value) + lower_types = (node.value.annotations or {}).get( + "lower_tf_tuple_element_types", (), + ) loc = node.loc or SourceLocation(file=self._filename, line=1, col=1, end_col=1) is_program_global = ( self._global_scope and self._symbols.current_scope.name == "global" @@ -5741,7 +5850,14 @@ def _visit_TupleAssign(self, node: TupleAssign) -> PineType: self._record_script_block_binding(sym, outer_symbol) self._note_member_writer(sym, name, node) - if (self._collection_scope_stack + if idx < len(lower_types): + inner = lower_types[idx] + spec = TypeSpec.array( + TypeSpec.primitive("int64") if inner == PineType.INT + else self._pine_type_to_spec(inner) + ) + self._record_collection_type(name, spec) + elif (self._collection_scope_stack and self._block_node_stack and self._symbols.current_scope.name != "global"): owner_id = id(self._block_node_stack[-1]) @@ -5854,10 +5970,13 @@ def _visit_FuncDef(self, node: FuncDef) -> PineType: # (e.g. ``getLineStyle(styleStr)`` where ``styleStr`` is a ``string`` param). loc = node.loc or SourceLocation(file=self._filename, line=1, col=1, end_col=1) param_hints = (node.annotations or {}).get("param_type_hints", []) + param_specs = self._param_type_specs_from_def(node) for i, param in enumerate(node.params): hint = param_hints[i] if i < len(param_hints) else None - pspec = self._type_spec_from_hint(hint) if hint else None + pspec = param_specs[i] ptype = self._type_hint_to_pine(hint) if hint else PineType.UNKNOWN + if is_builtin_enum(pspec): + ptype = BUILTIN_ENUM_STORAGE[pspec.name] sym = Symbol( name=param, pine_type=ptype, @@ -6179,6 +6298,12 @@ def _visit_FuncDef(self, node: FuncDef) -> PineType: terminal_direct_return_spec ) + if is_builtin_enum(terminal_direct_return_spec): + # A nominal result is independent of the primitive adapter used + # to store it, including a returned default parameter or wrapper. + body_type = BUILTIN_ENUM_STORAGE[terminal_direct_return_spec.name] + self._func_return_type_specs[node.name] = terminal_direct_return_spec + # Store return type self._func_return_types[node.name] = body_type @@ -6274,6 +6399,7 @@ def _visit_MethodDef(self, node) -> PineType: ) self._record_parameter_default_scalar_types(node) method_key = f"{node.type_name}.{node.name}" + declared_param_specs = self._param_type_specs_from_def(node) self._symbols.enter_scope(f"method_{node.type_name}_{node.name}") loc = node.loc or SourceLocation(file=self._filename, line=1, col=1, end_col=1) param_hints = (node.annotations or {}).get("param_type_hints", []) @@ -6286,7 +6412,9 @@ def _visit_MethodDef(self, node) -> PineType: # a regular UDF parameter; seeding it as FLOAT makes even a single # bool/int history call silently coerce through Series. ptype = self._type_hint_to_pine(hint) if hint else PineType.UNKNOWN - pspec = self._type_spec_from_hint(hint) if hint else None + pspec = declared_param_specs[i] + if is_builtin_enum(pspec): + ptype = BUILTIN_ENUM_STORAGE[pspec.name] udt_self = ( node.type_name if i == 0 and pspec is not None and pspec.kind == "udt" @@ -6340,6 +6468,9 @@ def _visit_MethodDef(self, node) -> PineType: ret_type = self._statement_value_type(node.body[-1], ret_type) if terminal_ret_expr is not None: terminal_spec = self._type_spec_from_expr(terminal_ret_expr) + if is_builtin_enum(terminal_spec): + return_type_spec = terminal_spec + ret_type = BUILTIN_ENUM_STORAGE[terminal_spec.name] nullable_selection = self._selection_contains_na(terminal_ret_expr) terminal_needs_structured_return = ( nullable_selection @@ -6509,12 +6640,15 @@ def _enter_top_level_branch_scope( def _certain_scalar_type(self, node: ASTNode) -> PineType: """Return a scalar type only when the syntax proves it. - Member access deliberately stays unknown: several namespace members - have legacy analyzer types that do not describe the value codegen - emits. Declaration hints, literals, builtin numeric series, + Uninventoried member access stays unknown: legacy namespace types + do not always describe emitted values. Inventoried typed constants, + declaration hints, literals, builtin numeric series, comparisons, logical results, arithmetic over proven numeric values, and the small set of scalar casts and builtin calls are reliable. """ + constant = self._builtin_constant_info(node) + if constant is not None: + return constant.storage_type if isinstance(node, NumberLiteral): return PineType.FLOAT if isinstance(node.value, float) else PineType.INT if isinstance(node, StringLiteral): @@ -7118,6 +7252,9 @@ def _operator_operand_type_uncached( query: OperatorTypeQuery, ) -> TypeSpec | None: """Resolve an operand using the current expression query's memo.""" + constant = self._builtin_constant_info(node) + if constant is not None: + return constant.type_spec if isinstance(node, Identifier) and parameter_types is not None: if node.name in parameter_types: ptype = parameter_types[node.name] @@ -9169,6 +9306,9 @@ def _visit_Identifier(self, node: Identifier) -> PineType: return PineType.FLOAT def _visit_MemberAccess(self, node: MemberAccess) -> PineType: + constant = self._builtin_constant_info(node) + if constant is not None: + return constant.storage_type from ..drawing_all import drawing_all_kind if drawing_all_kind(node) not in {None, "polyline"}: # Collections use TypeSpec for their shape; PineType stays scalar. diff --git a/pineforge_codegen/analyzer/call_handlers.py b/pineforge_codegen/analyzer/call_handlers.py index 7f25b6b2..4635541b 100644 --- a/pineforge_codegen/analyzer/call_handlers.py +++ b/pineforge_codegen/analyzer/call_handlers.py @@ -139,6 +139,9 @@ def infer(value): if expr is None: return PineType.UNKNOWN + constant = self._builtin_constant_info(expr) + if constant is not None: + return constant.storage_type if isinstance(expr, ExprStmt): return infer(expr.expr) if isinstance(expr, Identifier): @@ -1001,9 +1004,8 @@ def _handle_request_security_lower_tf(self, node: FuncCall) -> PineType: the codegen knows to emit a ``std::vector`` accumulator that gets cleared on sub-bar 0 and pushed-to per sub-bar. - UDT / color / string element types are deliberately rejected here - with a precise diagnostic; the runtime path only knows how to - accumulate ``double`` / ``int`` / ``bool``.""" + Scalar calls retain their existing numeric/bool arrays. Tuple calls + carry each primitive element's type to its own array accumulator.""" param_names = ["symbol", "timeframe", "expression", "ignore_invalid_symbol", "currency", "ignore_invalid_timeframe", "calc_bars_count"] @@ -1033,13 +1035,6 @@ def _handle_request_security_lower_tf(self, node: FuncCall) -> PineType: node.loc, ) - if isinstance(expr_node, TupleLiteral): - self._error( - "request.security_lower_tf does not support tuple expressions yet. " - "Issue separate request.security_lower_tf calls for each series.", - node.loc, - ) - for arg in node.args: if arg is not None and arg is not expr_node: self._visit(arg) @@ -1054,20 +1049,45 @@ def _handle_request_security_lower_tf(self, node: FuncCall) -> PineType: ta_end = len(self._ta_call_sites) security_ta_range = (ta_start, ta_end) if ta_end > ta_start else None + tuple_types = self._tuple_element_types_by_node.get(id(expr_node), ()) + if (isinstance(expr_node, FuncCall) + and isinstance(expr_node.callee, Identifier) + and self._func_returns_tuple.get(expr_node.callee.name, False)): + tuple_types = self._func_tuple_element_types.get(expr_node.callee.name, ()) + if PineType.UNKNOWN in tuple_types: + tuple_types = self._security_callsite_tuple_types( + expr_node.callee.name, expr_node, tuple_types, + ) + elif (isinstance(expr_node, FuncCall) + and isinstance(expr_node.callee, MemberAccess) + and isinstance(expr_node.callee.object, Identifier) + and expr_node.callee.object.name == "ta"): + count = TA_TUPLE_ELEMENT_COUNTS.get(expr_node.callee.member, 0) + if count: + tuple_types = tuple_types or (PineType.FLOAT,) * count + # Cache the resolved element type on the call node so the # ``_type_spec_from_expr`` pass can map ``request.security_lower_tf`` # to ``array`` without re-visiting the expression (which would # double-allocate TA call sites for expressions like ``ta.ema``). cached_anns = getattr(node, "annotations", None) or {} cached_anns["lower_tf_element_pine_type"] = expr_pine_type + if tuple_types: + cached_anns["lower_tf_tuple_element_types"] = tuple_types + self._security_tuple_shapes[id(node)] = len(tuple_types) node.annotations = cached_anns - if expr_pine_type not in (PineType.FLOAT, PineType.INT, PineType.BOOL, - PineType.NA, PineType.UNKNOWN): + element_types = tuple_types or (expr_pine_type,) + supported = {PineType.FLOAT, PineType.INT, PineType.BOOL, + PineType.NA, PineType.UNKNOWN} + if tuple_types: + supported |= {PineType.STRING, PineType.COLOR} + unsupported = next((item for item in element_types if item not in supported), None) + if unsupported is not None: element_label = { PineType.STRING: "string", PineType.COLOR: "color", - }.get(expr_pine_type, str(expr_pine_type)) + }.get(unsupported, str(unsupported)) self._error( "request.security_lower_tf element type '" + element_label + "' is not yet supported. Supported element types: float, int, bool. " @@ -1085,8 +1105,9 @@ def _handle_request_security_lower_tf(self, node: FuncCall) -> PineType: sec_id=sec_id, timeframe=context["timeframe"] if context is not None else tf_node, expression=expr_node, - returns_tuple=False, - tuple_size=0, + returns_tuple=bool(tuple_types), + tuple_size=len(tuple_types), + tuple_element_types=tuple(tuple_types), gaps=None, lookahead=None, ta_range=security_ta_range, diff --git a/pineforge_codegen/analyzer/operators.py b/pineforge_codegen/analyzer/operators.py index d5497b51..eb089201 100644 --- a/pineforge_codegen/analyzer/operators.py +++ b/pineforge_codegen/analyzer/operators.py @@ -1,6 +1,7 @@ """Pine operator overloads over semantic types, independent of C++ storage.""" from ..symbols import TypeSpec +from ..builtin_constants import is_builtin_enum # Only argument-independent scalar results belong here. The signature @@ -196,7 +197,8 @@ def resolve_operator( # Enum identities establish equality constraints; their other # operator contracts still have no proven domain here. operands = tuple( - None if actual is not None and actual.kind == "enum" else actual + None if actual is not None and actual.kind == "enum" + and not is_builtin_enum(actual) else actual for actual in operands ) if op == "?:": diff --git a/pineforge_codegen/analyzer/types.py b/pineforge_codegen/analyzer/types.py index 1c19d353..240a712e 100644 --- a/pineforge_codegen/analyzer/types.py +++ b/pineforge_codegen/analyzer/types.py @@ -47,6 +47,7 @@ ) from ..errors import Phase from ..limits import iter_ast_nodes +from ..builtin_constants import builtin_constant, is_builtin_enum from ..matrix_overloads import matrix_sum_has_rhs from ..reference_casts import na_reference_cast_kind from ..symbols import ( @@ -180,16 +181,32 @@ def _type_spec_from_hint(self, hint: str | None) -> TypeSpec | None: def _param_type_specs_from_def(self, func_def) -> list: """Per-parameter ``TypeSpec`` (or ``None``) from a function's DECLARED parameter type hints — the authoritative source for typed params - (``pivot hi``, ``string tf``, ``line[] arr``). Untyped params are - ``None`` here so regular-function call-site inference can fill them. + (``pivot hi``, ``string tf``, ``line[] arr``). An untyped builtin-enum + default keeps its nominal base type; other untyped params remain + ``None`` so regular-function call-site inference can fill them. """ hints = (getattr(func_def, "annotations", None) or {}).get("param_type_hints", []) specs: list = [] + defaults = (getattr(func_def, "annotations", None) or {}).get("param_defaults", []) for i in range(len(func_def.params)): hint = hints[i] if i < len(hints) else None - specs.append(self._type_spec_from_hint(hint) if hint else None) + spec = self._type_spec_from_hint(hint) if hint else None + if (spec is None and not hint and i < len(defaults) + and defaults[i] is not None): + inferred = self._operator_operand_type(defaults[i]) + if is_builtin_enum(inferred): + spec = inferred + specs.append(spec) return specs + def _builtin_constant_info(self, node): + if not (isinstance(node, MemberAccess) and isinstance(node.object, Identifier)): + return None + bindings = getattr(self, "_value_bindings", None) + if bindings is not None and bindings.declaration_bindings.get(id(node.object)) is not None: + return None + return builtin_constant(node.object.name, node.member) + def _template_args_from_call(self, node: FuncCall) -> list[str]: callee = node.callee ann = getattr(callee, "annotations", None) or {} @@ -317,7 +334,7 @@ def _selection_type_spec( *, nullable: bool | None = None, ) -> TypeSpec | None: - """Keep exact callable joins and the established script selection policy.""" + """Keep exact callable joins and script collection, string and enum joins.""" if nullable is None: nullable = any(self._selection_contains_na(node) for node, _ in branches) result: TypeSpec | None = None @@ -331,6 +348,8 @@ def _selection_type_spec( if spec is None: return None if (not nullable and spec.kind not in {"map", "matrix"} + and spec != TypeSpec.primitive("string") + and not is_builtin_enum(spec) and not isinstance( self._current_top_level_stmt, (FuncDef, MethodDef), )): @@ -384,6 +403,9 @@ def _type_spec_from_expr( ) -> TypeSpec | None: if value is None: return None + constant = self._builtin_constant_info(value) + if constant is not None: + return constant.type_spec from ..drawing_all import drawing_all_result_spec drawing_spec = drawing_all_result_spec(value) if drawing_spec is not None: @@ -1633,10 +1655,19 @@ def _check_node(self, node): if node.type_hint else self._analyzer._operator_operand_type(node.value, self._parameter_types) ) + if node.type_hint: + self._analyzer._check_nominal_slot( + node.value, spec, f"declaration '{node.name}'", self._parameter_types, + ) self._parameter_types[node.name] = spec return if isinstance(node, Assignment): self._check_node(node.value) + self._analyzer._check_nominal_slot( + node.value, + self._analyzer._operator_operand_type(node.target, self._parameter_types), + "assignment", self._parameter_types, + ) if node.op != ":=": self._analyzer._check_operator_operands( node, node.op[:-1], (node.target, node.value), self._parameter_types, @@ -1696,6 +1727,8 @@ def _check_node(self, node): for field, value in vars(node).items(): if field not in {"loc", "annotations"}: self._check_node(value) + if isinstance(node, FuncCall): + self._analyzer._check_builtin_enum_arguments(node, self._parameter_types) if isinstance(node, BinOp): op, operands = node.op, (node.left, node.right) elif isinstance(node, UnaryOp): diff --git a/pineforge_codegen/builtin_constants.py b/pineforge_codegen/builtin_constants.py new file mode 100644 index 00000000..06f23c30 --- /dev/null +++ b/pineforge_codegen/builtin_constants.py @@ -0,0 +1,85 @@ +"""Pine builtin constants: semantic types and exact scalar values. + +Visual arguments can be omitted by a renderer without changing the values of +the constants themselves. Nominal builtin enums keep their identity even when +the runtime adapter stores their payload in a primitive C++ type. +""" + +from dataclasses import dataclass + +from .symbols import PineType, TypeSpec + + +STRING_CONSTANT_VALUES = { + "size": {name: name for name in ("auto", "tiny", "small", "normal", "large", "huge")}, + "position": { + f"{vertical}_{horizontal}": f"{vertical}_{horizontal}" + for vertical in ("top", "middle", "bottom") + for horizontal in ("left", "center", "right") + }, + "line": { + "style_solid": "sol", "style_dotted": "dot", "style_dashed": "dsh", + "style_arrow_left": "al", "style_arrow_right": "ar", "style_arrow_both": "ab", + }, + "label": { + "style_none": "n", "style_xcross": "xcr", "style_cross": "cr", + "style_triangleup": "tup", "style_triangledown": "tdn", "style_flag": "flg", + "style_circle": "cir", "style_arrowup": "aup", "style_arrowdown": "adn", + "style_label_up": "lup", "style_label_down": "ldn", + "style_label_left": "llf", "style_label_right": "lrg", + "style_label_lower_left": "llwlf", "style_label_lower_right": "llwrg", + "style_label_upper_left": "luplf", "style_label_upper_right": "luprg", + "style_label_center": "lcn", "style_square": "sq", + "style_diamond": "dia", "style_text_outline": "to", + }, + "shape": { + "arrowdown": "shape_arrow_down", "arrowup": "shape_arrow_up", + "circle": "shape_circle", "cross": "shape_cross", "diamond": "shape_diamond", + "flag": "shape_flag", "labeldown": "shape_label_down", "labelup": "shape_label_up", + "square": "shape_square", "triangledown": "shape_triangle_down", + "triangleup": "shape_triangle_up", "xcross": "shape_xcross", + }, +} + +# Storage is an adapter detail, not an implicit Pine conversion to string. +BUILTIN_ENUM_STORAGE = {"sort_order": PineType.STRING} +ENUM_CONSTANT_VALUES = { + "order": ("sort_order", {"ascending": "ascending", "descending": "descending"}), +} + + +def is_builtin_enum(spec: TypeSpec | None) -> bool: + return (spec is not None and spec.kind == "enum" + and spec.name in BUILTIN_ENUM_STORAGE) + + +def contains_builtin_enum(spec: TypeSpec | None) -> bool: + if spec is None: + return False + return is_builtin_enum(spec) or any( + contains_builtin_enum(child) for child in (spec.element, spec.key, spec.value) + ) + + +@dataclass(frozen=True) +class BuiltinConstant: + type_spec: TypeSpec + value: str + + @property + def storage_type(self) -> PineType: + if is_builtin_enum(self.type_spec): + return BUILTIN_ENUM_STORAGE[self.type_spec.name] + return PineType.STRING + + +def builtin_constant(namespace: str, member: str) -> BuiltinConstant | None: + values = STRING_CONSTANT_VALUES.get(namespace, {}) + if member in values: + return BuiltinConstant(TypeSpec.primitive("string"), values[member]) + enum = ENUM_CONSTANT_VALUES.get(namespace) + if enum is not None: + type_name, values = enum + if member in values: + return BuiltinConstant(TypeSpec("enum", name=type_name), values[member]) + return None diff --git a/pineforge_codegen/codegen/base.py b/pineforge_codegen/codegen/base.py index 25480bfb..a77d6bbe 100644 --- a/pineforge_codegen/codegen/base.py +++ b/pineforge_codegen/codegen/base.py @@ -806,6 +806,7 @@ def func_var_storage(owner: str, raw_name: str) -> str: # authority for nested stateful-helper dispatch (see visit_call). self._current_instance_name: str | None = None self._instance_dispatch: dict[tuple[str | None, int], str] = {} + self._live_func_instances: set[str] | None = None self._fresh_instances: list[dict] = [] self._fresh_var_members: list[tuple[str, str, str]] = [] # Fresh fixnan members for context-sensitive helper instances (nested @@ -2075,6 +2076,7 @@ def natural_name(fname: str, cs_idx: int) -> str: interned: dict[tuple, dict] = {} worklist: list[dict] = [] seen_walk: set[str] = set() + instances: dict[str, dict] = {} fresh_counter = 0 # Seed with the natural clones the flat emission loop produces. @@ -2107,6 +2109,7 @@ def natural_name(fname: str, cs_idx: int) -> str: if inst["name"] in seen_walk: continue seen_walk.add(inst["name"]) + instances[inst["name"]] = inst body = func_bodies.get(inst["fname"]) if not body: continue @@ -2267,6 +2270,137 @@ def natural_name(fname: str, cs_idx: int) -> str: worklist.append(ginst) self._instance_dispatch[(inst["name"], id(callnode))] = ginst["name"] + # Definition-time calls reserve variants too. First find the paths + # rooted in executable statements, using the same natural/fresh + # dispatch graph as the call visitor. Resolvable natural variants of + # those callable owners are retained below. + for fi in ctx.func_infos: + instances.setdefault(self._func_cpp_base_name(fi.name), { + "fname": fi.name, "name": self._func_cpp_base_name(fi.name), + "call_site_idx": None, "ta_remap": {}, + }) + methods: dict[int, set[str]] = {} + for call_id, key in ctx.method_call_bindings: + methods.setdefault(call_id, set()).add(key) + + def targets(call, parent): + dispatch = self._instance_dispatch.get( + (parent["name"] if parent is not None else None, id(call)), + ) + if dispatch is not None: + return [dispatch] + bound = ctx.func_call_cs_map.get(id(call)) + if bound is not None: + names = [bound[0]] + elif id(call) in inherited_names: + names = [inherited_names[id(call)]] + elif isinstance(call.callee, Identifier) and call.callee.name in func_bodies: + names = [call.callee.name] + else: + names = sorted(methods.get(id(call), ())) + result = [] + for name in names: + if name in stateful and ctx.func_call_site_counts.get(name, 0) > 0: + index = ( + bound[1] if bound is not None + else (parent.get("call_site_idx") or 0) if parent is not None + else 0 + ) + result.append(natural_name(name, index)) + else: + result.append(self._func_cpp_base_name(name)) + return result + + roots = [ + stmt for stmt in ctx.ast.body + if not isinstance(stmt, (FuncDef, MethodDef)) + ] + pending = [ + target + for call in self._iter_func_calls(roots) + for target in targets(call, None) + ] + live: set[str] = set() + live_owners: set[str] = set() + used_ta: set[str] = set() + while pending: + if self._budget is not None: + self._budget.check(phase=Phase.CODEGEN) + name = pending.pop() + if name in live: + continue + live.add(name) + instance = instances.get(name) + if instance is None: + continue + owner = instance["fname"] + live_owners.add(owner) + used_ta.update(instance.get("ta_remap", {}).values()) + for call in self._iter_func_calls(func_bodies.get(owner, ())): + pending.extend(targets(call, instance)) + + # A reachable callable keeps its already-resolved natural variants, + # including state borrowed while analyzing an uncalled caller. Only + # definition-time paths whose state cannot be constructed need pruning. + # Check their whole dispatch closure so a retained body never names a + # dropped callee or TA member. Executable paths stay unconditional: the + # constructor guard must still report any invalid length they use. + sites_by_member = { + site.member_name: site for site in ctx.ta_call_sites + } + retainable: dict[str, bool] = {name: True for name in live} + retained_edges: dict[str, list[str]] = {} + + def can_retain(name: str) -> bool: + if name in retainable: + return retainable[name] + if self._budget is not None: + self._budget.check(phase=Phase.CODEGEN) + retainable[name] = False + instance = instances.get(name) + if instance is None or instance["fname"] not in live_owners: + return False + for member in instance.get("ta_remap", {}).values(): + site = sites_by_member.get(member) + if site is None or self._ta_dynamic_refusal(site) is not None: + return False + retained_edges[name] = [ + target + for call in self._iter_func_calls( + func_bodies.get(instance["fname"], ()) + ) + for target in targets(call, instance) + ] + retainable[name] = all( + can_retain(target) for target in retained_edges[name] + ) + return retainable[name] + + pending = [ + name for name, instance in instances.items() + if instance.get("call_site_idx") is not None + and not instance.get("fresh", False) + and name not in live and can_retain(name) + ] + while pending: + if self._budget is not None: + self._budget.check(phase=Phase.CODEGEN) + name = pending.pop() + if name in live: + continue + live.add(name) + used_ta.update(instances[name].get("ta_remap", {}).values()) + pending.extend(retained_edges.get(name, ())) + self._live_func_instances = live + self._dead_func_names.update(stateful - live_owners) + self._dead_ta_indices = { + index for index, site in enumerate(ctx.ta_call_sites) + if site.owner_func in stateful and site.member_name not in used_ta + } + self._fresh_instances[:] = [ + instance for instance in self._fresh_instances if instance["name"] in live + ] + def _callable_var_collection_spec( self, name: str, owner_func: str | None = None) -> TypeSpec | None: """Exact TypeSpec for a function-scoped persistent collection member. @@ -5657,17 +5791,27 @@ def _generate_cpp(self) -> str: # emits ``std::vector`` and the eval method pushes the # per-sub-bar value. Element type is inferred from the # expression — analyzer constrained it to int / float / bool. + tuple_types = item.get("tuple_element_types", ()) ctype = self._infer_cpp_type_for_security_elem(expr_node) - if ctype not in ("double", "int", "bool"): + if tuple_types: + accumulators = ", ".join( + f"std::vector<{part}>" + for part in self._lower_tf_tuple_cpp_types(tuple_types) + ) + result_type = f"std::tuple<{accumulators}>" + elif ctype not in ("double", "int", "bool"): # Defensive fallback — analyzer should already have # rejected unsupported types, but keep the codegen # well-defined if a future path slips through. ctype = "double" + result_type = f"std::vector<{ctype}>" + else: + result_type = f"std::vector<{ctype}>" self._security_ohlc_hist_fields_by_sec[sec_id] = ( self._collect_security_ohlc_hist_fields(state_expr) ) lines.append( - f" std::vector<{ctype}> _req_sec_lower_tf_{sec_id}{{}};" + f" {result_type} _req_sec_lower_tf_{sec_id}{{}};" ) for field in sorted( self._security_ohlc_hist_fields_by_sec.get(sec_id, ()) @@ -6344,6 +6488,10 @@ def allocate_legacy_flag(base: str) -> str: if (has_ta or has_series or needs_security_clone) and total_cs > 0: # Emit one variant per call site for cs_idx in range(total_cs): + if (self._live_func_instances is not None + and self._func_variant_name(fi.name, cs_idx) + not in self._live_func_instances): + continue self._emit_func_def(fi, lines, call_site_idx=cs_idx) lines.append("") else: diff --git a/pineforge_codegen/codegen/checked_settings.py b/pineforge_codegen/codegen/checked_settings.py index ce2b9bb9..f8096322 100644 --- a/pineforge_codegen/codegen/checked_settings.py +++ b/pineforge_codegen/codegen/checked_settings.py @@ -17,11 +17,11 @@ def _string_setting_arg(expr, lowered: str) -> str | None: - """Use the body's known string constants, never its inert visual codes. + """Use inventoried Pine strings, excluding nominal enum storage payloads. Name-echo strings are accepted only for inventoried currency/format members. - Unknown constants leave the checked input unsupported; legacy getters - and setters retain their existing execution semantics. + Visual constants share their typed value with the body. Unknown constants + leave the checked input unsupported. """ if isinstance(expr, MemberAccess): if isinstance(expr.object, Identifier): diff --git a/pineforge_codegen/codegen/constant_values.py b/pineforge_codegen/codegen/constant_values.py index 076a3d53..ac6f1c03 100644 --- a/pineforge_codegen/codegen/constant_values.py +++ b/pineforge_codegen/codegen/constant_values.py @@ -12,21 +12,27 @@ from ..security_contexts import ScriptIndex from ..signatures import INPUT_FUNCTIONS, MATH_CONSTANTS from ..symbols import PineType +from ..builtin_constants import STRING_CONSTANT_VALUES, builtin_constant from .color_constants import COLOR_PARAMETERS, color_arguments, literal_color_value from .constant_fold import fold_constant_expression from .strategy_declaration import _NAMED_CONSTANTS from .tables import ( ALERT_FREQ_VALUES, DRAWING_STYLE_NS, NAME_ECHO_STRING_MEMBERS, - ORDER_DIRECTION_MAP, ) _STRING_INPUT_CONSTANT_MEMBERS = { **NAME_ECHO_STRING_MEMBERS, "alert": ALERT_FREQ_VALUES, - "order": ORDER_DIRECTION_MAP, + **STRING_CONSTANT_VALUES, "session": frozenset({"regular", "extended"}), } +_SCALAR_CONSTANTS = { + **{f"math.{name}": value for name, value in MATH_CONSTANTS.items()}, + **{f"{namespace}.{member}": value + for namespace, members in STRING_CONSTANT_VALUES.items() + for member, value in members.items()}, +} @dataclass(frozen=True) @@ -171,8 +177,7 @@ def _scalar(self, expr, depth): if value is None or value.pine_type == PineType.COLOR: return None known[node.name] = value.value - value, proven = fold_constant_expression( - expr, known, {f"math.{name}": value for name, value in MATH_CONSTANTS.items()}) + value, proven = fold_constant_expression(expr, known, _SCALAR_CONSTANTS) kind = {bool: PineType.BOOL, int: PineType.INT, float: PineType.FLOAT, str: PineType.STRING}.get(type(value)) return ConstantValue(kind, value) if proven and kind is not None else None @@ -187,6 +192,9 @@ def expression(self, expr, depth=0): return ConstantValue(PineType.COLOR, literal_color_value(expr)) if (isinstance(expr, MemberAccess) and isinstance(expr.object, Identifier) and id(expr.object) not in self.index.refs): + constant = builtin_constant(expr.object.name, expr.member) + if constant is not None and constant.type_spec.kind == "primitive": + return ConstantValue(PineType.STRING, constant.value) value = literal_color_value(expr) if value is not None: return ConstantValue(PineType.COLOR, value) diff --git a/pineforge_codegen/codegen/security.py b/pineforge_codegen/codegen/security.py index 056756ab..297e3bdc 100644 --- a/pineforge_codegen/codegen/security.py +++ b/pineforge_codegen/codegen/security.py @@ -786,6 +786,24 @@ def _emit_runtime_security_templates(self, lines: list[str]) -> None: " };", )) + @staticmethod + def _lower_tf_tuple_cpp_types(element_types) -> list[str]: + return [ + "int64_t" if item == PineType.INT + else PINE_TYPE_TO_CPP.get(item, "double") + for item in element_types + ] + + @staticmethod + def _lower_tf_clear_cpp(item: dict) -> str: + name = f"_req_sec_lower_tf_{item['sec_id']}" + if item.get("returns_tuple"): + return ( + "std::apply([](auto&... _pf_arrays) { " + f"(_pf_arrays.clear(), ...); }}, {name});" + ) + return f"{name}.clear();" + def _runtime_security_result(self, item: dict, prefix: str) -> str: sec_id = item["sec_id"] if item.get("is_lower_tf_array"): @@ -4554,6 +4572,14 @@ def _collect_security_expr_hist_subscripts( helpers: list[str] = [] def add(n: Subscript) -> None: + if (n is node + and self._security_global_history_value(n) is n.object + and self._resolve_security_index_literal(n.index) != 0): + # A bare tuple-element history payload keeps its located + # refusal; offset zero reads the current element. Tuple slots + # also serve reads inside builder expressions, without + # widening the bare-history payload boundary. + return key = id(n) if helpers: in_helper.add(key) @@ -4604,7 +4630,11 @@ def walk(n) -> None: walk(x) walk(node) - return [n for n in out if id(n) not in in_helper or reached[id(n)] == 1] + return [ + n for n in out + if id(n) not in in_helper or reached[id(n)] == 1 + or self._security_global_history_value(n) is n.object + ] def _security_global_history_value(self, node) -> ASTNode | None: """The value of ``g`` in a payload's ``g[k]`` when ``g`` is a global @@ -4623,6 +4653,10 @@ def _security_global_history_value(self, node) -> ASTNode | None: return None name = node.object.name global_expr_map = getattr(self.ctx, "global_expr_map", {}) or {} + if (self._security_identifier_is_global_binding(node.object) + and name in self._direct_program_tuple_bindings + and name not in self._global_mutable_infos): + return node.object if ( not self._security_identifier_is_global_binding(node.object) or name not in global_expr_map @@ -4634,6 +4668,10 @@ def _security_global_history_value(self, node) -> ASTNode | None: seen = {name} while isinstance(value, Identifier): name = value.name + if (self._security_identifier_is_global_binding(value) + and name in self._direct_program_tuple_bindings + and name not in self._global_mutable_infos): + return node.object if (not self._security_identifier_is_global_binding(value) or name not in global_expr_map or name in seen or name in self._direct_program_tuple_binding_names @@ -4803,6 +4841,8 @@ def _emit_security_expr_hist_members( value = self._security_global_history_value(node) if self._is_session_flag(node.object): cpp_t = "bool" + elif value is node.object: + cpp_t = self._infer_type(value) elif value is not None: # A global's history is a double (a bool's a bool): its na # reads na, where typed from the global's ``int`` the first @@ -4812,12 +4852,14 @@ def _emit_security_expr_hist_members( cpp_t = "bool" if self._infer_type(value) == "bool" else "double" else: cpp_t = self._infer_type(node.object) - if cpp_t not in ("double", "int", "bool"): + if cpp_t not in ("double", "int", "int64_t", "bool", "std::string"): cpp_t = "double" name = f"_sec{sec_id}_expr_hist_{idx}" self._security_expr_hist_by_node[(sec_id, id(node))] = { "name": name, "type": cpp_t, + "object": node.object, + "tuple_global": value is node.object, } lines.append(f" Series<{cpp_t}> {name}{mbb_suffix};") @@ -6103,6 +6145,10 @@ def _security_global_value_inlinable(self, name: str) -> bool: requested bar at each read: its value, through the globals it reads, calls no multi-statement user function, whose statements -- and TA state -- the evaluator would emit again at every read.""" + if name in self._direct_program_tuple_bindings: + # The declaration is evaluated once in the evaluator prologue, + # including a multi-statement tuple helper. + return True cache = getattr(self, "_security_global_value_cache", None) if cache is None: cache = self._security_global_value_cache = {} @@ -6187,6 +6233,84 @@ def _security_pure_call(self, node: FuncCall) -> bool: and len(node.args) == len(info.node.params) and self._security_pure_body(info.node)) + def _security_needed_global_tuples(self, root) -> set[int]: + """Tuple declarations reached through the payload's resolved bindings.""" + needed: set[int] = set() + seen: set[int] = set() + pending = [root] + globals_ = getattr(self.ctx, "global_expr_map", {}) or {} + while pending: + if getattr(self, "_budget", None) is not None: + self._budget.check(phase=Phase.CODEGEN) + node = pending.pop() + if isinstance(node, (list, tuple)): + pending.extend(node) + continue + if not isinstance(node, ASTNode) or id(node) in seen: + continue + seen.add(id(node)) + if isinstance(node, Identifier) and self._security_identifier_is_global_binding(node): + binding = self._direct_program_tuple_bindings.get(node.name) + if binding is not None: + needed.add(id(binding[0])) + value = globals_.get(node.name) + if value is not None: + pending.append(value) + if isinstance(node, FuncCall): + key = self._security_user_call_key(node) + info = self._func_info_map.get(key) + if info is not None and info.node is not None: + pending.extend(info.node.body) + pending.extend(syntax_children(node)) + return needed + + def _security_global_tuple_element( + self, sec_id, name, ta_results, resolving, mutable_names, lines, + ) -> str: + """Sample a declaration once; all member reads share its tuple value.""" + cached = self._security_arg_global_values + if name in cached: + return cached[name] + declaration, _index = self._direct_program_tuple_bindings[name] + key = f"tuple:{id(declaration)}" + if key in resolving: + raise AssertionError("cyclic resolved tuple declaration") + if lines is None: + element = self._security_pure_user_tuple_element(name, declaration.value) + if element is not None: + _info, stack = self._security_bind_helper_args(declaration.value, ()) + return self._build_security_expr( + sec_id, element, None, ta_results, resolving, + mutable_names, stack, lines, + ) + raise AssertionError("tuple declaration requires a statement context") + resolving.add(key) + try: + value = self._build_security_expr( + sec_id, declaration.value, None, ta_results, resolving, + mutable_names, (), lines, + ) + finally: + resolving.remove(key) + site = self._get_ta_site(declaration.value) + fields = TA_TUPLE_FIELDS.get( + self._ta_name_from_site(site) if site is not None else "", (), + ) + if fields and value.isidentifier(): + # The TA prologue already sampled this tuple once. Read that + # result directly, preserving its existing field expressions. + temp = value + else: + temp = "_pf" + self._security_next_inline_name(sec_id, "tuple", "value") + lines.append(f" const auto {temp} = {value};") + for slot, target in enumerate(declaration.names): + if target != "_": + cached[target] = ( + f"{temp}.{fields[slot]}" if slot < len(fields) + else f"std::get<{slot}>({temp})" + ) + return cached[name] + def _security_pure_user_tuple_element(self, name: str, call): """The element expression a payload reads when it reads ``name``, a global of a tuple declaration ``[.., name, ..] = f(args)`` whose call @@ -8712,6 +8836,40 @@ def emit_security_ta(indices: list[int]) -> None: self._emit_security_rebinds(sec_id, info, lines, ta_results, indent=2, emitted_lines=lines) emit_security_ta(post_rebind_ta_indices) + needed_tuples = self._security_needed_global_tuples(expr_node) + for declaration in self.ctx.ast.body: + if isinstance(declaration, TupleAssign) and id(declaration) in needed_tuples: + target = next((name for name in declaration.names if name != "_"), None) + if target is not None: + self._security_global_tuple_element( + sec_id, target, ta_results, set(), security_mutable_names, lines, + ) + # Global tuple declarations run on every requested bar, even when + # a lazy payload does not read their history on that bar. + for (sid, _node_id), meta in self._security_expr_hist_by_node.items(): + obj = meta.get("object") + if (sid != sec_id or not isinstance(obj, Identifier) + or not meta.get("tuple_global")): + continue + value = self._security_arg_global_values.get(obj.name) + if value is None: + value = self._build_security_expr( + sec_id, obj, None, ta_results, set(), + security_mutable_names, (), lines, + ) + hist, cpp_t = meta["name"], meta["type"] + if cpp_t in ("int", "int64_t"): + value = na_preserving_int_cast(value, cpp_t) + elif cpp_t == "double": + value = ( + "[](auto _pf_v) { return is_na(_pf_v) ? na() " + f": static_cast(_pf_v); }}({value})" + ) + if (hist, cpp_t) not in self._security_global_hist_pushes: + self._security_global_hist_pushes.append((hist, cpp_t)) + lines.append( + f" if (is_complete) {{ {hist}_next = {value}; {hist}_set = true; }}" + ) # A global is an unconditional statement of the requested context, # even when the payload reads it only in an untaken selection arm. # Its sampled dependencies keep declaration order and one value. @@ -8727,6 +8885,7 @@ def emit_security_ta(indices: list[int]) -> None: and tuple_size and tuple_size > 0 and isinstance(expr_node, TupleLiteral) + and not item.get("is_lower_tf_array") ): # A tuple body destructures into per-element scalar members # ``_req_sec_{sec_id}_{i}`` (declared in ``base.py`` and reset in @@ -8773,6 +8932,41 @@ def emit_security_ta(indices: list[int]) -> None: emitted_lines=lines, ) if item.get("is_lower_tf_array"): + if returns_tuple: + # Evaluate the tuple once per intrabar, then distribute its + # elements to arrays of their own types. A stateful helper + # must not run once for every destructured element. + temp = f"_pf_lower_tuple_{sec_id}" + lines.append(f" const auto {temp} = {expr_cpp};") + lines.append( + f" if (security_lower_tf_sub_bar_index({sec_id}) == 0) " + f"{{ {self._lower_tf_clear_cpp(item)} }}" + ) + site = self._get_ta_site(expr_node) + fields = TA_TUPLE_FIELDS.get( + self._ta_name_from_site(site) if site is not None else "", (), + ) + for index, ctype in enumerate(self._lower_tf_tuple_cpp_types( + item["tuple_element_types"])): + value = ( + f"{temp}.{fields[index]}" if index < len(fields) + else f"std::get<{index}>({temp})" + ) + if ctype in ("int", "int64_t"): + value = na_preserving_int_cast(value, ctype) + elif ctype == "double": + value = ( + "[](auto _pf_v) { return is_na(_pf_v) ? na() " + f": static_cast(_pf_v); }}({value})" + ) + lines.append( + f" std::get<{index}>(_req_sec_lower_tf_{sec_id})" + f".push_back({value});" + ) + self._emit_security_ohlc_hist_pushes(sec_id, lines) + self._emit_security_ta_hist_pushes(sec_id, info, ta_results, lines) + lines.extend((" }", "")) + return # Match the accumulator's element type before adapting its value. # An integer sentinel must not pass through a floating NaN. ctype = self._infer_cpp_type_for_security_elem(expr_node) @@ -8906,7 +9100,7 @@ def _emit_security_evaluators(self, lines: list[str]) -> None: # gaps mode flushes between completions). Clearing the # vector is the right fallback. lines.append(f" case {sec_id}:") - lines.append(f" _req_sec_lower_tf_{sec_id}.clear();") + lines.append(f" {self._lower_tf_clear_cpp(item)}") for field in sorted(self._security_ohlc_hist_fields_by_sec.get(sec_id, ())): lines.append( f" {self._security_ohlc_hist_series_cpp(sec_id, field)}.clear();" @@ -9357,19 +9551,11 @@ def _build_security_expr( if expr_node.name in self._security_arg_global_values: return self._security_arg_global_values[expr_node.name] value = global_expr_map[expr_node.name] - if (expr_node.name in self._direct_program_tuple_binding_names - and self._security_ta_tuple_element_field(expr_node.name) is None): - # An element of a tuple declaration whose call is a pure - # user function: only that element is lowered, on the - # requested bar, with the call's arguments bound to the - # function's parameters. - element = self._security_pure_user_tuple_element( - expr_node.name, value) - if element is not None: - _info, bound_stack = self._security_bind_helper_args(value, ()) - return self._build_security_expr( - sec_id, element, ta_range, ta_results, resolving, - security_mutable_names, bound_stack, emitted_lines) + if expr_node.name in self._direct_program_tuple_bindings: + return self._security_global_tuple_element( + sec_id, expr_node.name, ta_results, resolving, + security_mutable_names, emitted_lines, + ) # A global bound to a user call is one value per requested # bar, however many reads the payload makes (``g - g[1]``): # every read inlined the call again, which advanced its TA @@ -9679,6 +9865,7 @@ def _build_security_expr( self._security_identifier_is_global_binding(expr_node.object) and expr_node.object.name in self._direct_program_tuple_binding_names and expr_node.object.name not in self._global_mutable_infos + and not self._security_reads_global_history(sec_id, expr_node) ): # Its value is the whole tuple: every earlier build # emitted the tuple's history, which did not compile. @@ -9752,7 +9939,8 @@ def _build_security_expr( # builtin keeps the chart's series and never gets here): # a double, which an integer slot of this evaluator # narrows na-preserving (``_security_emits_double``). - if self._infer_type(expr_node.object) == "std::string": + if (self._infer_type(expr_node.object) == "std::string" + and not meta.get("tuple_global")): # No string history on the requested clock; every # earlier build refused the read. self._codegen_error( @@ -10395,12 +10583,17 @@ def argument(value): emitted_lines, ) self._security_warn_chart_call(expr_node) - self._security_check_tuple_element_history(expr_node, sec_id, helper_binding_stack) + tuple_history = self._security_check_tuple_element_history( + expr_node, sec_id, helper_binding_stack, + ) chart: set[int] = set() if self._security_requested_calls: self._security_warn_global_history(expr_node, helper_binding_stack, sec_id) - chart_read = self._security_root_chart_read( - expr_node, helper_binding_stack, sec_id, security_mutable_names + chart_read = ( + "history it does not keep on the requested clock" + if tuple_history is not None else self._security_root_chart_read( + expr_node, helper_binding_stack, sec_id, security_mutable_names, + ) ) if chart_read is not None: if self._security_requested_used: @@ -10425,7 +10618,7 @@ def argument(value): def _security_check_tuple_element_history( self, root, sec_id: int, helper_binding_stack=None - ) -> None: + ) -> Subscript | None: """History of a tuple declaration's element read under a node the expression visitor renders, directly or through a global's value, reads the chart's series: warned. Refused, as the builder refuses the @@ -10433,12 +10626,14 @@ def _security_check_tuple_element_history( tuple's element outside a user call's arguments, its site in the requested history and reached by no multi-statement helper, whose struct result the evaluator then pushed into a ``Series`` - (``_emit_security_ta_hist_pushes`` skips it).""" + (``_emit_security_ta_hist_pushes`` skips it). Return the first chart + history read so a global referring to it keeps the same fallback.""" global_expr_map = getattr(self.ctx, "global_expr_map", {}) or {} info = self._security_info_without_methods(self._security_eval_info[sec_id]) hist = self._security_ta_hist_idx_by_sec.get(sec_id, ()) helper_reached = {key[0] for key in info.get("inline_helper_ta_indices", [])} followed: set[tuple[str, bool]] = set() + chart_history = None stack = [(root, False, helper_binding_stack)] while stack: n, in_call, bindings = stack.pop() @@ -10463,6 +10658,8 @@ def _security_check_tuple_element_history( and self._security_identifier_is_global_binding(n.object) and n.object.name in self._direct_program_tuple_binding_names and n.object.name not in self._global_mutable_infos + and not (self._security_reads_global_history(sec_id, n) + and self._security_fallback_owns(n, bindings or (), sec_id)) and self._resolve_security_index_literal(n.index, bindings) != 0): name = n.object.name site = self._get_ta_site(global_expr_map.get(name)) @@ -10474,6 +10671,8 @@ def _security_check_tuple_element_history( f"request.security payload reads the history of " f"'{name}', an element of a tuple declaration", ) + if chart_history is None: + chart_history = n warned = getattr(self, "_security_warned_tuple_history", None) if warned is None: warned = self._security_warned_tuple_history = set() @@ -10491,6 +10690,7 @@ def _security_check_tuple_element_history( stack.extend( (v, in_call, bindings) for k, v in vars(n).items() if k != "annotations" ) + return chart_history def _security_note_chart_read(self, node, reason: str) -> None: """Record the first thing the evaluator being emitted reads on the @@ -11233,6 +11433,16 @@ def _security_fallback_owns( history (re-evaluating ``g`` on the requested bar would pair it with the chart's ``g[1]``), and ``timeframe.*`` (beside the chart's ``timeframe.in_seconds()``).""" + # A helper's parameter is bound in both evaluator modes. The legacy + # fallback may keep a surrounding builtin's rendering, but it cannot + # resolve that parameter in the chart function's lexical scope. + if (isinstance(node, Identifier) + and not self._security_identifier_is_global_binding(node)): + binding = self._security_lookup_helper_binding_context( + node.name, helper_binding_stack, + ) + if binding is not None and self._security_bound_is_scalar(*binding): + return True selection = getattr(self, "_security_selection_context", None) if selection is not None: if id(node) in selection["reads"]: @@ -11262,6 +11472,13 @@ def _security_fallback_owns( and node.name in self._input_backed_vars) or self._security_source_input_call(node) is not None ) + # Cached current values belong to requested evaluation. A chart + # fallback still needs its global Series as a history receiver, even + # when the evaluator has also sampled that declaration's tuple. + if (isinstance(node, Identifier) + and self._security_identifier_is_global_binding(node) + and node.name in self._security_arg_global_values): + return True if self._get_ta_site(node) is not None: return True if isinstance(node, FuncCall): diff --git a/pineforge_codegen/codegen/tables.py b/pineforge_codegen/codegen/tables.py index baa0914a..acf8e833 100644 --- a/pineforge_codegen/codegen/tables.py +++ b/pineforge_codegen/codegen/tables.py @@ -22,6 +22,7 @@ from ..intrinsic_binding import validate_intrinsic_keywords from ..signatures import IntrinsicFunc from ..symbols import PineType +from ..builtin_constants import ENUM_CONSTANT_VALUES from .helpers import ( evaluate_args_once, na_preserving_int_cast, pine_index_int_cast, pine_truth_cast, @@ -654,8 +655,8 @@ def tz_time_field_lambda(field_expr: str, ts_arg: str, tz_arg: str) -> str: # order.* sort-direction constants — emitted as std::string literals. Unknown # member falls back to "ascending". ORDER_DIRECTION_MAP = { - "ascending": 'std::string("ascending")', - "descending": 'std::string("descending")', + member: f'std::string("{value}")' + for member, value in ENUM_CONSTANT_VALUES["order"][1].items() } NAME_ECHO_STRING_MEMBERS = { diff --git a/pineforge_codegen/codegen/types.py b/pineforge_codegen/codegen/types.py index 0a649e1b..4453cf59 100644 --- a/pineforge_codegen/codegen/types.py +++ b/pineforge_codegen/codegen/types.py @@ -51,6 +51,7 @@ from ..errors import CodegenInvariantError, Phase from ..external_requests import UNPINNED_ANNOTATION from ..limits import iter_ast_nodes +from ..builtin_constants import BUILTIN_ENUM_STORAGE, builtin_constant, is_builtin_enum from ..loop_types import FLOAT_REASSIGNMENT, LoopTypeSources, for_counter_type from ..matrix_overloads import matrix_sum_has_rhs from ..reference_casts import na_reference_cast_kind @@ -185,6 +186,8 @@ def _type_spec_to_cpp(self, spec: TypeSpec | None) -> str: """Render a TypeSpec as the equivalent C++ declaration string.""" if spec is None: return "double" + if is_builtin_enum(spec): + return PINE_TYPE_TO_CPP[BUILTIN_ENUM_STORAGE[spec.name]] if spec.kind == "primitive": return {"float": "double", "int": "int", "int64": "int64_t", "bool": "bool", "string": "std::string", @@ -2758,7 +2761,9 @@ def _selection_type_spec( spec = TypeSpec.primitive("color") if spec is None: return None - if not nullable and spec.kind not in {"map", "matrix"}: + if (not nullable and spec.kind not in {"map", "matrix"} + and spec != TypeSpec.primitive("string") + and not is_builtin_enum(spec)): return None result = ( spec @@ -3064,6 +3069,10 @@ def _type_spec_from_expr_raw( if plan is not None else spec) if self._builtin_sizing_constant(node) is not None: return TypeSpec.primitive("string") + if isinstance(node.object, Identifier): + constant = builtin_constant(node.object.name, node.member) + if constant is not None: + return constant.type_spec return None if isinstance(node, FuncCall): func_name, namespace = self._resolve_callee(node.callee) @@ -6543,6 +6552,11 @@ def _infer_type( return "std::string" if isinstance(node, MemberAccess) and isinstance(node.object, Identifier): ename = node.object.name + owner = self._type_spec_from_expr(node.object) + if owner is None: + constant = builtin_constant(ename, node.member) + if constant is not None: + return self._type_spec_to_cpp(constant.type_spec) if ename == "color": return "int64_t" if ename in self._enum_defs and node.member in self._enum_defs[ename]: diff --git a/pineforge_codegen/codegen/visit_expr.py b/pineforge_codegen/codegen/visit_expr.py index 194f0cda..aa0b941d 100644 --- a/pineforge_codegen/codegen/visit_expr.py +++ b/pineforge_codegen/codegen/visit_expr.py @@ -93,6 +93,7 @@ from ..errors import CodegenInvariantError, Phase from ..symbols import TypeSpec +from ..builtin_constants import builtin_constant from ..external_requests import UNPINNED_ANNOTATION from ..ast_nodes import ( ASTNode, @@ -1009,6 +1010,9 @@ def _visit_member_access(self, node: MemberAccess) -> str: return self._visit_expr(StringLiteral(value=sizing.value)) if isinstance(node.object, Identifier): ns = node.object.name + constant = builtin_constant(ns, node.member) + if constant is not None: + return f'std::string("{self._cpp_string_escape(constant.value)}")' if ns == "alert" and node.member in ALERT_FREQ_VALUES: # TradingView's const string values (its own tape spells them). return f'std::string("{ALERT_FREQ_VALUES[node.member]}")' @@ -1380,12 +1384,9 @@ def _visit_member_access(self, node: MemberAccess) -> str: if node.member in COLOR_CONST_MAP: return COLOR_CONST_MAP[node.member] return "0" - # Drawing style/visual CONSTANT member reads (line.style_solid, - # box.style_dashed, label.style_label_left, ...). These namespaces - # left SKIP_NAMESPACES (their FuncCall forms now route to the arena), - # but their constant members only ever feed dropped visual kwargs, so - # they still lower to "0". The function names arrive as FuncCalls - # (visit_call), never here. See spec §4.4. + # Inventoried constants were lowered above, independently of the + # renderer. Uninventoried members retain the legacy fallback here; + # namespace functions arrive as FuncCalls in visit_call. if ns in DRAWING_STYLE_NS: return "0" if ns in SKIP_NAMESPACES: diff --git a/pineforge_codegen/codegen/visit_stmt.py b/pineforge_codegen/codegen/visit_stmt.py index e367346c..b09f4c4b 100644 --- a/pineforge_codegen/codegen/visit_stmt.py +++ b/pineforge_codegen/codegen/visit_stmt.py @@ -1711,7 +1711,7 @@ def emit_call_tuple(call_expr: str, *, decompose: bool = False) -> None: # User-defined function returning a tuple: use C++17 structured bindings if isinstance(node.value, FuncCall): func_name, namespace = self._resolve_callee(node.value.callee) - if namespace == "request" and func_name == "security": + if namespace == "request" and func_name in ("security", "security_lower_tf"): # A request whose names are reassigned is read where it is # evaluated (external_requests): its data-missing stop. if UNPINNED_ANNOTATION in (node.value.annotations or {}): @@ -1768,6 +1768,14 @@ def _tuple_call_info(self, call: FuncCall): def _tuple_binding_cpp_types(self, node: TupleAssign) -> list[str]: """Exact supported tuple element types for later lexical operations.""" count = len(node.names) + lower_types = (node.value.annotations or {}).get( + "lower_tf_tuple_element_types", (), + ) + if lower_types: + return [ + f"std::vector<{ctype}>" + for ctype in self._lower_tf_tuple_cpp_types(lower_types) + ] if isinstance(node.value, (IfStmt, SwitchStmt)): return self._infer_selection_tuple_types(node.value, count) if not isinstance(node.value, FuncCall): diff --git a/pineforge_codegen/diagnostics_catalog.json b/pineforge_codegen/diagnostics_catalog.json index c6eef36f..0a4682e3 100644 --- a/pineforge_codegen/diagnostics_catalog.json +++ b/pineforge_codegen/diagnostics_catalog.json @@ -235,6 +235,7 @@ "PF-E2090": {"severity": "error", "area": "analysis", "message": "input.time() needs a constant default in Pine v6: timestamp() with a time zone and calendar numbers is not a constant. Use the date-string form, for example timestamp(\"01 Jan 2023 00:00 +0000\").", "hint": "Use a single date string in timestamp or an integer timestamp literal for the default.", "explanation": "The time-zone overload of timestamp, which takes a zone and calendar numbers, returns a simple int, while a Pine v6 input.time default requires a const int.", "args": {}, "user_message": "input.time() requires a constant default in Pine v6."}, "PF-E2091": {"severity": "error", "area": "analysis", "message": "plot() cannot have a linewidth below 1 in Pine v6: use a value of 1 or greater.", "hint": "Set the plot linewidth to an integer of at least 1.", "explanation": "A Pine v6 plot call cannot use an integer literal below one for linewidth; variable and calculated widths retain their existing handling.", "args": {}, "user_message": "plot() requires a linewidth of at least 1 in Pine v6."}, "PF-E2092": {"severity": "error", "area": "analysis", "message": "\"range\" cannot be used as a variable or function name in Pine v6: use another name.", "hint": "Rename the variable, function or method declaration.", "explanation": "Pine v6 reserves range as a variable and callable declaration name; parameters, user type fields and member access retain their existing handling.", "args": {}, "user_message": "Choose another variable, function or method name instead of range in Pine v6."}, + "PF-E2093": {"severity": "error", "area": "analysis", "message": "Cannot use {actual} as {expected} for {context}.", "hint": null, "explanation": "A built-in enum retains its nominal Pine type independently of its runtime storage. Assignments, arguments and defaults must use the expected semantic type.", "args": {"actual": {"kind": "identifier"}, "expected": {"kind": "identifier"}, "context": {"kind": "identifier"}}, "user_message": "Cannot use {actual} as {expected} for {context}."}, "PF-E2095": {"severity": "error", "area": "analysis", "message": "Cannot call ''{function_name}'' with an argument of type {argument_type} for {parameter_type} parameter ''{name}''.", "hint": null, "explanation": "A non-string value cannot be used for a string parameter. Pass a string value of the declared parameter type; a permitted generic na adopts that type.", "args": {"function_name": {"kind": "identifier"}, "argument_type": {"kind": "type"}, "parameter_type": {"kind": "type"}, "name": {"kind": "identifier"}}, "user_message": "Argument ''{name}'' of ''{function_name}'' has type {argument_type}, but requires {parameter_type}."}, "PF-E3001": {"severity": "error", "area": "request-security", "message": "request.security timeframe ''{name}'' cannot be resolved before the first bar: PineForge registers every requested timeframe before the script runs.", "hint": "Pass a literal, an input or a global to the helper that holds request.security.", "explanation": "The timeframe given to request.security is a name whose value PineForge cannot compute before the first bar; use a literal, an input or a global.", "args": {"name": {"kind": "identifier"}}, "user_message": "The timeframe given to request.security is a name whose value PineForge cannot compute before the first bar."}, "PF-E3002": {"severity": "error", "area": "request-security", "message": "request.security timeframe expression cannot be resolved before the first bar: PineForge registers every requested timeframe before the script runs.", "hint": "Pass a literal, an input or a global to the helper that holds request.security.", "explanation": "The timeframe expression given to request.security cannot be computed before the first bar; use a literal, an input or a global.", "args": {}, "user_message": "The timeframe expression given to request.security cannot be computed before the first bar."}, @@ -473,9 +474,9 @@ "PF-E5088": {"severity": "error", "area": "request-security", "message": "The branches of this {kind} return incompatible types: one returns a value and another returns nothing.", "hint": "Give every branch a value, or call the void function as a separate statement.", "explanation": "TradingView refuses a consumed if or switch with a value arm and a terminal void arm (CE10235). Statement selections, implicit missing arms and selections whose present arms all return values or all return nothing retain their existing behavior.", "args": {"kind": {"kind": "vocab", "values": ["if", "switch"]}}, "user_message": "The branches of this {kind} must all return compatible values."}, "PF-E5089": {"severity": "error", "area": "codegen", "message": "{function}: TradingView refuses keyword arguments for functions with a variable number of arguments.", "hint": "Pass every argument to {function} positionally.", "explanation": "TradingView refuses every keyword argument on variadic math functions; supply their arguments by position.", "args": {"function": {"kind": "identifier"}}, "user_message": "{function} accepts positional arguments only."}, "PF-E5090": {"severity": "error", "area": "codegen", "message": "{function}: unknown keyword argument ''{name}''; accepted keyword names: {accepted}.", "hint": "Use these parameter names for {function}: {accepted}.", "explanation": "The math function does not declare this keyword; use the parameter names recorded for that function.", "args": {"function": {"kind": "identifier"}, "name": {"kind": "identifier"}, "accepted": {"kind": "text"}}, "user_message": "Unknown keyword ''{name}'' for {function}; use {accepted}."}, - "PF-E5093": {"severity": "error", "area": "codegen", "message": "max_bars_back(): num must resolve to a positive compile-time integer.", "hint": null, "explanation": "A max_bars_back call needs a positive integer depth proven at compile time. Mutable, input-dependent, non-integer and nonpositive depths cannot size its history buffers.", "args": {}, "user_message": "max_bars_back requires a positive compile-time integer depth."}, "PF-E5091": {"severity": "error", "area": "codegen", "message": "Unsupported TA constructor length ''{a}'' for ta.{func_name}: it is neither a compile-time constant nor derived from an input, so PineForge cannot size the indicator buffer. TradingView accepts a series length for ta.{func_name}(); PineForge does not support it yet.", "hint": "Use a literal, an input.*() value, or arithmetic over those for TA constructor arguments.", "explanation": "A TA indicator's length is neither a constant nor derived from an input, so PineForge cannot size its buffer. TradingView compiles a series length for this function, so this is a limit of PineForge, not an error in the script; use a literal, an input or arithmetic on those.", "args": {"a": {"kind": "identifier"}, "func_name": {"kind": "identifier"}}, "user_message": "PineForge cannot size ta.{func_name} from a series length, which TradingView accepts."}, "PF-E5092": {"severity": "error", "area": "codegen", "message": "Unsupported TA constructor length ''{a}'' for ta.{func_name}: it is neither a compile-time constant nor derived from an input, so PineForge cannot size the indicator buffer. TradingView accepts a series length for ta.{func_name}(); PineForge does not support it yet.", "hint": "Use a literal, an input.*() value, or arithmetic over those for TA lengths.", "explanation": "A TA indicator's length is not a literal and not derived from an input, so PineForge cannot size its buffer. TradingView compiles a series length for this function, so this is a limit of PineForge, not an error in the script; use a literal, an input or arithmetic on those.", "args": {"a": {"kind": "identifier"}, "func_name": {"kind": "identifier"}}, "user_message": "PineForge cannot size ta.{func_name} from a series length, which TradingView accepts."}, + "PF-E5093": {"severity": "error", "area": "codegen", "message": "max_bars_back(): num must resolve to a positive compile-time integer.", "hint": null, "explanation": "A max_bars_back call needs a positive integer depth proven at compile time. Mutable, input-dependent, non-integer and nonpositive depths cannot size its history buffers.", "args": {}, "user_message": "max_bars_back requires a positive compile-time integer depth."}, "PF-E5094": {"severity": "error", "area": "codegen", "message": "Codegen option {option} must be an integer of at least {minimum}, or None.", "hint": null, "explanation": "An operator capacity must be an exact integer at or above its documented minimum, or None. Booleans, floating values, strings and smaller integers are invalid.", "args": {"option": {"kind": "identifier"}, "minimum": {"kind": "number"}}, "user_message": "Option {option} requires an integer of at least {minimum}, or None."}, "PF-E5095": {"severity": "error", "area": "codegen", "message": "History offset {offset} cannot fit history_capacity={capacity}.", "hint": null, "explanation": "A statically proven history offset requires more slots than the operator selected. Raise history_capacity to at least offset plus one, or use None.", "args": {"offset": {"kind": "number"}, "capacity": {"kind": "number"}}, "user_message": "History offset {offset} exceeds the selected capacity {capacity}."}, "PF-E5096": {"severity": "error", "area": "codegen", "message": "The default of a string input is a variable that PineForge cannot resolve to one constant string when it reads the input.", "hint": "Declare the variable once with a string literal and never assign to it, or write the literal in the call.", "explanation": "A variable used as the default of a string input needs one preceding global declaration with a literal initializer and no reassignment; an unproven value would otherwise become an empty default.", "args": {}, "user_message": "The default of a string input cannot be resolved to one constant string."}, diff --git a/pineforge_codegen/function_overloads.py b/pineforge_codegen/function_overloads.py index 06148d0e..4071207c 100644 --- a/pineforge_codegen/function_overloads.py +++ b/pineforge_codegen/function_overloads.py @@ -55,6 +55,7 @@ class _ScalarProof: _STRING_RETURNING_STR_FUNCTIONS = Analyzer._STRING_RETURNING_STR_FUNCTIONS _is_certainly_string = Analyzer._is_certainly_string + _builtin_constant_info = Analyzer._builtin_constant_info def __init__(self, program: Program, function_names: set[str]) -> None: self._symbols = SymbolTable() diff --git a/pineforge_codegen/lexer.py b/pineforge_codegen/lexer.py index aba38033..79f3e6d1 100644 --- a/pineforge_codegen/lexer.py +++ b/pineforge_codegen/lexer.py @@ -386,12 +386,24 @@ def _tokenize_line(self) -> None: indent_level += 1 raw = self.source[line_start: self.pos] - if " \t" in raw: + columns = raw.count("\t") * 4 + raw.count(" ") + mixed = "\t" in raw and " " in raw + aligned_block = ( + columns % 4 == 0 + and columns // 4 <= self.indent_stack[-1] + 1 + ) + if " \t" in raw and not aligned_block: self._emit_diagnostic( "Indentation cannot contain spaces followed by a tab.", self.line, self.col, self.col + 1, ) - if "\t" in raw: + if mixed and aligned_block and not self._in_continuation: + indent_level = columns // 4 + elif "\t" in raw: + # Keep the established treatment of incomplete mixed prefixes + # and first-body skipped widths, including pending logical-line + # continuations. Their TV grammar is not established by the + # nested, complete-level layouts. indent_level = raw.count("\t") else: indent_level = len(raw) // 4 diff --git a/pineforge_codegen/parser.py b/pineforge_codegen/parser.py index 2645bf0f..2ea987d0 100644 --- a/pineforge_codegen/parser.py +++ b/pineforge_codegen/parser.py @@ -835,12 +835,19 @@ def _looks_like_call_template_args(self) -> bool: return False def _parse_var_keyword_decl(self) -> VarDecl: - """Parse: var [type] name = expr or varip [type] name = expr""" + """Parse: (var | varip) [[qualifier] type] name = expr.""" start_tok = self._current() is_var = self._current().type == TokenType.VAR is_varip = self._current().type == TokenType.VARIP self._advance() + qualifier = None + if (self._check(TokenType.IDENT) + and self._current().value in ("const", "series", "simple") + and (self._peek().type in TYPE_KEYWORDS + or self._is_ident_typed_var_decl(offset=1))): + qualifier = self._advance().value + type_hint = None if self._current().type in TYPE_KEYWORDS: type_hint = self._parse_type_hint_string() @@ -873,6 +880,10 @@ def _parse_var_keyword_decl(self) -> VarDecl: name=name_tok.value, value=value, is_var=is_var, is_varip=is_varip, type_hint=type_hint, ) + if qualifier is not None: + node.annotations = {"qualifier": qualifier} + if self._library and qualifier == "const": + node.annotations["declared_const"] = True self._retain_declaration_names(node, name_tok) return self._set_loc(node, start_tok) diff --git a/pineforge_codegen/signatures.py b/pineforge_codegen/signatures.py index 0f4483d5..23b14941 100644 --- a/pineforge_codegen/signatures.py +++ b/pineforge_codegen/signatures.py @@ -24,6 +24,7 @@ class Param: pine_type: PineType = PineType.FLOAT default: Any = None # None = required, else default value is_series: bool = True # series vs simple/const + type_spec: TypeSpec | None = None # nominal identity beyond primitive storage @dataclass @@ -492,6 +493,14 @@ def _map(short_name: str, *sigs: FuncSig) -> None: "includes": _func("array.includes", _sig([("id", F), ("value", F)], ret=B)), "every": _func("array.every", _sig([("id", F)], ret=B)), "some": _func("array.some", _sig([("id", F)], ret=B)), + "sort": _func("array.sort", FuncSig(params=[ + Param("id", F), + Param("order", S, "ascending", type_spec=TypeSpec("enum", name="sort_order")), + ], return_type=VOID)), + "sort_indices": _func("array.sort_indices", FuncSig(params=[ + Param("id", F), + Param("order", S, "ascending", type_spec=TypeSpec("enum", name="sort_order")), + ], return_type=F)), } MATRIX_FUNCTIONS: dict[str, IntrinsicFunc] = { @@ -502,6 +511,10 @@ def _map(short_name: str, *sigs: FuncSig) -> None: "is_stochastic", "is_binary", "is_zero", ) } +MATRIX_FUNCTIONS["sort"] = _func("matrix.sort", FuncSig(params=[ + Param("id", F), Param("column", I, 0), + Param("order", S, "ascending", type_spec=TypeSpec("enum", name="sort_order")), +], return_type=VOID)) # matrix.new(rows, columns, initial_value): the row and column counts # default to 0. The record exists for keyword binding only (its element type @@ -641,6 +654,9 @@ def _input(short_name: str, *sigs: FuncSig, ("maxval", F, None), ("step", F, None)] + _C, ret=F)) _input("bool", _sig([("defval", B), ("title", S, "")] + _C, ret=B)) _input("string", _sig([("defval", S), ("title", S, ""), ("options", S, None)] + _C, ret=S)) +INPUT_FUNCTIONS["string"].primary.params[2].type_spec = TypeSpec.array( + TypeSpec.primitive("string") +) _input("source", _sig([("defval", F), ("title", S, "")] + _C, ret=F), default_qualifier="series") _input("color", _sig([("defval", I), ("title", S, "")] + _C, ret=I), diff --git a/pineforge_codegen/support_checker.py b/pineforge_codegen/support_checker.py index 74319c8e..23dc9c48 100644 --- a/pineforge_codegen/support_checker.py +++ b/pineforge_codegen/support_checker.py @@ -58,6 +58,7 @@ from .builtin_keywords import positional_builtin from .intrinsic_binding import intrinsic_keyword_diagnostics from .limits import iter_ast_nodes +from .builtin_constants import builtin_constant from .pine_spelling import expr_start from .external_requests import ( ABSENT_LOWERING, CHART_FALLBACK_ANNOTATION, FEED_LOWERING, FEED_WARNING_ANNOTATION, @@ -2276,6 +2277,7 @@ def _visit_MemberAccess(self, node: MemberAccess) -> None: if ( self._const_arg_ctx_depth == 0 and isinstance(node.object, Identifier) + and builtin_constant(node.object.name, node.member) is None and not (node.object.name == "alert" and node.member in ALERT_FREQ_MEMBERS) and not (node.object.name == "barmerge" diff --git a/tests/fixtures/diagnostic_codes_pin.json b/tests/fixtures/diagnostic_codes_pin.json index a59e136b..97c67d55 100644 --- a/tests/fixtures/diagnostic_codes_pin.json +++ b/tests/fixtures/diagnostic_codes_pin.json @@ -236,6 +236,7 @@ "PF-E2090": "dcc4d05f50b881e5345de8c73ec7b7951bedc86c30a24d40bc0077fca5242a0d", "PF-E2091": "2d1b04fd25a4198619628b94bf8f90b4002f33e4081c2b780fea510c49e0b44a", "PF-E2092": "74b483fe343d239ec254f03e839b257611db5295b0433eabd15072f919950f3d", + "PF-E2093": "b2d5cd10ea46245c7a9e31ff15db9db7d8538221d5443c9cc15ba0013a19e93f", "PF-E2095": "a2bf71e6ab48d39b72365b5a69960a8622c7b38deb0aace009215b46dc3cb89d", "PF-E3001": "585dad07e797bbe2cfffa9f45e14dee040be5570cc13f8e46fa4c31e953fc92a", "PF-E3002": "db089893825afe5ad1d91934c21107fc58451cadf3d71de2539b9c5805549433", @@ -474,9 +475,9 @@ "PF-E5088": "c2d0aa76b2718250efb6b1c0b586c88b564cdb5e48dd26fa54d402597b7f36e3", "PF-E5089": "9fc5cd13ce647daf93fe654d0788031f1ef4ef307740652dd0c7d330958c9fca", "PF-E5090": "04d16affd323ac6d3a797582dbaf54f2a307bb2f0bdd834f53fa66b68b672c56", - "PF-E5093": "27cbe0b1e2e66bbb020c5e46a4d3e6524f6c2ba8c9d602573f125eb83d226963", "PF-E5091": "8c57abd5175fc238a78bb69490c54c952d883a3366918da0eaf03f25c358a6e4", "PF-E5092": "7df7584244d6b96e5af1ba63cf0ad30ec122236d245505bb90da53a9ad1ab480", + "PF-E5093": "27cbe0b1e2e66bbb020c5e46a4d3e6524f6c2ba8c9d602573f125eb83d226963", "PF-E5094": "8586b9a7efbc037ea96ca98232e26eb026fb2ccdf637600980abafffe2d71eec", "PF-E5095": "b848739da40efbc09fa0a7206c09a9733f82889c4add617de251310c0f1abdee", "PF-E5096": "d9abe7c9d330cd37b1be38a5a9f506bc93d56fbaefdddbde310f192eafea341d", diff --git a/tests/fixtures/input_metadata_strings.pine b/tests/fixtures/input_metadata_strings.pine new file mode 100644 index 00000000..8b224a2b --- /dev/null +++ b/tests/fixtures/input_metadata_strings.pine @@ -0,0 +1,20 @@ +//@version=6 +strategy("Input metadata") +enum Side + long + short +length = input.int(14, "Length", minval=1) +threshold = input.float(-2.5, "Threshold") +enabled = input.bool(true, "Enabled") +source = input.source(hl2, "Source") +side = input.enum(Side.long, "Side") +symbol = input.symbol("NASDAQ:AAPL", "Symbol") +frequency = input.string(alert.freq_all, "Frequency", options=[alert.freq_all, alert.freq_once_per_bar]) +direction = input.string("ascending", "Direction", options=["ascending", "descending"]) +currency = input.string(currency.USD, "Currency", options=[currency.USD, currency.EUR]) +sessionChoice = input.string(session.regular, "Session", options=[session.regular, session.extended]) +formatChoice = input.string(format.price, "Format", options=[format.price, format.percent]) +sizeChoice = input.string(size.small, "Size", options=[size.small, size.tiny]) +positionChoice = input.string(position.top_right, "Position", options=[position.top_right, position.bottom_right]) +mixedChoice = input.string("literal", "Mixed", options=["literal", size.small]) +plot(ta.sma(source, length)) diff --git a/tests/gate-corpus/ok/input_metadata.pine b/tests/gate-corpus/err/input_metadata.pine similarity index 100% rename from tests/gate-corpus/ok/input_metadata.pine rename to tests/gate-corpus/err/input_metadata.pine diff --git a/tests/gate-corpus/err/const_ns_shape_free.pine b/tests/gate-corpus/ok/const_ns_shape_free.pine similarity index 100% rename from tests/gate-corpus/err/const_ns_shape_free.pine rename to tests/gate-corpus/ok/const_ns_shape_free.pine diff --git a/tests/test_builtin_style_values.py b/tests/test_builtin_style_values.py new file mode 100644 index 00000000..b2888ac3 --- /dev/null +++ b/tests/test_builtin_style_values.py @@ -0,0 +1,301 @@ +"""Builtin constant values keep their Pine types across value contexts. + +The exact public reporter scripts are exercised separately through the native +reproduction harness. These units pin the shared constant, inference and +nominal-type contracts; native rows observe the emitted translation unit. +""" + +import re + +import pytest + +from pineforge_codegen import transpile, transpile_full +from pineforge_codegen.analyzer import Analyzer +from pineforge_codegen.ast_nodes import Identifier, MemberAccess +from pineforge_codegen.codegen import CodeGen +from pineforge_codegen.errors import CompileError +from pineforge_codegen.lexer import Lexer +from pineforge_codegen.parser import Parser +from pineforge_codegen.symbols import PineType, TypeSpec +from tests._compile import compile_cpp, run_emitted_tu + + +HEADER = '//@version=6\nstrategy("builtin constant values")\n' + + +def analyze(body): + source = HEADER + body + return Analyzer(Parser(Lexer(source).tokenize(), source=source).parse()).analyze() + + +@pytest.mark.parametrize("namespace,member,value", [ + ("size", "tiny", "tiny"), ("size", "small", "small"), + ("size", "normal", "normal"), ("size", "large", "large"), + ("position", "top_right", "top_right"), + ("position", "bottom_left", "bottom_left"), + ("line", "style_dashed", "dsh"), ("line", "style_solid", "sol"), + ("label", "style_label_up", "lup"), + ("shape", "circle", "shape_circle"), ("shape", "cross", "shape_cross"), +]) +def test_constant_member_has_one_string_type_and_value(namespace, member, value): + context = analyze(f"held = {namespace}.{member}\n") + assert context.symbols.resolve("held").pine_type == PineType.STRING + generator = CodeGen(context) + expression = MemberAccess(object=Identifier(name=namespace), member=member) + assert generator._infer_type(expression) == "std::string" + assert generator._visit_member_access(expression) == f'std::string("{value}")' + + +STRING_DRIVER = r''' +#include +#include +int main() { + std::vector bars(3); + for (std::size_t i = 0; i < bars.size(); ++i) { + bars[i].open = 100.0; + bars[i].high = 103.0; + bars[i].low = 97.0; + bars[i].close = i % 2 == 0 ? 101.0 : 99.0; + bars[i].volume = 1.0; + bars[i].timestamp = 1704067200000LL + static_cast(i) * 3600000LL; + } + for (std::size_t count = 1; count <= bars.size(); ++count) { + GeneratedStrategy strategy; + strategy.run(bars.data(), count, "60", "60"); + if (!strategy.last_error().empty()) { + std::cerr << strategy.last_error() << "\n"; + return 14; + } + std::cout << strategy.held.size() << ":" << strategy.held << "\n"; + } +} +''' + + +@pytest.mark.parametrize("body,expected", [ + ( + "string held = size.small\nif close > open\n held := size.large\n", + "5:large\n5:small\n5:large\n", + ), + ( + "string held = position.top_right\n" + "if close > open\n held := position.bottom_left\n", + "11:bottom_left\n9:top_right\n11:bottom_left\n", + ), + ( + "held = close > open ? size.large : size.small\n", + "5:large\n5:small\n5:large\n", + ), + ( + 'styleOf(string choice) =>\n switch choice\n' + ' "Dashed" => line.style_dashed\n => line.style_solid\n' + 'held = styleOf(close > open ? "Dashed" : "Solid")\n', + "3:dsh\n3:sol\n3:dsh\n", + ), + ( + "held = switch\n close > open => shape.circle\n => shape.cross\n", + "12:shape_circle\n11:shape_cross\n12:shape_circle\n", + ), + ( + "echo(string sz) => sz\nheld = echo(size.small)\n", + "5:small\n5:small\n5:small\n", + ), + ( + "echo(string style) => style\nheld = echo(label.style_label_up)\n", + "3:lup\n3:lup\n3:lup\n", + ), +], ids=["reassignment", "position", "ternary", "function-return", "shape-switch", + "parameter", "label-parameter"]) +def test_string_constant_value_transport_runs(body, expected): + result = transpile_full(HEADER + body) + assert not any(diagnostic.code == "PF-W1078" for diagnostic in result["diagnostics"]) + assert run_emitted_tu(result["cpp"], STRING_DRIVER, opt="-O0", + label="builtin constant value transport") == expected + + +def test_nested_stateful_function_variants_keep_string_returns_and_parameters(): + body = ( + "styleOf(string choice) =>\n" + " average = ta.sma(close, 2)\n" + ' choice == "Dashed" ? line.style_dashed : line.style_solid\n' + "outer(string choice) => styleOf(choice)\n" + 'first = outer("Dashed")\nsecond = outer("Solid")\n' + "held = first + second\n" + ) + cpp = transpile(HEADER + body) + declarations = re.findall( + r"^\s*(\S+)\s+((?:styleOf|outer)(?:_cs\d+|__ni\d+)*)\(([^)]*)\)\s*\{", + cpp, re.MULTILINE, + ) + assert len(declarations) >= 3, declarations + assert all(result == "std::string" and "std::string" in parameters + for result, _name, parameters in declarations) + assert run_emitted_tu(cpp, STRING_DRIVER, opt="-O0", + label="nested style function variants") == ( + "6:dshsol\n6:dshsol\n6:dshsol\n" + ) + + +def test_direct_drawing_arguments_and_value_arguments_compile_together(): + cpp = transpile(HEADER + """ +string held = line.style_dashed +line.new(bar_index, high, bar_index + 1, high, style = line.style_solid) +line.new(bar_index, low, bar_index + 1, low, style = held) +box.new(bar_index, high, bar_index + 1, low, border_style = line.style_dashed) +label.new(bar_index, high, "x", style = label.style_label_up, size = size.small) +""") + compile_cpp(cpp, label="direct and stored drawing constants") + + +@pytest.mark.parametrize("call", [ + 'input.string(size.small, "Text size", options = [size.tiny, size.small, size.normal])', + 'input.string(options = [size.tiny, size.small, size.normal], title = "Text size", defval = size.small)', +]) +def test_size_input_binds_positional_and_named_constant_arguments(call): + result = transpile_full(HEADER + f"held = {call}\n") + assert result["inputs"] == [{ + "title": "Text size", "type": "string", "default": "small", + "options": ["tiny", "small", "normal"], "supported": True, + }] + + +def test_string_length_binds_constant_values_by_position_and_name(): + body = ( + "positional = str.length(size.small)\n" + "named = str.length(string = size.small)\n" + 'held = str.tostring(positional) + ":" + str.tostring(named)\n' + ) + assert run_emitted_tu(transpile(HEADER + body), STRING_DRIVER, opt="-O0", + label="named and positional string lengths") == "3:5:5\n" * 3 + + +@pytest.mark.parametrize("call", [ + "array.sort(values, direction)", "array.sort(values, order = direction)", + "values.sort(direction)", "values.sort(order = direction)", +]) +def test_sort_order_reaches_array_sort_in_each_binding(call): + body = ( + "values = array.from(3.0, 1.0, 2.0)\n" + "direction = order.descending\n" + call + "\n" + 'held = str.tostring(values.get(0)) + ":" + str.tostring(values.get(1)) ' + '+ ":" + str.tostring(values.get(2))\n' + ) + assert run_emitted_tu(transpile(HEADER + body), STRING_DRIVER, opt="-O0", + label="array sort order bindings") == "5:3:2:1\n" * 3 + + +@pytest.mark.parametrize("call", [ + "array.sort_indices(values, direction)", + "values.sort_indices(order = direction)", +]) +def test_sort_order_reaches_sort_indices(call): + body = ( + "values = array.from(3.0, 1.0, 2.0)\n" + "direction = order.descending\n" + f"indices = {call}\n" + 'held = str.tostring(indices.get(0)) + ":" + str.tostring(indices.get(1)) ' + '+ ":" + str.tostring(indices.get(2))\n' + ) + assert run_emitted_tu(transpile(HEADER + body), STRING_DRIVER, opt="-O0", + label="sort indices order bindings") == "5:0:2:1\n" * 3 + + +@pytest.mark.parametrize("call", [ + "matrix.sort(values, 0, direction)", + "values.sort(order = direction, column = 0)", +]) +def test_sort_order_reaches_matrix_sort_in_each_binding(call): + body = ( + "values = matrix.new(3, 1, 0.0)\n" + "values.set(0, 0, 3.0)\nvalues.set(1, 0, 1.0)\nvalues.set(2, 0, 2.0)\n" + "direction = order.descending\n" + call + "\n" + 'held = str.tostring(values.get(0, 0)) + ":" + str.tostring(values.get(1, 0)) ' + '+ ":" + str.tostring(values.get(2, 0))\n' + ) + assert run_emitted_tu(transpile(HEADER + body), STRING_DRIVER, opt="-O0", + label="matrix sort order bindings") == "5:3:2:1\n" * 3 + + +def test_sort_order_preserves_nominal_identity_and_native_selection(): + body = """pick(array values, direction = order.descending) => + direction == order.ascending ? values.min() : values.max() +values = array.from(3.0, 1.0, 2.0) +direction = close > open ? order.ascending : order.descending +alias = direction +selected = pick(values, alias) +byDefault = pick(values) +ascending = pick(values, direction = order.ascending) +held = str.tostring(selected) + ":" + str.tostring(byDefault) + ":" + str.tostring(ascending) +""" + context = analyze(body) + assert context.symbols.resolve("direction").type_spec == TypeSpec("enum", name="sort_order") + assert context.symbols.resolve("alias").type_spec == TypeSpec("enum", name="sort_order") + cpp = transpile(HEADER + body) + assert run_emitted_tu(cpp, STRING_DRIVER, opt="-O0", + label="sort order default and comparison") == ( + "5:1:3:1\n5:3:3:1\n5:1:3:1\n" + ) + + +def test_sort_order_identity_survives_default_returns_and_wrappers(): + body = ( + "identity(direction = order.descending) => direction\n" + "relay() => identity(order.ascending)\n" + "defaulted = identity()\nforwarded = relay()\n" + 'held = str.tostring(defaulted == order.descending) + ":" ' + '+ str.tostring(forwarded == order.ascending) + ":" ' + '+ str.tostring(defaulted != forwarded)\n' + ) + context = analyze(body) + for name in ("defaulted", "forwarded"): + assert context.symbols.resolve(name).type_spec == TypeSpec("enum", name="sort_order") + assert run_emitted_tu(transpile(HEADER + body), STRING_DRIVER, opt="-O0", + label="nominal sort order returns") == "14:true:true:true\n" * 3 + + +def test_sort_order_method_default_and_result_keep_their_nominal_type(): + body = ( + "type Bucket\n int marker\n" + "method selected(Bucket self, direction = order.descending) =>\n" + " self.marker > 0 ? direction : order.ascending\n" + "bucket = Bucket.new(1)\n" + "defaulted = bucket.selected()\n" + "explicit = bucket.selected(direction = order.ascending)\n" + "held = str.tostring(defaulted != explicit)\n" + ) + context = analyze(body) + for name in ("defaulted", "explicit"): + assert context.symbols.resolve(name).type_spec == TypeSpec("enum", name="sort_order") + assert run_emitted_tu(transpile(HEADER + body), STRING_DRIVER, opt="-O0", + label="nominal sort order method") == "4:true\n" * 3 + + +@pytest.mark.parametrize("body", [ + "string invalid = order.ascending\n", + "direction = order.ascending\ninvalid = direction == \"ascending\"\n", + "invalid = str.length(order.ascending)\n", + 'invalid = input.string(order.ascending, "Direction")\n', + 'consume(string value) => value\ninvalid = consume(order.ascending)\n', + 'consume(value) => str.length(value)\ninvalid = consume(order.ascending)\n', + 'consume(value) =>\n string held = value\n held\ninvalid = consume(order.ascending)\n', + "consume(direction = order.descending) => direction == order.ascending\n" + 'invalid = consume("ascending")\n', + "direction = order.ascending\ninvalid = direction + order.descending\n", + "direction = order.ascending\ninvalid = direction < order.descending\n", + 'values = array.from(1.0, 2.0)\narray.sort(values, "descending")\n', + 'invalid = input.string("ascending", "Direction", options = [order.ascending])\n', + 'type Config\n string direction\n' + 'direction = order.ascending\ninvalid = Config.new(direction)\n', + 'array invalid = array.from(order.ascending)\n', + "identity(direction = order.descending) => direction\n" + "string invalid = identity()\n", +], ids=["declaration", "comparison", "string-builtin", "string-input", "string-parameter", + "inferred-parameter", "local-declaration", "enum-parameter", "arithmetic", + "ordering", "sort-string", "input-options", "constructor-field", "collection-element", + "returned-enum"]) +def test_sort_order_cannot_implicitly_enter_another_value_domain(body): + with pytest.raises(CompileError) as error: + transpile(HEADER + body) + diagnostic = error.value.diagnostics[0] + assert diagnostic.location is not None + assert diagnostic.code != "PF-E0000" + assert "sort_order" in diagnostic.message diff --git a/tests/test_callable_ta_forwarded_length.py b/tests/test_callable_ta_forwarded_length.py index 10e82dca..77833618 100644 --- a/tests/test_callable_ta_forwarded_length.py +++ b/tests/test_callable_ta_forwarded_length.py @@ -11,6 +11,7 @@ from pineforge_codegen import transpile from pineforge_codegen.errors import CompileError +from tests._compile import compile_cpp from tests.test_runtime_var_initialization import _compile_and_run @@ -640,3 +641,16 @@ def test_recursive_stateful_callable_is_rejected_before_instance_expansion() -> CompileError, match="Recursive stateful callable cycle" ): transpile(source) + + +@pytest.mark.parametrize("average", ["rma", "ema", "sma"]) +def test_unreachable_forwarding_keeps_live_input_backed_lengths(average) -> None: + source = f'''//@version=6 +strategy("reachable input length") +filtered(float sample, int window) => ta.{average}(sample, window) +unreached(float sample, int window) => filtered(filtered(sample, window), window) +period = input.int(7, "Period", minval = 1) +observed = filtered(window = period, sample = close) +plot(observed) +''' + compile_cpp(transpile(source, check_support=True), label="reachable-input-backed-ta-length") diff --git a/tests/test_codegen_validation_fixes.py b/tests/test_codegen_validation_fixes.py index c32255cb..a0fba6cd 100644 --- a/tests/test_codegen_validation_fixes.py +++ b/tests/test_codegen_validation_fixes.py @@ -1360,14 +1360,13 @@ def test_str_tonumber_and_length_typed_numeric(): def test_input_string_unresolvable_default_is_safe(): - # input.string(size.tiny, ...) — size.tiny is unresolved; the default must - # NOT pass a null char* to get_input_string (strlen crash). It coerces to - # an empty string. + # The known Pine string size.tiny keeps its value and the original + # null-pointer safety guarantee. Retain this regression's historical id. cpp = _cpp( "sz = input.string(size.tiny, \"Label Size\", options=[size.tiny])\n" "plot(close)" ) - assert 'get_input_string("Label Size", std::string(""))' in cpp + assert 'get_input_string("Label Size", std::string("tiny"))' in cpp assert 'get_input_string("Label Size", 0)' not in cpp diff --git a/tests/test_diagnostic_notes.py b/tests/test_diagnostic_notes.py index 546959c8..ab442ee8 100644 --- a/tests/test_diagnostic_notes.py +++ b/tests/test_diagnostic_notes.py @@ -25,6 +25,26 @@ BEFORE = json.loads((FIXTURES / "catalog.before.json").read_text())["codes"] PRODUCT_BEFORE = json.loads((FIXTURES / "product.before.json").read_text()) CATALOG_ENTRY_ADDITIONS = { + # Reserved builtin-enum identity copied from the stock-generated catalog. + "PF-E2093": { + "severity": "error", + "area": "analysis", + "message": "Cannot use {actual} as {expected} for {context}.", + "hint": None, + "explanation": "A built-in enum retains its nominal Pine type independently of its runtime storage. Assignments, arguments and defaults must use the expected semantic type.", + "args": { + "actual": { + "kind": "identifier", + }, + "expected": { + "kind": "identifier", + }, + "context": { + "kind": "identifier", + }, + }, + "user_message": "Cannot use {actual} as {expected} for {context}.", + }, # Pine v6's order-condition migration has its own stable identity. "PF-E2081": { "severity": "error", @@ -208,6 +228,7 @@ def test_exact_catalog_delta_and_legacy_pin(): assert set(legacy) == set(BEFORE) assert set(current) == set(catalog) assert set(current) - set(legacy) == CATALOG_ADDITIONS + assert current["PF-E2093"] == "b2d5cd10ea46245c7a9e31ff15db9db7d8538221d5443c9cc15ba0013a19e93f" assert set(LOOKAHEAD_MIGRATION) == {"PF-E1085", "PF-W1085"} assert {code for code in legacy if legacy[code] != current[code]} == ( NOTE_CODES | set(LOOKAHEAD_MIGRATION)) @@ -264,6 +285,20 @@ def test_catalog_delta_rejects_malformed_candidate(mutation): _assert_catalog_delta(bad) +def test_catalog_delta_rejects_unknown_code(): + catalog = copy.deepcopy(diagnostics_catalog()["codes"]) + _assert_catalog_delta(catalog) + for code, template in ( + ("PF-E9999", "PF-E2093"), + ("PF-W9999", "PF-W1503"), + ): + assert code not in catalog + candidate = copy.deepcopy(catalog) + candidate[code] = copy.deepcopy(catalog[template]) + with pytest.raises(AssertionError): + _assert_catalog_delta(candidate) + + @pytest.mark.parametrize("code", ["PF-E1085", "PF-W1085"]) @pytest.mark.parametrize("field", [ "message", "hint", "explanation", "args", "severity", "area", "user_message", diff --git a/tests/test_input_manifest_values.py b/tests/test_input_manifest_values.py index 849ed9d8..7a68b586 100644 --- a/tests/test_input_manifest_values.py +++ b/tests/test_input_manifest_values.py @@ -13,6 +13,7 @@ from pineforge_codegen import transpile_full from pineforge_codegen.codegen.checked_settings import _numeric_setting_value +from pineforge_codegen.errors import CompileError ROOT = Path(__file__).resolve().parents[1] @@ -59,7 +60,9 @@ def test_enum_inputs_publish_their_choices_and_keep_the_member_default(): @pytest.mark.parametrize("literal,value", [ ("alert.freq_all", "all"), ("alert.freq_once_per_bar", "once_per_bar"), ("currency.USD", "USD"), ("format.price", "price"), - ("order.ascending", "ascending"), ("session.regular", "regular")]) + ("session.regular", "regular"), ("size.small", "small"), + ("position.top_right", "top_right"), ("line.style_dashed", "dsh"), + ("label.style_label_up", "lup"), ("shape.circle", "shape_circle")]) def test_builtin_string_constants_publish_their_runtime_values(literal, value): inputs = _inputs(f'a = input.string({literal}, "A", options=[{literal}, "other"])\n') assert inputs["A"]["default"] == value @@ -67,6 +70,11 @@ def test_builtin_string_constants_publish_their_runtime_values(literal, value): assert inputs["A"]["supported"] is True +def test_sort_order_is_not_a_string_input_default(): + with pytest.raises(CompileError, match="sort_order"): + _inputs('a = input.string(order.ascending, "A")\n') + + def test_named_string_constants_publish_their_values(): inputs = _inputs('SD = "abc"\na = input.string(SD, "A")\n' 'b = input.string("x", "B", options=["x", SD])\n') diff --git a/tests/test_input_metadata_receipt.py b/tests/test_input_metadata_receipt.py index deb8816c..03f2622b 100644 --- a/tests/test_input_metadata_receipt.py +++ b/tests/test_input_metadata_receipt.py @@ -178,19 +178,50 @@ def test_public_corpus_input_metadata_inventory_is_available(): def test_unrepresentable_strings_match_receipt_in_both_envelopes(): + # Preserve the original mixed fixture: its nominal sort_order default is + # TV-invalid as a string input, so both public APIs must refuse it. source = (ROOT / "tests" / "fixtures" / "input_metadata.pine").read_text() + with pytest.raises(CompileError, match="sort_order as string") as error: + transpile_full(source) + diagnostic = error.value.diagnostics[0] + assert diagnostic.code == "PF-E2093" + assert (diagnostic.location.line, diagnostic.location.col) == (13, 32) + assert diagnostic.args == { + "actual": "sort_order", "expected": "string", + "context": "argument 'defval' of 'input.string'", + } + envelope = json.loads(GLUE(source)) + assert envelope["ok"] is False + assert "cpp" not in envelope + assert "inputs" not in envelope + errors = [item for item in envelope["diagnostics"] if item["severity"] == "error"] + assert [(item["code"], item["line"], item["col"], item["args"]) for item in errors] == [ + (diagnostic.code, 13, 32, diagnostic.args), + ] + + +def test_supported_string_constants_match_receipt_in_both_envelopes(): + source = (ROOT / "tests" / "fixtures" / "input_metadata_strings.pine").read_text() inputs = transpile_full(source)["inputs"] - assert inputs == json.loads(GLUE(source))["inputs"] + envelope = json.loads(GLUE(source)) + assert envelope["ok"] is True + assert inputs == envelope["inputs"] by_key = {entry["title"]: entry for entry in inputs} - for key in ("Size", "Position"): - assert by_key[key]["supported"] is False - assert by_key[key]["default"] == "" - assert by_key[key]["options"] == [] - assert by_key["Mixed"]["supported"] is False - assert by_key["Mixed"]["default"] == "literal" - assert by_key["Mixed"]["options"] == [] - assert all(entry["supported"] is True for entry in inputs - if entry["title"] not in ("Size", "Position", "Mixed")) + assert list(by_key) == [ + "Length", "Threshold", "Enabled", "Source", "Side", "Symbol", + "Frequency", "Direction", "Currency", "Session", "Format", + "Size", "Position", "Mixed", + ] + for key, default, options in [ + ("Size", "small", ["small", "tiny"]), + ("Position", "top_right", ["top_right", "bottom_right"]), + ("Mixed", "literal", ["literal", "small"]), + ]: + assert by_key[key] == { + "title": key, "type": "string", "default": default, + "options": options, "supported": True, + } + assert all(entry["supported"] is True for entry in inputs) STRING_VALUES = [ diff --git a/tests/test_input_string_var_default.py b/tests/test_input_string_var_default.py index 145b84e9..d075b606 100644 --- a/tests/test_input_string_var_default.py +++ b/tests/test_input_string_var_default.py @@ -438,13 +438,12 @@ def test_a_variable_holding_a_string_constant_carries_it(): 'value = input.string(size.small, "Choice")\n', 'var D = size.small\nvalue = input.string(D, "Choice")\n', ]) -def test_visual_constants_keep_their_empty_default(body): - """Inert visual constants retain the safe legacy coercion and unsupported descriptor.""" +def test_visual_string_constants_keep_their_default_value(body): + """A direct constant or immutable carrier supplies the same supported value.""" full = transpile_full(HEAD + body) - assert 'get_input_string("Choice", std::string(""))' in full["cpp"] + assert 'get_input_string("Choice", std::string("small"))' in full["cpp"] assert full["inputs"] == [{ - "title": "Choice", "type": "string", "default": "", - "options": [], "supported": False, + "title": "Choice", "type": "string", "default": "small", "supported": True, }] diff --git a/tests/test_lower_timeframe_tuple_types.py b/tests/test_lower_timeframe_tuple_types.py new file mode 100644 index 00000000..00fac391 --- /dev/null +++ b/tests/test_lower_timeframe_tuple_types.py @@ -0,0 +1,52 @@ +"""Lower-timeframe tuple elements retain independent array types.""" + +import pytest + +from pineforge_codegen import transpile +from pineforge_codegen.analyzer import Analyzer +from pineforge_codegen.lexer import Lexer +from pineforge_codegen.parser import Parser +from pineforge_codegen.symbols import TypeSpec +from tests._compile import compile_cpp + + +@pytest.mark.parametrize("helper", [False, True], ids=["inline", "helper"]) +@pytest.mark.parametrize("named", [False, True], ids=["positional", "named"]) +def test_lower_timeframe_tuple_collection_types_are_per_element(helper, named): + payload = '[close, bar_index, close > open, "sample", color.blue]' + declaration = "" + if helper: + declaration = ( + "pack(float price, int ordinal, bool rising) =>\n" + ' [price, ordinal, rising, "sample", color.blue]\n' + ) + payload = "pack(close, bar_index, close > open)" + arguments = ( + f'expression = {payload}, timeframe = "15", symbol = syminfo.tickerid' + if named else f'syminfo.tickerid, "15", {payload}' + ) + source = ( + '//@version=6\nstrategy("typed lower tuple")\n' + declaration + + f"[prices, positions, rising, labels, colors] = request.security_lower_tf({arguments})\n" + + "plot(array.size(prices) + array.size(positions) + array.size(rising)" + + " + array.size(labels) + array.size(colors))\n" + ) + ast = Parser(Lexer(source).tokenize(), source=source).parse() + context = Analyzer(ast).analyze() + for name, inner in zip( + ("prices", "positions", "rising", "labels", "colors"), + ("float", "int64", "bool", "string", "color"), + ): + assert context.collection_types[name] == TypeSpec.array(TypeSpec.primitive(inner)) + compile_cpp(transpile(source, check_support=True), label="lower-tuple-per-element-types") + + +def test_lower_timeframe_tuple_arrays_keep_lexical_method_dispatch(): + source = """//@version=6 +strategy("local lower tuple") +readSize() => + [prices, rising] = request.security_lower_tf(syminfo.tickerid, "15", [close, close > open]) + prices.size() + rising.size() +plot(readSize()) +""" + compile_cpp(transpile(source, check_support=True), label="lower-tuple-lexical-array-dispatch") diff --git a/tests/test_mixed_block_indentation.py b/tests/test_mixed_block_indentation.py new file mode 100644 index 00000000..23d0790b --- /dev/null +++ b/tests/test_mixed_block_indentation.py @@ -0,0 +1,40 @@ +"""Mixed indentation preserves nested block structure and accepted wraps.""" + +import pytest + +from pineforge_codegen.ast_nodes import IfStmt, VarDecl +from pineforge_codegen.lexer import Lexer, TokenType +from pineforge_codegen.parser import Parser + + +@pytest.mark.parametrize( + "outer,inner", + [("\t", "\t "), (" ", " \t"), ("\t", "\t\t"), (" ", " ")], +) +def test_mixed_complete_levels_keep_nested_ast_and_dedent(outer, inner): + source = ( + "if close > open\n" + outer + "if high > close\n" + + inner + "observed = close\n" + + "after = open\n" + ) + tokens = Lexer(source).tokenize() + assert [token.line for token in tokens if token.type == TokenType.INDENT] == [2, 3] + assert [token.line for token in tokens if token.type == TokenType.DEDENT] == [4, 4] + program = Parser(tokens, source=source).parse() + outer_node, after = program.body + assert isinstance(outer_node, IfStmt) + (inner_node,) = outer_node.body + assert isinstance(inner_node, IfStmt) + (observed,) = inner_node.body + assert isinstance(observed, VarDecl) + assert observed.name == "observed" + assert isinstance(after, VarDecl) + assert after.name == "after" + + +@pytest.mark.parametrize("extra_spaces", [1, 2, 3]) +def test_previously_accepted_tab_first_incomplete_widths_keep_their_block(extra_spaces): + source = "if close > open\n\t" + " " * extra_spaces + "observed = close\n" + program = Parser(Lexer(source).tokenize(), source=source).parse() + (declaration,) = program.body[0].body + assert declaration.name == "observed" diff --git a/tests/test_persistent_declaration_qualifiers.py b/tests/test_persistent_declaration_qualifiers.py new file mode 100644 index 00000000..c47db1d6 --- /dev/null +++ b/tests/test_persistent_declaration_qualifiers.py @@ -0,0 +1,88 @@ +"""Persistence and qualifiers occupy separate positions in the declaration AST.""" + +import pytest + +from pineforge_codegen.ast_nodes import VarDecl +from pineforge_codegen.lexer import Lexer +from pineforge_codegen.parser import Parser + + +@pytest.mark.parametrize("storage", ["var", "varip"]) +@pytest.mark.parametrize( + "qualifier,hint,initializer,expected_hint", + [ + ("const", "string", '"x"', "string"), + ("const", "float", "1.5", "float"), + ("const", "int", "3", "int"), + ("const", "bool", "true", "bool"), + ("const", "color", "color.blue", "color"), + ("simple", "int", "3", "int"), + ("series", "float", "close", "float"), + ("series", "float[]", "array.new_float(0)", "array"), + ("series", "array", "array.new_float(0)", "array"), + ("series", "Sample", "Sample.new()", "Sample"), + ("series", "lib.Sample", "lib.newSample()", "lib.Sample"), + ], +) +def test_persistence_then_qualifier_keeps_the_declaration( + storage, qualifier, hint, initializer, expected_hint, +): + source = f"{storage} {qualifier} {hint} held = {initializer}\n" + program = Parser(Lexer(source).tokenize(), source=source).parse() + assert len(program.body) == 1 + declaration = program.body[0] + assert isinstance(declaration, VarDecl) + assert declaration.name == "held" + assert declaration.type_hint == expected_hint + assert (declaration.is_var, declaration.is_varip) == ( + storage == "var", storage == "varip", + ) + assert declaration.annotations["qualifier"] == qualifier + assert (declaration.loc.line, declaration.loc.col) == (1, 1) + ((name, location),) = declaration.annotations["declaration_names"] + assert name == "held" + assert (location.line, location.col) == (1, source.index("held") + 1) + + +@pytest.mark.parametrize("storage", ["var", "varip"]) +@pytest.mark.parametrize("word", ["const", "simple", "series"]) +@pytest.mark.parametrize("hint", ["", "int ", "Sample "]) +def test_contextual_qualifier_words_in_the_name_slot_are_names(storage, word, hint): + source = f"{storage} {hint}{word} = 3\n" + declaration = Parser(Lexer(source).tokenize(), source=source).parse().body[0] + assert isinstance(declaration, VarDecl) + assert declaration.name == word + assert declaration.type_hint == (hint.strip() or None) + assert "qualifier" not in declaration.annotations + + +@pytest.mark.parametrize("storage", ["var", "varip"]) +@pytest.mark.parametrize("word", ["const", "simple", "series"]) +def test_contextual_qualifier_word_as_a_type_keeps_its_type(storage, word): + source = f"{storage} {word} held = {word}.new()\n" + declaration = Parser(Lexer(source).tokenize(), source=source).parse().body[0] + assert declaration.type_hint == word + assert declaration.name == "held" + assert "qualifier" not in declaration.annotations + + +@pytest.mark.parametrize("storage", ["var", "varip"]) +@pytest.mark.parametrize("header", ["if close > open", "make() =>"]) +def test_local_persistent_qualified_declaration_keeps_storage(storage, header): + source = f"{header}\n {storage} const float held = 1.5\n" + declaration = Parser(Lexer(source).tokenize(), source=source).parse().body[0].body[0] + assert isinstance(declaration, VarDecl) + assert declaration.annotations["qualifier"] == "const" + assert (declaration.is_var, declaration.is_varip) == ( + storage == "var", storage == "varip", + ) + assert (declaration.loc.line, declaration.loc.col) == (2, 5) + + +@pytest.mark.parametrize("storage", ["var", "varip"]) +def test_library_persistent_const_keeps_constant_provenance(storage): + source = f"{storage} const float held = 1.5\n" + program = Parser(Lexer(source).tokenize(), source=source, library=True).parse() + declaration = program.body[0] + assert declaration.annotations["qualifier"] == "const" + assert declaration.annotations["declared_const"] is True diff --git a/tests/test_requested_na_helper_bindings.py b/tests/test_requested_na_helper_bindings.py new file mode 100644 index 00000000..22de8183 --- /dev/null +++ b/tests/test_requested_na_helper_bindings.py @@ -0,0 +1,52 @@ +"""Requested builtin observers retain their helper's parameter bindings.""" + +import pytest + +from pineforge_codegen import transpile +from tests._compile import compile_cpp +from tests.test_v6_declaration_request_compilation import ( + GLOBAL_TUPLE_HELPER, GLOBAL_TUPLE_HISTORY, TIMEFRAME_HELPER, TIME_HELPER, +) + + +@pytest.mark.parametrize( + "source", [TIMEFRAME_HELPER, GLOBAL_TUPLE_HELPER], + ids=["timeframe-change", "global-tuple"], +) +@pytest.mark.parametrize( + "definition", + [ + "valid(x) => nz(x)", + "valid(float x) => na(x) ? 0 : 1", + "valid(float x) => nz(source = x, replacement = 0.0)", + "valid(x) => x > 0 ? 1 : 0", + "valid(x) => math.abs(x)", + ], + ids=["nz", "typed-na", "typed-named-nz", "comparison-control", "math-control"], +) +def test_observer_bindings_compose_with_requested_context(source, definition): + source = source.replace("valid(x) => na(x) ? 0 : 1", definition, 1) + compile_cpp(transpile(source, check_support=True), label="requested-observer-binding") + + +@pytest.mark.parametrize( + "payload", + ["valid(close)", 'time("W")', 'timeframe.change("W") ? 1 : 0'], + ids=["helper-alone", "time-alone", "change-alone"], +) +def test_independent_requested_payload_parts_remain_compilable(payload): + source = TIME_HELPER.replace('valid(close) * time("W")', payload, 1) + compile_cpp(transpile(source, check_support=True), label="independent-requested-payload") + + +@pytest.mark.parametrize( + "declaration", + [ + '[upper, lower, ignored] = ta.macd(close, 12, 26, 9)', + '[upper, lower] = ta.supertrend(3.0, 10)', + ], + ids=["macd-control", "supertrend-control"], +) +def test_builtin_tuple_member_history_keeps_requested_storage(declaration): + source = GLOBAL_TUPLE_HISTORY.replace("[upper, lower] = channel()", declaration, 1) + compile_cpp(transpile(source, check_support=True), label="requested-builtin-tuple-history") diff --git a/tests/test_security_user_tuple_global.py b/tests/test_security_user_tuple_global.py index 4d856dfa..e0047fa6 100644 --- a/tests/test_security_user_tuple_global.py +++ b/tests/test_security_user_tuple_global.py @@ -4,8 +4,8 @@ bar when a request.security payload reads `g`, directly or through a function that reads the global: the global is a plain value of the requested context. A function with state (a ta.* call, a `var`, a request) advances that state once -per evaluation, so reading its tuple once per element would advance it twice; -that case, and the history of an element, stay refused. +per evaluation. Every element shares its declaration's requested-context +sample, including when the payload reads a member more than once. The value rows hold the requested series to numbers written here from the source and the 1-day alignment of a 15-minute chart (96 chart bars per day), @@ -16,11 +16,12 @@ from __future__ import annotations import math +import re import pytest from pineforge_codegen import transpile -from pineforge_codegen.errors import CompileError +from tests._compile import compile_cpp from tests._e2e import ( Build, chart_feed_head, execute_all, per_bar_mismatches, skip_unless_e2e_env, ) @@ -68,10 +69,19 @@ def test_a_pure_user_tuple_element_translates_in_a_payload(name, declarations, o @pytest.mark.parametrize("name,declarations,payload", STATEFUL, ids=[row[0] for row in STATEFUL]) -def test_a_stateful_user_tuple_element_stays_refused(name, declarations, payload): - with pytest.raises(CompileError, match="an element of a tuple declaration it cannot re-evaluate") as caught: - transpile(_source(declarations, payload, "close", "60")) - assert [d.code for d in caught.value.diagnostics] == ["PF-E3044"] +def test_a_stateful_user_tuple_declaration_is_sampled_once(name, declarations, payload): + cpp = transpile( + _source(declarations, f"{payload} + {payload} + b", "close", "60"), + check_support=True, + ) + evaluator = cpp.split( + "void _eval_security_0(const Bar& bar, bool is_complete) {", 1, + )[1].split("\n }\n", 1)[0] + samples = re.findall(r"const auto (_pf_sec0_tuple_\d+_value) = ", evaluator) + assert len(samples) == 1 + assert evaluator.count(f"std::get<0>({samples[0]})") == 2 + assert f"std::get<1>({samples[0]})" in evaluator + compile_cpp(cpp, label=f"requested-tuple-{name}") def _column(outcome, name): diff --git a/tests/test_string_settings_constants.py b/tests/test_string_settings_constants.py index d0799f3c..71b01a35 100644 --- a/tests/test_string_settings_constants.py +++ b/tests/test_string_settings_constants.py @@ -9,6 +9,7 @@ import pytest from pineforge_codegen import transpile +from pineforge_codegen.errors import CompileError from pineforge_codegen.ast_nodes import Identifier, MemberAccess, StringLiteral from pineforge_codegen.codegen.checked_settings import _string_setting_arg from pineforge_codegen.codegen.tables import ( @@ -28,20 +29,20 @@ "alert": ("freq_all", "all"), "currency": ("USD", "USD"), "format": ("mintick", "mintick"), - "order": ("ascending", "ascending"), + "order": ("ascending", None), "session": ("regular", "regular"), - "position": ("top_right", None), - "size": ("small", None), - "line": ("style_solid", None), + "position": ("top_right", "top_right"), + "size": ("small", "small"), + "line": ("style_solid", "sol"), "box": ("style_dashed", None), - "label": ("style_label_up", None), + "label": ("style_label_up", "lup"), "extend": ("right", None), "font": ("family_monospace", None), "hline": ("style_dotted", None), "location": ("abovebar", None), "plot": ("style_line", None), "scale": ("right", None), - "shape": ("triangleup", None), + "shape": ("triangleup", "shape_triangle_up"), "text": ("align_left", None), "xloc": ("bar_index", None), "yloc": ("price", None), @@ -165,6 +166,11 @@ def test_each_builtin_constant_family_is_string_or_unsupported(namespace, placem constant = f"{namespace}.{member}" arguments = (f'{constant}, "Choice"' if placement == "default" else f'"literal", "Choice", options=["literal", {constant}]') + if namespace == "order": + with pytest.raises(CompileError, match="sort_order"): + transpile(f'//@version=6\nstrategy("constants")\n' + f'choice = input.string({arguments})\n', check_support=False) + return cpp = transpile(f'//@version=6\nstrategy("constants")\n' f'choice = input.string({arguments})\nplot(close)\n', check_support=False) metadata = _metadata(cpp, "Choice") @@ -193,6 +199,12 @@ def test_unknown_builtin_members_do_not_fabricate_pine_strings(namespace): assert _string_setting_arg(expression, 'std::string("unknown")') is None +@pytest.mark.parametrize("member", ["ascending", "descending"]) +def test_nominal_enum_payload_is_not_a_string_setting_literal(member): + expression = MemberAccess(object=Identifier(name="order"), member=member) + assert _string_setting_arg(expression, f'std::string("{member}")') is None + + @pytest.mark.parametrize("optimization", ["-O0", "-O2"]) def test_unknown_name_echo_receipts_preserve_runtime_values(optimization): source = '''//@version=6 @@ -241,7 +253,7 @@ def test_known_constant_options_match_getters_and_legacy_values_are_unchanged(op source = '''//@version=6 strategy("constant settings") frequency = input.string(alert.freq_all, "Frequency", options=[alert.freq_all, alert.freq_once_per_bar]) -direction = input.string(order.ascending, "Direction", options=[order.ascending, order.descending]) +direction = input.string("ascending", "Direction", options=["ascending", "descending"]) currencyChoice = input.string(currency.USD, "Currency", options=[currency.USD, currency.EUR]) sessionChoice = input.string(session.regular, "Session", options=[session.regular, session.extended]) numberFormat = input.string(format.mintick, "Format", options=[format.mintick, format.percent]) @@ -264,8 +276,7 @@ class InputProbe : public GeneratedStrategy { InputProbe strategy; char error[256]{}; assert(strategy.read("Frequency", "all") == std::string("all")); - assert(strategy.read("Size", "") == std::string("")); - assert(strategy.read("Size", "") != std::string("small")); + assert(strategy.read("Size", "small") == std::string("small")); assert(strategy_set_input_checked(&strategy, "Frequency", "once_per_bar", error, sizeof(error)) == PF_SETTINGS_OK); assert(strategy.read("Frequency", "all") == std::string("once_per_bar")); @@ -286,13 +297,13 @@ class InputProbe : public GeneratedStrategy { sizeof(error)) == PF_SETTINGS_INVALID_ARGUMENT); assert(strategy.read("Frequency", "all") == std::string("once_per_bar")); assert(strategy_set_input_checked(&strategy, "Size", "small", error, - sizeof(error)) == PF_SETTINGS_UNSUPPORTED); - assert(strategy.read("Size", "") == std::string("")); + sizeof(error)) == PF_SETTINGS_OK); + assert(strategy.read("Size", "small") == std::string("small")); strategy_set_input(&strategy, "Size", "small"); assert(strategy.read("Size", "") == std::string("small")); assert(strategy_set_input_checked(&strategy, "Size", "tiny", error, - sizeof(error)) == PF_SETTINGS_UNSUPPORTED); - assert(strategy.read("Size", "") == std::string("small")); + sizeof(error)) == PF_SETTINGS_OK); + assert(strategy.read("Size", "small") == std::string("tiny")); std::cout << strategy._pf_settings_receipt(); } ''' @@ -310,9 +321,10 @@ class InputProbe : public GeneratedStrategy { assert inputs[name]["default"] == default assert inputs[name]["options"] == options assert inputs["Frequency"]["effective_value"] == "once_per_bar" - assert inputs["Size"]["supported"] is False - assert inputs["Size"]["default"] == "" and inputs["Size"]["options"] == [] - assert inputs["Size"]["effective_value"] == "small" + assert inputs["Size"]["supported"] is True + assert inputs["Size"]["default"] == "small" + assert inputs["Size"]["options"] == ["small", "tiny"] + assert inputs["Size"]["effective_value"] == "tiny" @pytest.mark.parametrize("sha12", SOURCE_INPUTS) @@ -334,7 +346,8 @@ def test_population_source_settings_receipt(sha12): assert all(isinstance(option, str) for option in entry["options"]) for name in SOURCE_INPUTS[sha12]: entry = inputs[name] - assert entry["supported"] is False - assert entry["default"] == entry["effective_value"] == "" - assert entry["options"] == [] - assert '"string", std::string(""), {}' in _metadata(cpp, name) + assert entry["supported"] is True + assert entry["default"] == entry["effective_value"] + assert entry["default"] != "" + assert all(isinstance(option, str) and option != "" for option in entry["options"]) + assert ", true, {}," in _metadata(cpp, name) diff --git a/tests/test_support_checker_const_namespaces.py b/tests/test_support_checker_const_namespaces.py index 4250040e..0b382db8 100644 --- a/tests/test_support_checker_const_namespaces.py +++ b/tests/test_support_checker_const_namespaces.py @@ -1,11 +1,9 @@ """Loud-rejection sweep for constant-only namespaces (audit items A1-A3, A6). -Constant-namespace members (plot.style_*, text.align_*, shape.*, ...) are -only legitimate as arguments to parse-and-skip visual calls or inside the -strategy() declaration. As FREE EXPRESSIONS they used to fall through codegen -to ``std::string("")`` while the analyzer typed them INT — a silent -mismatch. They must now reject with a clear CompileError, while the -argument-context uses keep transpiling. ``alert.freq_*`` are an exception: +Unlowered constant domains keep their existing explicit refusal, while +inventoried string values are ordinary expressions. Shape constants now share +their exact string values and types across value and visual-argument contexts. +``alert.freq_*`` are also strings: TradingView gives them const string values ("all", "once_per_bar", "once_per_bar_close"; tests/test_e2e_alert_freq_values.py), so they are ordinary values. ``barmerge.*`` are another: a script may hold and compare @@ -21,23 +19,27 @@ # --------------------------------------------------------------------------- -# Free-expression reads must reject +# Free-expression reads follow their value contract # --------------------------------------------------------------------------- -@pytest.mark.parametrize("expr", [ - "plot.style_line + 1", - "extend.both", - "font.family_monospace", - "hline.style_dashed", - "location.belowbar", - "scale.right", - "shape.triangleup", - "text.align_left", - "xloc.bar_index", - "yloc.price", +@pytest.mark.parametrize("expr,expected", [ + ("plot.style_line + 1", None), + ("extend.both", None), + ("font.family_monospace", None), + ("hline.style_dashed", None), + ("location.belowbar", None), + ("scale.right", None), + ("shape.triangleup", "shape_triangle_up"), + ("text.align_left", None), + ("xloc.bar_index", None), + ("yloc.price", None), ]) -def test_const_namespace_free_expression_rejected(expr): +def test_const_namespace_free_expression_contract(expr, expected): ns = expr.split(".")[0] + if expected is not None: + cpp = transpile(PRELUDE + f"x = {expr}\n") + assert f'x = std::string("{expected}");' in cpp + return with pytest.raises(CompileError, match=ns): transpile(PRELUDE + f"x = {expr}\n") diff --git a/tests/test_udt_lvalue_alias.py b/tests/test_udt_lvalue_alias.py index b74fda52..87407a47 100644 --- a/tests/test_udt_lvalue_alias.py +++ b/tests/test_udt_lvalue_alias.py @@ -249,6 +249,8 @@ def test_drawing_style_constant_into_string_param_coerced(): plot(close) """ cpp = transpile(src) - assert 'draw(std::string(""))' in cpp + # TradingView reference capture, entry 19: label.style_label_down=ldn. + # Source SHA-256: f5af7fd13cf1d6a645d13e36531caeffc995629fc810a95bbcc5dbb699cde973 + assert "draw(std::string(\"ldn\"))" in cpp # No bare null-pointer string construction. assert "draw(0)" not in cpp diff --git a/tests/test_v6_declaration_request_compilation.py b/tests/test_v6_declaration_request_compilation.py new file mode 100644 index 00000000..1b9ae392 --- /dev/null +++ b/tests/test_v6_declaration_request_compilation.py @@ -0,0 +1,260 @@ +"""Complete strategy compilation for qualified declarations and request payloads. + +These are compile regressions. TradingView value comparisons belong to the +parity population and are separate from this translation-unit check. +""" + +import pytest + +from pineforge_codegen import transpile +from tests._compile import compile_cpp, skip_if_no_compile_env + + +COLOR_DECLARATION = """//@version=6 +strategy("var const declaration", overlay = true, initial_capital = 10000) +var const color LINE_COLOR = color.blue +fast = ta.ema(close, 10) +slow = ta.ema(close, 30) +plot(fast, color = LINE_COLOR) +if ta.crossover(fast, slow) + strategy.entry("Long", strategy.long, qty = 0.01) +if ta.crossunder(fast, slow) + strategy.close("Long") +""" + +FLOAT_DECLARATION = """//@version=6 +strategy("var const float in a calculation", overlay = true, initial_capital = 10000) +var const float BAND = 0.002 +fast = ta.ema(close, 10) +slow = ta.ema(close, 30) +if fast > slow * (1 + BAND) + strategy.entry("Long", strategy.long, qty = 0.01) +if fast < slow + strategy.close("Long") +""" + +LOWER_TUPLE = """//@version=6 +strategy("security_lower_tf with a tuple", overlay = true, initial_capital = 10000) +[closes, opens] = request.security_lower_tf(syminfo.tickerid, "15", [close, open]) +float body = array.size(closes) > 0 ? array.last(closes) - array.first(opens) : close - open +fast = ta.ema(body, 5) +slow = ta.ema(body, 20) +if ta.crossover(fast, slow) + strategy.entry("Long", strategy.long, qty = 0.01) +if ta.crossunder(fast, slow) + strategy.close("Long") +""" + +MIXED_BLOCKS = """//@version=6 +strategy("Repro", overlay = true, initial_capital = 10000, default_qty_value = 0.01) +fast = ta.ema(close, 9) +slow = ta.ema(close, 21) +if ta.crossover(fast, slow) + if close > open + strategy.entry("Long", strategy.long) +if ta.crossunder(fast, slow) + strategy.entry("Short", strategy.short) +""" + +TIME_HELPER = """//@version=6 +strategy("Repro", overlay = true, initial_capital = 10000, default_qty_value = 0.01) +valid(x) => na(x) ? 0 : 1 +weekStart = request.security(syminfo.tickerid, "D", valid(close) * time("W")) +fast = ta.ema(close, 9) +slow = ta.ema(close, 21) +if ta.crossover(fast, slow) and time >= weekStart + strategy.entry("Long", strategy.long) +if ta.crossunder(fast, slow) + strategy.entry("Short", strategy.short) +""" + +TIMEFRAME_HELPER = """//@version=6 +strategy("Repro", overlay = true, initial_capital = 10000, default_qty_value = 0.01) +valid(x) => na(x) ? 0 : 1 +dailyScore = request.security(syminfo.tickerid, "D", valid(close) + (timeframe.change("W") ? 1 : 0)) +fast = ta.ema(close, 9) +slow = ta.ema(close, 21) +if ta.crossover(fast, slow) and dailyScore == 1 + strategy.entry("Long", strategy.long) +if ta.crossunder(fast, slow) + strategy.entry("Short", strategy.short) +""" + +GLOBAL_TUPLE_HELPER = """//@version=6 +strategy("Repro", overlay = true, initial_capital = 10000, default_qty_value = 0.01) +channel() => + upper = ta.highest(high, 20) + lower = ta.lowest(low, 20) + [upper, lower] +[upper, lower] = channel() +valid(x) => na(x) ? 0 : 1 +score() => valid(close) + valid(upper) +dailyScore = request.security(syminfo.tickerid, "D", score()) +fast = ta.ema(close, 9) +slow = ta.ema(close, 21) +if ta.crossover(fast, slow) and dailyScore == 2 + strategy.entry("Long", strategy.long) +if ta.crossunder(fast, slow) + strategy.entry("Short", strategy.short) +""" + +GLOBAL_TUPLE_HISTORY = """//@version=6 +strategy("Repro", overlay = true, initial_capital = 10000, default_qty_value = 0.01) +channel() => + upper = ta.highest(high, 20) + lower = ta.lowest(low, 20) + [upper, lower] +[upper, lower] = channel() +dailyAbove = request.security(syminfo.tickerid, "D", close > upper[1]) +fast = ta.ema(close, 9) +slow = ta.ema(close, 21) +if ta.crossover(fast, slow) and dailyAbove + strategy.entry("Long", strategy.long) +if ta.crossunder(fast, slow) + strategy.entry("Short", strategy.short) +""" + +UNUSED_FORWARDER = """//@version=6 +strategy("Repro", overlay = true, initial_capital = 10000, default_qty_value = 0.01) +smooth(src, len) => ta.rma(src, len) +smoothTwice(src, len) => smooth(smooth(src, len), len) +fast = smooth(close, 9) +slow = ta.ema(close, 21) +if ta.crossover(fast, slow) + strategy.entry("Long", strategy.long) +if ta.crossunder(fast, slow) + strategy.entry("Short", strategy.short) +""" + + +@pytest.mark.parametrize( + "source", + [ + pytest.param(COLOR_DECLARATION, id="color"), + pytest.param(FLOAT_DECLARATION, id="float"), + pytest.param( + COLOR_DECLARATION.replace("\nvar const ", "\nconst ", 1), + id="color-control", + ), + pytest.param( + FLOAT_DECLARATION.replace("\nvar const ", "\nconst ", 1), + id="float-control", + ), + pytest.param( + COLOR_DECLARATION.replace( + "var const color LINE_COLOR = color.blue", + "varip const int C = 3", + ).replace("color = LINE_COLOR", "color = color.blue"), + id="varip-int", + ), + ], +) +def test_persistent_qualified_declaration_compiles(source): + skip_if_no_compile_env() + compile_cpp(transpile(source, check_support=True), label="persistent-qualified-declaration") + + +@pytest.mark.parametrize( + "declaration", + [ + pytest.param( + '[closes, opens] = request.security_lower_tf(syminfo.tickerid, "15", [close, open])', + id="inline", + ), + pytest.param( + 'closeOpen() =>\n [close, open]\n' + '[closes, opens] = request.security_lower_tf(syminfo.tickerid, "15", closeOpen())', + id="helper", + ), + pytest.param( + 'closes = request.security_lower_tf(syminfo.tickerid, "15", close)\n' + 'opens = request.security_lower_tf(syminfo.tickerid, "15", open)', + id="separate-control", + ), + ], +) +def test_lower_timeframe_tuple_arrays_compile(declaration): + skip_if_no_compile_env() + source = LOWER_TUPLE.replace(LOWER_TUPLE.splitlines()[2], declaration, 1) + compile_cpp(transpile(source, check_support=True), label="lower-timeframe-tuple-arrays") + + +@pytest.mark.parametrize( + "outer,inner", + [ + pytest.param("\t", "\t ", id="tab-then-spaces"), + pytest.param(" ", " \t", id="spaces-then-tab"), + pytest.param("\t", "\t\t", id="tabs-control"), + pytest.param(" ", " ", id="spaces-control"), + ], +) +def test_mixed_nested_block_compiles(outer, inner): + skip_if_no_compile_env() + source = MIXED_BLOCKS.replace( + "\tif close > open\n\t strategy.entry", + outer + "if close > open\n" + inner + "strategy.entry", + 1, + ) + compile_cpp(transpile(source, check_support=True), label="mixed-nested-block") + + +@pytest.mark.parametrize( + "source", + [ + pytest.param(TIME_HELPER, id="time-na"), + pytest.param( + TIME_HELPER.replace("valid(x) => na(x) ? 0 : 1", "valid(x) => nz(x)"), + id="time-nz", + ), + pytest.param( + TIME_HELPER.replace("valid(x)", "valid(float x)", 1), + id="time-typed", + ), + pytest.param( + TIME_HELPER.replace('valid(close) * time("W")', '(na(close) ? 0 : 1) * time("W")'), + id="time-inline-control", + ), + pytest.param(TIMEFRAME_HELPER, id="timeframe-change"), + pytest.param(GLOBAL_TUPLE_HELPER, id="global-tuple-helper"), + pytest.param( + GLOBAL_TUPLE_HELPER.replace( + "[upper, lower] = channel()\nvalid(x)", "valid(x)", 1, + ).replace( + "score() => valid(close) + valid(upper)", + "score() =>\n [upper, lower] = channel()\n valid(close) + valid(upper)", + 1, + ), + id="local-tuple-control", + ), + pytest.param(GLOBAL_TUPLE_HISTORY, id="global-tuple-history"), + pytest.param( + GLOBAL_TUPLE_HISTORY.replace( + "[upper, lower] = channel()", "upper = ta.highest(high, 20)", 1, + ), + id="direct-history-control", + ), + ], +) +def test_requested_helpers_keep_parameter_and_tuple_bindings(source): + skip_if_no_compile_env() + compile_cpp(transpile(source, check_support=True), label="requested-helper-bindings") + + +@pytest.mark.parametrize( + "source", + [ + pytest.param(UNUSED_FORWARDER, id="uncalled"), + pytest.param( + UNUSED_FORWARDER.replace( + "smoothTwice(src, len) => smooth(smooth(src, len), len)\n", "", 1, + ), + id="removed-control", + ), + pytest.param( + UNUSED_FORWARDER + "plot(smoothTwice(close, 9))\n", id="called-control", + ), + ], +) +def test_unreachable_length_forwarding_does_not_allocate_ta(source): + skip_if_no_compile_env() + compile_cpp(transpile(source, check_support=True), label="unused-length-forwarder") diff --git a/tests/test_v6_reporter_controls.py b/tests/test_v6_reporter_controls.py new file mode 100644 index 00000000..2c03dcea --- /dev/null +++ b/tests/test_v6_reporter_controls.py @@ -0,0 +1,48 @@ +"""Previously accepted control forms keep their native compilation contract.""" + +import pytest + +from pineforge_codegen import transpile +from tests._compile import compile_cpp +from tests.test_i372_form_1 import BUILTIN_CONTROL, BUILTIN_SCRIPT, LOCAL_SCRIPT +from tests.test_input_string_var_default import ( + NONDEFAULT_BRANCH, NONDEFAULT_CONTROL, OPTIONS_CONTROL, VARIABLE_OPTIONS, +) +from tests.test_v6_declaration_request_compilation import COLOR_DECLARATION, UNUSED_FORWARDER + + +def test_conditional_history_call_control_keeps_both_written_sites(): + source = LOCAL_SCRIPT.replace( + "wrapper(x) =>\n s = x\n change(s)\nv = wrapper(close)", + "v = close > open ? change(close) : change(open)", + 1, + ) + compile_cpp(transpile(source, check_support=True), label="conditional-history-call-control") + + +@pytest.mark.parametrize("source", [BUILTIN_SCRIPT, BUILTIN_CONTROL]) +def test_builtin_derived_history_control_has_native_types(source): + compile_cpp(transpile(source, check_support=True), label="builtin-derived-history-control") + + +@pytest.mark.parametrize( + "source", + [NONDEFAULT_BRANCH, NONDEFAULT_CONTROL, VARIABLE_OPTIONS, OPTIONS_CONTROL], + ids=["default-var", "default-control", "options-var", "options-control"], +) +def test_reported_string_inputs_keep_native_compilation(source): + compile_cpp(transpile(source, check_support=True), label="string-input-var-control") + + +def test_persistent_declaration_without_qualifier_keeps_its_storage(): + source = COLOR_DECLARATION.replace("\nvar const color ", "\nvar color ", 1) + compile_cpp(transpile(source, check_support=True), label="persistent-color-control") + + +def test_uncalled_direct_ta_length_control_keeps_compiling(): + source = UNUSED_FORWARDER.replace( + "smoothTwice(src, len) => smooth(smooth(src, len), len)", + "unused(src, len) => ta.rma(src, len) * 2", + 1, + ) + compile_cpp(transpile(source, check_support=True), label="unused-direct-length-control")