Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,69 @@ supported as exact pairs; on the 0.x line they are independent. See the
uses their generated spelling. Declarations, calls and history storage use
the same allocated identities. Regenerate C++ and relink affected strategies.

- Requested helpers preserve declared string argument types when inferring
local variables, including string `na` values, so these expressions emit
string storage and string missing values.

- String `+`, `+=` and string-array joins recompose adjacent UTF-16 surrogate
halves, including across array parts and separators. Unpaired halves are
preserved; string search predicates compare UTF-16 units. Regenerate C++
and relink affected strategies.
- Persistent string locals initialized from helper parameters retain their
storage type in compound assignments and `request.security()` expressions.
- Array joins retain string return types through helper and method forwarding.
Named, mixed and reordered receiver/separator arguments use their declared
slots; an omitted separator is empty. Numeric elements use Pine number
formatting without fixed trailing zeros, and receiver and separator are
evaluated once in parameter order, including for empty arrays.
Builtin joins refuse boolean arrays during translation; authored methods
retain their own overloads and defaults.
- `str.length` validates named arguments against its declared `string`
parameter; unknown names produce the standard located argument diagnostic.
- `str.upper` and `str.lower` use pinned Unicode 14.0.0 mappings and properties,
including supplementary letters, expansions and contextual final sigma.
Translating Python versions and the native locale do not select the data.
The context crosses decimal runs; locale tailoring and normalization are
not applied. The data pin does not identify TradingView's whole dialect.
- `str.match` lowers its literals, character classes, ranges and repeated
atoms as Unicode characters, including complete supplementary characters,
so matches no longer depend on UTF-8 byte lengths. Word classes remain ASCII;
string lengths and substring offsets still count UTF-16 units.
Matches return the complete match while preserving user captures and
backreferences for pattern evaluation.
Regenerate C++ and relink affected strategies to use this correction.
- String input defaults, options and titles preserve embedded NUL bytes in
getters, metadata and checked receipts. Internal timeframe and footprint
column consumers carry literal byte lengths; their validation is unchanged.
Public C input setters retain their NUL-terminated contract.
- `str.split` applies string context to missing arguments, including literal
`na` in positional, named and helper calls.

- `str.pos` returns integer `na` when the substring is absent, so `na()` detects
the missing match and `nz()` uses its replacement (#382, item 5). It returned
`-1`; found positions retain their UTF-16 offsets. The TradingView expectation
comes from the reporter's readings. Regenerate C++ and relink affected
strategies.

- `str.substring` checks an explicit `na` end before boundary arithmetic and
returns the engine's canonical string `na` (stored as the empty string).
An `na` start becomes zero, including a wide integer `na` propagated from
`str.pos(...) + 1`, so the two-argument form returns the whole text. Finite
endpoints are ordered before computing the count; an end before the start
stops the run. UTF-16 slicing, end clamping and invalid-begin stops use the
existing helper. The oracle is the reporter's #382 item 9 readings.
- `str.substring` refuses a provably negative constant `begin_pos` during
analysis, including integer arithmetic, constant aliases and named arguments.
Dynamic negative starts retain their runtime error, and missing positions
retain their existing handling.
- `str.trim` additionally removes VT (U+000B), FF (U+000C), U+00A0, U+1680,
U+2003, U+2028, U+3000 and U+FEFF from the ends. Interior bytes and kept
characters are preserved; other Unicode spaces remain unchanged (#382, item 8).

- Reads of authored variables named like bar fields use their lexical
declarations and types. A string binding passed to `array.from()` previously
read the numeric bar field, which could fail to compile or produce a wrong
value. Leaving a block that shadows a builtin restores the builtin.
- A zone name the engine rejects now makes calendar-form `timestamp()` stop the
run at its first evaluation, instead of silently calculating a UTC value.
The paired engine reports a typed timezone error identifying the failed bar.
Expand Down
18 changes: 14 additions & 4 deletions pineforge_codegen/analyzer/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -6474,24 +6474,26 @@ def _visit_MethodDef(self, node) -> PineType:
def _top_level_branch_needs_lexical_scope(
self, body: list[ASTNode]
) -> bool:
"""Whether a top-level branch declares over a direct script binding.
"""Whether a top-level branch declares over a script binding or builtin.

Functions already own a symbol-table scope and ``for``/``for in``
create one explicitly. Top-level if/while/switch branches historically
reused the global Scope, so a local declaration could overwrite the
analyzer's outer Symbol even though codegen emits a C++ lexical local.
Add a child scope only for the collision shape handled by this change;
every unrelated program keeps the established analysis path.
Builtins need the same child scope as authored outer bindings, so
leaving a branch restores the builtin and its type.
"""
if self._collection_scope_stack:
return False
for stmt in body:
if (isinstance(stmt, VarDecl)
and stmt.name in self._direct_program_binding_names):
and (stmt.name in self._direct_program_binding_names
or stmt.name in BUILTIN_VARS)):
return True
if (isinstance(stmt, TupleAssign)
and any(
name in self._direct_program_binding_names
or name in BUILTIN_VARS
for name in stmt.names
if name != "_"
)):
Expand Down Expand Up @@ -7696,6 +7698,8 @@ def _visit_FuncCall(self, node: FuncCall) -> PineType:
if isinstance(obj, Identifier) and obj.name == "str":
for arg in node.args:
self._visit(arg)
for value in node.kwargs.values():
self._visit(value)
# Most str.* return a string, but predicates and index helpers
# are scalar. Keep this aligned with signatures.py and the C++
# emitter's _infer_type path.
Expand Down Expand Up @@ -8015,6 +8019,12 @@ def _visit_FuncCall(self, node: FuncCall) -> PineType:
)
return PineType.UNKNOWN
return receiver_pine_type
# Preserve fixed scalar array results in the coarse return-type
# visitor as well as TypeSpec inference. Authored methods above
# retain precedence over builtin names.
fixed_array_result = self._fixed_array_call_result(node, receiver_spec)
if fixed_array_result is not None:
return self._element_pine_type(fixed_array_result)
# Matrix method dispatch: ``m.get(0, 0)`` on ``matrix<int>`` must
# type as INT, not VOID, so ``v = m.get(...)`` propagates the
# element PineType. ``_type_spec_from_expr`` already carries the
Expand Down
4 changes: 3 additions & 1 deletion pineforge_codegen/analyzer/diagnostics.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ class DiagnosticsHelper:
the diagnostic surface easy to audit.
"""

def _error(self, message: str, loc: SourceLocation | None = None) -> None:
def _error(self, message: str, loc: SourceLocation | None = None,
*, hint: str | None = None) -> None:
"""Raise a compile error."""
if loc is None:
loc = SourceLocation(file=self._filename, line=1, col=1, end_col=1)
Expand All @@ -54,6 +55,7 @@ def _error(self, message: str, loc: SourceLocation | None = None) -> None:
phase=Phase.ANALYZER,
location=loc,
message=message,
hint=hint,
)
raise CompileError([diag])

Expand Down
3 changes: 3 additions & 0 deletions pineforge_codegen/analyzer/operators.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
# Only argument-independent scalar results belong here. The signature
# registry also contains storage-oriented approximations of polymorphic calls.
_FIXED_RESULT_CALLS = {
"array": frozenset({"join"}),
"str": frozenset({
"tostring", "format", "format_time", "length", "contains", "startswith",
"endswith", "pos", "substring", "replace", "replace_all", "lower",
Expand Down Expand Up @@ -53,6 +54,8 @@ def fixed_builtin_result(namespace: str, name: str, arity: int) -> TypeSpec | No
else "int" if name in {"barssince", "highestbars", "lowestbars"}
else "float"
)
elif namespace == "array":
result = "string"
elif namespace == "str":
result = (
"bool" if name in {"contains", "startswith", "endswith"}
Expand Down
51 changes: 50 additions & 1 deletion pineforge_codegen/analyzer/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,16 +46,18 @@
SwitchStmt, TupleAssign, TupleLiteral, UnaryOp, VarDecl,
)
from ..errors import Phase
from ..intrinsic_binding import array_method_domain_error
from ..limits import iter_ast_nodes
from ..matrix_overloads import matrix_sum_has_rhs
from ..reference_casts import na_reference_cast_kind
from ..pine_spelling import expr_start
from ..symbols import (
PineType,
TypeSpec,
method_receiver_type_name,
unify_type_specs,
)
from .operators import join_value_types, reference_value_type
from .operators import fixed_builtin_result, join_value_types, reference_value_type

# Drawing-objects-as-data type names (spec §4.1). Defined locally — the
# analyzer must not import from ``codegen`` (codegen imports analyzer, so the
Expand Down Expand Up @@ -196,6 +198,48 @@ def _template_args_from_call(self, node: FuncCall) -> list[str]:
raw = ann.get("template_args") or []
return [str(x).replace(" ", "") for x in raw]

def _fixed_array_call_result(
self, call: FuncCall, receiver_spec: TypeSpec | None,
) -> TypeSpec | None:
"""Validate a builtin receiver and read its fixed result after methods."""
callee = call.callee
if not isinstance(callee, MemberAccess):
return None
namespace_call = (
isinstance(callee.object, Identifier)
and callee.object.name == "array"
and self._symbols.resolve("array") is None
and self._value_bindings.declaration_bindings.get(
id(callee.object)
) is None
)
if not namespace_call and (
receiver_spec is None or receiver_spec.kind != "array"
):
return None
if callee.member == "join":
receiver = (
call.args[0] if call.args else call.kwargs.get("id")
) if namespace_call else callee.object
# A namespace receiver can be a constructor containing TA or
# request calls already visited above. Read its type without
# registering those calls again; preserve call-site type overlays.
query = (self if isinstance(self, OperatorTypeQuery)
else OperatorTypeQuery(self, {}))
actual_spec = (
query._type_spec_from_expr(receiver)
if namespace_call else receiver_spec
)
problem = array_method_domain_error(callee.member, actual_spec)
if problem is not None:
self._error(
problem[0], expr_start(receiver).loc or call.loc,
hint=problem[1],
)
return fixed_builtin_result(
"array", callee.member, len(call.args) + len(call.kwargs),
)

def _array_from_element_spec(self, value: ASTNode | None) -> TypeSpec | None:
"""Exact scalar element type for one ``array.from`` argument.

Expand Down Expand Up @@ -612,6 +656,11 @@ def direct_user_udt_ctor_name(node: ASTNode) -> str | None:
# Its body-derived return type is not available yet, but a
# same-named builtin must not lend it a false type.
return None
fixed_array_result = self._fixed_array_call_result(
value, typed_receiver_spec,
)
if fixed_array_result is not None:
return fixed_array_result
if (
typed_receiver_spec is not None
and typed_receiver_spec.kind == "udt"
Expand Down
62 changes: 62 additions & 0 deletions pineforge_codegen/analyzer/v6_forms.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@
from ..diagnostic_codes import render_diagnostic
from ..errors import CompileError, Diagnostic, Level, Phase, SourceLocation
from ..limits import TimeBudget, syntax_children
from ..loop_types import IntegerConstants, LoopTypeSources
from ..pine_spelling import expr_start
from ..signatures import STR_FUNCTIONS


def _refuse(code: str, node: ASTNode, filename: str,
Expand Down Expand Up @@ -114,3 +116,63 @@ def validate_v6_forms(program: Program, filename: str,
value for value in notes.get("param_defaults", ())
if isinstance(value, ASTNode)
)


def _substring_begin(node: FuncCall) -> ASTNode | None:
"""Read the begin slot of a call that binds to a declared signature."""
for signature in STR_FUNCTIONS["substring"].signatures:
names = [parameter.name for parameter in signature.params]
if (len(node.args) > len(names)
or set(node.kwargs) - set(names)
or set(names[:len(node.args)]) & set(node.kwargs)):
continue
bound = dict(zip(names, node.args))
bound.update(node.kwargs)
if any(parameter.default is None and parameter.name not in bound
for parameter in signature.params):
continue
return bound["begin_pos"]
# Malformed calls retain their existing argument-binding handling.
return None


def validate_v6_constant_arguments(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — validate_v6_constant_arguments()

fans out to 11 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

program: Program, bindings: dict[int, tuple[int, str] | None],
filename: str, budget: TimeBudget | None = None,
) -> None:
"""Check constant domains after lexical resolution, before call lowering."""
if program.version != 6:
return
constants = None
stack = [program]
while stack:
node = stack.pop()
if budget is not None:
budget.check(node.loc, Phase.ANALYZER)
notes = node.annotations or {}
if notes.get("pine_version") == 5:
continue
if (isinstance(node, FuncCall) and _callee_name(node) == "str.substring"
and bindings.get(id(node.callee.object)) is None):
begin = _substring_begin(node)
if begin is not None:
if constants is None:
constants = IntegerConstants(LoopTypeSources(program), bindings, budget)
value = constants.value(begin)
if value is not None and value < 0:
raise CompileError([Diagnostic(
level=Level.ERROR,
phase=Phase.ANALYZER,
location=(expr_start(begin).loc or begin.loc or node.loc
or SourceLocation(filename, 1, 1, 1)),
message=(
"str.substring() requires 'begin_pos' "
"to be greater than or equal to 0."
),
)])
stack.extend(syntax_children(node))
if isinstance(node, (FuncDef, MethodDef)):
stack.extend(
value for value in notes.get("param_defaults", ())
if isinstance(value, ASTNode)
)
5 changes: 4 additions & 1 deletion pineforge_codegen/analyzer/value_bindings.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ def _missing(self, name: str, scope: _Scope, position: int,
def resolve(self) -> None:
if self.program.version != 6:
return # Analyzer owns missing/unsupported-version diagnostics.
from .v6_forms import validate_v6_forms
from .v6_forms import validate_v6_constant_arguments, validate_v6_forms
validate_v6_forms(self.program, self._filename, self.budget)
# Import after Analyzer has loaded; reuse its namespace inventory.
from .base import _BUILTIN_NAMESPACE_NAMES
Expand Down Expand Up @@ -266,6 +266,9 @@ def resolve(self) -> None:
)
node.annotations = {**(node.annotations or {}), _VALUE_USE: origin}
self.check_use(node)
validate_v6_constant_arguments(
self.program, self.declaration_bindings, self._filename, self.budget,
)

def _body(self, body: list, scope: _Scope) -> None:
for stmt in body:
Expand Down
Loading
Loading