Skip to content

fix: merge US stock 3m bars returned by the Yahoo chart - #250

Closed
Dev-next-gen wants to merge 1 commit into
OpenByteInc:mainfrom
Dev-next-gen:fix/us-stock-yahoo-3m-merge
Closed

Dev-next-gen wants to merge 1 commit into
OpenByteInc:mainfrom
Dev-next-gen:fix/us-stock-yahoo-3m-merge

Conversation

@Dev-next-gen

Copy link
Copy Markdown
Contributor

Summary

I noticed that asking USStockDataSource.get_kline for the 3m timeframe gives back one-minute bars. 3m maps to Yahoo's 1m interval, and MERGE_FACTOR_MAP says those bars must be grouped by three, but only the yfinance DataFrame fallback applied that merge. The Nasdaq intraday fallback merges on its own too. The Yahoo chart endpoint is tried first, so in the usual case where it answers, the bars went out unmerged: a request for two 3m bars returned the last two minutes, each labelled as a 3m candle. A 3m backtest or strategy on a US stock was really running on 1m data.

Changes

  • get_kline now runs _merge_every_n_sorted_bars on the Yahoo chart result when the timeframe has a merge factor. It's the helper the yfinance path already uses, so all three sources now return the same shape for 3m. No other timeframe has a merge factor in this class, so 1m, 5m, 1H, 1D and the rest are unchanged.
  • I added a regression test in tests/test_us_stock_intraday_window.py. It feeds six one-minute Yahoo bars and checks that get_kline("NVDA", "3m", 2) returns two candles starting at 13:30 and 13:33, with the right OHLCV values.

The only output that changes is get_kline for 3m when the Yahoo chart answers. It now matches what the method already returned whenever it fell back to yfinance or Nasdaq, so callers see the same 3m bars whichever source answers.

Test plan

  • Tested locally with docker compose up -d --build
  • Backend logs show no errors
  • Relevant pytest tests pass

Without the fix, the new test fails with [1789392840, 1789392900] == [1789392600, 1789392780] (13:34 and 13:35 come back instead of 13:30 and 13:33). With the fix, it passes. The full backend suite passes (2793 passed, 55 skipped), and ruff check app scripts tests is clean.

API documentation (if routes/schemas changed)

No routes or schemas changed.

Found by a defect-hunting pipeline I build and run (Dev-next-gen), using Claude Code with Anthropic's Claude Opus 5.

The 3m timeframe requests 1m bars and relies on MERGE_FACTOR_MAP to group them, but only the yfinance fallback applied the merge. When the Yahoo chart endpoint answered, which is the normal case, get_kline returned raw one-minute bars labelled as 3m.
@brokermr810

Copy link
Copy Markdown
Collaborator

Thanks for identifying the Yahoo 3m data issue. We confirmed that the normal Yahoo chart path returned raw 1m bars, but merging after the chart response had already been tail-truncated could shift candle boundaries during a partial session (for example, 13:32/13:35 instead of 13:30/13:33). A corrected implementation has landed on main in d14b7d5: it aggregates fixed three-minute timestamp buckets, rejects buckets with missing minutes, and applies the requested result limit only after aggregation. Closing this PR as superseded by that fix. Thanks again for the useful report and regression case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants