fix: merge US stock 3m bars returned by the Yahoo chart - #250
Closed
Dev-next-gen wants to merge 1 commit into
Closed
Dev-next-gen wants to merge 1 commit into
Dev-next-gen wants to merge 1 commit into
Conversation
The 3m timeframe requests 1m bars and relies on MERGE_FACTOR_MAP to group them, but only the yfinance fallback applied the merge. When the Yahoo chart endpoint answered, which is the normal case, get_kline returned raw one-minute bars labelled as 3m.
Collaborator
|
Thanks for identifying the Yahoo 3m data issue. We confirmed that the normal Yahoo chart path returned raw 1m bars, but merging after the chart response had already been tail-truncated could shift candle boundaries during a partial session (for example, 13:32/13:35 instead of 13:30/13:33). A corrected implementation has landed on main in d14b7d5: it aggregates fixed three-minute timestamp buckets, rejects buckets with missing minutes, and applies the requested result limit only after aggregation. Closing this PR as superseded by that fix. Thanks again for the useful report and regression case. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
I noticed that asking
USStockDataSource.get_klinefor the3mtimeframe gives back one-minute bars.3mmaps to Yahoo's1minterval, andMERGE_FACTOR_MAPsays those bars must be grouped by three, but only the yfinance DataFrame fallback applied that merge. The Nasdaq intraday fallback merges on its own too. The Yahoo chart endpoint is tried first, so in the usual case where it answers, the bars went out unmerged: a request for two 3m bars returned the last two minutes, each labelled as a 3m candle. A 3m backtest or strategy on a US stock was really running on 1m data.Changes
get_klinenow runs_merge_every_n_sorted_barson the Yahoo chart result when the timeframe has a merge factor. It's the helper the yfinance path already uses, so all three sources now return the same shape for3m. No other timeframe has a merge factor in this class, so1m,5m,1H,1Dand the rest are unchanged.tests/test_us_stock_intraday_window.py. It feeds six one-minute Yahoo bars and checks thatget_kline("NVDA", "3m", 2)returns two candles starting at 13:30 and 13:33, with the right OHLCV values.The only output that changes is
get_klinefor3mwhen the Yahoo chart answers. It now matches what the method already returned whenever it fell back to yfinance or Nasdaq, so callers see the same3mbars whichever source answers.Test plan
docker compose up -d --buildWithout the fix, the new test fails with
[1789392840, 1789392900] == [1789392600, 1789392780](13:34 and 13:35 come back instead of 13:30 and 13:33). With the fix, it passes. The full backend suite passes (2793 passed, 55 skipped), andruff check app scripts testsis clean.API documentation (if routes/schemas changed)
No routes or schemas changed.
Found by a defect-hunting pipeline I build and run (Dev-next-gen), using Claude Code with Anthropic's Claude Opus 5.