QuantCandle Engine: High-Frequency Feature Engineering

A real-time data pipeline for stationary Bitcoin ML features.

The QuantCandle Engine is a backend service designed to solve the "non-stationarity" problem in financial machine learning. It consumes live 1m K-Line streams via WebSockets, applies vectorized mathematical transformations in real-time, and maintains a high-concurrency data bridge for the QuantCandle API.

Core Functionalities

• Real-Time Ingestion: Leverages ThreadedWebsocketManager for non-blocking Binance 1m K-Line streams.
• Gap-Resilient Pipeline: Features a backfilling logic that identifies historical data gaps using SQL window functions (LEAD) and patches them via the Binance REST API to ensure calculation continuity.
• Vectorized Engineering: Transforms raw OHLCV data into scale-invariant features using NumPy, Pandas, and TA-Lib.
• Decoupled Persistence: Maintains an optimized SQLite database with independent tables for raw data (ohlcv_1m) and engineered features (features_1m).

Tech Stack

• Language: Python 3.12+
• Package Manager: uv
• Data Processing: Pandas, NumPy, TA-Lib
• Connectivity: python-binance (WebSocket + REST)
• Storage: SQLite3 with vectorized update_db_pandas logic

---

Feature Dictionary

1. Primary Features

• log_returns: Continuous compounded returns, standardized.

$$ \large r_t = \ln \left({\frac{P_t}{P_{t-1}}}\right) $$

• parkinson_volatility Volatility estimator based on High-Low range.

$$ \large \sigma = \sqrt{ \frac{1}{4 N ln \left( 2.0 \right)} \sum \ln {\left( \frac{High_i}{Low_i} \right)^{2}}} $$

2. Candle Geometry (Input)

• log_upper_wick:

$$ \large W_{upper} = \ln{(High)} - \ln{(max(O_t, C_t))} $$

• log_lower_wick:

$$ \large W_{lower} = \ln{(min(O_t, C_t))} - \ln{(Low)} $$

• ibs (Internal Bar Strength): Position of the Close relative to the High-Low range. Naturally stationary [0, 1].

$$ \large \text{IBS} = \frac{C_t - L_t}{H_t - L_t} $$

3. Time Encoding (Cyclical Inputs)

Time is decomposed into cyclical sine/cosine pairs to preserve temporal continuity (e.g., minute 59 is close to minute 0).

• minute/hour/day... sin/cos:

$$ \large x_{sin} = \sin\left(\frac{2\pi t}{T}\right), \quad x_{cos} = \cos\left(\frac{2\pi t}{T}\right) $$

(Where T is the period, e.g., 60 for minutes, 24 for hours).

4. Volatility (Risk)

• parkinson_volatility: A high-efficiency volatility estimator using the High/Low range.
• bbands: Measures the relative width of the bands (volatility expansion/contraction).

$$ \large \text{B}_{width} = \frac {\text{Upper} - \text{Lower}}{\text{Middle}} $$

5. Momentum (Trend)

• ppo_line: Percentage Price Oscillator. Normalized version of MACD, making it naturally stationary.

$$ \large \text{PPO} = \frac{EMA_{fast} - EMA_{slow}}{EMA_{slow}} $$

• ppo_signal: Exponential moving average of the ppo_line.
• ppo_histogram: Divergence between PPO line and signal.
• imbalance: Measures market aggressiveness.

$$ \large \text{Imbalance} = \frac{\text{Taker Buy Volume} - \text{Take Sell Volume}}{\text{Total Volume}} $$

6. Liquidity (Flow)

Volume is normalized relative to a trailing window to detect anomalies relative to recent activity.

• relative_volume:

$$ \large \frac{V_t}{\mu_{window}} $$

(Where $\mu_{window}$ is the rolling mean of the period defined in your .env configuration)

• relative_taker_buy_volume: Taker buy volume normalized by its window period moving average.

7. Bitcoin Cycle (Halving)

• halving_sin/cos: A long-term cyclical feature representing Bitcoin's ~4-year supply shock cycle.

---

Architecture & Deployment

The engine operates on an event-driven loop: it triggers on every closed candle (kline['x']), commits raw data, verifies continuity via backfilling, and pushes new features into the database.

Running Locally

This project uses uv for lightning-fast environment management.

git clone https://github.com/StarlitVienna/quantcandle-engine.git
cd quantcandle-engine
uv run python main.py

• Kaggle Dataset: Bitcoin Quant Core
• Live API: QuantCandle Data on RapidAPI