00bb06434e
fix: collector WebSocket trade parsing + DuckDB 1.5 API compatibility
...
data/collectors/hyperliquid.py:
- Fixed _handle_trades to handle HL WebSocket trade data as list[dict]
(each trade dict has its own 'coin' field) instead of assuming single dict
- Fixed _poll_funding and _poll_open_interest to guard against
data[0] being list vs dict (mainnet vs testnet response format difference)
data/duckdb_load.py:
- Replaced deprecated duckdb.from_sequence() with executemany() for
l2_snapshots, trades, and funding table inserts (DuckDB 1.5.x API)
Tick runner verified on real mainnet BTC data:
- 324 events (45 L2 + 279 trades), VPIN-gated A-S MM, 1 trade, 98 cancels
- HFT dashboard generated with 8 panels from real tick data
2026-08-11 14:45:48 +08:00
20ee340cef
feat: VBT visualization + validation pipeline, HFT tick viz, DuckDB loader
...
Track 1 — VBT Candle-Frequency Pipeline:
- backtests/vbt_validator.py: VBTValidator with 11 checks — timestamp monotonicity,
duplicates, NaN, data gaps, lookahead bias, signal alignment, density,
coincident entry/exit, min trade count, fee application, benchmark comparison.
ValidationReport dataclass with errors/warnings/stats. Validates VBT results
or raw signal arrays.
- backtests/vbt_viz.py: VBTVisualizer with 10+ Plotly chart methods — equity
curve with benchmark, drawdown, rolling Sharpe/Sortino/vol, trade markers,
returns distribution with normal fit, monthly PnL heatmap, gross vs net,
holding periods, parameter sensitivity heatmaps, dashboard compositor,
HTML save (self-contained, CDN Plotly). All methods handle empty/null inputs.
- backtests/vbt_report.py: Markdown + HTML report generator — structured
sections for implementation summary, performance metrics, cost analysis,
validation results, signal analysis, known limitations, next steps.
batch_report() for mass report generation from results directory.
- backtests/vbt_runner.py: Added run_benchmark() (buy-and-hold VBT portfolio),
validate() (integrated VBTValidator), run_with_report() (fetch→validate→
backtest→visualize→save in one call).
Track 2 — HFT Tick Pipeline:
- backtests/tick_viz.py: 9-panel HFT dashboard — price+trade markers,
spread dynamics, top-of-book depth, microprice vs mid, OBI/OFI panel,
VPIN toxicity with thresholds, event timeline (PnL from tick_runner),
markout curves at 6 horizons. Parquet→pandas→Plotly pipeline.
Dark-themed HTML output for microstructure review.
- data/duckdb_load.py: Parquet→DuckDB loader — creates l2_snapshots,
trades, funding tables with schema. Pre-computed 1s rollup views for
microprice, OFI, trade imbalance. Markout queries directly in SQL.
Incremental loading with load_state tracking.
CLI Integration:
- cli.py: Added 'report' (full VBT report), 'validate' (check existing
results), 'hft' (tick dashboard generation) commands. Fixed argparse
help string escaping.
355 tests passing (34 new).
2026-08-11 12:22:11 +08:00
ramseshk