Commit Graph

142 Commits

Author SHA1 Message Date
ramseshk 20ee340cef feat: VBT visualization + validation pipeline, HFT tick viz, DuckDB loader
Track 1 — VBT Candle-Frequency Pipeline:
- backtests/vbt_validator.py: VBTValidator with 11 checks — timestamp monotonicity,
  duplicates, NaN, data gaps, lookahead bias, signal alignment, density,
  coincident entry/exit, min trade count, fee application, benchmark comparison.
  ValidationReport dataclass with errors/warnings/stats. Validates VBT results
  or raw signal arrays.
- backtests/vbt_viz.py: VBTVisualizer with 10+ Plotly chart methods — equity
  curve with benchmark, drawdown, rolling Sharpe/Sortino/vol, trade markers,
  returns distribution with normal fit, monthly PnL heatmap, gross vs net,
  holding periods, parameter sensitivity heatmaps, dashboard compositor,
  HTML save (self-contained, CDN Plotly). All methods handle empty/null inputs.
- backtests/vbt_report.py: Markdown + HTML report generator — structured
  sections for implementation summary, performance metrics, cost analysis,
  validation results, signal analysis, known limitations, next steps.
  batch_report() for mass report generation from results directory.
- backtests/vbt_runner.py: Added run_benchmark() (buy-and-hold VBT portfolio),
  validate() (integrated VBTValidator), run_with_report() (fetch→validate→
  backtest→visualize→save in one call).

Track 2 — HFT Tick Pipeline:
- backtests/tick_viz.py: 9-panel HFT dashboard — price+trade markers,
  spread dynamics, top-of-book depth, microprice vs mid, OBI/OFI panel,
  VPIN toxicity with thresholds, event timeline (PnL from tick_runner),
  markout curves at 6 horizons. Parquet→pandas→Plotly pipeline.
  Dark-themed HTML output for microstructure review.
- data/duckdb_load.py: Parquet→DuckDB loader — creates l2_snapshots,
  trades, funding tables with schema. Pre-computed 1s rollup views for
  microprice, OFI, trade imbalance. Markout queries directly in SQL.
  Incremental loading with load_state tracking.

CLI Integration:
- cli.py: Added 'report' (full VBT report), 'validate' (check existing
  results), 'hft' (tick dashboard generation) commands. Fixed argparse
  help string escaping.

355 tests passing (34 new).
2026-08-11 12:22:11 +08:00
ramseshk 09cb0d42b5 feat: queue-aware paper fills, kill switch, systemd services, WQI+FundingArb in paper trader
- live/paper_trader.py: replaced random 5% fill probability in simulate_avellaneda()
  with QueueAwareFillModel — fills only when aggressor volume exceeds depth ahead,
  regime-adaptive quote placement (tight in LOW_VOL, wide in HIGH_VOL). Integrated
  WQI Predictor and Funding Rate Arb as new strategies with signal generation.
  Dashboard metrics now include WQI summaries, funding arb status, and fill model
  throughput stats (fill rate, fills vs skips). 14 strategies total.
- scripts/kill_switch.py: emergency kill switch — flattens all positions, cancels
  all open orders, verifies account is flat. Supports --dry-run, --mainnet, retry
  logic, L1 action signing. Reads private key from HL_PRIVATE_KEY env or ~/.hl/key.
- infrastructure/systemd/: three service unit files for production deployment:
  ftdt-collector (data collection), ftdt-paper (trading node v2), ftdt-dashboard
  (FastAPI backend). Includes memory/cpu limits, auto-restart, log rotation.

321 tests passing.
2026-08-11 11:28:01 +08:00
ramseshk 3073415d33 feat: funding arb strategy, queue-aware paper fills, WQI live integration
- strategies/funding_arb_strategy.py: full backtestable funding rate carry module
  with entry/exit thresholds, position tracking, funding payment accounting,
  basis stop-loss, max-hold timeout. Includes backtest_funding_arb() and
  run_funding_discovery() for threshold optimization
- live/node_v2.py: replaced naive random fills with QueueAwareFillModel (sim/fills.py)
  with queue-priority simulation; integrated WQI predictor and funding arb strategies;
  per-coin WQI signal generation every 3 ticks; funding arb metrics in dashboard
- cli.py: added 'funding' command for funding rate distribution analysis and
  threshold backtesting
- tests/test_funding_arb.py: 20 tests covering entry/exit logic, fee accounting,
  signal generation, backtesting, and node integration

321 tests passing (20 new).
2026-08-11 11:15:25 +08:00
ramseshk 50d63e1ecc feat: HFT infrastructure — tick backtest runner, VPIN-gated A-S maker, WQI predictor, queue-aware fills
- backtests/tick_runner.py: TickBacktestRunner replays stored Parquet L2/trade events
  through sim/engine.py with queue position modeling, producing PnL breakdowns,
  equity curves, VPIN curves, and QuantVerdict significance reports
- VPINGatedASMaker: VPIN-toxicity-gated A-S market maker with inventory skew
  and dynamic spread widening; blocks quoting when VPIN >= alarm threshold
- sim/engine.py: Added SimConfig.from_fee_tier() factory — constructs sim
  config from Hyperliquid fee tier (VIP + staking)
- sim/fills.py: Added QueueAwareFillModel — realistic queue-priority fill
  simulation replacing random fills in paper trading
- strategies/wqi_predictor.py: WQI z-score directional strategy with
  adverse selection gating, timeout exit, stop-loss, and take-profit
- cli.py: Added 'tick', 'markout' analysis, and 'discover' signal-discovery
  commands for end-to-end tick-level HFT research pipeline

301 tests passing (23 new).
2026-08-11 10:43:51 +08:00
ramseshk cdc7a01986 feat: queue simulation validation — VBT vs realistic event-driven PnL comparison
Track 4 — quant/sim_validate.py:
  SimulationValidator: runs VBT backtest, then replays signals through
  Phase 3 event-driven simulator with queue position, partial fills,
  adverse selection, and cancel latency.

  Compares VBT (optimistic candle fill) vs simulator (realistic) PnL.
  BatchSimValidator: runs multiple param combos through the pipeline.

Grid MM simulation results (all DISCARD):
  VBT S=+0.78, ret=+1.5% → Sim S=0.00, ret=0.0%, 100% adverse
  VBT S=-0.03, ret=-0.4% → Sim S=0.00, ret=0.0%, 100% adverse
  VBT S=-2.74, ret=-6.1% → Sim S=0.00, ret=0.0%, 100% adverse

  Grid fills happen during momentum (price crossing levels),
  not mean reversion. Adverse selection rate approaches 100%.
  VBT candle fills are fundamentally optimistic for grid strategies.

269 tests passing.
2026-08-10 16:48:31 +08:00
ramseshk 7517163142 feat: configurable grid params + auto walk-forward optimizer
Track 1 — Grid param sweep in vbt_runner:
  - _generate_signals accepts params dict: grid_levels, spacing_bps, rebalance_every
  - run_strategy passes params through to signals
  - _strategy_params reflects actual runtime params
  - Grid param sweep results: spacing is critical, levels don't matter
    Tight spacing (1-2bps) = 1 trade, positive EV
    Wide spacing (20bps+) = many trades, negative EV
    Candle simulation can't model grid MM fills accurately

Track 6 — quant/optimizer.py:
  - ParanOptimizer: automated IS/OOS parameter walk-forward
  - add_param() to define parameter grid
  - Composite score: Sharpe × sqrt(trades) for robustness
  - IS optimization per window, OOS testing per window
  - WFParamWindow + OptimizerReport with consistency + stable params

Grid MM walk-forward results (3 windows):
  W0: IS S=-3.75 → OOS S=+2.23 (+12.7%, 1t)
  W1: IS S=+2.52 → OOS S=-3.49 (-19.0%, 11t)
  W2: IS S=-3.30 → OOS S=0.00 (0t)
  Consistency: 33.3%, Stable params: {levels=5, spacing=1bps, rebalance=5}

  Verdict: Candle-based grid MM is fundamentally unreliable.
  Real fills require queue simulation with L2 data.
2026-08-10 16:47:09 +08:00
ramseshk ad4036713e feat: recent-window backtests (week/month/year) — 207 clean results
Archived 639 old full-history results. Re-ran sweep on time-windowed
bar limits matching practical trading horizons:

  Last week bars:  1h=168, 4h=42,  1d=7
  Last month bars: 1h=720, 4h=180, 1d=30
  Last year bars:  1h=5000, 4h=2190, 1d=365

270/270 sweep succeeded, 63 zero-trade files archived.
207 results with real trades remain.

Key findings on RECENT (year) data only:

  grid_mm:    S=13.8 (1h), S=9.3 (4h), S=8.1 (1d)
  as_mm:      S=14.8 (1h), S=8.0 (4h), S=3.3 (1d)
  composite:  S=3.2 (1h), S=2.3 (4h), S=0.7 (1d)
  mean_rev:   1d only viable (S=3.5)

  Directional strategies (momentum, obi, iceberg, hurst_vpin)
  all show near-zero or negative Sharpe on recent data.

  Market-making dominates in sideways crypto markets.

Added --window flag to sweep_runner for week/month/year presets
2026-08-10 16:35:25 +08:00
ramseshk 543537e33f feat: quant validation framework — DSR, PSR, Haircut, regimes, walk-forward
Three-module quant framework replacing 'sort by Sharpe' with proper
statistical validation:

quant/significance.py (15 tests):
  - deflated_sharpe_ratio(): adjusts for N trials (Harvey & Liu 2015)
  - probabilistic_sharpe_ratio(): P(True SR > benchmark) given T, skew, kurt
  - sharpe_haircut(): expected OOS Sharpe after selection bias deflation
  - QuantVerdict: DEPLOY / SIMULATE / DISCARD with 5-point scoring
  - validate_strategy(): one-shot validation function

quant/regimes.py (8 tests):
  - classify_regime(): trending_up/down, ranging, volatile
  - RegimeClassifier: stateful rolling-window classifier
  - conditional_performance(): per-regime trade statistics

quant/walkforward.py (5 tests):
  - WalkForwardRunner: sequential IS/OOS window optimization
  - WFWindow/WFReport: structured walk-forward results
  - consistency score, performance decay, concatenated OOS equity
  - significance_report() integration

Walk-forward results (real HL data with date-sliced windows):
  grid_mm 1h:    2/4 pos, OOS S=-0.45,  74t, haircut=-22.66 → DISCARD
  momentum 4h:   2/4 pos, OOS S=-1.47, 116t, haircut=-45.35 → DISCARD
  composite_mm 1h: 2/4 pos, OOS S=+2.97, 6t, haircut=+43.25 → SIMULATE

28 tests total
2026-08-10 16:20:56 +08:00
ramseshk 268fe606fa fix: clear stale __pycache__ causing 'datetime is not defined' error 2026-08-10 14:03:56 +08:00
ramseshk e71b47ac56 refactor: VBT sidebar — cleaner UX with grouped sections + collapsible fee tiers
Sidebar reorganized into clear sections:
  [FILTERS] — Strategy + Asset + Interval + Sort
  ▶ Fee Tiers — collapsible (hidden by default), shows summary
    T0 + staking tier name when collapsed
  [NEW BACKTEST] — compact: Strategy (short names) + Interval + Limit
    Row 2: Coin + ▶ Run button (side by side) + ↻ Refresh
  [RESULTS] — count header + list + pagination

Removed:
  - Full-width run strategy row (now inline with interval/limit)
  - Duplicate fee tier labels in select options
  - Separate run/refresh button row (now same row as coin)

Added:
  - toggleFeeTiers() — expand/collapse fee settings
  - updateFeeSummary() — shows current tier in collapsed label
  - Fee changes auto-recalculate (no manual onchange needed)
  - Section headers: small uppercase labels
2026-08-10 12:58:37 +08:00
ramseshk 0486e8c93f feat: VBT overview + light mode + /api/vbt/summary
dashboard/static/vbt.html — complete rewrite:
  - CSS variables for theming (--bg, --text, --blue, etc.)
  - Light mode toggle (☀/☾) persisted in localStorage
  - Plotly charts update colors on theme switch
  - Overview panel on load: strategy cards + heatmap
  - Strategy cards: best Sharpe, best return, top combo,
    +Sharpe %, run count. Click to filter.
  - Heatmap: color-coded strategy×interval table
    (green>2, green>0.5, yellow>0, orange>-0.5, red)
  - Back-to-overview button on detail view
  - Default sort: Sharpe (was date)

dashboard/server.py:
  - New /api/vbt/summary endpoint:
    Per-strategy best (Sharpe, Return, Calmar, trades,
    positive-Sharpe %, top combo)
    Heatmap: strategy×interval best Sharpe matrix

Tests: 241 passing (excl. Hawkes)
2026-08-10 12:38:31 +08:00
ramseshk b7c7fbb0af feat: VBT dashboard overview — strategy cards + heatmap + /api/vbt/summary
dashboard/server.py — new /api/vbt/summary endpoint:
  - Aggregated per-strategy best stats (Sharpe, Return, Calmar, trades,
    positive-Sharpe %, top combo details)
  - Heatmap: strategy × interval matrix with best Sharpe per cell

dashboard/static/vbt.html — overview panel replaces empty state:
  - Strategy cards grid: best Sharpe, best return, top combo,
    positive-Sharpe %, run count. Click to filter sidebar.
  - Heatmap: color-coded strategy × interval table (green=good,
    red=bad). Hover for tooltip, click to load result.
  - Back-to-overview button when viewing a strategy detail
  - Overview re-shown on page load, auto-loads from /api/vbt/summary

Dashboard opens to overview → select strategy → drill into details
2026-08-10 12:03:49 +08:00
ramseshk 140b0cc360 feat: full-information live terminal — all 10 modules with coefficients
live/monitor_service.py — rewritten state() includes all 10:
  - Hawkes: 3x3 alpha excitation matrix, mu baseline per type, beta decay
  - Dealer GEX: net GEX (), pin levels, direction signal
  - Tick Regime: current/next tick size, boundary price, bars-to-cross
  - Triangular Arb: venue count, opportunities, best spread bps
  - Sequencer: stale-state detection, P50/P99 latency, event count
  - Full: prices, microstructure, HLP, whipsaw, liq, term, spoof

live.html — complete rewrite: Bloomberg-terminal 4-column grid
  - Panel 1: Prices (mid, mark, oracle, premium, spread, funding)
  - Panel 2: Microstructure (OBI, best bid/ask, toxicity bar, branching ratio, sequencer latency P50/P99)
  - Panel 3: Composite Signals (HLP, Whipsaw w/countdown, Term Structure w/z-score, GEX w/signal, Tick Regime, Spoof probability bar)
  - Panel 4: Dealer GEX (net GEX bar, pin levels, direction)
  - Full-width: HLP Vault (total delta, assets, toxicity, per-coin signals)
  - Wide: Hawkes Coefficients (3x3 alpha matrix, mu per type, beta)
  - Liq Waterfall + Tri Arb + Sequencer + Tick Regime
  - Term Structure + Funding Whipsaw with countdown
2026-08-10 10:54:54 +08:00
ramseshk e0be9f4d40 feat: advanced microstructure — sequencer latency, dealer GEX, tick regime, triangular arb
4 new modules with 20 tests:

#22 Sequencer Latency Detector (live/monitors/sequencer_latency.py):
  Detects stale-state windows between WebSocket and REST API.
  - WebSocket vs REST timestamp delta tracking
  - Transport latency percentiles (p50, p99)
  - Liquidation-triggered stale-state detection
  - Stale asset identification for cross-margin arbitrage

#25 Dealer GEX (microstructure/dealer_gex.py):
  Dealer Gamma Exposure estimation via Black-Scholes.
  - Per-strike gamma × OI × spot² GEX computation
  - Pin level detection (strikes where dealers are long gamma)
  - Net GEX aggregation
  - Signal: fade_breakout (long gamma pinning) vs ride_momentum
    (short gamma amplification)
  - nearest_pin() for distance-to-magnet calculation

#30 Tick-Size Regime Exploitation (microstructure/tick_regime.py):
  Detects when asset price approaches tick-size boundaries.
  - Hyperliquid tick schedule (BTC 0.1/0.5, ETH 0.01/0.05, SOL 0.001/0.005)
  - Boundary approach detection with configurable threshold
  - Linear trend estimation for expected bars-to-cross
  - Signal: widen_quotes or tighten_quotes with urgency classification
  - Per-coin state tracking

#33 Triangular Latency Arb (live/monitors/triangular_arb.py):
  Cross-venue A→B→C triangular arbitrage detection.
  - Internal triangular: BTC-USDT → ETH-BTC → ETH-USDT
  - Cross-venue: price discrepancy across slow/fast venues
  - Latency gap detection between venue pairs
  - Implied cross-rate computation vs direct quote
  - Minimum spread threshold gating
2026-08-10 10:34:56 +08:00
ramseshk 27c096dc9e feat: live monitoring dashboard — Bloomberg-terminal UI for all advanced modules
live/monitor_service.py (LiveMonitorService):
  Background service running all 6 advanced monitors plus microstructure.
  Polls Hyperliquid REST API every 4s for prices, books, funding.
  Exposes unified state() method for the live dashboard API.

dashboard/server.py:
  Added /api/monitors/status?coin=BTC — combined state of all monitors
  Added /api/monitors/hlp — HLP vault state
  Added /api/monitors/funding — funding whipsaw signal
  Added /api/monitors/spoof — spoof detector summary
  Added /live route serving the live dashboard
  Monitor service auto-starts on first API request

dashboard/static/live.html:
  Professional dark-themed live monitoring dashboard.
  Three-column grid layout with real-time polling:
  - Price panel: mid, mark, oracle, premium, spread, funding
  - Microstructure panel: OBI, VPIN/toxicity bar, bid/ask depths
  - Composite signal panel: HLP signal, funding whipsaw signal,
    term structure signal, spoof probability bar, branching ratio
  - HLP Vault panel: total delta, assets tracked, toxicity %,
    overextended assets, per-coin rebalancing signals
  - Bottom row: Spoof Detector, Liquidation Waterfall,
    Term Structure
  - Coin selector (BTC/ETH) with color-coded signal rows
  - Auto-refreshes every 3s with live indicator

Access: http://localhost:9175/live
2026-08-07 18:00:48 +08:00
ramseshk 5304534e38 feat: advanced microstructure modules — HLP, Hawkes, Whipsaw, Term Structure, Liq Waterfall, Spoof Detector
6 new modules with 46 new tests (230 total):

#21 HLP Vault Monitor (live/monitors/hlp_vault.py):
  Tracks Hyperliquid's native protocol market maker at address
  0xfefefe... Queries clearinghouseState + metaAndAssetCtxs.
  - Delta exposure per asset (notional + PnL)
  - Overextension detection (notional exceeds  M threshold)
  - Rebalancing signals: fade_short when HLP too short,
    fade_long when HLP too long (front-run forced rebalancing)
  - Toxicity score: HLP losing money = absorbing informed flow
  - Historical delta tracking

#29 Hawkes Processes (microstructure/hawkes.py):
  Multivariate Hawkes calibrator for limit order book dynamics.
  - MLE calibration via SGD gradient descent on log-likelihood
  - Branching ratio enforcement (alpha/beta < 0.99 for stationarity)
  - Intensity computation λ_i(t) with cross-excitation
  - Activity forecasting (expected event count in horizon)
  - Synthetic event generator (Ogata thinning)
  - Pure functions: hawkes_intensity, hawkes_log_likelihood,
    generate_hawkes_events

#23 Funding Whipsaw Trader (live/strategies/funding_whipsaw.py):
  Premium index decay trading in final 60s of funding epoch.
  - Detects deterministic convergence of premium→0 at settlement
  - Time-scaled position sizing (larger closer to settlement)
  - Auto-close after funding epoch completes
  - Confidence scoring based on premium magnitude

#32 Term Structure Monitor (live/monitors/term_structure.py):
  Perp/quarterly/bi-quarterly futures basis curve trading.
  - Quarterly-perp basis with z-score anomaly detection
  - BiQ-quarterly curve steepness monitoring
  - Fair quarterly price via interest rate parity + funding carry
  - Calendar spread signals: buy_basis, sell_basis,
    curve_steepener, curve_flattener

#24 Liquidation Waterfall (live/monitors/liq_waterfall.py):
  Cross-margin liquidation order prediction.
  - Margin ratio tracking (equity / maintenance margin)
  - Danger/critical level classification
  - Asset liquidation priority: maintenance / book_liquidity ratio
    (least liquid asset relative to margin = dumped first)
  - Strategy output: widen_spreads on target, tighten on rest

#31 Spoof Detector (microstructure/spoof_detector.py):
  Adversarial ML-style spoofing pattern recognition.
  - Rule 1: Large order far from mid, cancelled immediately
  - Rule 2: Cancel right before trade approaches price level
  - Rule 3: Oversized order with no fill within short lifetime
  - Spoof probability (rolling window ratio)
  - Cancel-to-fill ratio monitoring
2026-08-07 17:52:20 +08:00
ramseshk 31c1fe7fbe chore: sweep all strategies on HFT/intraday/intraweek intervals — 639 clean results
Archived all old historical/ files (zero trade prices, broken).
Removed 67 zero-trade sweep files.

Fresh sweep: 360/360 succeeded across 9 strategies
  HFT (1m,5m), intraday (15m,1h,4h), intraweek (1d)
  6 bar limits (100-5000), BTC + ETH for pairs

Key results by timeframe:

HFT (1m/5m, 235 files):
  hurst_vpin 1m 200b     S=82.05  1t   +0.60%
  hurst_vpin 1m 100b     S=62.06  1t   +0.22%
  momentum    1m 100b     S=31.96  2t   +0.16%
  NOTE: 1m results are noisy, single-trade Sharpe unreliable

INTRADAY (15m/1h/4h, 303 files):
  hurst_vpin 15m 100b     S=19.38  1t   +0.58%
  grid_mm    1h  100b     S= 8.56  4t   +2.50%
  obi        15m 100b     S= 6.95  2t   +0.29%
  momentum   4h  100b     S= 2.91  3t   +2.70%

INTRAWEEK (1d, 101 files):
  hurst_vpin 1d  100b     S= 1.17  1t   +5.03%
  grid_mm    1d  5000b    S= 0.78  1t   +446%*
  momentum   1d  5000b    S= 0.51  66t  +56%
  mean_rev   1d  5000b    S= 0.69  111t +37%

* grid_mm 1d returns suspiciously high — likely signal artifact

VBT Dashboard: 639 results, paginated, searchable
2026-08-07 17:34:55 +08:00
ramseshk 2aca789581 fix: NaN win_rate/profit_factor/expectancy crash JSON serialization
Root cause: VBT produces NaN for win_rate, profit_factor, and
expectancy when all trades have zero PnL (no winners, no losers).
JSON.dumps() rejects NaN/Inf values with 'ValueError: Out of range
float values are not JSON compliant'.

Fixes:
- Add _sanitize_nan(): recursive NaN/Inf → 0.0 for JSON safety
- Apply at end of _normalize_vbt_fields (after field fixes)
- Fix win_rate check: use 'is None' instead of 'not' (0.0 is falsy)
- Previously crashing files now work:
  grid_mm_BTC_4h_100b, hurst_vpin_BTC_1m_100b, etc.
2026-08-07 16:46:14 +08:00
ramseshk 745174f0e6 fix: iceberg strategy — bool dtype + consecutive spike detection
Root cause: iceberg signal generator produced object-dtype entries
that crashed VectorBT's numba JIT compiler with 'non-precise type
array(pyobject, 1d, C)'. All 28 sweep combos returned 0 trades.

Fixes:
- iceberg: lower vol spike threshold (1.3→1.15), require 2/3
  consecutive same-direction spikes (not just single bar)
- exit when spike subsides (not arbitrary 5-bar hold)
- .astype(bool) on all entries/exits before returning from
  _generate_signals, preventing numba JIT errors

Results (36/36 succeeded):
  iceberg 1d 2000b BTC  S=0.18  85t  ret=9.38%  (best)
  iceberg 15m  100b BTC  S=-36.34 4t  ret=-0.18% (worst)
  Consistently negative Sharpe except 1d interval —
  volume-spike following loses on sub-daily timescales
2026-08-07 16:27:06 +08:00
ramseshk 78a170a42b feat: VBT sweep runner — batch backtest all strategy/interval/limit/coin combos
backtests/sweep_runner.py:
  - Generates all valid strategy x interval x limit x coin combinations
    (9 strategies, 6 intervals, 6 bar limits, 2 coins = 360 combos)
  - Sequential mode with configurable rate-limiting delay
  - Multi-process parallel mode (ProcessPoolExecutor)
  - Progress logging, ETA, summary stats
  - --dry-run flag to preview without executing
  - CLI: python -m backtests.sweep_runner --workers 2 --delay 1.0

Results from initial sweep (262/360 succeeded):
  Best performers:
    grid_mm 1h 100b    S=5.08  ret=1.45%  4 trades
    composite_mm 4h    S=3.50  ret=1.50%  1 trade
    momentum 4h  200b  S=3.12  ret=5.94%  7 trades
    hurst_vpin 4h 100b S=2.88  ret=1.89%  1 trade

  Regime issues found:
    - 1m interval: universally extreme negative Sharpe
      (VBT can't model sub-minute HFT dynamics)
    - iceberg: generated 0 trades across all 28 runs (signal bug)
    - as_mm: many combos with 'no results' (A-S in low vol)
    - mean_rev: consistent negative across all combos

Archived 67 old broken files, kept 310 clean sweep results
2026-08-07 16:14:13 +08:00
ramseshk 7d5b05b640 chore: archive 7 broken backtest files with no equity curves
Archived files (backtests/results/archive/):
  as_mm_BTC_20260806-081121.json  — 506 trades, no EC, no PnL
  as_mm_ETH_20260806-081122.json  — 505 trades, no EC, no PnL
  as_mm_HYPE_20260806-081123.json — 510 trades, no EC, no PnL
  as_mm_VVV_20260806-081123.json  — 512 trades, no EC, no PnL
  hurst_vpin_BTC_20260806-071308.json — 47 trades, no EC
  spx_reversion_1h_20260806-073445.json — 38 trades, no EC
  spx_reversion_30m_20260806-073445.json — 22 trades, no EC

Remaining: 102 files with valid equity curves (720+ points)
and meaningful Sharpe ratios + trade data
2026-08-07 15:52:04 +08:00
ramseshk 0162e83138 feat: Hyperliquid fee schedule — all tiers, staking, maker rebates
config/fee_tiers.py — complete rewrite:
  - 7 perps fee tiers (T0-T6) matching HL docs:
    T0: 0.045/0.015% → T6: 0.024/0.000%
  - 7 spot fee tiers (T0-T6):
    T0: 0.070/0.040% → T6: 0.025/0.000%
  - 7 staking tiers (none → diamond):
    multiplier 1.00 → 0.60 (40% discount)
  - 3 maker rebate tiers (>0.5%, >1.5%, >3% maker ratio)
    extra -0.001% to -0.003% on positive maker rates
  - compute_trade_fees() — per-trade fee breakdown
  - fee_tier_from_volume(), staking_tier_from_hype()
  - STRATEGY_FEE_MODELS: maker/taker classification per strategy

backtests/vbt_runner.py:
  - Accept vip_tier, staking_tier, maker_rebate_tier at init
  - Auto-detect fee model per strategy (maker vs taker)
  - compute_trade_fees() for per-trade fee calculation
  - Include fee_info in result JSON

dashboard/server.py:
  - /api/vbt/run accepts fee_tier/aking_tier/maker_rebate params
  - Trade normalization uses proper HL fee schedule per strategy
  - /api/vbt/result/{filename}/recalc — recalc trades with new tiers
  - /api/vbt/fee_tiers — get full fee schedule as JSON

dashboard/static/vbt.html:
  - Fee tier selector (T0-T6) + staking tier selector
  - Auto-recalculate on tier change when a result is selected
  - Fee rate shown in trade log header (e.g. 0.045%)
2026-08-07 15:44:43 +08:00
ramseshk 9d817ac2fa feat: VBT trade log — show asset, entry/exit prices, Hyperliquid fees
Backend (vbt_runner.py):
  - Add asset (BTC/ETH) to each trade record
  - Compute per-trade fee using HL taker rate (0.05%)
    entry_fee = size * entry_px * fee_rate
    exit_fee = size * exit_px * fee_rate
  - Add pnl_gross (before fees) and pnl_net (after fees)
  - Add fee_rate field for transparency

Server (server.py):
  - Normalize old backtest trades: add missing asset, fee,
    pnl_net, pnl_gross fields
  - Holyliquid default fee rate: 0.05% taker

Frontend (vbt.html):
  - Trade log table now shows:
    Time | Side + Asset | Size | Entry | Exit | Fee | PnL (net) | Duration
  - Asset shown as inline badge in Side column
  - Fee column with explicit USD amount
  - PnL now explicitly labeled 'net' (after fees)
  - Fallback to old 'pnl' field for legacy backtest files
2026-08-07 15:35:33 +08:00
ramseshk 9ee13a45bb fix: LTTB downsampling crashes on string timestamps + abs→Math.abs
- _lttb_downsample: equity curve timestamps are ISO strings,
  not numeric. Rewrote to use array indices for triangle area
  instead of trying to multiply string * float.
- Fixed IndexError from reusing 'a' variable incorrectly
- Also fixed earlier: abs(dd) → Math.abs(dd) in vbt.html
2026-08-07 15:26:42 +08:00
ramseshk 870df57051 fix: dashboard errors — Math.abs, API routing, fetchHistorical guard
Fixes three bugs in dashboard:

1. vbt.html: bare abs(dd) → Math.abs(dd) — fixes ReferenceError
2. server.py: Add /cv/ routing for Next.js quant dashboard
   - Mount _next static assets at /cv/_next (not /cv/ which eats API routes)
   - Add /cv/api/* routes for backtests/historical, detail, recalc, risk
   - Add /cv/ws WebSocket endpoints for live/paper metrics
   - Add /cv/ catchall for Next.js HTML pages
3. dashboard-next/src/lib/api.ts: add res.ok guard to fetchHistorical()
   — prevents SyntaxError when API returns HTML error pages
4. sim/maker.py: guard observe() against zero mid_price
2026-08-07 15:19:18 +08:00
ramseshk e58c5951b7 feat: Phase 5 — integration layer (analytics pipeline, production node v2, CLI) + 12 tests
live/integrator.py (AnalyticsPipeline):
  Real-time pipeline: data → microstructure → signals.
  Accumulates book snapshots + trades, computes OBI, VPIN, microprice,
  spread, depth, trade imbalance, HFT regime, and emits composite
  signal with confidence and breakdown. Per-coin isolation.

live/node_v2.py (ProductionNode):
  Rebuilt production node integrating ALL Phase 1-4 modules:
  - REST data fetching (order book, mark prices, funding rates)
  - AnalyticsPipeline per coin for real-time microstructure signals
  - Treasury for position/capital/PnL/breaker management
  - ToxicityFilter integration via HlMakerPool makers
  - HlMakerPool for per-coin A-S quoting
  - CrossVenueMonitor, FundingBasisMonitor, LiquidationRiskOverlay
  - Paper trading with probabilistic fill simulation
  - Dashboard metrics JSON output (equity, treasury, analytics, maker)
  - Periodic status logging

cli.py (unified CLI):
  Subcommands integrating all modules:
    collect   — Run Hyperliquid data collector to Parquet
    analyze   — Run microstructure analytics on stored data
    simulate  — Run market-making simulator on stored data
    run       — Start production trading node (paper or live)
    backtest  — Run VectorBT backtest

12 integration tests (all pass):
  - AnalyticsPipeline: empty, book, trade, VPIN, emit, regime, isolation
  - ProductionNode: creation, tick cycle (3 ticks), metrics JSON output
  - CLI: import verification

Total test suite: 184 tests, all passing.
2026-08-07 14:54:47 +08:00
ramseshk 4f66ef36a9 feat: Phase 4 — controlled strategy deployment module + 38 tests
New live/ sub-modules for production-ready market making:

live/filters/toxicity.py (ToxicityFilter):
  VPIN-based pre-trade filter. Accumulates buy/sell volume, computes
  VPIN via microstructure module, produces quoting decision:
    - allow_quoting: bool
    - size_multiplier: 0.0–1.0 (graduated reduction approaching alarm)
    - granular thresholds (threshold vs alarm) with smooth reduction

live/treasury.py (Treasury):
  Central capital/risk management — single source of truth:
  - Position tracking per coin (opening, closing, average entry)
  - Realized + unrealized PnL computation
  - Pre-trade constraint checks (inventory limits, fee estimates)
  - Circuit breaker (drawdown, trade count, toxic fill rate, API errors)
  - Liquidation distance monitoring
  - Automatic cooldown reset after trip expiry

live/makers/hl_btc_eth.py:
  HlMaker — per-coin market maker integrating:
    - AvellanedaStoikovMaker (Phase 3) for optimal quotes
    - ToxicityFilter for pre-trade gating
    - Treasury for position/risk checks
  HlMakerPool — manages multiple HlMaker instances with shared treasury
    and coordinated observe_all()/quote_all()

live/monitors/cross_venue.py (CrossVenueMonitor):
  Cross-exchange lead-lag detection via cross-correlation at multiple
  lags. Spot premium (basis proxy) computation. Multi-venue summary.

live/monitors/funding_basis.py (FundingBasisMonitor):
  Funding regime classification, momentum detection, carry PnL
  estimation, basis spread analysis. Uses microstructure/funding.py.

live/monitors/liq_risk.py (LiquidationRiskOverlay):
  Per-position liquidation distance monitoring with tiered warnings
  (safe/warning/danger/critical). Recommended position reduction.

38 tests across 4 files (all pass):
  test_live_filters.py (5)
  test_live_maker.py (9)
  test_live_monitors.py (11)
  test_live_treasury.py (13)

Total test suite: 172 tests, all passing.
2026-08-07 14:47:08 +08:00
ramseshk 639dd4fb6d feat: Phase 3 — event-driven market-making simulator + 53 tests
New sim/ module — 7 files + init, replays stored L2/trade data
through a realistic market-making simulation:

sim/engine.py (SimulationEngine):
  Event-driven core — processes L2 updates, trades, mark prices
  sequentially. Orchestrates queue model, maker quotes, fill sim,
  constraints, scenarios. Supports periodic re-quoting and
  stale order cancellation.

sim/queue.py (QueueModel):
  Price-time FIFO queue per price level. Tracks where maker orders
  sit in queue. Simulates order eating by aggressor trades.
  fill_probability() — Poisson thinning model for fill odds.

sim/maker.py:
  AvellanedaStoikovMaker — stochastic control quoting with
    aeta, k, tau parameters. Reservation price based on inventory.
    quote() and quote_with_skew() with configurable inventory tilt.
  GridMaker — evenly-spaced grid quoting at N levels.

sim/fills.py:
  FillSimulator — partial fills, adverse selection probability,
    cancel latency (gaussian RTT). FillEvent/CancelEvent tracking.
  adverse_selection_intensity() — measures post-fill price moves.

sim/constraints.py:
  InventoryConstraint — long/short/net/gross position limits.
  FundingConstraint — hourly funding cost estimation.
  FeeSchedule — maker/taker fee calculation.
  LiquidationRisk — liquidation price and safety distance.
  CircuitBreaker — PnL, trade count, toxic rate, slippage trips.
  ConstraintManager — unified pre-trade constraint check.

sim/scenario.py:
  ScenarioEngine — randomized exchange downtimes, latency spikes,
    volatility bursts. State query per sim_time for spread/trade-rate.

sim/reporter.py:
  PnLReporter — component-level PnL breakdown:
    spread_capture, inventory_pnl, fees, funding, adverse_selection.
  SimulationStats — trade counts, fill rates, drawdown, sharpe.
  Equity curve tracking and max drawdown computation.

53 new tests across 4 files (all pass):
  test_sim_queue.py (12) — order placement, FIFO, fills, cancels
  test_sim_maker.py (9) — A-S quotes, inventory skew, grid maker
  test_sim_constraints.py (14) — limits, funding, fees, liquidation, breakers
  test_sim_reporter.py (12) — PnL components, equity curve, stats
  test_sim_engine.py (6) — full engine integration

Total test suite: 134 tests, all passing.
2026-08-07 14:39:59 +08:00
ramseshk fcfc136384 feat: Phase 2 — microstructure analytics + 81 tests
New microstructure/ module with pure-function analytics:

microstructure/book.py:
  microprice() — depth-weighted mid price
  mid_price() — simple bid/ask midpoint
  order_book_imbalance() — ranged [-1, 1] volume skew
  depth_imbalance() — imbalance at fixed price distance
  spread_stats() — spread, spread_bps, mid, bid, ask
  depth_resiliency() — bid/ask volume within impact radius
  queue_depletion_prob() — Poisson fill probability at level
  batch_book_stats() — aggregate stats across snapshots

microstructure/trades.py:
  classify_lee_ready() — Lee-Ready aggressor classification
  classify_bulk_lee_ready() — batch classification with mids/bids/asks
  compute_markouts() — forward mid-price change at configurable horizons
  markout_summary() — mean/std/t-stat per side per horizon
  trade_volume_profile() — size bucket distribution
  trade_arrival_rate() — rolling trades/sec with burst detection

microstructure/toxicity.py:
  compute_vpin() — volume-synchronized informed trading probability
  compute_vpin_time_series() — rolling VPIN with alarm threshold
  fill_toxicity() — adverse price movement post-trade
  adverse_selection_ratio() — per-side adverse selection
  liquidation_clustering() — cluster detection in liquidation events

microstructure/funding.py:
  funding_regime() — classify regime (neutral/positive/negative/high)
  funding_predictability() — AR(1) autocorrelation analysis
  funding_carry_pnl() — cumulative carry PnL estimation
  basis_spread() — perp premium over spot (bps)
  basis_convergence_speed() — mean-reversion half-life via AR(1)

microstructure/signals.py:
  composite_signal() — weighted OBI + trade + VPIN + funding signal
  SignalPipeline — stateful pipeline accumulating book/trade updates
  detect_hft_regime() — regime classifier for HFT strategy selection

Bug fixes in Phase 1:
  - data/latency.py: proper linear-interpolation percentiles
  - data/normalizer.py: UTC timezone for naive datetimes
  - data/normalizer.py: detect_sequence_gap returns gap-1 (missing count)
  - microstructure/toxicity.py: consistent vpin_value key in compute_vpin

81 tests across 4 test files (store, normalizer, latency, microstructure)
2026-08-07 14:34:18 +08:00
ramseshk a7f811eb81 feat: Phase 1 — real-time & historical data system
New data/ module with:
- data/store.py: Parquet-based raw message storage with background writer
  thread. Messages partitioned by channel/coin/date. Thread-safe queue.
  Supports pyarrow Parquet with zstd compression. Includes read_range()
  helper for replay.

- data/collectors/hyperliquid.py: HL WebSocket + REST collector
  - WebSocket: l2Book (full book reconstruction), trades, allMids (mark prices)
  - REST pollers: funding rates, predicted funding, open interest, liquidations
  - Per-coin OrderBook class with snapshot/update reconstruction
  - Sequence gap detection with per-coin re-snapshot on gap
  - Latency tracking (exchange transport, signal, order, roundtrip)
  - Periodic stats reporter (book stats + latency summary every 60s)
  - CLI entrypoint: python -m data.collectors.hyperliquid --coins BTC ETH

- data/normalizer.py: Timestamp normalization (ms, s, ISO strings from
  HL/Binance/Bybit/OKX/Coinbase/Deribit) + SequenceTracker with gap detection

- data/latency.py: Rolling-window latency metrics (p50/p90/p95/p99) for
  transport, signal computation, order submission, and roundtrip

- Added pyarrow + aiohttp to requirements.txt
2026-08-07 14:28:21 +08:00
ramseshk b13ce68fef fix: inverted condition in loadResults — new API response format was falling into wrong branch 2026-08-07 14:17:03 +08:00
ramseshk 6889e06a86 feat: VBT dashboard overhaul — pagination, caching, LTTB downsampling, deep links, export, more metrics
- Merge vbt_server.py into server.py (eliminate duplicated VBT API)
- Add server-side pagination (offset/limit) with metadata (total, has_more)
- Add server-side ?asset= filtering to results endpoint
- Add JSON file caching with 5s TTL to avoid re-parsing on every request
- Add LTTB (Largest-Triangle-Three-Buckets) downsampling for equity curves
- Add pre-computed drawdown curve to result detail response
- Add /api/vbt/result/{filename}/csv endpoint for trade export
- Add Calmar ratio and expectancy to results metadata
- Rebuild vbt.html frontend with:
  - URL hash deep-linking (#filename) for bookmarkable views
  - JSON and CSV export buttons in detail panel
  - More metrics: Calmar, Sortino, Expectancy, End Equity (10 total)
  - Running backtest progress indicator with elapsed seconds
  - Pagination controls (prev/next) with page info
  - Filter/sort changes auto-apply (no manual refresh needed)
  - Better error states with retry buttons
  - Run strategy selector independent of filter
2026-08-07 14:14:40 +08:00
ramseshk 92ba6a564a chore: ignore generated VBT backtest result files 2026-08-07 12:59:34 +08:00
ramseshk 887a33f278 feat: trade log table, strategy params panel, B+W color scheme
Dashboard:
- Trade log table: all trades with time, side, size, entry/exit price, PnL, duration
  in scrollable panel below charts
- Strategy params panel: displays all coefficients (z_entry, gamma, obi_entry,
  grid_levels, etc.) for the selected strategy
- Color scheme: professional black/white
  • positive: #03A9F4 (light blue)
  • negative: #FF5252 (red)
  • neutral: #777 (gray)
  • backgrounds: #0a0a0a / #111 / #181818
  • borders: #222 / #333

VBT runner:
- _extract_metrics now captures trades from pf.trades.records_readable
  (Avg Entry Price, Avg Exit Price, PnL, Return, Duration, Direction)
- _strategy_params() returns key coefficients per strategy type
- _empty_result includes empty trades/params

New vbt_server.py: minimal standalone dashboard (no live trading machinery,
no memory guard, no broadcast loop) — avoids crashing issues
2026-08-07 12:53:52 +08:00
ramseshk 121c67ae5f feat: VBT dashboard — asset badges, interval/bar selectors, sort/filter
Dashboard (vbt.html):
- Interval selector: 1m, 5m, 15m, 1h, 4h, 1d (all Hyperliquid intervals)
- Candle limit selector: 100-5000 bars (6 levels)
- Asset selector: auto/BTC/ETH/SOL for run
- Strategy filter dropdown
- Sort dropdown: Latest, Sharpe, Return%, Min DD, Trades
- Asset badge on every result item in sidebar
- Asset interval filter for results list
- Improved layout: compact 3-row control panel

API (server.py):
- /api/vbt/results: new sort param (sharpe/return/dd/trades/date)
  new interval filter, asset field with _infer_asset()
- /api/vbt/run: new coin param, interval already supported
  coin suffix in saved filenames
- _infer_asset(): maps strategy names to BTC/ETH/BTC-ETH/SOL

Verified: sort=sharpe shows A-S S=+11.37, interval=1h filters
correctly, 7 dashboard controls rendered, asset badges on all items
2026-08-07 12:41:08 +08:00
ramseshk 623345c4d7 fix: normalize old backtest field names in VBT dashboard API
Old files used pnl_pct (not total_return_pct), max_dd (decimal,
not max_drawdown_pct %), num_periods (not n_bars), no profit_factor.
Added _normalize_vbt_fields() that:
- Maps pnl_pct/ann_return_pct → total_return_pct
- Converts max_dd (decimal) → max_drawdown_pct (percentage)
- Maps num_periods → n_bars
- Computes profit_factor from trades (gross_wins / gross_losses)
- Computes win_rate from trades if missing

Both /api/vbt/results and /api/vbt/result/{filename} now normalise.
Verified: old Cartea-Jaimungal file now shows ret=0.82%, pf=1.18, bars=720
2026-08-07 12:35:18 +08:00
ramseshk 737b24895c fix: VBT dashboard — proper metrics display + redesigned UI
- Fixed total_return_pct, profit_factor, n_bars showing 0 in detail view
  by using ?? operator instead of || 0 and fixing renderDetail logic
- Run Backtest now renders result directly from API response
  (no re-fetch race condition)
- Redesigned UI: monospace trading terminal aesthetic
  - Darker palette (#090d14 background, #0d1321 cards)
  - Indigo histogram, proper grid layout
  - Subtle borders (1px #1a2332), better spacing
  - Status indicator with pulse animation
  - Sidebar shows Sharpe, Return%, Profit Factor per result
  - 8 metric cards: Return, Sharpe, DD, Win Rate, PF, Trades,
    End Equity, Sortino
  - Smaller, cleaner fonts, monospace throughout
- Bumped memory guard to 2GB to prevent dashboard getting killed
2026-08-07 12:31:24 +08:00
ramseshk 3606e7f92e feat: proper Grid MM, Composite MM, Hurst/VPIN, Iceberg, A-S strategies
New strategies (strategies/nt/):
- GridMMNT: symmetric limit order grid around mid-price, captures spread from
  oscillation. Simulates fills from candle high/low. Rebuilds grid every 20 bars.
- CompositeMMNT: weighted ensemble of OBI (30%) + A-S inventory skew (40%) +
  Hurst/VPIN (30%). Votes: +1 long, -1 short, 0 neutral. Entry |score| > 0.5.
- IcebergNT: volume spike detection for whale accumulation. Dual-mode:
  candle proxy (volume > avg*2.5, >= 3 consecutive same-direction) and
  L2 wall detection (single level > avg*3). Exit on stop-loss/time/spike-fade.

Fixed strategies:
- Hurst/VPIN VBT: added proper VPIN proxy from candle volume (buy_vol when close
  > open, sell_vol when close < open). 50-bar rolling VPIN window. Signal:
  H>0.55 AND VPIN>0.25 AND |direction|>0.05. Exit: H<0.45 or direction flips.
- Hurst/VPIN paper trader: added HurstVPINLive integration (was missing entirely)
- A-S VBT: replaced placeholder spread filter with proper A-S simulation using
  reservation price formula (mid - q*gamma*sigma^2*tau), inventory tracking
- A-S NT formula: fixed to standard: mid - q*gamma*sigma^2*tau (was scaled by
  notional and gamma_scale improperly)
- Iceberg VBT: new volume spike detection replacing the old trend proxy

Registry: all 7 strategies now ✅ (pairs, hurst_vpin, as_mm, obi, grid_mm,
         composite_mm, iceberg)

VBT backtest results (500 BTC 1h bars):
  pairs:       -2.81%  13 trades   38% win
  hurst_vpin:  -0.77%   1 trade    (VPIN now active, very selective)
  as_mm:       -16.38%  73 trades  29% win
  obi:         -7.19%  15 trades    7% win
  grid_mm:     -4.79%  22 trades  33% win
  iceberg:     0 trades (threshold strict for 1h BTC data)
2026-08-07 11:42:32 +08:00
ramseshk 37da46a016 feat: proper Order Book Imbalance strategy for BTC-USD on HL
strategies/nt/obi_nt.py:
- Dual-mode OBI: candle proxy (backtest) + real L2 orderbook (live)
- Volume-based imbalance: buy_vol / (buy_vol + sell_vol) over rolling window
- Entry when |imbalance| > 0.35, exit on reversion < 0.10
- Stop-loss 2%, take-profit 0.5%, cooldown 3 bars
- compute_signal(price, orderbook=None) for paper trader integration

backtests/vbt_runner.py:
- Replaced placeholder z-score with proper volume-based OBI
- Buy vol = volume where close > open, sell vol = volume where close < open
- Rolling window imbalance computation
- Parameter sweep support with 12 combos tested

Registered across: deploy.py, nt_runner.py, dashboard, strategies/nt/__init__

Verified:
- VectorBT OBI backtest: 15 trades, -7.2% on default (window=20)
- Param sweep best: w=30 t=0.35 → sharpe -0.82, 49% win, 23% DD
- Real L2 orderbook signal: BUY obi=0.880 (bids 88% of depth)
- NT backtest engine: 201 bars, 8 days, 236ms
2026-08-07 10:50:43 +08:00
ramseshk 879372f69e merge: resolve conflicts, keep local framework changes 2026-08-06 17:52:55 +08:00
ramseshk 9cf871be46 Fix order pricing: 1-tick advantage at best bid/ask + process guard
A-S was quoting at best bid/ask (0% win) — orders filled but
0.04% round-trip maker fee exceeded spread capture.
Now: bid+1 / ask-1 = captures spread minus 1 tick each side.

Signal-driven strategies: same 1-tick pricing instead of
0.03% offset that crossed the book or sat too far away.

Added fcntl file lock to prevent duplicate live nodes.
Added IOC fallback (market-crossing) when post-only rejected.

A-S win rate: 0% → 25% (first 8 trades with new pricing)
2026-08-06 09:47:21 +00:00
ramseshk 6934bfdaa0 feat: VectorBT results dashboard with Plotly charts
Dashboard (dashboard/):
- New /api/vbt/results — list VBT backtest results with full metrics
- New /api/vbt/result/{file} — load result + equity curve (auto-decimated >500pts)
- New /api/vbt/run — run backtests on-demand from the UI
- New /api/vbt/sweep — parameter sweep as heatmap data
- New /api/vbt/strategies — list available strategy keys
- New /vbt — interactive HTML dashboard (Plotly.js):
  - Equity curve chart with area fill
  - Drawdown waterfall chart
  - Returns distribution histogram
  - Metric cards: Sharpe, Sortino, max DD, win rate, profit factor
  - Strategy filter sidebar
  - One-click backtest runner
- Fix BACKTEST_DIR auto-detection for local/dev paths

API verified: all 5 endpoints tested against live data
2026-08-06 17:43:47 +08:00
ramseshk 39545ac94b fix: NT backtest engine venue registration and bar precision
- Fix add_venue call with required OmsType, AccountType, Money params
- Fix Bar volume precision to match instrument size_precision
- Fix subscribe_bars to use BarType not InstrumentId
- Fix _submit_order to gracefully handle NT internal API
- All tests pass: VBT, NT, signals, paper exec, param sweep
2026-08-06 17:33:52 +08:00
ramseshk f5ffe4baee feat: NautilusTrader + VectorBT unified framework for Hyperliquid
Add complete framework for testing and deploying quant strategies:

Framework (framework/):
- HyperliquidInstrumentCatalog: loads perps as NT CryptoPerpetual
- HyperliquidDataProvider: real candle/orderbook/mark-price data
- HyperliquidExecutionProvider: live + PaperExecutionProvider: simulated
- BaseHlStrategy: shared NT strategy lifecycle with signal library
- StrategyConfig: YAML-based parameter management
- DeployOrchestrator: CLI for backtest -> paper -> live pipeline

Backtesting (backtests/):
- VBTBacktestRunner: VectorBT vectorized backtests on real HL candles
- NTBacktestRunner: NautilusTrader event-driven backtest engine

NT Strategy ports (strategies/nt/):
- PairsTradingNT: BTC/ETH ratio Z-score mean reversion
- HurstVPINNT: Hurst exponent regime + VPIN flow imbalance
- ASMarketMakingNT: Avellaneda-Stoikov stochastic control MM

E2E verified: real HL candles fetch, VectorBT backtest (Sharpe 5.2
on Hurst/VPIN), instrument catalog, deploy CLI --list, strategy signals.
Existing live/node.py and paper_trader.py unchanged.
2026-08-06 17:23:49 +08:00
ramseshk 8461ed5097 Live open orders/positions + A-S gamma fix + MR 60-tick window
1. Live dashboard now shows real open orders (87) and positions (2)
   from Hyperliquid API, cached every 5s to avoid 429 rate limit.

2. A-S gamma scaling: gamma*500K gives ~0 skew at max inventory
   (was bash.003, functionally identical to naive dual-quote).

3. Mean Reversion: 60-tick window with 0.5σ threshold
   (20s of 1s ticks was noise, not mean-reverting).

4. Sizes reduced for margin safety (wallet 86, 9 concurrent orders).

5. Kalman win_rate bug fixed: added net_pnl/gross_pnl field support.
2026-08-06 09:13:51 +00:00
ramseshk 2429394cd8 Deep audit fixes: A-S gamma scaling + Mean Rev window
1. A-S reservation price now uses gamma*500000 scaling.
   Before: bash.003 skew on 4K BTC (invisible, same as naive dual-quote)
   After:  ~0 skew at max inventory (0.05% of mid — enough to suppress one side)

2. Mean Reversion: 20-tick → 60-tick window, threshold 1.0σ → 0.5σ.
   20 seconds of 1s ticks is noise, not mean-reverting.
   60 seconds captures real short-term reversion dynamics.

Fill attribution verified: BTC sizes differ by 50 μBTC, ETH by 0.0025 — all above matching tolerance.
Orderbook null guards present — no crash on failed fetch.
2026-08-06 08:34:37 +00:00
ramseshk a6905f2691 Fix win_rate() for Kalman Pairs: add net_pnl/gross_pnl field support
Bug: win_rate() only checked pnl_net/pnl_gross/pnl fields,
but Kalman backtests save trades with net_pnl/gross_pnl (underscore-first).
Result: all 4 Kalman assets showed 0% win on 27-35 trades.

After fix:
  BTC: 0% → 45% (16/35)
  ETH: 0% → 47% (16/34)
  HYPE: 0% → 51% (14/27)
  VVV: 0% → 57% (19/33)

Also corrected paper trader coin assignments for Mean Reversion
and Momentum Breakout (was BTC, should be ETH).
2026-08-06 08:26:35 +00:00
ramseshk 37b8496dc2 Optimal position sizing: 4x BTC, 40x ETH utilization
Strategy          Old→New Notional   Capital Utilization
─────────────────────────────────────────────────────────
OBI (BTC)          3→1           12%→51%  (4x)
Iceberg (BTC)      3→4           13%→54%  (4x)
Funding (BTC)      4→8           14%→58%  (4x)
A-S MM (BTC)       5→1           15%→61%  (4x)
Hurst VPIN (BTC)   5→4           15%→64%  (4x)
Momentum (ETH)     →8             1%→38%  (40x)
Mean Reversion (ETH) →3           1%→43%  (45x)
Kalman Pairs (ETH) 0→8           10%→48%  (5x)
Pairs Trading (ETH) 1→2          11%→52%  (5x)

ETH strategies were using <1% of capital — essentially generating no PnL.
Kelly-based optimal sizing: 40-65% utilization is the sweet spot for
balancing return vs drawdown at 00/strategy scale.
2026-08-06 08:16:48 +00:00
ramseshk 74113ab624 A-S MM backtest: 4 assets with FIFO round-trip PnL
Results on real 5m candle data (7 days):
  BTC: +0.65% PnL | 506 matched | 72% win | 1044 fills
  ETH: 0.00% PnL | 505 matched | 57% win
  HYPE: 0.00% PnL | 510 matched | 61% win
  VVV: 0.00% PnL | 512 matched | 42% win

Side-selection via reservation price reduces adverse fills.
BTC shows clear edge: spreads are wider in absolute terms.
2026-08-06 08:11:46 +00:00
ramseshk f9bed72b1c Proper A-S: side selection via reservation price (not spread formula)
The AS optimal spread formula gives absurd spreads at crypto scale.
Real market makers quote at the MARKET spread (best bid/ask) and use
AS to decide WHEN to quote based on inventory-adjusted fair value:
  r = s - q * gamma * sigma^2 * tau

If r < best_bid (long-biased) → stop quoting bid
If r > best_ask (short-biased) → stop quoting ask
If circuit breaker active → pause both sides

Decoupled: spread is market-driven, inventory skew is AS-driven.
2026-08-06 08:04:49 +00:00