rams/ftdt-quant-lab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
130 Commits 1 Branch 0 Tags
140b0cc3607fc5ab9989aadeae74af14c3f50fad
Commit Graph

7 Commits

Author SHA1 Message Date
ramseshk e0be9f4d40 feat: advanced microstructure — sequencer latency, dealer GEX, tick regime, triangular arb 
4 new modules with 20 tests:

#22 Sequencer Latency Detector (live/monitors/sequencer_latency.py):
  Detects stale-state windows between WebSocket and REST API.
  - WebSocket vs REST timestamp delta tracking
  - Transport latency percentiles (p50, p99)
  - Liquidation-triggered stale-state detection
  - Stale asset identification for cross-margin arbitrage

#25 Dealer GEX (microstructure/dealer_gex.py):
  Dealer Gamma Exposure estimation via Black-Scholes.
  - Per-strike gamma × OI × spot² GEX computation
  - Pin level detection (strikes where dealers are long gamma)
  - Net GEX aggregation
  - Signal: fade_breakout (long gamma pinning) vs ride_momentum
    (short gamma amplification)
  - nearest_pin() for distance-to-magnet calculation

#30 Tick-Size Regime Exploitation (microstructure/tick_regime.py):
  Detects when asset price approaches tick-size boundaries.
  - Hyperliquid tick schedule (BTC 0.1/0.5, ETH 0.01/0.05, SOL 0.001/0.005)
  - Boundary approach detection with configurable threshold
  - Linear trend estimation for expected bars-to-cross
  - Signal: widen_quotes or tighten_quotes with urgency classification
  - Per-coin state tracking

#33 Triangular Latency Arb (live/monitors/triangular_arb.py):
  Cross-venue A→B→C triangular arbitrage detection.
  - Internal triangular: BTC-USDT → ETH-BTC → ETH-USDT
  - Cross-venue: price discrepancy across slow/fast venues
  - Latency gap detection between venue pairs
  - Implied cross-rate computation vs direct quote
  - Minimum spread threshold gating
 2026-08-10 10:34:56 +08:00
ramseshk 5304534e38 feat: advanced microstructure modules — HLP, Hawkes, Whipsaw, Term Structure, Liq Waterfall, Spoof Detector 
6 new modules with 46 new tests (230 total):

#21 HLP Vault Monitor (live/monitors/hlp_vault.py):
  Tracks Hyperliquid's native protocol market maker at address
  0xfefefe... Queries clearinghouseState + metaAndAssetCtxs.
  - Delta exposure per asset (notional + PnL)
  - Overextension detection (notional exceeds  M threshold)
  - Rebalancing signals: fade_short when HLP too short,
    fade_long when HLP too long (front-run forced rebalancing)
  - Toxicity score: HLP losing money = absorbing informed flow
  - Historical delta tracking

#29 Hawkes Processes (microstructure/hawkes.py):
  Multivariate Hawkes calibrator for limit order book dynamics.
  - MLE calibration via SGD gradient descent on log-likelihood
  - Branching ratio enforcement (alpha/beta < 0.99 for stationarity)
  - Intensity computation λ_i(t) with cross-excitation
  - Activity forecasting (expected event count in horizon)
  - Synthetic event generator (Ogata thinning)
  - Pure functions: hawkes_intensity, hawkes_log_likelihood,
    generate_hawkes_events

#23 Funding Whipsaw Trader (live/strategies/funding_whipsaw.py):
  Premium index decay trading in final 60s of funding epoch.
  - Detects deterministic convergence of premium→0 at settlement
  - Time-scaled position sizing (larger closer to settlement)
  - Auto-close after funding epoch completes
  - Confidence scoring based on premium magnitude

#32 Term Structure Monitor (live/monitors/term_structure.py):
  Perp/quarterly/bi-quarterly futures basis curve trading.
  - Quarterly-perp basis with z-score anomaly detection
  - BiQ-quarterly curve steepness monitoring
  - Fair quarterly price via interest rate parity + funding carry
  - Calendar spread signals: buy_basis, sell_basis,
    curve_steepener, curve_flattener

#24 Liquidation Waterfall (live/monitors/liq_waterfall.py):
  Cross-margin liquidation order prediction.
  - Margin ratio tracking (equity / maintenance margin)
  - Danger/critical level classification
  - Asset liquidation priority: maintenance / book_liquidity ratio
    (least liquid asset relative to margin = dumped first)
  - Strategy output: widen_spreads on target, tighten on rest

#31 Spoof Detector (microstructure/spoof_detector.py):
  Adversarial ML-style spoofing pattern recognition.
  - Rule 1: Large order far from mid, cancelled immediately
  - Rule 2: Cancel right before trade approaches price level
  - Rule 3: Oversized order with no fill within short lifetime
  - Spoof probability (rolling window ratio)
  - Cancel-to-fill ratio monitoring
 2026-08-07 17:52:20 +08:00
ramseshk e58c5951b7 feat: Phase 5 — integration layer (analytics pipeline, production node v2, CLI) + 12 tests 
live/integrator.py (AnalyticsPipeline):
  Real-time pipeline: data → microstructure → signals.
  Accumulates book snapshots + trades, computes OBI, VPIN, microprice,
  spread, depth, trade imbalance, HFT regime, and emits composite
  signal with confidence and breakdown. Per-coin isolation.

live/node_v2.py (ProductionNode):
  Rebuilt production node integrating ALL Phase 1-4 modules:
  - REST data fetching (order book, mark prices, funding rates)
  - AnalyticsPipeline per coin for real-time microstructure signals
  - Treasury for position/capital/PnL/breaker management
  - ToxicityFilter integration via HlMakerPool makers
  - HlMakerPool for per-coin A-S quoting
  - CrossVenueMonitor, FundingBasisMonitor, LiquidationRiskOverlay
  - Paper trading with probabilistic fill simulation
  - Dashboard metrics JSON output (equity, treasury, analytics, maker)
  - Periodic status logging

cli.py (unified CLI):
  Subcommands integrating all modules:
    collect   — Run Hyperliquid data collector to Parquet
    analyze   — Run microstructure analytics on stored data
    simulate  — Run market-making simulator on stored data
    run       — Start production trading node (paper or live)
    backtest  — Run VectorBT backtest

12 integration tests (all pass):
  - AnalyticsPipeline: empty, book, trade, VPIN, emit, regime, isolation
  - ProductionNode: creation, tick cycle (3 ticks), metrics JSON output
  - CLI: import verification

Total test suite: 184 tests, all passing.
 2026-08-07 14:54:47 +08:00
ramseshk 4f66ef36a9 feat: Phase 4 — controlled strategy deployment module + 38 tests 
New live/ sub-modules for production-ready market making:

live/filters/toxicity.py (ToxicityFilter):
  VPIN-based pre-trade filter. Accumulates buy/sell volume, computes
  VPIN via microstructure module, produces quoting decision:
    - allow_quoting: bool
    - size_multiplier: 0.0–1.0 (graduated reduction approaching alarm)
    - granular thresholds (threshold vs alarm) with smooth reduction

live/treasury.py (Treasury):
  Central capital/risk management — single source of truth:
  - Position tracking per coin (opening, closing, average entry)
  - Realized + unrealized PnL computation
  - Pre-trade constraint checks (inventory limits, fee estimates)
  - Circuit breaker (drawdown, trade count, toxic fill rate, API errors)
  - Liquidation distance monitoring
  - Automatic cooldown reset after trip expiry

live/makers/hl_btc_eth.py:
  HlMaker — per-coin market maker integrating:
    - AvellanedaStoikovMaker (Phase 3) for optimal quotes
    - ToxicityFilter for pre-trade gating
    - Treasury for position/risk checks
  HlMakerPool — manages multiple HlMaker instances with shared treasury
    and coordinated observe_all()/quote_all()

live/monitors/cross_venue.py (CrossVenueMonitor):
  Cross-exchange lead-lag detection via cross-correlation at multiple
  lags. Spot premium (basis proxy) computation. Multi-venue summary.

live/monitors/funding_basis.py (FundingBasisMonitor):
  Funding regime classification, momentum detection, carry PnL
  estimation, basis spread analysis. Uses microstructure/funding.py.

live/monitors/liq_risk.py (LiquidationRiskOverlay):
  Per-position liquidation distance monitoring with tiered warnings
  (safe/warning/danger/critical). Recommended position reduction.

38 tests across 4 files (all pass):
  test_live_filters.py (5)
  test_live_maker.py (9)
  test_live_monitors.py (11)
  test_live_treasury.py (13)

Total test suite: 172 tests, all passing.
 2026-08-07 14:47:08 +08:00
ramseshk 639dd4fb6d feat: Phase 3 — event-driven market-making simulator + 53 tests 
New sim/ module — 7 files + init, replays stored L2/trade data
through a realistic market-making simulation:

sim/engine.py (SimulationEngine):
  Event-driven core — processes L2 updates, trades, mark prices
  sequentially. Orchestrates queue model, maker quotes, fill sim,
  constraints, scenarios. Supports periodic re-quoting and
  stale order cancellation.

sim/queue.py (QueueModel):
  Price-time FIFO queue per price level. Tracks where maker orders
  sit in queue. Simulates order eating by aggressor trades.
  fill_probability() — Poisson thinning model for fill odds.

sim/maker.py:
  AvellanedaStoikovMaker — stochastic control quoting with
    aeta, k, tau parameters. Reservation price based on inventory.
    quote() and quote_with_skew() with configurable inventory tilt.
  GridMaker — evenly-spaced grid quoting at N levels.

sim/fills.py:
  FillSimulator — partial fills, adverse selection probability,
    cancel latency (gaussian RTT). FillEvent/CancelEvent tracking.
  adverse_selection_intensity() — measures post-fill price moves.

sim/constraints.py:
  InventoryConstraint — long/short/net/gross position limits.
  FundingConstraint — hourly funding cost estimation.
  FeeSchedule — maker/taker fee calculation.
  LiquidationRisk — liquidation price and safety distance.
  CircuitBreaker — PnL, trade count, toxic rate, slippage trips.
  ConstraintManager — unified pre-trade constraint check.

sim/scenario.py:
  ScenarioEngine — randomized exchange downtimes, latency spikes,
    volatility bursts. State query per sim_time for spread/trade-rate.

sim/reporter.py:
  PnLReporter — component-level PnL breakdown:
    spread_capture, inventory_pnl, fees, funding, adverse_selection.
  SimulationStats — trade counts, fill rates, drawdown, sharpe.
  Equity curve tracking and max drawdown computation.

53 new tests across 4 files (all pass):
  test_sim_queue.py (12) — order placement, FIFO, fills, cancels
  test_sim_maker.py (9) — A-S quotes, inventory skew, grid maker
  test_sim_constraints.py (14) — limits, funding, fees, liquidation, breakers
  test_sim_reporter.py (12) — PnL components, equity curve, stats
  test_sim_engine.py (6) — full engine integration

Total test suite: 134 tests, all passing.
 2026-08-07 14:39:59 +08:00
ramseshk fcfc136384 feat: Phase 2 — microstructure analytics + 81 tests 
New microstructure/ module with pure-function analytics:

microstructure/book.py:
  microprice() — depth-weighted mid price
  mid_price() — simple bid/ask midpoint
  order_book_imbalance() — ranged [-1, 1] volume skew
  depth_imbalance() — imbalance at fixed price distance
  spread_stats() — spread, spread_bps, mid, bid, ask
  depth_resiliency() — bid/ask volume within impact radius
  queue_depletion_prob() — Poisson fill probability at level
  batch_book_stats() — aggregate stats across snapshots

microstructure/trades.py:
  classify_lee_ready() — Lee-Ready aggressor classification
  classify_bulk_lee_ready() — batch classification with mids/bids/asks
  compute_markouts() — forward mid-price change at configurable horizons
  markout_summary() — mean/std/t-stat per side per horizon
  trade_volume_profile() — size bucket distribution
  trade_arrival_rate() — rolling trades/sec with burst detection

microstructure/toxicity.py:
  compute_vpin() — volume-synchronized informed trading probability
  compute_vpin_time_series() — rolling VPIN with alarm threshold
  fill_toxicity() — adverse price movement post-trade
  adverse_selection_ratio() — per-side adverse selection
  liquidation_clustering() — cluster detection in liquidation events

microstructure/funding.py:
  funding_regime() — classify regime (neutral/positive/negative/high)
  funding_predictability() — AR(1) autocorrelation analysis
  funding_carry_pnl() — cumulative carry PnL estimation
  basis_spread() — perp premium over spot (bps)
  basis_convergence_speed() — mean-reversion half-life via AR(1)

microstructure/signals.py:
  composite_signal() — weighted OBI + trade + VPIN + funding signal
  SignalPipeline — stateful pipeline accumulating book/trade updates
  detect_hft_regime() — regime classifier for HFT strategy selection

Bug fixes in Phase 1:
  - data/latency.py: proper linear-interpolation percentiles
  - data/normalizer.py: UTC timezone for naive datetimes
  - data/normalizer.py: detect_sequence_gap returns gap-1 (missing count)
  - microstructure/toxicity.py: consistent vpin_value key in compute_vpin

81 tests across 4 test files (store, normalizer, latency, microstructure)
 2026-08-07 14:34:18 +08:00
ramseshk 50f8f4f970 Refactor: review, fix, and test entire codebase 
Live node:
  - Fix null-handling for open_ords and get_fills requests
  - Cap equity_history, strategy_equity at 600-1000 entries (memory leak fix)
  - Dynamic strategy count in startup log
  - Loop error recovery: catch exceptions, backoff 5s, continue

Dashboard server:
  - Fix backtest detail API: check HISTORICAL_DIR first
  - This was causing all historical detail views to show zeros

Tests (5 suites, all passing):
  1. Signal generation: Mean Reversion VWAP + Momentum + Pairs + OBI
  2. Backtest: SPX mean reversion on 500-point series
  3. Hurst/VPIN: 15 signals from 280 dollar bars
  4. Memory guard: RSS monitoring, GC thresholds
  5. Dashboard API: historical listing + SPX detail

38 backtests on dashboard, 2 SPX entries with real trade data.
 2026-08-06 07:52:02 +00:00